2024-05-08T05:43:58.109 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-08T05:43:58.110 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_vshankar@teuthology --description fs:workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/direct_io}} --archive /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697251 --name vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi --verbose -- /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697251/orig.config.yaml 2024-05-08T05:43:58.161 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T05:43:58.252 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697251 branch: wip-57676-3 description: fs:workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/direct_io}} email: vshankar@redhat.com first_in_suite: false job_id: '7697251' kernel: client: branch: testing flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-57676-3 ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: bee299f4ac344e4c45e550d7a80abcadd7bc3d5a subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt dry_run: true install: ceph: flavor: default sha1: bee299f4ac344e4c45e550d7a80abcadd7bc3d5a kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-57676-3 sha1: bee299f4ac344e4c45e550d7a80abcadd7bc3d5a owner: scheduled_vshankar@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 5998 sha1: bee299f4ac344e4c45e550d7a80abcadd7bc3d5a sleep_before_teardown: 0 subset: 915/1024 suite: fs:workload suite_branch: wip-57676-3 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bee299f4ac344e4c45e550d7a80abcadd7bc3d5a syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi123.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKp3RfdXpwAxXRSW8LIssyBDMHM4Gn0NJNcACp55JeNVYmaYyv+JM5NVVNcfjwxN0lUDRnwnOpDjBzoQOznCcrg= smithi163.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKO66P7YXTVr8v7p8S9JWwdpCR9Q7gqr+EHYer4AvO0ILywjCzw133shhcn1R7uM564cLRPu1s6SMDRDOLMLhI4= smithi192.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMJ/MGQ13gaqyla9BkLvTpD5o04DT3Lr3FDBgmLJDU5qKXRqcnm5pWtGNeCSkkFffAIc5jeUQDdSVuMbnLbDRaQ= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - direct_io teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-08_04:00:39 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-08T05:43:58.252 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/qa; will attempt to use it 2024-05-08T05:43:58.253 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/qa/tasks 2024-05-08T05:43:58.253 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-08T05:43:58.254 INFO:teuthology.task.internal:Checking packages... 2024-05-08T05:43:58.276 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bee299f4ac344e4c45e550d7a80abcadd7bc3d5a' 2024-05-08T05:43:58.276 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-08T05:43:58.276 INFO:teuthology.packaging:ref: None 2024-05-08T05:43:58.276 INFO:teuthology.packaging:tag: None 2024-05-08T05:43:58.276 INFO:teuthology.packaging:branch: wip-57676-3 2024-05-08T05:43:58.276 INFO:teuthology.packaging:sha1: bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:43:58.276 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-57676-3 2024-05-08T05:43:58.495 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3538.gbee299f4 2024-05-08T05:43:58.496 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-08T05:43:58.503 INFO:teuthology.task.internal:no buildpackages task found 2024-05-08T05:43:58.503 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-08T05:43:58.522 INFO:teuthology.task.internal:Saving configuration 2024-05-08T05:43:58.540 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-08T05:43:58.546 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T05:43:58.572 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697251', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 05:38:15.277893', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKp3RfdXpwAxXRSW8LIssyBDMHM4Gn0NJNcACp55JeNVYmaYyv+JM5NVVNcfjwxN0lUDRnwnOpDjBzoQOznCcrg='} 2024-05-08T05:43:58.593 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi163.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697251', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 05:38:15.273519', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKO66P7YXTVr8v7p8S9JWwdpCR9Q7gqr+EHYer4AvO0ILywjCzw133shhcn1R7uM564cLRPu1s6SMDRDOLMLhI4='} 2024-05-08T05:43:58.613 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi192.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697251', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 05:38:15.275454', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMJ/MGQ13gaqyla9BkLvTpD5o04DT3Lr3FDBgmLJDU5qKXRqcnm5pWtGNeCSkkFffAIc5jeUQDdSVuMbnLbDRaQ='} 2024-05-08T05:43:58.613 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-08T05:43:58.619 INFO:teuthology.task.internal:roles: ubuntu@smithi123.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-08T05:43:58.619 INFO:teuthology.task.internal:roles: ubuntu@smithi163.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-08T05:43:58.619 INFO:teuthology.task.internal:roles: ubuntu@smithi192.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-08T05:43:58.619 INFO:teuthology.run_tasks:Running task console_log... 2024-05-08T05:43:58.712 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f99481450d0>, signals=[15]) 2024-05-08T05:43:58.713 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-08T05:43:58.719 INFO:teuthology.task.internal:Opening connections... 2024-05-08T05:43:58.719 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi123.front.sepia.ceph.com 2024-05-08T05:43:58.721 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:43:58.799 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi163.front.sepia.ceph.com 2024-05-08T05:43:58.800 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:43:58.873 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi192.front.sepia.ceph.com 2024-05-08T05:43:58.873 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:43:58.948 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-08T05:43:58.955 DEBUG:teuthology.orchestra.run.smithi123:> uname -m 2024-05-08T05:43:58.972 INFO:teuthology.orchestra.run.smithi123.stdout:x86_64 2024-05-08T05:43:58.973 DEBUG:teuthology.orchestra.run.smithi123:> cat /etc/os-release 2024-05-08T05:43:59.029 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="CentOS Stream" 2024-05-08T05:43:59.029 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="9" 2024-05-08T05:43:59.030 INFO:teuthology.orchestra.run.smithi123.stdout:ID="centos" 2024-05-08T05:43:59.030 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="rhel fedora" 2024-05-08T05:43:59.030 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="9" 2024-05-08T05:43:59.030 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el9" 2024-05-08T05:43:59.030 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-08T05:43:59.030 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31" 2024-05-08T05:43:59.030 INFO:teuthology.orchestra.run.smithi123.stdout:LOGO="fedora-logo-icon" 2024-05-08T05:43:59.030 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-08T05:43:59.030 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://centos.org/" 2024-05-08T05:43:59.030 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T05:43:59.030 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-08T05:43:59.030 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-08T05:43:59.031 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2024-05-08T05:43:59.054 DEBUG:teuthology.orchestra.run.smithi163:> uname -m 2024-05-08T05:43:59.072 INFO:teuthology.orchestra.run.smithi163.stdout:x86_64 2024-05-08T05:43:59.073 DEBUG:teuthology.orchestra.run.smithi163:> cat /etc/os-release 2024-05-08T05:43:59.129 INFO:teuthology.orchestra.run.smithi163.stdout:NAME="CentOS Stream" 2024-05-08T05:43:59.129 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION="9" 2024-05-08T05:43:59.129 INFO:teuthology.orchestra.run.smithi163.stdout:ID="centos" 2024-05-08T05:43:59.129 INFO:teuthology.orchestra.run.smithi163.stdout:ID_LIKE="rhel fedora" 2024-05-08T05:43:59.129 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION_ID="9" 2024-05-08T05:43:59.129 INFO:teuthology.orchestra.run.smithi163.stdout:PLATFORM_ID="platform:el9" 2024-05-08T05:43:59.129 INFO:teuthology.orchestra.run.smithi163.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-08T05:43:59.130 INFO:teuthology.orchestra.run.smithi163.stdout:ANSI_COLOR="0;31" 2024-05-08T05:43:59.130 INFO:teuthology.orchestra.run.smithi163.stdout:LOGO="fedora-logo-icon" 2024-05-08T05:43:59.130 INFO:teuthology.orchestra.run.smithi163.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-08T05:43:59.130 INFO:teuthology.orchestra.run.smithi163.stdout:HOME_URL="https://centos.org/" 2024-05-08T05:43:59.130 INFO:teuthology.orchestra.run.smithi163.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T05:43:59.130 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-08T05:43:59.130 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-08T05:43:59.130 INFO:teuthology.lock.ops:Updating smithi163.front.sepia.ceph.com on lock server 2024-05-08T05:43:59.154 DEBUG:teuthology.orchestra.run.smithi192:> uname -m 2024-05-08T05:43:59.171 INFO:teuthology.orchestra.run.smithi192.stdout:x86_64 2024-05-08T05:43:59.172 DEBUG:teuthology.orchestra.run.smithi192:> cat /etc/os-release 2024-05-08T05:43:59.228 INFO:teuthology.orchestra.run.smithi192.stdout:NAME="CentOS Stream" 2024-05-08T05:43:59.228 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION="9" 2024-05-08T05:43:59.228 INFO:teuthology.orchestra.run.smithi192.stdout:ID="centos" 2024-05-08T05:43:59.228 INFO:teuthology.orchestra.run.smithi192.stdout:ID_LIKE="rhel fedora" 2024-05-08T05:43:59.228 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION_ID="9" 2024-05-08T05:43:59.228 INFO:teuthology.orchestra.run.smithi192.stdout:PLATFORM_ID="platform:el9" 2024-05-08T05:43:59.228 INFO:teuthology.orchestra.run.smithi192.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-08T05:43:59.228 INFO:teuthology.orchestra.run.smithi192.stdout:ANSI_COLOR="0;31" 2024-05-08T05:43:59.228 INFO:teuthology.orchestra.run.smithi192.stdout:LOGO="fedora-logo-icon" 2024-05-08T05:43:59.229 INFO:teuthology.orchestra.run.smithi192.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-08T05:43:59.229 INFO:teuthology.orchestra.run.smithi192.stdout:HOME_URL="https://centos.org/" 2024-05-08T05:43:59.229 INFO:teuthology.orchestra.run.smithi192.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T05:43:59.229 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-08T05:43:59.229 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-08T05:43:59.229 INFO:teuthology.lock.ops:Updating smithi192.front.sepia.ceph.com on lock server 2024-05-08T05:43:59.254 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-08T05:43:59.265 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-08T05:43:59.292 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-08T05:43:59.293 DEBUG:teuthology.orchestra.run.smithi123:> test '!' -e /home/ubuntu/cephtest 2024-05-08T05:43:59.295 DEBUG:teuthology.orchestra.run.smithi163:> test '!' -e /home/ubuntu/cephtest 2024-05-08T05:43:59.297 DEBUG:teuthology.orchestra.run.smithi192:> test '!' -e /home/ubuntu/cephtest 2024-05-08T05:43:59.313 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-08T05:43:59.319 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-08T05:43:59.319 DEBUG:teuthology.orchestra.run.smithi123:> test -z $(ls -A /var/lib/ceph) 2024-05-08T05:43:59.352 DEBUG:teuthology.orchestra.run.smithi163:> test -z $(ls -A /var/lib/ceph) 2024-05-08T05:43:59.354 DEBUG:teuthology.orchestra.run.smithi192:> test -z $(ls -A /var/lib/ceph) 2024-05-08T05:43:59.444 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-08T05:43:59.512 INFO:teuthology.run_tasks:Running task kernel... 2024-05-08T05:43:59.526 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-05-08T05:43:59.526 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-05-08T05:43:59.526 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-05-08T05:43:59.527 DEBUG:teuthology.orchestra.run.smithi123:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T05:43:59.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:43:59.543 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-05-08T05:43:59.709 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-05-08T05:43:59.709 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-05-08T05:43:59.709 DEBUG:teuthology.orchestra.run.smithi123:> uname -r 2024-05-08T05:43:59.726 INFO:teuthology.orchestra.run.smithi123.stdout:5.14.0-437.el9.x86_64 2024-05-08T05:43:59.726 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-05-08T05:43:59.727 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-08T05:43:59.727 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum upgrade -y linux-firmware 2024-05-08T05:44:02.316 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - BaseOS 8.2 MB/s | 8.1 MB 00:00 2024-05-08T05:44:09.633 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - AppStream 3.1 MB/s | 19 MB 00:06 2024-05-08T05:44:17.125 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - CRB 1.6 MB/s | 6.2 MB 00:03 2024-05-08T05:44:18.525 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - Extras packages 71 kB/s | 16 kB 00:00 2024-05-08T05:44:19.787 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-05-08T05:44:25.001 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-05-08T05:44:27.119 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-05-08T05:44:27.120 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-05-08T05:44:27.120 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-05-08T05:44:27.236 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-05-08T05:44:27.236 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-05-08T05:44:27.395 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-05-08T05:44:27.396 DEBUG:teuthology.orchestra.run.smithi123:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-08T05:44:27.398 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi123.front.sepia.ceph.com to complete... 2024-05-08T05:44:39.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08 05:44:39 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-08T05:44:39.830 INFO:teuthology.orchestra.run.smithi123.stderr:FINISHED --2024-05-08 05:44:39-- 2024-05-08T05:44:39.830 INFO:teuthology.orchestra.run.smithi123.stderr:Total wall clock time: 12s 2024-05-08T05:44:39.830 INFO:teuthology.orchestra.run.smithi123.stderr:Downloaded: 1 files, 341M in 12s (28.3 MB/s) 2024-05-08T05:44:39.833 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-05-08T05:44:39.833 DEBUG:teuthology.orchestra.run.smithi123:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-08T05:44:41.971 INFO:teuthology.orchestra.run.smithi123.stdout:Verifying... ######################################## 2024-05-08T05:44:42.190 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing... ######################################## 2024-05-08T05:44:42.308 INFO:teuthology.orchestra.run.smithi123.stdout:Updating / installing... 2024-05-08T05:44:50.303 INFO:teuthology.orchestra.run.smithi123.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-05-08T05:45:28.979 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi123.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-08T05:45:28.979 DEBUG:teuthology.orchestra.run.smithi123:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-08T05:45:29.054 INFO:teuthology.orchestra.run.smithi123.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-05-08T05:45:29.054 INFO:teuthology.orchestra.run.smithi123.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-05-08T05:45:29.054 INFO:teuthology.orchestra.run.smithi123.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-05-08T05:45:29.055 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-05-08T05:45:29.055 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-05-08T05:45:29.055 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-05-08T05:45:29.055 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-05-08T05:45:29.055 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-05-08T05:45:29.055 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-05-08T05:45:29.055 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-08T05:45:29.055 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-08T05:45:29.055 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-08T05:45:29.055 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-08T05:45:29.055 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-08T05:45:29.055 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-08T05:45:29.055 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-08T05:45:29.055 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-08T05:45:29.056 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-08T05:45:29.056 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-08T05:45:29.056 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-08T05:45:29.056 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-08T05:45:29.056 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-08T05:45:29.056 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-08T05:45:29.056 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-08T05:45:29.056 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-08T05:45:29.056 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-08T05:45:29.056 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-08T05:45:29.056 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-08T05:45:29.056 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-08T05:45:29.056 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-05-08T05:45:29.057 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-05-08T05:45:29.057 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-05-08T05:45:29.057 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-08T05:45:29.057 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-05-08T05:45:29.057 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-08T05:45:29.057 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-08T05:45:29.057 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-05-08T05:45:29.057 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-05-08T05:45:29.057 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-05-08T05:45:29.057 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-05-08T05:45:29.057 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-05-08T05:45:29.057 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-05-08T05:45:29.057 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-05-08T05:45:29.058 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-05-08T05:45:29.058 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-08T05:45:29.058 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-05-08T05:45:29.058 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-05-08T05:45:29.058 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-05-08T05:45:29.058 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-05-08T05:45:29.058 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-05-08T05:45:29.058 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-05-08T05:45:29.058 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-05-08T05:45:29.058 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-05-08T05:45:29.058 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-05-08T05:45:29.058 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-05-08T05:45:29.058 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-05-08T05:45:29.059 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-05-08T05:45:29.059 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-05-08T05:45:29.059 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-05-08T05:45:29.059 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-05-08T05:45:29.059 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-05-08T05:45:29.059 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-05-08T05:45:29.059 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-05-08T05:45:29.059 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-05-08T05:45:29.059 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-05-08T05:45:29.059 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-05-08T05:45:29.059 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-05-08T05:45:29.059 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-05-08T05:45:29.059 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-05-08T05:45:29.060 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-05-08T05:45:29.060 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-05-08T05:45:29.060 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-05-08T05:45:29.060 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-05-08T05:45:29.060 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-05-08T05:45:29.060 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-05-08T05:45:29.060 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-05-08T05:45:29.060 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-05-08T05:45:29.060 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-05-08T05:45:29.060 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-05-08T05:45:29.060 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-05-08T05:45:29.060 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-05-08T05:45:29.060 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-05-08T05:45:29.060 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-05-08T05:45:29.061 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-05-08T05:45:29.061 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-05-08T05:45:29.061 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-05-08T05:45:29.061 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-05-08T05:45:29.061 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-05-08T05:45:29.061 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-05-08T05:45:29.061 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-08T05:45:29.061 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-05-08T05:45:29.061 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-05-08T05:45:29.061 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-05-08T05:45:29.061 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-05-08T05:45:29.061 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-05-08T05:45:29.061 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-05-08T05:45:29.061 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-05-08T05:45:29.062 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-05-08T05:45:29.062 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-05-08T05:45:29.062 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-05-08T05:45:29.062 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-05-08T05:45:29.062 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-05-08T05:45:29.062 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-05-08T05:45:29.062 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-05-08T05:45:29.062 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-05-08T05:45:29.062 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-05-08T05:45:29.062 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-05-08T05:45:29.062 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-05-08T05:45:29.062 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-05-08T05:45:29.062 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-05-08T05:45:29.062 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-05-08T05:45:29.063 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-05-08T05:45:29.063 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-05-08T05:45:29.063 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-08T05:45:29.063 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-08T05:45:29.063 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-05-08T05:45:29.063 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-05-08T05:45:29.063 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-05-08T05:45:29.063 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-05-08T05:45:29.063 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-05-08T05:45:29.063 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-05-08T05:45:29.063 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-05-08T05:45:29.063 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-05-08T05:45:29.063 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-05-08T05:45:29.063 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-05-08T05:45:29.064 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-05-08T05:45:29.064 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-05-08T05:45:29.064 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-05-08T05:45:29.064 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-08T05:45:29.064 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-08T05:45:29.064 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-05-08T05:45:29.064 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-05-08T05:45:29.064 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-05-08T05:45:29.064 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-05-08T05:45:29.064 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-05-08T05:45:29.064 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-05-08T05:45:29.064 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-05-08T05:45:29.064 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-05-08T05:45:29.065 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-05-08T05:45:29.065 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-05-08T05:45:29.065 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-05-08T05:45:29.065 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-05-08T05:45:29.065 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-05-08T05:45:29.066 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-05-08T05:45:29.066 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-05-08T05:45:29.066 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-05-08T05:45:29.066 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-05-08T05:45:29.066 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-05-08T05:45:29.067 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-05-08T05:45:29.067 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-05-08T05:45:29.067 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-05-08T05:45:29.067 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-05-08T05:45:29.067 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-05-08T05:45:29.067 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-05-08T05:45:29.067 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-05-08T05:45:29.067 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-05-08T05:45:29.067 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-05-08T05:45:29.067 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-08T05:45:29.067 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-08T05:45:29.067 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-05-08T05:45:29.067 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-05-08T05:45:29.068 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-05-08T05:45:29.068 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-05-08T05:45:29.068 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-05-08T05:45:29.068 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-05-08T05:45:29.068 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-05-08T05:45:29.068 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-05-08T05:45:29.068 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-05-08T05:45:29.068 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-05-08T05:45:29.068 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-05-08T05:45:29.068 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-08T05:45:29.068 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-05-08T05:45:29.068 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-05-08T05:45:29.068 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-05-08T05:45:29.068 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-05-08T05:45:29.068 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-05-08T05:45:29.069 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-05-08T05:45:29.069 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-05-08T05:45:29.069 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-08T05:45:29.069 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-05-08T05:45:29.069 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-05-08T05:45:29.069 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-05-08T05:45:29.069 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-05-08T05:45:29.069 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-05-08T05:45:29.069 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-05-08T05:45:29.069 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-08T05:45:29.069 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-08T05:45:29.069 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-08T05:45:29.069 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-05-08T05:45:29.069 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-08T05:45:29.070 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-05-08T05:45:29.070 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-08T05:45:29.070 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-08T05:45:29.070 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-08T05:45:29.070 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-08T05:45:29.070 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-08T05:45:29.070 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-08T05:45:29.070 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-05-08T05:45:29.070 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-05-08T05:45:29.070 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-05-08T05:45:29.070 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-05-08T05:45:29.070 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-05-08T05:45:29.070 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-05-08T05:45:29.070 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-05-08T05:45:29.071 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-08T05:45:29.071 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-08T05:45:29.071 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-08T05:45:29.071 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-08T05:45:29.071 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-08T05:45:29.071 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-05-08T05:45:29.071 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-05-08T05:45:29.071 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-05-08T05:45:29.071 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-08T05:45:29.071 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-08T05:45:29.071 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-08T05:45:29.071 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-08T05:45:29.071 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-08T05:45:29.071 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-08T05:45:29.072 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-05-08T05:45:29.072 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-05-08T05:45:29.072 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-08T05:45:29.072 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-05-08T05:45:29.072 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-05-08T05:45:29.072 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-05-08T05:45:29.072 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-05-08T05:45:29.072 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-08T05:45:29.072 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-08T05:45:29.072 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-08T05:45:29.072 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-05-08T05:45:29.072 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-05-08T05:45:29.072 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-05-08T05:45:29.073 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-08T05:45:29.073 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-05-08T05:45:29.073 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-05-08T05:45:29.073 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-05-08T05:45:29.073 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-08T05:45:29.073 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-05-08T05:45:29.073 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-05-08T05:45:29.073 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-08T05:45:29.073 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-05-08T05:45:29.073 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-08T05:45:29.073 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-05-08T05:45:29.073 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-05-08T05:45:29.073 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-05-08T05:45:29.073 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-05-08T05:45:29.073 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-05-08T05:45:29.074 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-05-08T05:45:29.074 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-05-08T05:45:29.074 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-05-08T05:45:29.074 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-05-08T05:45:29.074 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-05-08T05:45:29.074 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-05-08T05:45:29.074 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-05-08T05:45:29.074 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-05-08T05:45:29.074 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-05-08T05:45:29.074 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-05-08T05:45:29.074 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-05-08T05:45:29.074 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-08T05:45:29.074 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-05-08T05:45:29.075 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-05-08T05:45:29.075 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-08T05:45:29.075 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-05-08T05:45:29.075 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-05-08T05:45:29.075 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-05-08T05:45:29.075 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-05-08T05:45:29.075 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-05-08T05:45:29.075 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-05-08T05:45:29.075 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-05-08T05:45:29.075 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-05-08T05:45:29.075 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-05-08T05:45:29.075 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-05-08T05:45:29.075 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-05-08T05:45:29.075 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-08T05:45:29.076 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-08T05:45:29.076 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-05-08T05:45:29.076 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-05-08T05:45:29.076 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-05-08T05:45:29.076 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-05-08T05:45:29.076 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-05-08T05:45:29.076 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-05-08T05:45:29.076 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-05-08T05:45:29.076 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-05-08T05:45:29.076 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-08T05:45:29.076 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-08T05:45:29.077 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-05-08T05:45:29.077 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-05-08T05:45:29.077 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-05-08T05:45:29.077 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-05-08T05:45:29.077 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-05-08T05:45:29.077 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-05-08T05:45:29.077 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-05-08T05:45:29.077 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-05-08T05:45:29.077 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-05-08T05:45:29.077 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-08T05:45:29.077 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-05-08T05:45:29.078 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-08T05:45:29.078 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-08T05:45:29.078 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-08T05:45:29.078 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-08T05:45:29.078 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-08T05:45:29.078 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-08T05:45:29.078 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-08T05:45:29.078 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-08T05:45:29.078 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-08T05:45:29.078 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-05-08T05:45:29.078 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-05-08T05:45:29.078 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-05-08T05:45:29.078 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-08T05:45:29.078 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-05-08T05:45:29.079 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-05-08T05:45:29.079 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-05-08T05:45:29.079 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-05-08T05:45:29.079 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-05-08T05:45:29.079 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-08T05:45:29.079 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-05-08T05:45:29.079 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-05-08T05:45:29.079 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-05-08T05:45:29.079 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-05-08T05:45:29.079 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-05-08T05:45:29.079 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-05-08T05:45:29.079 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-05-08T05:45:29.079 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-05-08T05:45:29.079 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-05-08T05:45:29.079 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-05-08T05:45:29.080 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-05-08T05:45:29.080 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-05-08T05:45:29.080 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-05-08T05:45:29.080 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-05-08T05:45:29.080 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-05-08T05:45:29.080 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-05-08T05:45:29.080 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-05-08T05:45:29.080 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-05-08T05:45:29.080 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-05-08T05:45:29.080 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-08T05:45:29.080 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-05-08T05:45:29.080 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-05-08T05:45:29.080 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-05-08T05:45:29.080 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-05-08T05:45:29.081 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-05-08T05:45:29.081 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-05-08T05:45:29.081 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-05-08T05:45:29.081 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-05-08T05:45:29.081 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-05-08T05:45:29.081 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-05-08T05:45:29.081 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-05-08T05:45:29.081 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-05-08T05:45:29.081 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-05-08T05:45:29.081 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-05-08T05:45:29.081 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-05-08T05:45:29.081 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-05-08T05:45:29.081 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-05-08T05:45:29.081 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-05-08T05:45:29.081 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-05-08T05:45:29.082 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-08T05:45:29.082 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-05-08T05:45:29.082 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-05-08T05:45:29.082 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-08T05:45:29.082 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-05-08T05:45:29.082 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-05-08T05:45:29.082 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-05-08T05:45:29.082 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-05-08T05:45:29.082 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-05-08T05:45:29.082 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-05-08T05:45:29.082 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-05-08T05:45:29.082 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-05-08T05:45:29.082 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-05-08T05:45:29.083 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-05-08T05:45:29.083 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-05-08T05:45:29.083 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-05-08T05:45:29.083 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-05-08T05:45:29.083 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-05-08T05:45:29.083 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-08T05:45:29.083 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-05-08T05:45:29.083 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-05-08T05:45:29.083 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-05-08T05:45:29.083 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-05-08T05:45:29.083 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-05-08T05:45:29.083 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-05-08T05:45:29.083 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-05-08T05:45:29.083 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-05-08T05:45:29.083 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-05-08T05:45:29.084 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-05-08T05:45:29.084 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-05-08T05:45:29.084 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-05-08T05:45:29.084 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-05-08T05:45:29.084 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-05-08T05:45:29.084 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-05-08T05:45:29.084 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-05-08T05:45:29.084 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-05-08T05:45:29.084 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-05-08T05:45:29.084 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-05-08T05:45:29.084 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-05-08T05:45:29.084 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-05-08T05:45:29.084 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-05-08T05:45:29.084 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-05-08T05:45:29.085 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-05-08T05:45:29.085 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-05-08T05:45:29.085 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-05-08T05:45:29.085 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-05-08T05:45:29.085 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-05-08T05:45:29.085 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-05-08T05:45:29.085 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-05-08T05:45:29.085 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-05-08T05:45:29.085 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-05-08T05:45:29.085 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-05-08T05:45:29.085 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-05-08T05:45:29.085 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-05-08T05:45:29.085 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-05-08T05:45:29.086 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-05-08T05:45:29.086 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-05-08T05:45:29.086 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-05-08T05:45:29.086 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-05-08T05:45:29.086 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-05-08T05:45:29.086 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-05-08T05:45:29.086 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-05-08T05:45:29.086 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-05-08T05:45:29.086 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-05-08T05:45:29.086 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-05-08T05:45:29.086 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-05-08T05:45:29.086 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-05-08T05:45:29.086 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-08T05:45:29.087 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-08T05:45:29.087 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-08T05:45:29.087 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-08T05:45:29.087 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-08T05:45:29.087 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-08T05:45:29.087 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-08T05:45:29.087 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-08T05:45:29.087 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-08T05:45:29.087 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-08T05:45:29.087 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-08T05:45:29.087 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-08T05:45:29.087 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-05-08T05:45:29.087 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-05-08T05:45:29.088 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-05-08T05:45:29.088 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-08T05:45:29.088 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-05-08T05:45:29.088 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-05-08T05:45:29.088 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-05-08T05:45:29.088 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-05-08T05:45:29.088 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-05-08T05:45:29.088 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-05-08T05:45:29.088 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-05-08T05:45:29.088 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-05-08T05:45:29.088 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-05-08T05:45:29.089 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-05-08T05:45:29.089 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-05-08T05:45:29.089 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-05-08T05:45:29.089 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-05-08T05:45:29.089 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-05-08T05:45:29.089 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-05-08T05:45:29.089 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-05-08T05:45:29.089 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-05-08T05:45:29.089 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-05-08T05:45:29.089 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-05-08T05:45:29.089 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-05-08T05:45:29.089 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-05-08T05:45:29.089 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-05-08T05:45:29.089 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-05-08T05:45:29.090 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-05-08T05:45:29.090 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-05-08T05:45:29.090 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-05-08T05:45:29.090 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-05-08T05:45:29.090 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-08T05:45:29.090 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-08T05:45:29.090 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-05-08T05:45:29.090 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-08T05:45:29.090 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-08T05:45:29.090 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-08T05:45:29.090 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-08T05:45:29.090 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-08T05:45:29.090 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-08T05:45:29.091 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-08T05:45:29.091 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-08T05:45:29.091 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-08T05:45:29.091 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-08T05:45:29.091 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-08T05:45:29.091 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-08T05:45:29.091 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-08T05:45:29.091 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-08T05:45:29.091 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-08T05:45:29.091 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-08T05:45:29.091 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-08T05:45:29.091 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-08T05:45:29.091 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-08T05:45:29.092 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-08T05:45:29.092 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-08T05:45:29.092 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-08T05:45:29.092 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-08T05:45:29.092 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-05-08T05:45:29.092 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-05-08T05:45:29.092 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-08T05:45:29.092 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-05-08T05:45:29.092 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-05-08T05:45:29.092 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-05-08T05:45:29.092 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-05-08T05:45:29.092 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-08T05:45:29.092 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-05-08T05:45:29.092 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-08T05:45:29.093 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-08T05:45:29.093 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-08T05:45:29.093 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-08T05:45:29.093 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-08T05:45:29.093 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-05-08T05:45:29.093 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-05-08T05:45:29.093 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-08T05:45:29.093 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-05-08T05:45:29.093 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-08T05:45:29.093 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-05-08T05:45:29.093 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-08T05:45:29.093 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-05-08T05:45:29.093 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-08T05:45:29.093 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-05-08T05:45:29.094 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-05-08T05:45:29.094 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-08T05:45:29.094 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-05-08T05:45:29.094 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-08T05:45:29.094 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-05-08T05:45:29.094 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-08T05:45:29.094 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-05-08T05:45:29.094 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-08T05:45:29.094 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-05-08T05:45:29.094 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-08T05:45:29.094 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-05-08T05:45:29.094 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-05-08T05:45:29.094 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-08T05:45:29.095 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-08T05:45:29.095 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-05-08T05:45:29.095 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-05-08T05:45:29.095 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-05-08T05:45:29.095 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-08T05:45:29.095 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-05-08T05:45:29.095 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-05-08T05:45:29.095 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-05-08T05:45:29.095 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-05-08T05:45:29.095 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-08T05:45:29.095 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-05-08T05:45:29.095 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-08T05:45:29.095 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-08T05:45:29.095 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-08T05:45:29.096 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-05-08T05:45:29.096 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-08T05:45:29.096 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-05-08T05:45:29.096 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-05-08T05:45:29.096 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-05-08T05:45:29.096 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-08T05:45:29.096 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-05-08T05:45:29.096 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-05-08T05:45:29.096 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-05-08T05:45:29.096 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-05-08T05:45:29.096 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-05-08T05:45:29.096 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-05-08T05:45:29.096 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-05-08T05:45:29.096 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-05-08T05:45:29.096 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-08T05:45:29.097 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-08T05:45:29.097 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-08T05:45:29.097 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-08T05:45:29.097 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-08T05:45:29.097 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-08T05:45:29.097 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-08T05:45:29.097 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-08T05:45:29.097 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-08T05:45:29.097 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-05-08T05:45:29.097 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-05-08T05:45:29.097 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-05-08T05:45:29.097 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-05-08T05:45:29.097 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-05-08T05:45:29.097 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-05-08T05:45:29.098 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-05-08T05:45:29.098 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-05-08T05:45:29.098 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-05-08T05:45:29.098 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-05-08T05:45:29.098 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-05-08T05:45:29.098 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-05-08T05:45:29.098 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-05-08T05:45:29.098 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-05-08T05:45:29.098 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-05-08T05:45:29.098 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-05-08T05:45:29.098 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-05-08T05:45:29.098 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-05-08T05:45:29.099 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-05-08T05:45:29.099 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-08T05:45:29.099 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-05-08T05:45:29.099 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-05-08T05:45:29.099 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-05-08T05:45:29.099 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-05-08T05:45:29.099 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-05-08T05:45:29.099 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-05-08T05:45:29.099 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-05-08T05:45:29.099 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-05-08T05:45:29.099 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-05-08T05:45:29.099 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-05-08T05:45:29.099 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-05-08T05:45:29.099 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-05-08T05:45:29.100 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-05-08T05:45:29.100 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-05-08T05:45:29.100 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-05-08T05:45:29.100 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-05-08T05:45:29.100 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-05-08T05:45:29.100 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-05-08T05:45:29.100 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-05-08T05:45:29.100 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-05-08T05:45:29.100 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-05-08T05:45:29.100 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-05-08T05:45:29.100 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-05-08T05:45:29.100 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-08T05:45:29.100 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-05-08T05:45:29.100 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-05-08T05:45:29.101 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-05-08T05:45:29.101 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-05-08T05:45:29.101 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-05-08T05:45:29.101 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-05-08T05:45:29.101 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-05-08T05:45:29.101 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-08T05:45:29.101 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-05-08T05:45:29.101 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-05-08T05:45:29.101 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-05-08T05:45:29.101 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-05-08T05:45:29.101 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-05-08T05:45:29.101 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-05-08T05:45:29.101 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-08T05:45:29.101 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-08T05:45:29.102 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-05-08T05:45:29.102 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-05-08T05:45:29.102 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-05-08T05:45:29.102 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-05-08T05:45:29.102 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-05-08T05:45:29.102 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-08T05:45:29.102 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-05-08T05:45:29.102 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-08T05:45:29.102 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-08T05:45:29.102 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-08T05:45:29.102 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-05-08T05:45:29.102 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-05-08T05:45:29.102 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-05-08T05:45:29.103 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-05-08T05:45:29.103 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-05-08T05:45:29.103 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-08T05:45:29.103 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-05-08T05:45:29.103 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-05-08T05:45:29.103 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-05-08T05:45:29.103 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-05-08T05:45:29.103 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-05-08T05:45:29.103 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-08T05:45:29.103 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-08T05:45:29.103 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-08T05:45:29.103 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-05-08T05:45:29.103 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-08T05:45:29.103 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-05-08T05:45:29.104 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-05-08T05:45:29.104 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-05-08T05:45:29.104 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-05-08T05:45:29.104 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-08T05:45:29.104 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-08T05:45:29.104 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-08T05:45:29.104 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-08T05:45:29.104 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-05-08T05:45:29.104 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-08T05:45:29.104 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-08T05:45:29.104 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-08T05:45:29.104 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-08T05:45:29.104 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-05-08T05:45:29.104 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-08T05:45:29.105 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-05-08T05:45:29.105 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-08T05:45:29.105 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-05-08T05:45:29.105 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-08T05:45:29.105 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-05-08T05:45:29.105 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-08T05:45:29.105 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-05-08T05:45:29.105 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-08T05:45:29.105 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-08T05:45:29.105 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-08T05:45:29.105 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-05-08T05:45:29.105 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-05-08T05:45:29.105 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-08T05:45:29.105 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-08T05:45:29.106 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-08T05:45:29.106 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-08T05:45:29.106 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-08T05:45:29.106 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-05-08T05:45:29.106 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-08T05:45:29.106 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-08T05:45:29.106 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-05-08T05:45:29.106 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-08T05:45:29.106 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-08T05:45:29.106 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-08T05:45:29.106 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-08T05:45:29.106 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-08T05:45:29.106 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-05-08T05:45:29.106 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-08T05:45:29.107 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-08T05:45:29.107 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-05-08T05:45:29.107 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-05-08T05:45:29.107 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-08T05:45:29.107 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-05-08T05:45:29.107 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-08T05:45:29.107 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-08T05:45:29.107 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-08T05:45:29.107 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-08T05:45:29.107 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-08T05:45:29.107 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-08T05:45:29.107 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-08T05:45:29.107 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-08T05:45:29.107 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-08T05:45:29.108 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-08T05:45:29.108 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-05-08T05:45:29.108 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-05-08T05:45:29.108 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-08T05:45:29.108 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-05-08T05:45:29.108 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-05-08T05:45:29.108 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-08T05:45:29.108 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-08T05:45:29.108 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-08T05:45:29.108 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-08T05:45:29.108 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-08T05:45:29.108 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-08T05:45:29.108 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-05-08T05:45:29.108 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-08T05:45:29.109 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-08T05:45:29.109 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-05-08T05:45:29.109 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-05-08T05:45:29.109 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-05-08T05:45:29.109 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-08T05:45:29.109 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-05-08T05:45:29.109 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-05-08T05:45:29.109 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-05-08T05:45:29.109 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-08T05:45:29.109 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-05-08T05:45:29.109 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-08T05:45:29.109 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-05-08T05:45:29.109 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-08T05:45:29.109 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-05-08T05:45:29.109 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-08T05:45:29.110 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-05-08T05:45:29.110 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-08T05:45:29.110 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-05-08T05:45:29.110 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-08T05:45:29.110 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-05-08T05:45:29.110 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-08T05:45:29.110 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-05-08T05:45:29.110 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-08T05:45:29.110 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-08T05:45:29.110 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-08T05:45:29.110 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-08T05:45:29.110 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-08T05:45:29.110 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-05-08T05:45:29.111 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-05-08T05:45:29.111 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-08T05:45:29.111 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-08T05:45:29.111 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-08T05:45:29.111 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-05-08T05:45:29.111 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-08T05:45:29.111 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-08T05:45:29.111 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-08T05:45:29.111 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-08T05:45:29.111 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-08T05:45:29.111 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-08T05:45:29.111 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-05-08T05:45:29.111 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-08T05:45:29.111 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-05-08T05:45:29.111 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-08T05:45:29.112 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-08T05:45:29.112 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-05-08T05:45:29.112 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-08T05:45:29.112 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-08T05:45:29.112 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-05-08T05:45:29.112 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-08T05:45:29.112 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-05-08T05:45:29.112 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-08T05:45:29.112 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-05-08T05:45:29.112 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-08T05:45:29.112 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-08T05:45:29.112 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-05-08T05:45:29.112 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-08T05:45:29.112 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-08T05:45:29.113 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-08T05:45:29.113 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-08T05:45:29.113 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-08T05:45:29.113 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-05-08T05:45:29.113 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-08T05:45:29.113 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-05-08T05:45:29.113 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-08T05:45:29.113 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-08T05:45:29.113 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-08T05:45:29.113 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-08T05:45:29.113 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-05-08T05:45:29.113 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-08T05:45:29.113 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-05-08T05:45:29.114 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-08T05:45:29.114 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-05-08T05:45:29.114 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-08T05:45:29.114 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-05-08T05:45:29.114 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-08T05:45:29.114 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-08T05:45:29.114 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-05-08T05:45:29.114 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-08T05:45:29.114 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-08T05:45:29.114 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-08T05:45:29.114 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-08T05:45:29.114 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-05-08T05:45:29.114 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-08T05:45:29.114 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-08T05:45:29.115 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-08T05:45:29.115 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-05-08T05:45:29.115 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-08T05:45:29.115 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-08T05:45:29.115 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-08T05:45:29.115 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-08T05:45:29.115 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-05-08T05:45:29.115 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-08T05:45:29.115 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-05-08T05:45:29.115 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-08T05:45:29.115 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-05-08T05:45:29.115 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-08T05:45:29.115 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-08T05:45:29.115 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-05-08T05:45:29.115 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-08T05:45:29.116 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-08T05:45:29.116 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-05-08T05:45:29.116 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-08T05:45:29.116 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-05-08T05:45:29.116 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-05-08T05:45:29.116 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-08T05:45:29.116 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-05-08T05:45:29.116 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-08T05:45:29.116 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-05-08T05:45:29.116 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-05-08T05:45:29.116 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-08T05:45:29.116 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-05-08T05:45:29.116 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-05-08T05:45:29.116 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-05-08T05:45:29.116 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-05-08T05:45:29.116 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-08T05:45:29.117 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-08T05:45:29.117 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-08T05:45:29.117 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-05-08T05:45:29.117 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-05-08T05:45:29.117 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-05-08T05:45:29.117 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-05-08T05:45:29.117 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-05-08T05:45:29.117 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-08T05:45:29.117 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-08T05:45:29.117 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-05-08T05:45:29.117 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-05-08T05:45:29.117 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-08T05:45:29.117 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-08T05:45:29.117 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-08T05:45:29.117 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-05-08T05:45:29.118 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-05-08T05:45:29.118 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-05-08T05:45:29.118 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-05-08T05:45:29.118 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-05-08T05:45:29.118 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-05-08T05:45:29.118 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-05-08T05:45:29.118 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-05-08T05:45:29.118 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-05-08T05:45:29.118 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-05-08T05:45:29.118 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-05-08T05:45:29.118 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-05-08T05:45:29.118 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-05-08T05:45:29.118 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-05-08T05:45:29.119 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-05-08T05:45:29.119 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-05-08T05:45:29.119 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-05-08T05:45:29.119 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-05-08T05:45:29.119 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-08T05:45:29.119 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-05-08T05:45:29.119 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-08T05:45:29.119 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-08T05:45:29.119 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-08T05:45:29.119 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-08T05:45:29.119 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-05-08T05:45:29.119 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-05-08T05:45:29.119 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-05-08T05:45:29.119 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-05-08T05:45:29.119 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-05-08T05:45:29.119 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-05-08T05:45:29.120 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-05-08T05:45:29.120 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-05-08T05:45:29.120 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-05-08T05:45:29.120 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-05-08T05:45:29.120 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-08T05:45:29.120 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-08T05:45:29.120 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-08T05:45:29.120 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-05-08T05:45:29.120 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-08T05:45:29.120 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-05-08T05:45:29.120 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-05-08T05:45:29.120 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-05-08T05:45:29.120 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-05-08T05:45:29.120 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-08T05:45:29.120 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-05-08T05:45:29.121 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-05-08T05:45:29.121 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-05-08T05:45:29.121 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-08T05:45:29.121 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-05-08T05:45:29.121 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-05-08T05:45:29.121 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-05-08T05:45:29.121 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-05-08T05:45:29.121 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-05-08T05:45:29.121 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-05-08T05:45:29.121 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-05-08T05:45:29.121 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-05-08T05:45:29.121 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-05-08T05:45:29.121 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-05-08T05:45:29.121 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-05-08T05:45:29.122 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-05-08T05:45:29.122 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-05-08T05:45:29.122 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-05-08T05:45:29.122 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-08T05:45:29.122 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-05-08T05:45:29.122 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-08T05:45:29.122 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-05-08T05:45:29.122 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-05-08T05:45:29.122 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-05-08T05:45:29.122 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-05-08T05:45:29.122 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-08T05:45:29.122 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-05-08T05:45:29.122 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-05-08T05:45:29.123 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-08T05:45:29.123 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-05-08T05:45:29.123 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-05-08T05:45:29.123 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-08T05:45:29.123 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-05-08T05:45:29.123 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-08T05:45:29.123 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-08T05:45:29.123 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-08T05:45:29.123 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-08T05:45:29.123 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-08T05:45:29.123 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-08T05:45:29.123 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-08T05:45:29.123 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-08T05:45:29.123 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-08T05:45:29.123 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-05-08T05:45:29.124 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-05-08T05:45:29.124 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-08T05:45:29.124 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-05-08T05:45:29.124 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-08T05:45:29.124 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-08T05:45:29.124 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-08T05:45:29.124 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-08T05:45:29.124 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-08T05:45:29.124 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-08T05:45:29.124 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-08T05:45:29.124 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-08T05:45:29.124 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-08T05:45:29.124 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-05-08T05:45:29.124 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-05-08T05:45:29.124 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-08T05:45:29.125 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-08T05:45:29.125 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-05-08T05:45:29.125 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-05-08T05:45:29.125 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-08T05:45:29.125 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-08T05:45:29.125 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-05-08T05:45:29.125 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-05-08T05:45:29.125 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-05-08T05:45:29.125 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-05-08T05:45:29.125 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-05-08T05:45:29.125 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-05-08T05:45:29.125 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-05-08T05:45:29.125 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-05-08T05:45:29.125 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-08T05:45:29.125 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-05-08T05:45:29.125 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-05-08T05:45:29.126 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-08T05:45:29.126 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-08T05:45:29.126 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-08T05:45:29.126 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-08T05:45:29.126 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-08T05:45:29.126 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-08T05:45:29.126 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-08T05:45:29.126 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-08T05:45:29.126 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-08T05:45:29.126 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-08T05:45:29.126 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-05-08T05:45:29.126 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-08T05:45:29.126 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-08T05:45:29.126 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-08T05:45:29.127 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-08T05:45:29.127 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-08T05:45:29.127 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-08T05:45:29.127 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-05-08T05:45:29.127 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-08T05:45:29.127 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-08T05:45:29.127 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-08T05:45:29.127 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-08T05:45:29.127 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-08T05:45:29.127 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-08T05:45:29.127 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-08T05:45:29.127 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-08T05:45:29.127 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-08T05:45:29.127 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-08T05:45:29.127 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-08T05:45:29.128 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-08T05:45:29.128 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-05-08T05:45:29.128 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-05-08T05:45:29.128 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-05-08T05:45:29.128 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-05-08T05:45:29.128 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-05-08T05:45:29.128 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-05-08T05:45:29.128 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-05-08T05:45:29.128 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-05-08T05:45:29.128 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-08T05:45:29.128 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-05-08T05:45:29.128 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-05-08T05:45:29.128 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-08T05:45:29.128 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-08T05:45:29.129 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-05-08T05:45:29.129 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-05-08T05:45:29.129 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-08T05:45:29.129 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-05-08T05:45:29.129 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-05-08T05:45:29.129 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-08T05:45:29.129 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-05-08T05:45:29.129 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-05-08T05:45:29.129 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-05-08T05:45:29.129 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-08T05:45:29.129 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-05-08T05:45:29.129 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-08T05:45:29.129 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-05-08T05:45:29.129 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-05-08T05:45:29.129 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-08T05:45:29.130 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-05-08T05:45:29.130 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-08T05:45:29.130 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-08T05:45:29.130 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-08T05:45:29.130 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-08T05:45:29.130 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-08T05:45:29.130 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-05-08T05:45:29.130 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-05-08T05:45:29.130 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-08T05:45:29.130 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-08T05:45:29.130 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-08T05:45:29.130 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-08T05:45:29.130 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-05-08T05:45:29.130 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-05-08T05:45:29.131 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-08T05:45:29.131 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-05-08T05:45:29.131 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-05-08T05:45:29.131 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-08T05:45:29.131 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-08T05:45:29.131 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-05-08T05:45:29.131 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-05-08T05:45:29.131 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-05-08T05:45:29.131 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-05-08T05:45:29.131 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-05-08T05:45:29.131 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-05-08T05:45:29.131 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-05-08T05:45:29.131 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-05-08T05:45:29.131 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-05-08T05:45:29.131 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-05-08T05:45:29.132 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-08T05:45:29.132 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-08T05:45:29.132 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-05-08T05:45:29.132 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-05-08T05:45:29.132 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-05-08T05:45:29.132 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-08T05:45:29.132 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-05-08T05:45:29.132 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-05-08T05:45:29.132 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-05-08T05:45:29.132 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-08T05:45:29.132 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-05-08T05:45:29.132 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-05-08T05:45:29.132 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-08T05:45:29.132 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-08T05:45:29.132 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-08T05:45:29.133 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-05-08T05:45:29.133 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-08T05:45:29.133 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-05-08T05:45:29.133 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-08T05:45:29.133 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-05-08T05:45:29.133 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-05-08T05:45:29.133 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-08T05:45:29.133 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-05-08T05:45:29.133 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-05-08T05:45:29.133 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-05-08T05:45:29.133 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-08T05:45:29.133 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-08T05:45:29.133 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-05-08T05:45:29.133 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-08T05:45:29.133 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-05-08T05:45:29.134 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-05-08T05:45:29.134 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-08T05:45:29.134 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-08T05:45:29.134 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-08T05:45:29.134 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-08T05:45:29.134 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-08T05:45:29.134 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-05-08T05:45:29.134 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-05-08T05:45:29.134 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-05-08T05:45:29.134 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-05-08T05:45:29.134 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-08T05:45:29.134 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-05-08T05:45:29.134 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-08T05:45:29.134 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-05-08T05:45:29.135 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-05-08T05:45:29.135 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-05-08T05:45:29.135 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-05-08T05:45:29.135 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-05-08T05:45:29.135 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-08T05:45:29.135 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-05-08T05:45:29.135 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-08T05:45:29.135 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-05-08T05:45:29.135 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-05-08T05:45:29.135 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-05-08T05:45:29.135 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-05-08T05:45:29.135 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-05-08T05:45:29.135 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-05-08T05:45:29.135 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-08T05:45:29.135 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-05-08T05:45:29.136 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-05-08T05:45:29.136 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-05-08T05:45:29.136 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-08T05:45:29.136 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-08T05:45:29.136 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-08T05:45:29.136 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-08T05:45:29.136 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-08T05:45:29.136 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-08T05:45:29.136 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-08T05:45:29.136 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-08T05:45:29.136 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-08T05:45:29.136 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-08T05:45:29.136 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-08T05:45:29.137 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-08T05:45:29.137 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-08T05:45:29.137 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-08T05:45:29.137 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-08T05:45:29.137 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-08T05:45:29.137 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-08T05:45:29.137 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-08T05:45:29.137 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-05-08T05:45:29.137 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-05-08T05:45:29.137 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-08T05:45:29.137 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-05-08T05:45:29.137 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-05-08T05:45:29.137 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-05-08T05:45:29.137 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-05-08T05:45:29.138 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-05-08T05:45:29.138 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-08T05:45:29.138 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-05-08T05:45:29.138 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-05-08T05:45:29.138 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-08T05:45:29.138 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-05-08T05:45:29.138 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-05-08T05:45:29.138 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-08T05:45:29.138 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-05-08T05:45:29.138 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-08T05:45:29.138 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-05-08T05:45:29.138 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-05-08T05:45:29.138 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-05-08T05:45:29.138 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-05-08T05:45:29.139 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-05-08T05:45:29.139 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-05-08T05:45:29.139 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-05-08T05:45:29.139 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-05-08T05:45:29.139 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-05-08T05:45:29.139 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-08T05:45:29.139 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-05-08T05:45:29.139 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-05-08T05:45:29.139 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-05-08T05:45:29.139 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-08T05:45:29.139 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-05-08T05:45:29.139 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-05-08T05:45:29.139 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-05-08T05:45:29.139 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-05-08T05:45:29.139 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-05-08T05:45:29.140 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-05-08T05:45:29.140 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-05-08T05:45:29.140 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-05-08T05:45:29.140 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-05-08T05:45:29.140 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-05-08T05:45:29.140 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-05-08T05:45:29.140 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-08T05:45:29.140 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-05-08T05:45:29.140 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-05-08T05:45:29.140 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-05-08T05:45:29.140 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-08T05:45:29.140 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-08T05:45:29.140 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-05-08T05:45:29.140 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-05-08T05:45:29.140 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-08T05:45:29.140 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-05-08T05:45:29.141 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-05-08T05:45:29.141 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-05-08T05:45:29.141 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-05-08T05:45:29.141 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-05-08T05:45:29.141 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-05-08T05:45:29.141 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-05-08T05:45:29.141 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-08T05:45:29.141 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-05-08T05:45:29.141 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-05-08T05:45:29.141 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-05-08T05:45:29.141 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-08T05:45:29.141 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-08T05:45:29.141 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-05-08T05:45:29.141 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-08T05:45:29.141 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-08T05:45:29.142 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-08T05:45:29.142 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-05-08T05:45:29.142 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-05-08T05:45:29.142 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-05-08T05:45:29.142 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-05-08T05:45:29.142 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-08T05:45:29.142 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-05-08T05:45:29.142 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-08T05:45:29.142 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-08T05:45:29.142 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-08T05:45:29.142 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-05-08T05:45:29.142 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-05-08T05:45:29.142 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-05-08T05:45:29.142 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-05-08T05:45:29.142 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-05-08T05:45:29.143 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-05-08T05:45:29.143 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-05-08T05:45:29.143 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-05-08T05:45:29.143 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-05-08T05:45:29.143 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-05-08T05:45:29.143 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-05-08T05:45:29.143 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-05-08T05:45:29.143 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-05-08T05:45:29.143 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-05-08T05:45:29.143 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-08T05:45:29.143 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-05-08T05:45:29.143 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-05-08T05:45:29.143 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-08T05:45:29.143 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-08T05:45:29.143 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-05-08T05:45:29.144 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-08T05:45:29.144 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-05-08T05:45:29.144 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-05-08T05:45:29.144 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-08T05:45:29.144 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-05-08T05:45:29.144 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-05-08T05:45:29.144 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-08T05:45:29.144 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-08T05:45:29.144 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-08T05:45:29.144 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-08T05:45:29.144 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-08T05:45:29.144 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-08T05:45:29.144 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-08T05:45:29.144 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-05-08T05:45:29.145 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-08T05:45:29.145 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-08T05:45:29.145 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-08T05:45:29.145 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-08T05:45:29.145 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-08T05:45:29.145 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-05-08T05:45:29.145 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-05-08T05:45:29.145 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-08T05:45:29.145 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-08T05:45:29.145 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-08T05:45:29.145 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-05-08T05:45:29.145 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-05-08T05:45:29.145 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-08T05:45:29.145 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-08T05:45:29.145 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-05-08T05:45:29.145 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-08T05:45:29.146 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-05-08T05:45:29.146 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-05-08T05:45:29.146 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-08T05:45:29.146 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-05-08T05:45:29.146 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-05-08T05:45:29.146 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-05-08T05:45:29.146 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-05-08T05:45:29.146 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-05-08T05:45:29.146 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-05-08T05:45:29.146 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-08T05:45:29.146 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-05-08T05:45:29.146 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-05-08T05:45:29.146 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-05-08T05:45:29.146 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-05-08T05:45:29.147 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-05-08T05:45:29.147 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-08T05:45:29.147 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-08T05:45:29.147 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-05-08T05:45:29.147 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-05-08T05:45:29.147 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-05-08T05:45:29.147 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-05-08T05:45:29.147 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-05-08T05:45:29.147 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-05-08T05:45:29.147 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-08T05:45:29.147 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-08T05:45:29.147 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-08T05:45:29.147 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-08T05:45:29.147 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-08T05:45:29.147 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-08T05:45:29.147 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-08T05:45:29.148 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-08T05:45:29.148 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-08T05:45:29.148 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-08T05:45:29.148 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-08T05:45:29.148 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-05-08T05:45:29.148 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-05-08T05:45:29.148 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-08T05:45:29.148 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-08T05:45:29.148 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-08T05:45:29.148 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-05-08T05:45:29.148 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-05-08T05:45:29.148 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-08T05:45:29.148 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-08T05:45:29.148 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-05-08T05:45:29.148 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-05-08T05:45:29.149 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-08T05:45:29.149 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-08T05:45:29.149 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-05-08T05:45:29.149 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-05-08T05:45:29.149 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-05-08T05:45:29.149 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-08T05:45:29.149 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-08T05:45:29.149 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-08T05:45:29.149 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-08T05:45:29.149 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-05-08T05:45:29.149 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-08T05:45:29.149 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-05-08T05:45:29.149 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-08T05:45:29.149 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-08T05:45:29.150 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-08T05:45:29.150 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-05-08T05:45:29.150 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-05-08T05:45:29.150 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-05-08T05:45:29.150 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-08T05:45:29.150 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-05-08T05:45:29.150 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-05-08T05:45:29.150 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-05-08T05:45:29.150 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-05-08T05:45:29.150 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-08T05:45:29.150 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-05-08T05:45:29.150 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-05-08T05:45:29.150 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-08T05:45:29.151 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-05-08T05:45:29.151 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-05-08T05:45:29.151 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-05-08T05:45:29.151 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-05-08T05:45:29.151 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-05-08T05:45:29.151 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-05-08T05:45:29.151 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-05-08T05:45:29.151 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-05-08T05:45:29.151 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-05-08T05:45:29.151 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-05-08T05:45:29.151 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-05-08T05:45:29.151 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-05-08T05:45:29.151 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-05-08T05:45:29.151 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-05-08T05:45:29.152 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-05-08T05:45:29.152 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-05-08T05:45:29.152 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-05-08T05:45:29.152 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-05-08T05:45:29.152 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-05-08T05:45:29.152 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-05-08T05:45:29.152 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-05-08T05:45:29.152 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-08T05:45:29.152 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-05-08T05:45:29.152 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-05-08T05:45:29.152 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-05-08T05:45:29.152 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-05-08T05:45:29.152 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-05-08T05:45:29.152 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-05-08T05:45:29.152 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-05-08T05:45:29.153 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-05-08T05:45:29.153 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-05-08T05:45:29.153 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-05-08T05:45:29.153 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-05-08T05:45:29.153 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-05-08T05:45:29.153 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-05-08T05:45:29.153 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-05-08T05:45:29.153 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-05-08T05:45:29.153 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-05-08T05:45:29.153 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-05-08T05:45:29.153 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-05-08T05:45:29.153 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-05-08T05:45:29.153 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-05-08T05:45:29.153 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-05-08T05:45:29.153 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-05-08T05:45:29.154 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-05-08T05:45:29.154 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-05-08T05:45:29.154 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-05-08T05:45:29.154 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-05-08T05:45:29.154 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-05-08T05:45:29.154 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-08T05:45:29.154 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-05-08T05:45:29.154 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-08T05:45:29.154 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-05-08T05:45:29.154 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-08T05:45:29.154 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-05-08T05:45:29.154 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-05-08T05:45:29.154 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-05-08T05:45:29.154 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-05-08T05:45:29.155 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-05-08T05:45:29.155 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-05-08T05:45:29.155 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-05-08T05:45:29.155 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-05-08T05:45:29.155 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-05-08T05:45:29.155 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-05-08T05:45:29.155 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-05-08T05:45:29.155 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-05-08T05:45:29.155 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-05-08T05:45:29.155 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-05-08T05:45:29.155 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-05-08T05:45:29.155 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-05-08T05:45:29.155 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-05-08T05:45:29.155 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-05-08T05:45:29.156 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-05-08T05:45:29.156 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-05-08T05:45:29.156 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-05-08T05:45:29.156 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-05-08T05:45:29.156 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-05-08T05:45:29.156 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-05-08T05:45:29.156 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-05-08T05:45:29.156 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-05-08T05:45:29.156 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-05-08T05:45:29.156 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-05-08T05:45:29.156 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-05-08T05:45:29.156 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-05-08T05:45:29.156 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-05-08T05:45:29.156 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-05-08T05:45:29.156 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-05-08T05:45:29.157 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-05-08T05:45:29.157 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-05-08T05:45:29.157 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-05-08T05:45:29.157 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-05-08T05:45:29.157 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-05-08T05:45:29.157 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-05-08T05:45:29.157 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-05-08T05:45:29.157 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-05-08T05:45:29.157 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-05-08T05:45:29.157 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-05-08T05:45:29.157 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-05-08T05:45:29.157 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-05-08T05:45:29.157 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-05-08T05:45:29.157 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-05-08T05:45:29.157 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-05-08T05:45:29.158 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-05-08T05:45:29.158 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-05-08T05:45:29.158 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-05-08T05:45:29.158 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-05-08T05:45:29.158 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-05-08T05:45:29.158 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-05-08T05:45:29.158 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-05-08T05:45:29.158 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-05-08T05:45:29.158 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-05-08T05:45:29.158 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-05-08T05:45:29.158 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-05-08T05:45:29.158 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-08T05:45:29.158 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-05-08T05:45:29.158 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-05-08T05:45:29.159 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-05-08T05:45:29.159 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-08T05:45:29.159 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-05-08T05:45:29.159 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-08T05:45:29.159 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-05-08T05:45:29.159 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-08T05:45:29.159 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-05-08T05:45:29.159 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-08T05:45:29.159 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-08T05:45:29.159 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-08T05:45:29.159 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-05-08T05:45:29.159 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-05-08T05:45:29.159 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-05-08T05:45:29.159 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-05-08T05:45:29.159 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-05-08T05:45:29.160 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-05-08T05:45:29.160 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-05-08T05:45:29.160 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-05-08T05:45:29.160 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-05-08T05:45:29.160 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-05-08T05:45:29.160 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-05-08T05:45:29.160 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-05-08T05:45:29.160 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-05-08T05:45:29.160 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-05-08T05:45:29.160 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-05-08T05:45:29.160 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-05-08T05:45:29.160 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-05-08T05:45:29.160 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-05-08T05:45:29.160 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-05-08T05:45:29.161 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-05-08T05:45:29.161 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-05-08T05:45:29.161 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-05-08T05:45:29.161 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-05-08T05:45:29.161 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-05-08T05:45:29.161 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-05-08T05:45:29.161 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-05-08T05:45:29.161 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-05-08T05:45:29.161 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-05-08T05:45:29.161 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-05-08T05:45:29.161 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-08T05:45:29.161 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-05-08T05:45:29.161 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-05-08T05:45:29.161 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-05-08T05:45:29.161 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-05-08T05:45:29.162 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-05-08T05:45:29.162 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-05-08T05:45:29.162 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-05-08T05:45:29.162 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-05-08T05:45:29.162 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-05-08T05:45:29.162 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-05-08T05:45:29.162 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-05-08T05:45:29.162 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-05-08T05:45:29.162 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-05-08T05:45:29.162 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-05-08T05:45:29.162 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-05-08T05:45:29.162 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-05-08T05:45:29.162 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-05-08T05:45:29.162 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-08T05:45:29.163 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-05-08T05:45:29.163 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-05-08T05:45:29.163 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-05-08T05:45:29.163 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-05-08T05:45:29.163 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-05-08T05:45:29.163 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-05-08T05:45:29.163 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-05-08T05:45:29.163 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-05-08T05:45:29.163 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-05-08T05:45:29.163 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-05-08T05:45:29.163 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-05-08T05:45:29.163 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-05-08T05:45:29.163 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-05-08T05:45:29.163 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-08T05:45:29.163 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-05-08T05:45:29.164 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-05-08T05:45:29.164 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-05-08T05:45:29.164 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-05-08T05:45:29.164 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-05-08T05:45:29.164 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-05-08T05:45:29.164 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-05-08T05:45:29.164 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-05-08T05:45:29.164 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-05-08T05:45:29.164 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-05-08T05:45:29.164 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-05-08T05:45:29.164 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-05-08T05:45:29.164 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-05-08T05:45:29.164 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-05-08T05:45:29.164 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-05-08T05:45:29.165 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-05-08T05:45:29.165 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-05-08T05:45:29.165 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-05-08T05:45:29.165 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-05-08T05:45:29.165 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-05-08T05:45:29.165 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-05-08T05:45:29.165 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-05-08T05:45:29.165 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-08T05:45:29.165 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-08T05:45:29.165 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-08T05:45:29.165 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-08T05:45:29.165 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-08T05:45:29.165 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-08T05:45:29.165 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-08T05:45:29.165 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-08T05:45:29.166 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-08T05:45:29.166 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-08T05:45:29.166 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-08T05:45:29.166 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-08T05:45:29.166 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-08T05:45:29.166 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-08T05:45:29.166 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-08T05:45:29.166 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-08T05:45:29.166 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-08T05:45:29.166 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-08T05:45:29.166 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-08T05:45:29.166 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-08T05:45:29.166 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-05-08T05:45:29.166 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-08T05:45:29.167 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-08T05:45:29.167 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-08T05:45:29.167 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-05-08T05:45:29.167 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-05-08T05:45:29.167 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-05-08T05:45:29.167 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-05-08T05:45:29.167 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-05-08T05:45:29.167 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-05-08T05:45:29.167 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-05-08T05:45:29.167 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-05-08T05:45:29.167 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-05-08T05:45:29.167 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-05-08T05:45:29.167 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-05-08T05:45:29.167 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-05-08T05:45:29.167 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-05-08T05:45:29.168 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-05-08T05:45:29.168 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-05-08T05:45:29.168 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-05-08T05:45:29.168 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-05-08T05:45:29.168 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-05-08T05:45:29.168 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-05-08T05:45:29.168 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-05-08T05:45:29.168 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-08T05:45:29.168 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-05-08T05:45:29.168 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-08T05:45:29.168 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-05-08T05:45:29.168 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-05-08T05:45:29.168 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-05-08T05:45:29.168 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-05-08T05:45:29.168 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-05-08T05:45:29.169 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-05-08T05:45:29.169 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-05-08T05:45:29.169 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-05-08T05:45:29.169 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-05-08T05:45:29.169 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-05-08T05:45:29.169 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-05-08T05:45:29.169 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-05-08T05:45:29.169 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-08T05:45:29.169 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-08T05:45:29.169 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-08T05:45:29.169 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-08T05:45:29.169 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-08T05:45:29.169 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-08T05:45:29.169 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-08T05:45:29.169 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-08T05:45:29.170 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-08T05:45:29.170 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-08T05:45:29.170 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-08T05:45:29.170 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-08T05:45:29.170 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-08T05:45:29.170 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-08T05:45:29.170 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-08T05:45:29.170 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-08T05:45:29.170 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-08T05:45:29.170 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-08T05:45:29.170 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-08T05:45:29.170 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-08T05:45:29.170 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-08T05:45:29.170 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-05-08T05:45:29.171 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-05-08T05:45:29.171 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-05-08T05:45:29.171 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-05-08T05:45:29.171 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-05-08T05:45:29.171 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-05-08T05:45:29.171 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-05-08T05:45:29.171 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-05-08T05:45:29.171 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-05-08T05:45:29.171 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-05-08T05:45:29.171 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-05-08T05:45:29.171 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-05-08T05:45:29.171 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-05-08T05:45:29.171 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-05-08T05:45:29.171 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-05-08T05:45:29.172 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-05-08T05:45:29.172 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-08T05:45:29.172 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-05-08T05:45:29.172 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-05-08T05:45:29.172 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-05-08T05:45:29.172 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-05-08T05:45:29.172 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-05-08T05:45:29.172 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-08T05:45:29.172 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-05-08T05:45:29.172 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-05-08T05:45:29.172 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-05-08T05:45:29.172 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-08T05:45:29.172 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-08T05:45:29.172 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-08T05:45:29.172 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-08T05:45:29.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-08T05:45:29.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-08T05:45:29.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-08T05:45:29.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-08T05:45:29.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-08T05:45:29.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-08T05:45:29.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-08T05:45:29.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-08T05:45:29.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-08T05:45:29.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-08T05:45:29.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-08T05:45:29.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-08T05:45:29.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-08T05:45:29.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-08T05:45:29.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-08T05:45:29.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-08T05:45:29.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-08T05:45:29.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-08T05:45:29.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-08T05:45:29.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-08T05:45:29.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-05-08T05:45:29.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-05-08T05:45:29.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-08T05:45:29.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-05-08T05:45:29.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-05-08T05:45:29.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-05-08T05:45:29.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-05-08T05:45:29.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-08T05:45:29.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-05-08T05:45:29.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-05-08T05:45:29.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-05-08T05:45:29.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-05-08T05:45:29.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-05-08T05:45:29.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-05-08T05:45:29.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-05-08T05:45:29.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-05-08T05:45:29.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-05-08T05:45:29.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-05-08T05:45:29.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-05-08T05:45:29.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-05-08T05:45:29.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-05-08T05:45:29.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-05-08T05:45:29.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-08T05:45:29.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-08T05:45:29.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-08T05:45:29.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-08T05:45:29.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-08T05:45:29.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-08T05:45:29.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-08T05:45:29.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-08T05:45:29.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-08T05:45:29.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-08T05:45:29.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-08T05:45:29.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-08T05:45:29.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-08T05:45:29.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-08T05:45:29.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-08T05:45:29.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-05-08T05:45:29.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-08T05:45:29.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-08T05:45:29.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-08T05:45:29.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-08T05:45:29.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-08T05:45:29.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-08T05:45:29.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-08T05:45:29.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-08T05:45:29.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-08T05:45:29.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-08T05:45:29.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-08T05:45:29.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-08T05:45:29.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-08T05:45:29.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-08T05:45:29.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-05-08T05:45:29.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-05-08T05:45:29.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-08T05:45:29.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-08T05:45:29.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-08T05:45:29.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-08T05:45:29.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-08T05:45:29.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-08T05:45:29.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-08T05:45:29.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-08T05:45:29.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-08T05:45:29.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-08T05:45:29.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-08T05:45:29.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-08T05:45:29.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-08T05:45:29.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-05-08T05:45:29.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-08T05:45:29.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-05-08T05:45:29.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-05-08T05:45:29.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-08T05:45:29.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-08T05:45:29.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-05-08T05:45:29.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-05-08T05:45:29.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-08T05:45:29.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-08T05:45:29.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-05-08T05:45:29.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-05-08T05:45:29.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-08T05:45:29.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-05-08T05:45:29.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-08T05:45:29.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-08T05:45:29.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-05-08T05:45:29.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-05-08T05:45:29.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-05-08T05:45:29.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-08T05:45:29.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-05-08T05:45:29.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-08T05:45:29.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-05-08T05:45:29.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-05-08T05:45:29.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-05-08T05:45:29.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-05-08T05:45:29.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-05-08T05:45:29.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-05-08T05:45:29.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-05-08T05:45:29.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-05-08T05:45:29.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-05-08T05:45:29.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-05-08T05:45:29.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-05-08T05:45:29.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-05-08T05:45:29.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-05-08T05:45:29.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-05-08T05:45:29.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-08T05:45:29.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-08T05:45:29.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-08T05:45:29.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-05-08T05:45:29.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-05-08T05:45:29.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-05-08T05:45:29.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-05-08T05:45:29.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-08T05:45:29.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-05-08T05:45:29.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-05-08T05:45:29.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-08T05:45:29.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-05-08T05:45:29.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-05-08T05:45:29.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-08T05:45:29.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-05-08T05:45:29.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-08T05:45:29.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-05-08T05:45:29.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-08T05:45:29.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-08T05:45:29.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-05-08T05:45:29.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-05-08T05:45:29.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-05-08T05:45:29.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-05-08T05:45:29.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-05-08T05:45:29.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-05-08T05:45:29.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-05-08T05:45:29.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-05-08T05:45:29.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-05-08T05:45:29.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-05-08T05:45:29.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-05-08T05:45:29.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-05-08T05:45:29.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-05-08T05:45:29.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-05-08T05:45:29.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-05-08T05:45:29.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-05-08T05:45:29.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-05-08T05:45:29.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-05-08T05:45:29.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-05-08T05:45:29.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-05-08T05:45:29.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-05-08T05:45:29.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-05-08T05:45:29.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-05-08T05:45:29.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-05-08T05:45:29.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-05-08T05:45:29.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-05-08T05:45:29.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-05-08T05:45:29.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-05-08T05:45:29.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-05-08T05:45:29.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-05-08T05:45:29.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-05-08T05:45:29.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-05-08T05:45:29.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-05-08T05:45:29.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-05-08T05:45:29.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-05-08T05:45:29.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-05-08T05:45:29.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-05-08T05:45:29.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-05-08T05:45:29.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-05-08T05:45:29.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-05-08T05:45:29.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-05-08T05:45:29.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-05-08T05:45:29.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-05-08T05:45:29.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-05-08T05:45:29.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-05-08T05:45:29.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-05-08T05:45:29.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-05-08T05:45:29.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-05-08T05:45:29.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-05-08T05:45:29.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-05-08T05:45:29.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-05-08T05:45:29.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-05-08T05:45:29.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-05-08T05:45:29.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-05-08T05:45:29.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-05-08T05:45:29.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-05-08T05:45:29.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-05-08T05:45:29.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-05-08T05:45:29.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-05-08T05:45:29.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-05-08T05:45:29.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-05-08T05:45:29.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-05-08T05:45:29.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-05-08T05:45:29.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-05-08T05:45:29.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-05-08T05:45:29.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-05-08T05:45:29.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-05-08T05:45:29.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-05-08T05:45:29.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-05-08T05:45:29.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-05-08T05:45:29.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-05-08T05:45:29.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-05-08T05:45:29.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-05-08T05:45:29.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-05-08T05:45:29.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-05-08T05:45:29.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-05-08T05:45:29.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-05-08T05:45:29.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-05-08T05:45:29.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-05-08T05:45:29.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-05-08T05:45:29.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-05-08T05:45:29.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-05-08T05:45:29.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-05-08T05:45:29.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-05-08T05:45:29.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-05-08T05:45:29.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-05-08T05:45:29.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-05-08T05:45:29.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-05-08T05:45:29.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-05-08T05:45:29.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-05-08T05:45:29.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-05-08T05:45:29.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-05-08T05:45:29.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-05-08T05:45:29.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-05-08T05:45:29.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-05-08T05:45:29.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-05-08T05:45:29.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-05-08T05:45:29.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-05-08T05:45:29.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-05-08T05:45:29.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-05-08T05:45:29.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-05-08T05:45:29.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-05-08T05:45:29.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-05-08T05:45:29.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-05-08T05:45:29.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-05-08T05:45:29.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-05-08T05:45:29.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-05-08T05:45:29.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-05-08T05:45:29.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-05-08T05:45:29.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-05-08T05:45:29.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-05-08T05:45:29.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-05-08T05:45:29.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-05-08T05:45:29.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-05-08T05:45:29.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-05-08T05:45:29.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-08T05:45:29.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-08T05:45:29.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-05-08T05:45:29.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-05-08T05:45:29.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-08T05:45:29.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-05-08T05:45:29.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-05-08T05:45:29.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-05-08T05:45:29.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-05-08T05:45:29.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-05-08T05:45:29.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-05-08T05:45:29.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-08T05:45:29.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-08T05:45:29.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-05-08T05:45:29.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-08T05:45:29.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-08T05:45:29.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-05-08T05:45:29.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-08T05:45:29.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-05-08T05:45:29.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-05-08T05:45:29.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-05-08T05:45:29.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-08T05:45:29.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-05-08T05:45:29.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-05-08T05:45:29.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-05-08T05:45:29.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-05-08T05:45:29.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-05-08T05:45:29.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-05-08T05:45:29.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-05-08T05:45:29.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-05-08T05:45:29.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-05-08T05:45:29.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-05-08T05:45:29.193 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-05-08T05:45:29.193 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-05-08T05:45:29.193 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-05-08T05:45:29.193 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-05-08T05:45:29.193 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-05-08T05:45:29.193 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-05-08T05:45:29.197 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-05-08T05:45:29.197 DEBUG:teuthology.orchestra.run.smithi123:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-08T05:45:29.220 INFO:teuthology.orchestra.run.smithi123.stdout:postinstall scriptlet (using /bin/sh): 2024-05-08T05:45:29.220 INFO:teuthology.orchestra.run.smithi123.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-08T05:45:29.220 INFO:teuthology.orchestra.run.smithi123.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-05-08T05:45:29.220 INFO:teuthology.orchestra.run.smithi123.stdout:fi 2024-05-08T05:45:29.220 INFO:teuthology.orchestra.run.smithi123.stdout:for file in vmlinuz System.map config; do 2024-05-08T05:45:29.220 INFO:teuthology.orchestra.run.smithi123.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-05-08T05:45:29.220 INFO:teuthology.orchestra.run.smithi123.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-05-08T05:45:29.220 INFO:teuthology.orchestra.run.smithi123.stdout: fi 2024-05-08T05:45:29.220 INFO:teuthology.orchestra.run.smithi123.stdout:done 2024-05-08T05:45:29.221 INFO:teuthology.orchestra.run.smithi123.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-05-08T05:45:29.221 INFO:teuthology.orchestra.run.smithi123.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-05-08T05:45:29.221 INFO:teuthology.orchestra.run.smithi123.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-05-08T05:45:29.221 INFO:teuthology.orchestra.run.smithi123.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-05-08T05:45:29.221 INFO:teuthology.orchestra.run.smithi123.stdout:fi 2024-05-08T05:45:29.221 INFO:teuthology.orchestra.run.smithi123.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-05-08T05:45:29.221 INFO:teuthology.orchestra.run.smithi123.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-05-08T05:45:29.221 INFO:teuthology.orchestra.run.smithi123.stdout:fi 2024-05-08T05:45:29.221 INFO:teuthology.orchestra.run.smithi123.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-08T05:45:29.221 INFO:teuthology.orchestra.run.smithi123.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-08T05:45:29.221 INFO:teuthology.orchestra.run.smithi123.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-05-08T05:45:29.221 INFO:teuthology.orchestra.run.smithi123.stdout:fi 2024-05-08T05:45:29.222 DEBUG:teuthology.orchestra.run.smithi123:> sudo rpm -qi grub2-tools 2024-05-08T05:45:29.300 INFO:teuthology.orchestra.run.smithi123.stdout:Name : grub2-tools 2024-05-08T05:45:29.300 INFO:teuthology.orchestra.run.smithi123.stdout:Epoch : 1 2024-05-08T05:45:29.300 INFO:teuthology.orchestra.run.smithi123.stdout:Version : 2.06 2024-05-08T05:45:29.300 INFO:teuthology.orchestra.run.smithi123.stdout:Release : 68.el9 2024-05-08T05:45:29.300 INFO:teuthology.orchestra.run.smithi123.stdout:Architecture: x86_64 2024-05-08T05:45:29.300 INFO:teuthology.orchestra.run.smithi123.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-08T05:45:29.300 INFO:teuthology.orchestra.run.smithi123.stdout:Group : Unspecified 2024-05-08T05:45:29.300 INFO:teuthology.orchestra.run.smithi123.stdout:Size : 8264254 2024-05-08T05:45:29.301 INFO:teuthology.orchestra.run.smithi123.stdout:License : GPLv3+ 2024-05-08T05:45:29.301 INFO:teuthology.orchestra.run.smithi123.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-08T05:45:29.301 INFO:teuthology.orchestra.run.smithi123.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-08T05:45:29.301 INFO:teuthology.orchestra.run.smithi123.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-08T05:45:29.301 INFO:teuthology.orchestra.run.smithi123.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-08T05:45:29.301 INFO:teuthology.orchestra.run.smithi123.stdout:Packager : builder@centos.org 2024-05-08T05:45:29.301 INFO:teuthology.orchestra.run.smithi123.stdout:Vendor : CentOS 2024-05-08T05:45:29.301 INFO:teuthology.orchestra.run.smithi123.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-08T05:45:29.301 INFO:teuthology.orchestra.run.smithi123.stdout:Summary : Support tools for GRUB. 2024-05-08T05:45:29.301 INFO:teuthology.orchestra.run.smithi123.stdout:Description : 2024-05-08T05:45:29.301 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T05:45:29.301 INFO:teuthology.orchestra.run.smithi123.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-08T05:45:29.301 INFO:teuthology.orchestra.run.smithi123.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-08T05:45:29.301 INFO:teuthology.orchestra.run.smithi123.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-08T05:45:29.301 INFO:teuthology.orchestra.run.smithi123.stdout:hardware devices. 2024-05-08T05:45:29.302 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T05:45:29.302 INFO:teuthology.orchestra.run.smithi123.stdout:This subpackage provides tools for support of all platforms. 2024-05-08T05:45:29.302 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-08T05:45:29.303 INFO:teuthology.task.kernel:Updating grub on smithi123 to boot 6.9.0-rc3-g806ec36b2112 2024-05-08T05:45:29.303 DEBUG:teuthology.orchestra.run.smithi123:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-08T05:45:29.840 INFO:teuthology.orchestra.run.smithi123.stderr:Generating grub configuration file ... 2024-05-08T05:45:31.194 INFO:teuthology.orchestra.run.smithi123.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-08T05:45:31.224 INFO:teuthology.orchestra.run.smithi123.stderr:done 2024-05-08T05:45:31.226 DEBUG:teuthology.orchestra.run.smithi123:> mktemp 2024-05-08T05:45:31.266 INFO:teuthology.orchestra.run.smithi123.stdout:/tmp/tmp.d84ig3Ie6x 2024-05-08T05:45:31.266 DEBUG:teuthology.orchestra.run.smithi123:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.d84ig3Ie6x 2024-05-08T05:45:31.342 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0666 /tmp/tmp.d84ig3Ie6x 2024-05-08T05:45:31.482 DEBUG:teuthology.orchestra.remote:smithi123:/tmp/tmp.d84ig3Ie6x is 6KB 2024-05-08T05:45:31.530 DEBUG:teuthology.orchestra.run.smithi123:> rm -fr /tmp/tmp.d84ig3Ie6x 2024-05-08T05:45:31.544 DEBUG:teuthology.orchestra.run.smithi123:> sudo /bin/ls /boot/loader/entries || true 2024-05-08T05:45:31.606 INFO:teuthology.orchestra.run.smithi123.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-08T05:45:31.606 INFO:teuthology.orchestra.run.smithi123.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-08T05:45:31.606 INFO:teuthology.orchestra.run.smithi123.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-08T05:45:31.606 INFO:teuthology.orchestra.run.smithi123.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-05-08T05:45:31.607 DEBUG:teuthology.orchestra.run.smithi123:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-05-08T05:45:31.787 DEBUG:teuthology.orchestra.run.smithi123:> sudo shutdown -r now 2024-05-08T05:46:01.790 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-08T05:46:01.790 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-05-08T05:46:01.791 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:46:20.318 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.123 2024-05-08T05:46:29.319 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-05-08T05:46:29.320 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:46:32.386 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.123 2024-05-08T05:46:44.387 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-05-08T05:46:44.388 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:47:16.356 DEBUG:teuthology.orchestra.run.smithi123:> true 2024-05-08T05:47:16.899 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-05-08T05:47:16.899 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-05-08T05:47:16.899 DEBUG:teuthology.orchestra.run.smithi123:> uname -r 2024-05-08T05:47:16.939 INFO:teuthology.orchestra.run.smithi123.stdout:6.9.0-rc3-g806ec36b2112 2024-05-08T05:47:16.939 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-05-08T05:47:16.939 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-08T05:47:16.939 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-08T05:47:17.940 DEBUG:teuthology.parallel:result is None 2024-05-08T05:47:17.941 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-08T05:47:17.948 INFO:teuthology.task.internal:Creating test directory... 2024-05-08T05:47:17.949 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T05:47:17.954 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T05:47:17.956 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T05:47:17.993 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-08T05:47:18.000 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-08T05:47:18.011 INFO:teuthology.task.internal:Creating archive directory... 2024-05-08T05:47:18.011 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T05:47:18.040 DEBUG:teuthology.orchestra.run.smithi163:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T05:47:18.042 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T05:47:18.171 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-08T05:47:18.178 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-08T05:47:18.178 DEBUG:teuthology.orchestra.run.smithi163:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T05:47:18.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:47:18.194 DEBUG:teuthology.orchestra.run.smithi192:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T05:47:18.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:47:18.211 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T05:47:18.218 DEBUG:teuthology.orchestra.run.smithi163:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T05:47:18.237 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T05:47:18.278 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T05:47:18.287 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T05:47:18.300 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T05:47:18.306 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T05:47:18.366 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T05:47:18.418 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T05:47:18.422 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-08T05:47:18.431 INFO:teuthology.task.internal:Configuring sudo... 2024-05-08T05:47:18.431 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T05:47:18.435 DEBUG:teuthology.orchestra.run.smithi163:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T05:47:18.438 DEBUG:teuthology.orchestra.run.smithi192:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T05:47:18.493 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-08T05:47:18.503 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-08T05:47:18.503 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T05:47:18.507 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T05:47:18.509 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T05:47:18.541 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T05:47:18.655 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T05:47:18.737 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T05:47:18.808 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T05:47:18.888 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-05-08T05:47:18.888 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T05:47:18.976 DEBUG:teuthology.orchestra.run.smithi163:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T05:47:19.020 DEBUG:teuthology.orchestra.run.smithi163:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T05:47:19.077 DEBUG:teuthology.orchestra.run.smithi163:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T05:47:19.161 DEBUG:teuthology.orchestra.run.smithi163:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T05:47:19.202 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T05:47:19.202 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T05:47:19.267 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T05:47:19.310 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T05:47:19.360 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T05:47:19.443 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T05:47:19.485 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T05:47:19.485 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T05:47:19.552 DEBUG:teuthology.orchestra.run.smithi123:> sudo service rsyslog restart 2024-05-08T05:47:19.559 DEBUG:teuthology.orchestra.run.smithi163:> sudo service rsyslog restart 2024-05-08T05:47:19.562 DEBUG:teuthology.orchestra.run.smithi192:> sudo service rsyslog restart 2024-05-08T05:47:19.599 INFO:teuthology.orchestra.run.smithi163.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T05:47:19.629 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T05:47:19.661 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T05:47:20.087 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-08T05:47:20.095 INFO:teuthology.task.internal:Starting timer... 2024-05-08T05:47:20.095 INFO:teuthology.run_tasks:Running task pcp... 2024-05-08T05:47:20.112 INFO:teuthology.run_tasks:Running task selinux... 2024-05-08T05:47:20.123 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-08T05:47:20.123 DEBUG:teuthology.orchestra.run.smithi123:> sudo service auditd rotate 2024-05-08T05:47:20.230 INFO:teuthology.orchestra.run.smithi123.stdout:Rotating logs: 2024-05-08T05:47:20.234 DEBUG:teuthology.orchestra.run.smithi163:> sudo service auditd rotate 2024-05-08T05:47:20.287 INFO:teuthology.orchestra.run.smithi163.stdout:Rotating logs: 2024-05-08T05:47:20.289 DEBUG:teuthology.orchestra.run.smithi192:> sudo service auditd rotate 2024-05-08T05:47:20.360 INFO:teuthology.orchestra.run.smithi192.stdout:Rotating logs: 2024-05-08T05:47:20.362 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-08T05:47:20.363 DEBUG:teuthology.orchestra.run.smithi123:> /usr/sbin/getenforce 2024-05-08T05:47:20.419 INFO:teuthology.orchestra.run.smithi123.stdout:Permissive 2024-05-08T05:47:20.419 DEBUG:teuthology.orchestra.run.smithi163:> /usr/sbin/getenforce 2024-05-08T05:47:20.441 INFO:teuthology.orchestra.run.smithi163.stdout:Permissive 2024-05-08T05:47:20.442 DEBUG:teuthology.orchestra.run.smithi192:> /usr/sbin/getenforce 2024-05-08T05:47:20.472 INFO:teuthology.orchestra.run.smithi192.stdout:Permissive 2024-05-08T05:47:20.472 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi123.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi163.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi192.front.sepia.ceph.com': 'permissive'} 2024-05-08T05:47:20.473 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T05:47:20.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:47:20.544 DEBUG:teuthology.orchestra.run.smithi163:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T05:47:20.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:47:20.571 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T05:47:20.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:47:20.598 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-08T05:47:20.598 DEBUG:teuthology.orchestra.run.smithi123:> sudo /usr/sbin/setenforce permissive 2024-05-08T05:47:20.714 DEBUG:teuthology.orchestra.run.smithi163:> sudo /usr/sbin/setenforce permissive 2024-05-08T05:47:20.753 DEBUG:teuthology.orchestra.run.smithi192:> sudo /usr/sbin/setenforce permissive 2024-05-08T05:47:20.783 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-08T05:47:20.792 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-08T05:47:20.799 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-08T05:47:20.934 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-08T05:47:20.949 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-08T05:47:20.951 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi123.front.sepia.ceph.com,smithi163.front.sepia.ceph.com,smithi192.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-08T05:52:55.399 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi123.front.sepia.ceph.com'), Remote(name='ubuntu@smithi163.front.sepia.ceph.com'), Remote(name='ubuntu@smithi192.front.sepia.ceph.com')] 2024-05-08T05:52:55.400 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-05-08T05:52:55.402 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:52:55.512 DEBUG:teuthology.orchestra.run.smithi123:> true 2024-05-08T05:52:55.670 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-05-08T05:52:55.670 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-05-08T05:52:55.671 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:52:55.742 DEBUG:teuthology.orchestra.run.smithi163:> true 2024-05-08T05:52:55.821 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-05-08T05:52:55.822 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-05-08T05:52:55.822 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:52:55.898 DEBUG:teuthology.orchestra.run.smithi192:> true 2024-05-08T05:52:55.979 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-05-08T05:52:55.979 INFO:teuthology.run_tasks:Running task clock... 2024-05-08T05:52:55.992 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-08T05:52:55.992 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T05:52:55.992 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T05:52:56.000 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T05:52:56.000 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T05:52:56.002 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T05:52:56.002 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T05:52:56.029 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T05:52:56.043 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T05:52:56.048 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T05:52:56.062 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T05:52:56.066 INFO:teuthology.orchestra.run.smithi163.stderr:sudo: ntpd: command not found 2024-05-08T05:52:56.077 INFO:teuthology.orchestra.run.smithi163.stdout:506 Cannot talk to daemon 2024-05-08T05:52:56.086 INFO:teuthology.orchestra.run.smithi192.stderr:sudo: ntpd: command not found 2024-05-08T05:52:56.088 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T05:52:56.091 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T05:52:56.097 INFO:teuthology.orchestra.run.smithi192.stdout:506 Cannot talk to daemon 2024-05-08T05:52:56.104 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T05:52:56.110 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T05:52:56.123 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T05:52:56.130 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T05:52:56.160 INFO:teuthology.orchestra.run.smithi163.stderr:bash: line 1: ntpq: command not found 2024-05-08T05:52:56.163 INFO:teuthology.orchestra.run.smithi163.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T05:52:56.164 INFO:teuthology.orchestra.run.smithi163.stdout:=============================================================================== 2024-05-08T05:52:56.164 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:52:56.164 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:52:56.164 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:52:56.164 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:52:56.182 INFO:teuthology.orchestra.run.smithi192.stderr:bash: line 1: ntpq: command not found 2024-05-08T05:52:56.186 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T05:52:56.186 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2024-05-08T05:52:56.186 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:52:56.187 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:52:56.187 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:52:56.187 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:52:56.200 INFO:teuthology.orchestra.run.smithi123.stderr:sudo: ntpd: command not found 2024-05-08T05:52:56.228 INFO:teuthology.orchestra.run.smithi123.stdout:506 Cannot talk to daemon 2024-05-08T05:52:56.268 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T05:52:56.307 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T05:52:56.442 INFO:teuthology.orchestra.run.smithi123.stderr:bash: line 1: ntpq: command not found 2024-05-08T05:52:56.451 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T05:52:56.451 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2024-05-08T05:52:56.451 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:52:56.451 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:52:56.451 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:52:56.451 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:52:56.452 INFO:teuthology.run_tasks:Running task install... 2024-05-08T05:52:56.461 DEBUG:teuthology.task.install:project ceph 2024-05-08T05:52:56.461 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bee299f4ac344e4c45e550d7a80abcadd7bc3d5a'}} 2024-05-08T05:52:56.461 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bee299f4ac344e4c45e550d7a80abcadd7bc3d5a'} 2024-05-08T05:52:56.461 INFO:teuthology.task.install:Using flavor: default 2024-05-08T05:52:56.467 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-08T05:52:56.467 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-08T05:52:56.467 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bee299f4ac344e4c45e550d7a80abcadd7bc3d5a', 'tag': None, 'wait_for_package': False} 2024-05-08T05:52:56.468 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:52:56.469 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bee299f4ac344e4c45e550d7a80abcadd7bc3d5a', 'tag': None, 'wait_for_package': False} 2024-05-08T05:52:56.470 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:52:56.470 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bee299f4ac344e4c45e550d7a80abcadd7bc3d5a', 'tag': None, 'wait_for_package': False} 2024-05-08T05:52:56.471 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:52:56.617 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/ 2024-05-08T05:52:56.618 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3538.gbee299f4 2024-05-08T05:52:56.693 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/ 2024-05-08T05:52:56.694 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3538.gbee299f4 2024-05-08T05:52:56.700 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/ 2024-05-08T05:52:56.700 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3538.gbee299f4 2024-05-08T05:52:56.804 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T05:52:56.804 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-05-08T05:52:56.804 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T05:52:56.804 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T05:52:56.804 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T05:52:56.805 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T05:52:56.830 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T05:52:56.830 DEBUG:teuthology.orchestra.run.smithi192:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T05:52:56.870 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T05:52:56.870 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T05:52:56.870 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T05:52:56.894 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T05:52:56.894 DEBUG:teuthology.orchestra.run.smithi123:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T05:52:56.895 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T05:52:56.896 DEBUG:teuthology.orchestra.run.smithi163:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T05:52:56.897 DEBUG:teuthology.orchestra.run.smithi192:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T05:52:56.959 DEBUG:teuthology.orchestra.run.smithi163:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T05:52:56.970 DEBUG:teuthology.orchestra.run.smithi192:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T05:52:57.019 DEBUG:teuthology.orchestra.run.smithi123:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T05:52:57.032 DEBUG:teuthology.orchestra.run.smithi163:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T05:52:57.033 INFO:teuthology.orchestra.run.smithi192.stdout:check_obsoletes = 1 2024-05-08T05:52:57.035 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum clean all 2024-05-08T05:52:57.095 INFO:teuthology.orchestra.run.smithi163.stdout:check_obsoletes = 1 2024-05-08T05:52:57.096 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum clean all 2024-05-08T05:52:57.126 DEBUG:teuthology.orchestra.run.smithi123:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T05:52:57.203 INFO:teuthology.orchestra.run.smithi123.stdout:check_obsoletes = 1 2024-05-08T05:52:57.207 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum clean all 2024-05-08T05:52:57.259 INFO:teuthology.orchestra.run.smithi192.stdout:54 files removed 2024-05-08T05:52:57.287 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T05:52:57.324 INFO:teuthology.orchestra.run.smithi163.stdout:54 files removed 2024-05-08T05:52:57.352 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T05:52:57.539 INFO:teuthology.orchestra.run.smithi123.stdout:54 files removed 2024-05-08T05:52:57.582 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T05:52:57.937 INFO:teuthology.orchestra.run.smithi192.stdout:ceph packages for x86_64 215 kB/s | 84 kB 00:00 2024-05-08T05:52:58.027 INFO:teuthology.orchestra.run.smithi163.stdout:ceph packages for x86_64 204 kB/s | 84 kB 00:00 2024-05-08T05:52:58.313 INFO:teuthology.orchestra.run.smithi192.stdout:ceph noarch packages 35 kB/s | 12 kB 00:00 2024-05-08T05:52:58.361 INFO:teuthology.orchestra.run.smithi163.stdout:ceph noarch packages 41 kB/s | 12 kB 00:00 2024-05-08T05:52:58.697 INFO:teuthology.orchestra.run.smithi192.stdout:ceph source packages 5.4 kB/s | 1.9 kB 00:00 2024-05-08T05:52:58.929 INFO:teuthology.orchestra.run.smithi163.stdout:ceph source packages 3.6 kB/s | 1.9 kB 00:00 2024-05-08T05:52:58.931 INFO:teuthology.orchestra.run.smithi123.stdout:ceph packages for x86_64 92 kB/s | 84 kB 00:00 2024-05-08T05:52:59.391 INFO:teuthology.orchestra.run.smithi123.stdout:ceph noarch packages 33 kB/s | 12 kB 00:00 2024-05-08T05:52:59.729 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.1 MB 00:00 2024-05-08T05:52:59.999 INFO:teuthology.orchestra.run.smithi123.stdout:ceph source packages 3.6 kB/s | 1.9 kB 00:00 2024-05-08T05:53:00.321 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - BaseOS 5.1 MB/s | 8.1 MB 00:01 2024-05-08T05:53:01.663 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-05-08T05:53:02.756 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - BaseOS 3.0 MB/s | 8.1 MB 00:02 2024-05-08T05:53:04.156 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - AppStream 7.0 MB/s | 19 MB 00:02 2024-05-08T05:53:06.968 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - AppStream 6.4 MB/s | 19 MB 00:02 2024-05-08T05:53:09.922 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - CRB 1.3 MB/s | 6.2 MB 00:04 2024-05-08T05:53:11.672 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 16 kB 00:00 2024-05-08T05:53:12.415 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - CRB 1.3 MB/s | 6.2 MB 00:04 2024-05-08T05:53:12.827 INFO:teuthology.orchestra.run.smithi163.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:01 2024-05-08T05:53:13.835 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - CRB 2.1 MB/s | 6.2 MB 00:02 2024-05-08T05:53:14.024 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-05-08T05:53:15.070 INFO:teuthology.orchestra.run.smithi192.stdout:Extra Packages for Enterprise Linux 23 MB/s | 21 MB 00:00 2024-05-08T05:53:15.408 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - Extras packages 63 kB/s | 16 kB 00:00 2024-05-08T05:53:16.782 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-05-08T05:53:18.091 INFO:teuthology.orchestra.run.smithi163.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-05-08T05:53:20.268 INFO:teuthology.orchestra.run.smithi192.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-05-08T05:53:20.350 INFO:teuthology.orchestra.run.smithi163.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T05:53:20.350 INFO:teuthology.orchestra.run.smithi163.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T05:53:20.361 INFO:teuthology.orchestra.run.smithi163.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-08T05:53:20.362 INFO:teuthology.orchestra.run.smithi163.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-08T05:53:20.363 INFO:teuthology.orchestra.run.smithi163.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-08T05:53:20.363 INFO:teuthology.orchestra.run.smithi163.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-08T05:53:20.365 INFO:teuthology.orchestra.run.smithi163.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-08T05:53:20.367 INFO:teuthology.orchestra.run.smithi163.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-08T05:53:20.368 INFO:teuthology.orchestra.run.smithi163.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-08T05:53:20.428 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2024-05-08T05:53:20.449 INFO:teuthology.orchestra.run.smithi163.stdout:======================================================================================= 2024-05-08T05:53:20.449 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repository Size 2024-05-08T05:53:20.449 INFO:teuthology.orchestra.run.smithi163.stdout:======================================================================================= 2024-05-08T05:53:20.449 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2024-05-08T05:53:20.449 INFO:teuthology.orchestra.run.smithi163.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-08T05:53:20.449 INFO:teuthology.orchestra.run.smithi163.stdout: ceph x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 6.6 k 2024-05-08T05:53:20.449 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-base x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 5.4 M 2024-05-08T05:53:20.449 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-fuse x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 890 k 2024-05-08T05:53:20.449 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 145 k 2024-05-08T05:53:20.449 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 1.5 M 2024-05-08T05:53:20.450 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 142 k 2024-05-08T05:53:20.450 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 3.5 M 2024-05-08T05:53:20.450 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 7.4 M 2024-05-08T05:53:20.450 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-rook noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 49 k 2024-05-08T05:53:20.450 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-radosgw x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 11 M 2024-05-08T05:53:20.450 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-test x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 49 M 2024-05-08T05:53:20.450 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-volume noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 273 k 2024-05-08T05:53:20.450 INFO:teuthology.orchestra.run.smithi163.stdout: cephadm noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 763 k 2024-05-08T05:53:20.450 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-mirror x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 225 k 2024-05-08T05:53:20.451 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-shell x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 27 k 2024-05-08T05:53:20.451 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-top noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 18 k 2024-05-08T05:53:20.451 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-08T05:53:20.451 INFO:teuthology.orchestra.run.smithi163.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-08T05:53:20.451 INFO:teuthology.orchestra.run.smithi163.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-08T05:53:20.451 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs-devel x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 32 k 2024-05-08T05:53:20.451 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs2 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 747 k 2024-05-08T05:53:20.451 INFO:teuthology.orchestra.run.smithi163.stdout: librados-devel x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 127 k 2024-05-08T05:53:20.451 INFO:teuthology.orchestra.run.smithi163.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-08T05:53:20.451 INFO:teuthology.orchestra.run.smithi163.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-08T05:53:20.451 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-08T05:53:20.451 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-08T05:53:20.452 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-08T05:53:20.452 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-08T05:53:20.452 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cephfs x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 161 k 2024-05-08T05:53:20.452 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rados x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 321 k 2024-05-08T05:53:20.452 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rbd x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 299 k 2024-05-08T05:53:20.452 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rgw x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 100 k 2024-05-08T05:53:20.452 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-fuse x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 85 k 2024-05-08T05:53:20.452 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-mirror x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 3.1 M 2024-05-08T05:53:20.452 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-nbd x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 171 k 2024-05-08T05:53:20.452 INFO:teuthology.orchestra.run.smithi163.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-08T05:53:20.452 INFO:teuthology.orchestra.run.smithi163.stdout:Upgrading: 2024-05-08T05:53:20.452 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-08T05:53:20.452 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-08T05:53:20.452 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-08T05:53:20.453 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-08T05:53:20.453 INFO:teuthology.orchestra.run.smithi163.stdout: librados2 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 3.4 M 2024-05-08T05:53:20.453 INFO:teuthology.orchestra.run.smithi163.stdout: librbd1 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 3.1 M 2024-05-08T05:53:20.453 INFO:teuthology.orchestra.run.smithi163.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-08T05:53:20.453 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-08T05:53:20.453 INFO:teuthology.orchestra.run.smithi163.stdout:Installing dependencies: 2024-05-08T05:53:20.453 INFO:teuthology.orchestra.run.smithi163.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-08T05:53:20.453 INFO:teuthology.orchestra.run.smithi163.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-08T05:53:20.453 INFO:teuthology.orchestra.run.smithi163.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-08T05:53:20.453 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-common x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 21 M 2024-05-08T05:53:20.453 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 34 k 2024-05-08T05:53:20.453 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mds x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 2.4 M 2024-05-08T05:53:20.453 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 274 k 2024-05-08T05:53:20.453 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mon x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 4.7 M 2024-05-08T05:53:20.454 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-osd x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 17 M 2024-05-08T05:53:20.454 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 16 k 2024-05-08T05:53:20.454 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-selinux x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 25 k 2024-05-08T05:53:20.454 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-08T05:53:20.454 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-08T05:53:20.454 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-08T05:53:20.455 INFO:teuthology.orchestra.run.smithi163.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-08T05:53:20.455 INFO:teuthology.orchestra.run.smithi163.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-08T05:53:20.455 INFO:teuthology.orchestra.run.smithi163.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-08T05:53:20.455 INFO:teuthology.orchestra.run.smithi163.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-08T05:53:20.455 INFO:teuthology.orchestra.run.smithi163.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-08T05:53:20.455 INFO:teuthology.orchestra.run.smithi163.stdout: libcephsqlite x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 163 k 2024-05-08T05:53:20.456 INFO:teuthology.orchestra.run.smithi163.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-08T05:53:20.456 INFO:teuthology.orchestra.run.smithi163.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-08T05:53:20.456 INFO:teuthology.orchestra.run.smithi163.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-08T05:53:20.456 INFO:teuthology.orchestra.run.smithi163.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-08T05:53:20.456 INFO:teuthology.orchestra.run.smithi163.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-08T05:53:20.456 INFO:teuthology.orchestra.run.smithi163.stdout: libradosstriper1 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 500 k 2024-05-08T05:53:20.457 INFO:teuthology.orchestra.run.smithi163.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-08T05:53:20.457 INFO:teuthology.orchestra.run.smithi163.stdout: librgw2 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 5.4 M 2024-05-08T05:53:20.457 INFO:teuthology.orchestra.run.smithi163.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-08T05:53:20.457 INFO:teuthology.orchestra.run.smithi163.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-08T05:53:20.457 INFO:teuthology.orchestra.run.smithi163.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-08T05:53:20.457 INFO:teuthology.orchestra.run.smithi163.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-08T05:53:20.457 INFO:teuthology.orchestra.run.smithi163.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-08T05:53:20.457 INFO:teuthology.orchestra.run.smithi163.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-08T05:53:20.457 INFO:teuthology.orchestra.run.smithi163.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-08T05:53:20.457 INFO:teuthology.orchestra.run.smithi163.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-08T05:53:20.458 INFO:teuthology.orchestra.run.smithi163.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-08T05:53:20.458 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-08T05:53:20.458 INFO:teuthology.orchestra.run.smithi163.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-08T05:53:20.458 INFO:teuthology.orchestra.run.smithi163.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-08T05:53:20.458 INFO:teuthology.orchestra.run.smithi163.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-08T05:53:20.458 INFO:teuthology.orchestra.run.smithi163.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-08T05:53:20.458 INFO:teuthology.orchestra.run.smithi163.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-08T05:53:20.459 INFO:teuthology.orchestra.run.smithi163.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-08T05:53:20.459 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-08T05:53:20.459 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-argparse x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 45 k 2024-05-08T05:53:20.459 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-common x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 135 k 2024-05-08T05:53:20.459 INFO:teuthology.orchestra.run.smithi163.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-08T05:53:20.459 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-08T05:53:20.459 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-08T05:53:20.460 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-08T05:53:20.460 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-08T05:53:20.460 INFO:teuthology.orchestra.run.smithi163.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-08T05:53:20.460 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-08T05:53:20.460 INFO:teuthology.orchestra.run.smithi163.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-08T05:53:20.460 INFO:teuthology.orchestra.run.smithi163.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-08T05:53:20.460 INFO:teuthology.orchestra.run.smithi163.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-08T05:53:20.460 INFO:teuthology.orchestra.run.smithi163.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-08T05:53:20.460 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-08T05:53:20.460 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-08T05:53:20.460 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-08T05:53:20.460 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-08T05:53:20.460 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-08T05:53:20.461 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-08T05:53:20.461 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-08T05:53:20.461 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-08T05:53:20.461 INFO:teuthology.orchestra.run.smithi163.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-08T05:53:20.461 INFO:teuthology.orchestra.run.smithi163.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-08T05:53:20.461 INFO:teuthology.orchestra.run.smithi163.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-08T05:53:20.461 INFO:teuthology.orchestra.run.smithi163.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-08T05:53:20.461 INFO:teuthology.orchestra.run.smithi163.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-08T05:53:20.461 INFO:teuthology.orchestra.run.smithi163.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-08T05:53:20.461 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-08T05:53:20.461 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-08T05:53:20.461 INFO:teuthology.orchestra.run.smithi163.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-08T05:53:20.462 INFO:teuthology.orchestra.run.smithi163.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-08T05:53:20.462 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-08T05:53:20.462 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-08T05:53:20.462 INFO:teuthology.orchestra.run.smithi163.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-08T05:53:20.462 INFO:teuthology.orchestra.run.smithi163.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-08T05:53:20.462 INFO:teuthology.orchestra.run.smithi163.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-08T05:53:20.462 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-08T05:53:20.462 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-08T05:53:20.462 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-08T05:53:20.462 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-08T05:53:20.462 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-08T05:53:20.462 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-08T05:53:20.462 INFO:teuthology.orchestra.run.smithi163.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-08T05:53:20.463 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-08T05:53:20.463 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-08T05:53:20.463 INFO:teuthology.orchestra.run.smithi163.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-08T05:53:20.463 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-08T05:53:20.463 INFO:teuthology.orchestra.run.smithi163.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-08T05:53:20.463 INFO:teuthology.orchestra.run.smithi163.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-08T05:53:20.463 INFO:teuthology.orchestra.run.smithi163.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-08T05:53:20.463 INFO:teuthology.orchestra.run.smithi163.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-08T05:53:20.463 INFO:teuthology.orchestra.run.smithi163.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-08T05:53:20.463 INFO:teuthology.orchestra.run.smithi163.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-08T05:53:20.463 INFO:teuthology.orchestra.run.smithi163.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-08T05:53:20.463 INFO:teuthology.orchestra.run.smithi163.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-08T05:53:20.463 INFO:teuthology.orchestra.run.smithi163.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-08T05:53:20.464 INFO:teuthology.orchestra.run.smithi163.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-08T05:53:20.464 INFO:teuthology.orchestra.run.smithi163.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-08T05:53:20.464 INFO:teuthology.orchestra.run.smithi163.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-08T05:53:20.464 INFO:teuthology.orchestra.run.smithi163.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-08T05:53:20.464 INFO:teuthology.orchestra.run.smithi163.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-08T05:53:20.464 INFO:teuthology.orchestra.run.smithi163.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-08T05:53:20.464 INFO:teuthology.orchestra.run.smithi163.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-08T05:53:20.464 INFO:teuthology.orchestra.run.smithi163.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-08T05:53:20.464 INFO:teuthology.orchestra.run.smithi163.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-08T05:53:20.464 INFO:teuthology.orchestra.run.smithi163.stdout:Installing weak dependencies: 2024-05-08T05:53:20.464 INFO:teuthology.orchestra.run.smithi163.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-08T05:53:20.464 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T05:53:20.464 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2024-05-08T05:53:20.464 INFO:teuthology.orchestra.run.smithi163.stdout:======================================================================================= 2024-05-08T05:53:20.464 INFO:teuthology.orchestra.run.smithi163.stdout:Install 148 Packages 2024-05-08T05:53:20.464 INFO:teuthology.orchestra.run.smithi163.stdout:Upgrade 8 Packages 2024-05-08T05:53:20.465 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T05:53:20.465 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 226 M 2024-05-08T05:53:20.465 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading Packages: 2024-05-08T05:53:20.959 INFO:teuthology.orchestra.run.smithi163.stdout:(1/156): ceph-19.0.0-3538.gbee299f4.el9.x86_64. 36 kB/s | 6.6 kB 00:00 2024-05-08T05:53:21.225 INFO:teuthology.orchestra.run.smithi163.stdout:(2/156): ceph-fuse-19.0.0-3538.gbee299f4.el9.x8 3.3 MB/s | 890 kB 00:00 2024-05-08T05:53:21.359 INFO:teuthology.orchestra.run.smithi163.stdout:(3/156): ceph-base-19.0.0-3538.gbee299f4.el9.x8 9.2 MB/s | 5.4 MB 00:00 2024-05-08T05:53:21.384 INFO:teuthology.orchestra.run.smithi163.stdout:(4/156): ceph-immutable-object-cache-19.0.0-353 912 kB/s | 145 kB 00:00 2024-05-08T05:53:21.510 INFO:teuthology.orchestra.run.smithi163.stdout:(5/156): ceph-mds-19.0.0-3538.gbee299f4.el9.x86 16 MB/s | 2.4 MB 00:00 2024-05-08T05:53:21.769 INFO:teuthology.orchestra.run.smithi163.stdout:(6/156): ceph-mgr-19.0.0-3538.gbee299f4.el9.x86 3.9 MB/s | 1.5 MB 00:00 2024-05-08T05:53:21.836 INFO:teuthology.orchestra.run.smithi163.stdout:(7/156): ceph-mon-19.0.0-3538.gbee299f4.el9.x86 14 MB/s | 4.7 MB 00:00 2024-05-08T05:53:22.478 INFO:teuthology.orchestra.run.smithi163.stdout:(8/156): ceph-radosgw-19.0.0-3538.gbee299f4.el9 17 MB/s | 11 MB 00:00 2024-05-08T05:53:22.519 INFO:teuthology.orchestra.run.smithi192.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T05:53:22.519 INFO:teuthology.orchestra.run.smithi192.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T05:53:22.520 INFO:teuthology.orchestra.run.smithi163.stdout:(9/156): ceph-selinux-19.0.0-3538.gbee299f4.el9 606 kB/s | 25 kB 00:00 2024-05-08T05:53:22.530 INFO:teuthology.orchestra.run.smithi192.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-08T05:53:22.530 INFO:teuthology.orchestra.run.smithi192.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-08T05:53:22.531 INFO:teuthology.orchestra.run.smithi192.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-08T05:53:22.532 INFO:teuthology.orchestra.run.smithi192.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-08T05:53:22.534 INFO:teuthology.orchestra.run.smithi192.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-08T05:53:22.536 INFO:teuthology.orchestra.run.smithi192.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-08T05:53:22.537 INFO:teuthology.orchestra.run.smithi192.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-08T05:53:22.597 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-05-08T05:53:22.612 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================= 2024-05-08T05:53:22.612 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-05-08T05:53:22.612 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================= 2024-05-08T05:53:22.612 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-05-08T05:53:22.613 INFO:teuthology.orchestra.run.smithi192.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-08T05:53:22.613 INFO:teuthology.orchestra.run.smithi192.stdout: ceph x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 6.6 k 2024-05-08T05:53:22.613 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 5.4 M 2024-05-08T05:53:22.613 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 890 k 2024-05-08T05:53:22.613 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 145 k 2024-05-08T05:53:22.613 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 1.5 M 2024-05-08T05:53:22.613 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 142 k 2024-05-08T05:53:22.613 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 3.5 M 2024-05-08T05:53:22.613 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 7.4 M 2024-05-08T05:53:22.613 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 49 k 2024-05-08T05:53:22.613 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 11 M 2024-05-08T05:53:22.613 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 49 M 2024-05-08T05:53:22.614 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-volume noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 273 k 2024-05-08T05:53:22.614 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 763 k 2024-05-08T05:53:22.614 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-mirror x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 225 k 2024-05-08T05:53:22.614 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-shell x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 27 k 2024-05-08T05:53:22.614 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-top noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 18 k 2024-05-08T05:53:22.614 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-08T05:53:22.614 INFO:teuthology.orchestra.run.smithi192.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-08T05:53:22.614 INFO:teuthology.orchestra.run.smithi192.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-08T05:53:22.614 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 32 k 2024-05-08T05:53:22.614 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 747 k 2024-05-08T05:53:22.614 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 127 k 2024-05-08T05:53:22.614 INFO:teuthology.orchestra.run.smithi192.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-08T05:53:22.614 INFO:teuthology.orchestra.run.smithi192.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-08T05:53:22.615 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-08T05:53:22.615 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-08T05:53:22.615 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-08T05:53:22.615 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-08T05:53:22.615 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 161 k 2024-05-08T05:53:22.615 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 321 k 2024-05-08T05:53:22.615 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 299 k 2024-05-08T05:53:22.615 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 100 k 2024-05-08T05:53:22.615 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 85 k 2024-05-08T05:53:22.615 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 3.1 M 2024-05-08T05:53:22.615 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 171 k 2024-05-08T05:53:22.615 INFO:teuthology.orchestra.run.smithi192.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-08T05:53:22.615 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrading: 2024-05-08T05:53:22.615 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-08T05:53:22.616 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-08T05:53:22.616 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-08T05:53:22.616 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-08T05:53:22.616 INFO:teuthology.orchestra.run.smithi192.stdout: librados2 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 3.4 M 2024-05-08T05:53:22.616 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 3.1 M 2024-05-08T05:53:22.616 INFO:teuthology.orchestra.run.smithi192.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-08T05:53:22.616 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-08T05:53:22.616 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-05-08T05:53:22.616 INFO:teuthology.orchestra.run.smithi192.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-08T05:53:22.616 INFO:teuthology.orchestra.run.smithi192.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-08T05:53:22.616 INFO:teuthology.orchestra.run.smithi192.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-08T05:53:22.616 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 21 M 2024-05-08T05:53:22.616 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 34 k 2024-05-08T05:53:22.617 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 2.4 M 2024-05-08T05:53:22.617 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 274 k 2024-05-08T05:53:22.617 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 4.7 M 2024-05-08T05:53:22.617 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 17 M 2024-05-08T05:53:22.617 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 16 k 2024-05-08T05:53:22.617 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 25 k 2024-05-08T05:53:22.617 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-08T05:53:22.617 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-08T05:53:22.617 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-08T05:53:22.617 INFO:teuthology.orchestra.run.smithi192.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-08T05:53:22.617 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-08T05:53:22.617 INFO:teuthology.orchestra.run.smithi192.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-08T05:53:22.617 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-08T05:53:22.617 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-08T05:53:22.618 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 163 k 2024-05-08T05:53:22.618 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-08T05:53:22.618 INFO:teuthology.orchestra.run.smithi192.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-08T05:53:22.618 INFO:teuthology.orchestra.run.smithi192.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-08T05:53:22.618 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-08T05:53:22.618 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-08T05:53:22.618 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 500 k 2024-05-08T05:53:22.618 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-08T05:53:22.618 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 5.4 M 2024-05-08T05:53:22.618 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-08T05:53:22.618 INFO:teuthology.orchestra.run.smithi192.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-08T05:53:22.618 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-08T05:53:22.618 INFO:teuthology.orchestra.run.smithi192.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-08T05:53:22.619 INFO:teuthology.orchestra.run.smithi192.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-08T05:53:22.619 INFO:teuthology.orchestra.run.smithi192.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-08T05:53:22.619 INFO:teuthology.orchestra.run.smithi192.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-08T05:53:22.619 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-08T05:53:22.619 INFO:teuthology.orchestra.run.smithi192.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-08T05:53:22.619 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-08T05:53:22.619 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-08T05:53:22.619 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-08T05:53:22.620 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-08T05:53:22.620 INFO:teuthology.orchestra.run.smithi192.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-08T05:53:22.620 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-08T05:53:22.620 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-08T05:53:22.620 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-08T05:53:22.620 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 45 k 2024-05-08T05:53:22.620 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 135 k 2024-05-08T05:53:22.620 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-08T05:53:22.620 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-08T05:53:22.620 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-08T05:53:22.620 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-08T05:53:22.620 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-08T05:53:22.620 INFO:teuthology.orchestra.run.smithi192.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-08T05:53:22.620 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-08T05:53:22.621 INFO:teuthology.orchestra.run.smithi192.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-08T05:53:22.621 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-08T05:53:22.621 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-08T05:53:22.621 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-08T05:53:22.621 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-08T05:53:22.621 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-08T05:53:22.621 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-08T05:53:22.621 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-08T05:53:22.621 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-08T05:53:22.621 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-08T05:53:22.621 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-08T05:53:22.621 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-08T05:53:22.621 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-08T05:53:22.622 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-08T05:53:22.622 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-08T05:53:22.622 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-08T05:53:22.622 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-08T05:53:22.622 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-08T05:53:22.622 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-08T05:53:22.622 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-08T05:53:22.622 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-08T05:53:22.622 INFO:teuthology.orchestra.run.smithi192.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-08T05:53:22.622 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-08T05:53:22.622 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-08T05:53:22.622 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-08T05:53:22.622 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-08T05:53:22.623 INFO:teuthology.orchestra.run.smithi192.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-08T05:53:22.623 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-08T05:53:22.623 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-08T05:53:22.623 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-08T05:53:22.623 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-08T05:53:22.623 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-08T05:53:22.623 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-08T05:53:22.623 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-08T05:53:22.623 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-08T05:53:22.623 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-08T05:53:22.623 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-08T05:53:22.623 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-08T05:53:22.623 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-08T05:53:22.623 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-08T05:53:22.624 INFO:teuthology.orchestra.run.smithi192.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-08T05:53:22.624 INFO:teuthology.orchestra.run.smithi192.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-08T05:53:22.624 INFO:teuthology.orchestra.run.smithi192.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-08T05:53:22.624 INFO:teuthology.orchestra.run.smithi192.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-08T05:53:22.624 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-08T05:53:22.624 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-08T05:53:22.624 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-08T05:53:22.624 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-08T05:53:22.624 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-08T05:53:22.624 INFO:teuthology.orchestra.run.smithi192.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-08T05:53:22.624 INFO:teuthology.orchestra.run.smithi192.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-08T05:53:22.624 INFO:teuthology.orchestra.run.smithi192.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-08T05:53:22.624 INFO:teuthology.orchestra.run.smithi192.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-08T05:53:22.624 INFO:teuthology.orchestra.run.smithi192.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-08T05:53:22.625 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-08T05:53:22.625 INFO:teuthology.orchestra.run.smithi192.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-08T05:53:22.625 INFO:teuthology.orchestra.run.smithi192.stdout:Installing weak dependencies: 2024-05-08T05:53:22.625 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-08T05:53:22.625 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T05:53:22.625 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-05-08T05:53:22.625 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================= 2024-05-08T05:53:22.625 INFO:teuthology.orchestra.run.smithi192.stdout:Install 148 Packages 2024-05-08T05:53:22.625 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrade 8 Packages 2024-05-08T05:53:22.625 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T05:53:22.626 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 226 M 2024-05-08T05:53:22.626 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-05-08T05:53:22.968 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-05-08T05:53:23.960 INFO:teuthology.orchestra.run.smithi192.stdout:(1/156): ceph-19.0.0-3538.gbee299f4.el9.x86_64. 12 kB/s | 6.6 kB 00:00 2024-05-08T05:53:25.671 INFO:teuthology.orchestra.run.smithi123.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T05:53:25.673 INFO:teuthology.orchestra.run.smithi123.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T05:53:25.694 INFO:teuthology.orchestra.run.smithi123.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-08T05:53:25.696 INFO:teuthology.orchestra.run.smithi123.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-08T05:53:25.697 INFO:teuthology.orchestra.run.smithi123.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-08T05:53:25.699 INFO:teuthology.orchestra.run.smithi123.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-08T05:53:25.701 INFO:teuthology.orchestra.run.smithi123.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-08T05:53:25.705 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-08T05:53:25.706 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-08T05:53:25.727 INFO:teuthology.orchestra.run.smithi192.stdout:(2/156): ceph-fuse-19.0.0-3538.gbee299f4.el9.x8 504 kB/s | 890 kB 00:01 2024-05-08T05:53:25.777 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-05-08T05:53:25.794 INFO:teuthology.orchestra.run.smithi163.stdout:(10/156): ceph-test-19.0.0-3538.gbee299f4.el9.x 15 MB/s | 49 MB 00:03 2024-05-08T05:53:25.799 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-05-08T05:53:25.801 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-05-08T05:53:25.801 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-05-08T05:53:25.801 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-05-08T05:53:25.801 INFO:teuthology.orchestra.run.smithi123.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-08T05:53:25.802 INFO:teuthology.orchestra.run.smithi123.stdout: ceph x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 6.6 k 2024-05-08T05:53:25.802 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 5.4 M 2024-05-08T05:53:25.802 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 890 k 2024-05-08T05:53:25.802 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 145 k 2024-05-08T05:53:25.802 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 1.5 M 2024-05-08T05:53:25.802 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 142 k 2024-05-08T05:53:25.802 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 3.5 M 2024-05-08T05:53:25.802 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 7.4 M 2024-05-08T05:53:25.802 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 49 k 2024-05-08T05:53:25.802 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 11 M 2024-05-08T05:53:25.802 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 49 M 2024-05-08T05:53:25.802 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 273 k 2024-05-08T05:53:25.802 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 763 k 2024-05-08T05:53:25.803 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 225 k 2024-05-08T05:53:25.803 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-shell x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 27 k 2024-05-08T05:53:25.803 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 18 k 2024-05-08T05:53:25.803 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-08T05:53:25.803 INFO:teuthology.orchestra.run.smithi123.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-08T05:53:25.803 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-08T05:53:25.803 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 32 k 2024-05-08T05:53:25.803 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 747 k 2024-05-08T05:53:25.803 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 127 k 2024-05-08T05:53:25.803 INFO:teuthology.orchestra.run.smithi123.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-08T05:53:25.803 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-08T05:53:25.803 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-08T05:53:25.803 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-08T05:53:25.803 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-08T05:53:25.803 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-08T05:53:25.804 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 161 k 2024-05-08T05:53:25.804 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 321 k 2024-05-08T05:53:25.804 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 299 k 2024-05-08T05:53:25.804 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 100 k 2024-05-08T05:53:25.804 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 85 k 2024-05-08T05:53:25.804 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 3.1 M 2024-05-08T05:53:25.804 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 171 k 2024-05-08T05:53:25.804 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-08T05:53:25.804 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrading: 2024-05-08T05:53:25.804 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-08T05:53:25.804 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-08T05:53:25.804 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-08T05:53:25.804 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-08T05:53:25.804 INFO:teuthology.orchestra.run.smithi123.stdout: librados2 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 3.4 M 2024-05-08T05:53:25.805 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 3.1 M 2024-05-08T05:53:25.805 INFO:teuthology.orchestra.run.smithi123.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-08T05:53:25.805 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-08T05:53:25.805 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-05-08T05:53:25.805 INFO:teuthology.orchestra.run.smithi123.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-08T05:53:25.805 INFO:teuthology.orchestra.run.smithi123.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-08T05:53:25.805 INFO:teuthology.orchestra.run.smithi123.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-08T05:53:25.805 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 21 M 2024-05-08T05:53:25.805 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 34 k 2024-05-08T05:53:25.805 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 2.4 M 2024-05-08T05:53:25.805 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 274 k 2024-05-08T05:53:25.805 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 4.7 M 2024-05-08T05:53:25.805 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 17 M 2024-05-08T05:53:25.805 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 16 k 2024-05-08T05:53:25.805 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 25 k 2024-05-08T05:53:25.806 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-08T05:53:25.806 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-08T05:53:25.806 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-08T05:53:25.806 INFO:teuthology.orchestra.run.smithi123.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-08T05:53:25.806 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-08T05:53:25.806 INFO:teuthology.orchestra.run.smithi123.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-08T05:53:25.806 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-08T05:53:25.806 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-08T05:53:25.806 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 163 k 2024-05-08T05:53:25.806 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-08T05:53:25.806 INFO:teuthology.orchestra.run.smithi123.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-08T05:53:25.806 INFO:teuthology.orchestra.run.smithi123.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-08T05:53:25.806 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-08T05:53:25.806 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-08T05:53:25.806 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 500 k 2024-05-08T05:53:25.807 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-08T05:53:25.807 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 5.4 M 2024-05-08T05:53:25.807 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-08T05:53:25.807 INFO:teuthology.orchestra.run.smithi123.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-08T05:53:25.807 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-08T05:53:25.807 INFO:teuthology.orchestra.run.smithi123.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-08T05:53:25.807 INFO:teuthology.orchestra.run.smithi123.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-08T05:53:25.807 INFO:teuthology.orchestra.run.smithi123.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-08T05:53:25.807 INFO:teuthology.orchestra.run.smithi123.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-08T05:53:25.807 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-08T05:53:25.807 INFO:teuthology.orchestra.run.smithi123.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-08T05:53:25.807 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-08T05:53:25.807 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-08T05:53:25.807 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-08T05:53:25.807 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-08T05:53:25.808 INFO:teuthology.orchestra.run.smithi123.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-08T05:53:25.808 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-08T05:53:25.808 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-08T05:53:25.808 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-08T05:53:25.808 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 45 k 2024-05-08T05:53:25.808 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 135 k 2024-05-08T05:53:25.808 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-08T05:53:25.808 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-08T05:53:25.808 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-08T05:53:25.808 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-08T05:53:25.808 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-08T05:53:25.808 INFO:teuthology.orchestra.run.smithi123.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-08T05:53:25.808 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-08T05:53:25.808 INFO:teuthology.orchestra.run.smithi123.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-08T05:53:25.808 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-08T05:53:25.809 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-08T05:53:25.809 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-08T05:53:25.809 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-08T05:53:25.809 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-08T05:53:25.809 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-08T05:53:25.809 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-08T05:53:25.809 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-08T05:53:25.809 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-08T05:53:25.809 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-08T05:53:25.809 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-08T05:53:25.809 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-08T05:53:25.809 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-08T05:53:25.809 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-08T05:53:25.809 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-08T05:53:25.809 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-08T05:53:25.810 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-08T05:53:25.810 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-08T05:53:25.810 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-08T05:53:25.810 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-08T05:53:25.810 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-08T05:53:25.810 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-08T05:53:25.810 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-08T05:53:25.810 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-08T05:53:25.810 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-08T05:53:25.810 INFO:teuthology.orchestra.run.smithi123.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-08T05:53:25.810 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-08T05:53:25.810 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-08T05:53:25.810 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-08T05:53:25.810 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-08T05:53:25.810 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-08T05:53:25.811 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-08T05:53:25.811 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-08T05:53:25.811 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-08T05:53:25.811 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-08T05:53:25.811 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-08T05:53:25.811 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-08T05:53:25.811 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-08T05:53:25.811 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-08T05:53:25.811 INFO:teuthology.orchestra.run.smithi123.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-08T05:53:25.811 INFO:teuthology.orchestra.run.smithi123.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-08T05:53:25.811 INFO:teuthology.orchestra.run.smithi123.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-08T05:53:25.811 INFO:teuthology.orchestra.run.smithi123.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-08T05:53:25.811 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-08T05:53:25.811 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-08T05:53:25.811 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-08T05:53:25.811 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-08T05:53:25.812 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-08T05:53:25.812 INFO:teuthology.orchestra.run.smithi123.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-08T05:53:25.812 INFO:teuthology.orchestra.run.smithi123.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-08T05:53:25.812 INFO:teuthology.orchestra.run.smithi123.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-08T05:53:25.812 INFO:teuthology.orchestra.run.smithi123.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-08T05:53:25.812 INFO:teuthology.orchestra.run.smithi123.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-08T05:53:25.812 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-08T05:53:25.812 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-08T05:53:25.812 INFO:teuthology.orchestra.run.smithi123.stdout:Installing weak dependencies: 2024-05-08T05:53:25.812 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-08T05:53:25.812 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T05:53:25.812 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-05-08T05:53:25.812 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-05-08T05:53:25.812 INFO:teuthology.orchestra.run.smithi123.stdout:Install 148 Packages 2024-05-08T05:53:25.812 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrade 8 Packages 2024-05-08T05:53:25.812 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T05:53:25.813 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 226 M 2024-05-08T05:53:25.813 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-05-08T05:53:25.852 INFO:teuthology.orchestra.run.smithi192.stdout:(3/156): ceph-immutable-object-cache-19.0.0-353 1.1 MB/s | 145 kB 00:00 2024-05-08T05:53:25.961 INFO:teuthology.orchestra.run.smithi163.stdout:(11/156): ceph-osd-19.0.0-3538.gbee299f4.el9.x8 4.0 MB/s | 17 MB 00:04 2024-05-08T05:53:25.986 INFO:teuthology.orchestra.run.smithi163.stdout:(12/156): cephfs-mirror-19.0.0-3538.gbee299f4.e 1.1 MB/s | 225 kB 00:00 2024-05-08T05:53:26.012 INFO:teuthology.orchestra.run.smithi163.stdout:(13/156): cephfs-shell-19.0.0-3538.gbee299f4.el 542 kB/s | 27 kB 00:00 2024-05-08T05:53:26.028 INFO:teuthology.orchestra.run.smithi163.stdout:(14/156): libcephfs-devel-19.0.0-3538.gbee299f4 757 kB/s | 32 kB 00:00 2024-05-08T05:53:26.087 INFO:teuthology.orchestra.run.smithi163.stdout:(15/156): libcephsqlite-19.0.0-3538.gbee299f4.e 2.7 MB/s | 163 kB 00:00 2024-05-08T05:53:26.162 INFO:teuthology.orchestra.run.smithi163.stdout:(16/156): librados-devel-19.0.0-3538.gbee299f4. 1.7 MB/s | 127 kB 00:00 2024-05-08T05:53:26.213 INFO:teuthology.orchestra.run.smithi163.stdout:(17/156): libcephfs2-19.0.0-3538.gbee299f4.el9. 3.6 MB/s | 747 kB 00:00 2024-05-08T05:53:26.255 INFO:teuthology.orchestra.run.smithi163.stdout:(18/156): libradosstriper1-19.0.0-3538.gbee299f 5.3 MB/s | 500 kB 00:00 2024-05-08T05:53:26.322 INFO:teuthology.orchestra.run.smithi163.stdout:(19/156): python3-ceph-argparse-19.0.0-3538.gbe 677 kB/s | 45 kB 00:00 2024-05-08T05:53:26.414 INFO:teuthology.orchestra.run.smithi163.stdout:(20/156): python3-ceph-common-19.0.0-3538.gbee2 1.4 MB/s | 135 kB 00:00 2024-05-08T05:53:26.489 INFO:teuthology.orchestra.run.smithi163.stdout:(21/156): python3-cephfs-19.0.0-3538.gbee299f4. 2.1 MB/s | 161 kB 00:00 2024-05-08T05:53:26.698 INFO:teuthology.orchestra.run.smithi163.stdout:(22/156): ceph-common-19.0.0-3538.gbee299f4.el9 3.6 MB/s | 21 MB 00:05 2024-05-08T05:53:26.723 INFO:teuthology.orchestra.run.smithi163.stdout:(23/156): python3-rados-19.0.0-3538.gbee299f4.e 1.3 MB/s | 321 kB 00:00 2024-05-08T05:53:26.761 INFO:teuthology.orchestra.run.smithi192.stdout:(4/156): ceph-base-19.0.0-3538.gbee299f4.el9.x8 1.6 MB/s | 5.4 MB 00:03 2024-05-08T05:53:26.773 INFO:teuthology.orchestra.run.smithi163.stdout:(24/156): python3-rgw-19.0.0-3538.gbee299f4.el9 1.9 MB/s | 100 kB 00:00 2024-05-08T05:53:26.798 INFO:teuthology.orchestra.run.smithi163.stdout:(25/156): python3-rbd-19.0.0-3538.gbee299f4.el9 2.9 MB/s | 299 kB 00:00 2024-05-08T05:53:26.832 INFO:teuthology.orchestra.run.smithi163.stdout:(26/156): rbd-fuse-19.0.0-3538.gbee299f4.el9.x8 1.4 MB/s | 85 kB 00:00 2024-05-08T05:53:26.845 INFO:teuthology.orchestra.run.smithi123.stdout:(1/156): ceph-19.0.0-3538.gbee299f4.el9.x86_64. 21 kB/s | 6.6 kB 00:00 2024-05-08T05:53:26.908 INFO:teuthology.orchestra.run.smithi163.stdout:(27/156): rbd-nbd-19.0.0-3538.gbee299f4.el9.x86 2.2 MB/s | 171 kB 00:00 2024-05-08T05:53:26.974 INFO:teuthology.orchestra.run.smithi163.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3538.g 514 kB/s | 34 kB 00:00 2024-05-08T05:53:27.075 INFO:teuthology.orchestra.run.smithi163.stdout:(29/156): librgw2-19.0.0-3538.gbee299f4.el9.x86 6.2 MB/s | 5.4 MB 00:00 2024-05-08T05:53:27.100 INFO:teuthology.orchestra.run.smithi163.stdout:(30/156): ceph-mgr-cephadm-19.0.0-3538.gbee299f 1.1 MB/s | 142 kB 00:00 2024-05-08T05:53:27.428 INFO:teuthology.orchestra.run.smithi192.stdout:(5/156): ceph-mds-19.0.0-3538.gbee299f4.el9.x86 1.5 MB/s | 2.4 MB 00:01 2024-05-08T05:53:27.470 INFO:teuthology.orchestra.run.smithi192.stdout:(6/156): ceph-mgr-19.0.0-3538.gbee299f4.el9.x86 2.1 MB/s | 1.5 MB 00:00 2024-05-08T05:53:27.634 INFO:teuthology.orchestra.run.smithi163.stdout:(31/156): rbd-mirror-19.0.0-3538.gbee299f4.el9. 3.7 MB/s | 3.1 MB 00:00 2024-05-08T05:53:27.679 INFO:teuthology.orchestra.run.smithi123.stdout:(2/156): ceph-fuse-19.0.0-3538.gbee299f4.el9.x8 1.0 MB/s | 890 kB 00:00 2024-05-08T05:53:27.792 INFO:teuthology.orchestra.run.smithi163.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3538.gbe 1.7 MB/s | 274 kB 00:00 2024-05-08T05:53:27.846 INFO:teuthology.orchestra.run.smithi123.stdout:(3/156): ceph-immutable-object-cache-19.0.0-353 875 kB/s | 145 kB 00:00 2024-05-08T05:53:27.909 INFO:teuthology.orchestra.run.smithi163.stdout:(33/156): ceph-mgr-diskprediction-local-19.0.0- 9.2 MB/s | 7.4 MB 00:00 2024-05-08T05:53:27.934 INFO:teuthology.orchestra.run.smithi163.stdout:(34/156): ceph-mgr-rook-19.0.0-3538.gbee299f4.e 348 kB/s | 49 kB 00:00 2024-05-08T05:53:27.977 INFO:teuthology.orchestra.run.smithi163.stdout:(35/156): ceph-prometheus-alerts-19.0.0-3538.gb 246 kB/s | 16 kB 00:00 2024-05-08T05:53:28.052 INFO:teuthology.orchestra.run.smithi163.stdout:(36/156): ceph-mgr-dashboard-19.0.0-3538.gbee29 3.6 MB/s | 3.5 MB 00:00 2024-05-08T05:53:28.085 INFO:teuthology.orchestra.run.smithi163.stdout:(37/156): ceph-volume-19.0.0-3538.gbee299f4.el9 1.8 MB/s | 273 kB 00:00 2024-05-08T05:53:28.119 INFO:teuthology.orchestra.run.smithi163.stdout:(38/156): cephadm-19.0.0-3538.gbee299f4.el9.noa 5.3 MB/s | 763 kB 00:00 2024-05-08T05:53:28.144 INFO:teuthology.orchestra.run.smithi163.stdout:(39/156): cephfs-top-19.0.0-3538.gbee299f4.el9. 198 kB/s | 18 kB 00:00 2024-05-08T05:53:28.286 INFO:teuthology.orchestra.run.smithi163.stdout:(40/156): fuse-2.9.9-15.el9.x86_64.rpm 400 kB/s | 80 kB 00:00 2024-05-08T05:53:28.345 INFO:teuthology.orchestra.run.smithi163.stdout:(41/156): libquadmath-11.4.1-3.el9.x86_64.rpm 975 kB/s | 195 kB 00:00 2024-05-08T05:53:28.412 INFO:teuthology.orchestra.run.smithi163.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.0 MB/s | 253 kB 00:00 2024-05-08T05:53:28.470 INFO:teuthology.orchestra.run.smithi163.stdout:(43/156): libgfortran-11.4.1-3.el9.x86_64.rpm 2.2 MB/s | 803 kB 00:00 2024-05-08T05:53:28.496 INFO:teuthology.orchestra.run.smithi163.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-05-08T05:53:28.529 INFO:teuthology.orchestra.run.smithi163.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-05-08T05:53:28.563 INFO:teuthology.orchestra.run.smithi163.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-05-08T05:53:28.596 INFO:teuthology.orchestra.run.smithi163.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-05-08T05:53:28.604 INFO:teuthology.orchestra.run.smithi123.stdout:(4/156): ceph-base-19.0.0-3538.gbee299f4.el9.x8 2.6 MB/s | 5.4 MB 00:02 2024-05-08T05:53:28.630 INFO:teuthology.orchestra.run.smithi163.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 4.3 MB/s | 1.2 MB 00:00 2024-05-08T05:53:28.705 INFO:teuthology.orchestra.run.smithi163.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 107 kB 00:00 2024-05-08T05:53:28.739 INFO:teuthology.orchestra.run.smithi163.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 3.8 MB/s | 689 kB 00:00 2024-05-08T05:53:28.781 INFO:teuthology.orchestra.run.smithi163.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 5.0 MB/s | 943 kB 00:00 2024-05-08T05:53:28.806 INFO:teuthology.orchestra.run.smithi163.stdout:(52/156): elfutils-libelf-devel-0.190-2.el9.x86 214 kB/s | 21 kB 00:00 2024-05-08T05:53:28.848 INFO:teuthology.orchestra.run.smithi163.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 490 kB/s | 33 kB 00:00 2024-05-08T05:53:28.882 INFO:teuthology.orchestra.run.smithi163.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.2 MB/s | 313 kB 00:00 2024-05-08T05:53:28.888 INFO:teuthology.orchestra.run.smithi123.stdout:(5/156): ceph-mgr-19.0.0-3538.gbee299f4.el9.x86 5.4 MB/s | 1.5 MB 00:00 2024-05-08T05:53:28.924 INFO:teuthology.orchestra.run.smithi163.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 242 kB/s | 18 kB 00:00 2024-05-08T05:53:28.940 INFO:teuthology.orchestra.run.smithi163.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 190 kB/s | 11 kB 00:00 2024-05-08T05:53:29.033 INFO:teuthology.orchestra.run.smithi163.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-05-08T05:53:29.049 INFO:teuthology.orchestra.run.smithi163.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 417 kB/s | 45 kB 00:00 2024-05-08T05:53:29.080 INFO:teuthology.orchestra.run.smithi123.stdout:(6/156): ceph-mds-19.0.0-3538.gbee299f4.el9.x86 1.9 MB/s | 2.4 MB 00:01 2024-05-08T05:53:29.083 INFO:teuthology.orchestra.run.smithi163.stdout:(59/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-05-08T05:53:29.142 INFO:teuthology.orchestra.run.smithi163.stdout:(60/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 437 kB/s | 25 kB 00:00 2024-05-08T05:53:29.175 INFO:teuthology.orchestra.run.smithi163.stdout:(61/156): librdkafka-1.6.1-102.el9.x86_64.rpm 4.5 MB/s | 662 kB 00:00 2024-05-08T05:53:29.217 INFO:teuthology.orchestra.run.smithi163.stdout:(62/156): libtool-2.4.6-45.el9.x86_64.rpm 3.4 MB/s | 582 kB 00:00 2024-05-08T05:53:29.242 INFO:teuthology.orchestra.run.smithi163.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 473 kB/s | 47 kB 00:00 2024-05-08T05:53:29.301 INFO:teuthology.orchestra.run.smithi163.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-05-08T05:53:29.335 INFO:teuthology.orchestra.run.smithi163.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 382 kB/s | 35 kB 00:00 2024-05-08T05:53:29.360 INFO:teuthology.orchestra.run.smithi163.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.3 MB/s | 188 kB 00:00 2024-05-08T05:53:29.536 INFO:teuthology.orchestra.run.smithi163.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 9.1 MB/s | 1.6 MB 00:00 2024-05-08T05:53:29.597 INFO:teuthology.orchestra.run.smithi123.stdout:(7/156): ceph-mon-19.0.0-3538.gbee299f4.el9.x86 6.6 MB/s | 4.7 MB 00:00 2024-05-08T05:53:29.678 INFO:teuthology.orchestra.run.smithi163.stdout:(68/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-05-08T05:53:29.711 INFO:teuthology.orchestra.run.smithi163.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 5.1 MB/s | 911 kB 00:00 2024-05-08T05:53:29.728 INFO:teuthology.orchestra.run.smithi192.stdout:(7/156): ceph-mon-19.0.0-3538.gbee299f4.el9.x86 2.0 MB/s | 4.7 MB 00:02 2024-05-08T05:53:29.795 INFO:teuthology.orchestra.run.smithi163.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-05-08T05:53:29.903 INFO:teuthology.orchestra.run.smithi163.stdout:(71/156): openblas-openmp-0.3.21-2.el9.x86_64.r 7.9 MB/s | 4.7 MB 00:00 2024-05-08T05:53:29.987 INFO:teuthology.orchestra.run.smithi163.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-05-08T05:53:30.020 INFO:teuthology.orchestra.run.smithi163.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 4.5 MB/s | 1.0 MB 00:00 2024-05-08T05:53:30.096 INFO:teuthology.orchestra.run.smithi163.stdout:(74/156): postgresql-server-13.14-1.el9.x86_64. 15 MB/s | 5.8 MB 00:00 2024-05-08T05:53:30.188 INFO:teuthology.orchestra.run.smithi163.stdout:(75/156): python3-devel-3.9.18-3.el9.x86_64.rpm 1.4 MB/s | 244 kB 00:00 2024-05-08T05:53:30.229 INFO:teuthology.orchestra.run.smithi163.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 1.8 MB/s | 248 kB 00:00 2024-05-08T05:53:30.280 INFO:teuthology.orchestra.run.smithi163.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 279 kB/s | 26 kB 00:00 2024-05-08T05:53:30.305 INFO:teuthology.orchestra.run.smithi163.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 254 kB/s | 19 kB 00:00 2024-05-08T05:53:30.406 INFO:teuthology.orchestra.run.smithi163.stdout:(79/156): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-05-08T05:53:30.431 INFO:teuthology.orchestra.run.smithi163.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 277 kB/s | 35 kB 00:00 2024-05-08T05:53:30.456 INFO:teuthology.orchestra.run.smithi163.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 980 kB/s | 172 kB 00:00 2024-05-08T05:53:30.548 INFO:teuthology.orchestra.run.smithi163.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-05-08T05:53:30.573 INFO:teuthology.orchestra.run.smithi163.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.9 MB/s | 222 kB 00:00 2024-05-08T05:53:30.640 INFO:teuthology.orchestra.run.smithi163.stdout:(84/156): python3-prettytable-0.7.2-27.el9.noar 634 kB/s | 42 kB 00:00 2024-05-08T05:53:30.665 INFO:teuthology.orchestra.run.smithi163.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 660 kB/s | 77 kB 00:00 2024-05-08T05:53:30.749 INFO:teuthology.orchestra.run.smithi163.stdout:(86/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-05-08T05:53:30.788 INFO:teuthology.orchestra.run.smithi123.stdout:(8/156): ceph-radosgw-19.0.0-3538.gbee299f4.el9 8.9 MB/s | 11 MB 00:01 2024-05-08T05:53:30.799 INFO:teuthology.orchestra.run.smithi163.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 1.7 MB/s | 269 kB 00:00 2024-05-08T05:53:30.824 INFO:teuthology.orchestra.run.smithi163.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-05-08T05:53:30.858 INFO:teuthology.orchestra.run.smithi163.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-05-08T05:53:30.872 INFO:teuthology.orchestra.run.smithi123.stdout:(9/156): ceph-selinux-19.0.0-3538.gbee299f4.el9 306 kB/s | 25 kB 00:00 2024-05-08T05:53:30.883 INFO:teuthology.orchestra.run.smithi163.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 609 kB/s | 51 kB 00:00 2024-05-08T05:53:30.909 INFO:teuthology.orchestra.run.smithi163.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 640 kB/s | 54 kB 00:00 2024-05-08T05:53:30.959 INFO:teuthology.orchestra.run.smithi163.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 554 kB/s | 42 kB 00:00 2024-05-08T05:53:30.984 INFO:teuthology.orchestra.run.smithi163.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 587 kB/s | 44 kB 00:00 2024-05-08T05:53:31.035 INFO:teuthology.orchestra.run.smithi163.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 716 kB/s | 36 kB 00:00 2024-05-08T05:53:31.085 INFO:teuthology.orchestra.run.smithi163.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 1.7 MB/s | 221 kB 00:00 2024-05-08T05:53:31.102 INFO:teuthology.orchestra.run.smithi163.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 709 kB/s | 47 kB 00:00 2024-05-08T05:53:31.169 INFO:teuthology.orchestra.run.smithi163.stdout:(97/156): uuid-1.6.2-55.el9.x86_64.rpm 866 kB/s | 58 kB 00:00 2024-05-08T05:53:31.234 INFO:teuthology.orchestra.run.smithi163.stdout:(98/156): socat-1.7.4.1-5.el9.x86_64.rpm 2.4 MB/s | 305 kB 00:00 2024-05-08T05:53:31.253 INFO:teuthology.orchestra.run.smithi163.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 94 kB 00:00 2024-05-08T05:53:31.270 INFO:teuthology.orchestra.run.smithi163.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-05-08T05:53:31.328 INFO:teuthology.orchestra.run.smithi163.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 595 kB/s | 45 kB 00:00 2024-05-08T05:53:31.381 INFO:teuthology.orchestra.run.smithi123.stdout:(10/156): ceph-common-19.0.0-3538.gbee299f4.el9 4.4 MB/s | 21 MB 00:04 2024-05-08T05:53:31.464 INFO:teuthology.orchestra.run.smithi123.stdout:(11/156): cephfs-mirror-19.0.0-3538.gbee299f4.e 2.7 MB/s | 225 kB 00:00 2024-05-08T05:53:31.487 INFO:teuthology.orchestra.run.smithi192.stdout:(8/156): ceph-common-19.0.0-3538.gbee299f4.el9. 2.6 MB/s | 21 MB 00:08 2024-05-08T05:53:31.537 INFO:teuthology.orchestra.run.smithi192.stdout:(9/156): ceph-selinux-19.0.0-3538.gbee299f4.el9 507 kB/s | 25 kB 00:00 2024-05-08T05:53:31.548 INFO:teuthology.orchestra.run.smithi123.stdout:(12/156): cephfs-shell-19.0.0-3538.gbee299f4.el 327 kB/s | 27 kB 00:00 2024-05-08T05:53:31.554 INFO:teuthology.orchestra.run.smithi163.stdout:(102/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-05-08T05:53:31.665 INFO:teuthology.orchestra.run.smithi123.stdout:(13/156): libcephfs-devel-19.0.0-3538.gbee299f4 274 kB/s | 32 kB 00:00 2024-05-08T05:53:31.857 INFO:teuthology.orchestra.run.smithi123.stdout:(14/156): libcephfs2-19.0.0-3538.gbee299f4.el9. 3.8 MB/s | 747 kB 00:00 2024-05-08T05:53:31.973 INFO:teuthology.orchestra.run.smithi123.stdout:(15/156): libcephsqlite-19.0.0-3538.gbee299f4.e 1.4 MB/s | 163 kB 00:00 2024-05-08T05:53:32.046 INFO:teuthology.orchestra.run.smithi163.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 29 kB/s | 22 kB 00:00 2024-05-08T05:53:32.065 INFO:teuthology.orchestra.run.smithi123.stdout:(16/156): librados-devel-19.0.0-3538.gbee299f4. 1.4 MB/s | 127 kB 00:00 2024-05-08T05:53:32.104 INFO:teuthology.orchestra.run.smithi163.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 996 kB/s | 548 kB 00:00 2024-05-08T05:53:32.182 INFO:teuthology.orchestra.run.smithi123.stdout:(17/156): libradosstriper1-19.0.0-3538.gbee299f 4.2 MB/s | 500 kB 00:00 2024-05-08T05:53:32.205 INFO:teuthology.orchestra.run.smithi163.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 194 kB/s | 19 kB 00:00 2024-05-08T05:53:32.522 INFO:teuthology.orchestra.run.smithi163.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 650 kB/s | 309 kB 00:00 2024-05-08T05:53:32.597 INFO:teuthology.orchestra.run.smithi163.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-05-08T05:53:32.622 INFO:teuthology.orchestra.run.smithi163.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 259 kB/s | 26 kB 00:00 2024-05-08T05:53:32.664 INFO:teuthology.orchestra.run.smithi163.stdout:(109/156): protobuf-compiler-3.14.0-13.el9.x86_ 646 kB/s | 863 kB 00:01 2024-05-08T05:53:32.689 INFO:teuthology.orchestra.run.smithi163.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 527 kB/s | 49 kB 00:00 2024-05-08T05:53:32.723 INFO:teuthology.orchestra.run.smithi163.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 671 kB/s | 67 kB 00:00 2024-05-08T05:53:32.798 INFO:teuthology.orchestra.run.smithi163.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.5 MB/s | 839 kB 00:00 2024-05-08T05:53:32.907 INFO:teuthology.orchestra.run.smithi163.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 2.9 MB/s | 548 kB 00:00 2024-05-08T05:53:32.932 INFO:teuthology.orchestra.run.smithi163.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 325 kB/s | 43 kB 00:00 2024-05-08T05:53:33.033 INFO:teuthology.orchestra.run.smithi163.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 323 kB/s | 32 kB 00:00 2024-05-08T05:53:33.058 INFO:teuthology.orchestra.run.smithi163.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 120 kB/s | 15 kB 00:00 2024-05-08T05:53:33.083 INFO:teuthology.orchestra.run.smithi163.stdout:(117/156): luarocks-3.9.2-1.el9.noarch.rpm 361 kB/s | 151 kB 00:00 2024-05-08T05:53:33.107 INFO:teuthology.orchestra.run.smithi123.stdout:(18/156): librgw2-19.0.0-3538.gbee299f4.el9.x86 5.8 MB/s | 5.4 MB 00:00 2024-05-08T05:53:33.141 INFO:teuthology.orchestra.run.smithi163.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-05-08T05:53:33.184 INFO:teuthology.orchestra.run.smithi163.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-05-08T05:53:33.225 INFO:teuthology.orchestra.run.smithi123.stdout:(19/156): python3-ceph-argparse-19.0.0-3538.gbe 390 kB/s | 45 kB 00:00 2024-05-08T05:53:33.234 INFO:teuthology.orchestra.run.smithi163.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 384 kB/s | 35 kB 00:00 2024-05-08T05:53:33.284 INFO:teuthology.orchestra.run.smithi163.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.5 MB/s | 303 kB 00:00 2024-05-08T05:53:33.318 INFO:teuthology.orchestra.run.smithi163.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 1.5 MB/s | 206 kB 00:00 2024-05-08T05:53:33.342 INFO:teuthology.orchestra.run.smithi123.stdout:(20/156): python3-ceph-common-19.0.0-3538.gbee2 1.1 MB/s | 135 kB 00:00 2024-05-08T05:53:33.460 INFO:teuthology.orchestra.run.smithi163.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 823 kB/s | 144 kB 00:00 2024-05-08T05:53:33.485 INFO:teuthology.orchestra.run.smithi163.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 64 kB/s | 11 kB 00:00 2024-05-08T05:53:33.512 INFO:teuthology.orchestra.run.smithi192.stdout:(10/156): ceph-osd-19.0.0-3538.gbee299f4.el9.x8 2.8 MB/s | 17 MB 00:06 2024-05-08T05:53:33.535 INFO:teuthology.orchestra.run.smithi163.stdout:(125/156): python3-grpcio-1.46.7-10.el9.x86_64. 6.8 MB/s | 2.0 MB 00:00 2024-05-08T05:53:33.561 INFO:teuthology.orchestra.run.smithi163.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-05-08T05:53:33.586 INFO:teuthology.orchestra.run.smithi163.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 230 kB/s | 23 kB 00:00 2024-05-08T05:53:33.628 INFO:teuthology.orchestra.run.smithi163.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-05-08T05:53:33.653 INFO:teuthology.orchestra.run.smithi163.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 213 kB/s | 20 kB 00:00 2024-05-08T05:53:33.713 INFO:teuthology.orchestra.run.smithi192.stdout:(11/156): ceph-radosgw-19.0.0-3538.gbee299f4.el 2.7 MB/s | 11 MB 00:03 2024-05-08T05:53:33.720 INFO:teuthology.orchestra.run.smithi163.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 504 kB/s | 46 kB 00:00 2024-05-08T05:53:33.730 INFO:teuthology.orchestra.run.smithi123.stdout:(21/156): python3-cephfs-19.0.0-3538.gbee299f4. 416 kB/s | 161 kB 00:00 2024-05-08T05:53:33.754 INFO:teuthology.orchestra.run.smithi163.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 786 kB/s | 79 kB 00:00 2024-05-08T05:53:33.771 INFO:teuthology.orchestra.run.smithi192.stdout:(12/156): cephfs-shell-19.0.0-3538.gbee299f4.el 466 kB/s | 27 kB 00:00 2024-05-08T05:53:33.787 INFO:teuthology.orchestra.run.smithi163.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 5.1 MB/s | 1.0 MB 00:00 2024-05-08T05:53:33.813 INFO:teuthology.orchestra.run.smithi163.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 626 kB/s | 58 kB 00:00 2024-05-08T05:53:33.834 INFO:teuthology.orchestra.run.smithi123.stdout:(22/156): python3-rados-19.0.0-3538.gbee299f4.e 3.1 MB/s | 321 kB 00:00 2024-05-08T05:53:33.846 INFO:teuthology.orchestra.run.smithi192.stdout:(13/156): libcephfs-devel-19.0.0-3538.gbee299f4 423 kB/s | 32 kB 00:00 2024-05-08T05:53:33.854 INFO:teuthology.orchestra.run.smithi163.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-05-08T05:53:33.880 INFO:teuthology.orchestra.run.smithi163.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 178 kB/s | 16 kB 00:00 2024-05-08T05:53:33.905 INFO:teuthology.orchestra.run.smithi163.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 977 kB/s | 90 kB 00:00 2024-05-08T05:53:33.930 INFO:teuthology.orchestra.run.smithi192.stdout:(14/156): cephfs-mirror-19.0.0-3538.gbee299f4.e 540 kB/s | 225 kB 00:00 2024-05-08T05:53:33.934 INFO:teuthology.orchestra.run.smithi123.stdout:(23/156): python3-rbd-19.0.0-3538.gbee299f4.el9 2.9 MB/s | 299 kB 00:00 2024-05-08T05:53:33.947 INFO:teuthology.orchestra.run.smithi163.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 306 kB/s | 28 kB 00:00 2024-05-08T05:53:33.972 INFO:teuthology.orchestra.run.smithi163.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 333 kB/s | 31 kB 00:00 2024-05-08T05:53:34.005 INFO:teuthology.orchestra.run.smithi192.stdout:(15/156): libcephfs2-19.0.0-3538.gbee299f4.el9. 4.6 MB/s | 747 kB 00:00 2024-05-08T05:53:34.023 INFO:teuthology.orchestra.run.smithi163.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-05-08T05:53:34.048 INFO:teuthology.orchestra.run.smithi163.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 589 kB/s | 59 kB 00:00 2024-05-08T05:53:34.051 INFO:teuthology.orchestra.run.smithi123.stdout:(24/156): python3-rgw-19.0.0-3538.gbee299f4.el9 859 kB/s | 100 kB 00:00 2024-05-08T05:53:34.064 INFO:teuthology.orchestra.run.smithi192.stdout:(16/156): libcephsqlite-19.0.0-3538.gbee299f4.e 1.2 MB/s | 163 kB 00:00 2024-05-08T05:53:34.065 INFO:teuthology.orchestra.run.smithi163.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 388 kB/s | 36 kB 00:00 2024-05-08T05:53:34.089 INFO:teuthology.orchestra.run.smithi192.stdout:(17/156): librados-devel-19.0.0-3538.gbee299f4. 1.5 MB/s | 127 kB 00:00 2024-05-08T05:53:34.107 INFO:teuthology.orchestra.run.smithi163.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 609 kB/s | 51 kB 00:00 2024-05-08T05:53:34.148 INFO:teuthology.orchestra.run.smithi163.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-05-08T05:53:34.151 INFO:teuthology.orchestra.run.smithi123.stdout:(25/156): rbd-fuse-19.0.0-3538.gbee299f4.el9.x8 861 kB/s | 85 kB 00:00 2024-05-08T05:53:34.165 INFO:teuthology.orchestra.run.smithi163.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 893 kB/s | 90 kB 00:00 2024-05-08T05:53:34.208 INFO:teuthology.orchestra.run.smithi163.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-05-08T05:53:34.233 INFO:teuthology.orchestra.run.smithi163.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 238 kB/s | 20 kB 00:00 2024-05-08T05:53:34.275 INFO:teuthology.orchestra.run.smithi163.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-05-08T05:53:34.341 INFO:teuthology.orchestra.run.smithi163.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-05-08T05:53:34.381 INFO:teuthology.orchestra.run.smithi192.stdout:(18/156): libradosstriper1-19.0.0-3538.gbee299f 1.5 MB/s | 500 kB 00:00 2024-05-08T05:53:34.473 INFO:teuthology.orchestra.run.smithi192.stdout:(19/156): python3-ceph-argparse-19.0.0-3538.gbe 492 kB/s | 45 kB 00:00 2024-05-08T05:53:34.573 INFO:teuthology.orchestra.run.smithi192.stdout:(20/156): python3-ceph-common-19.0.0-3538.gbee2 1.3 MB/s | 135 kB 00:00 2024-05-08T05:53:34.674 INFO:teuthology.orchestra.run.smithi192.stdout:(21/156): python3-cephfs-19.0.0-3538.gbee299f4. 1.6 MB/s | 161 kB 00:00 2024-05-08T05:53:34.817 INFO:teuthology.orchestra.run.smithi163.stdout:(149/156): NetworkManager-1.47.5-1.el9.x86_64.r 4.9 MB/s | 2.3 MB 00:00 2024-05-08T05:53:34.849 INFO:teuthology.orchestra.run.smithi192.stdout:(22/156): python3-rados-19.0.0-3538.gbee299f4.e 1.8 MB/s | 321 kB 00:00 2024-05-08T05:53:35.033 INFO:teuthology.orchestra.run.smithi192.stdout:(23/156): python3-rbd-19.0.0-3538.gbee299f4.el9 1.6 MB/s | 299 kB 00:00 2024-05-08T05:53:35.059 INFO:teuthology.orchestra.run.smithi163.stdout:(150/156): NetworkManager-libnm-1.47.5-1.el9.x8 7.6 MB/s | 1.8 MB 00:00 2024-05-08T05:53:35.100 INFO:teuthology.orchestra.run.smithi192.stdout:(24/156): librgw2-19.0.0-3538.gbee299f4.el9.x86 5.3 MB/s | 5.4 MB 00:01 2024-05-08T05:53:35.117 INFO:teuthology.orchestra.run.smithi163.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 717 kB/s | 42 kB 00:00 2024-05-08T05:53:35.125 INFO:teuthology.orchestra.run.smithi192.stdout:(25/156): python3-rgw-19.0.0-3538.gbee299f4.el9 1.1 MB/s | 100 kB 00:00 2024-05-08T05:53:35.150 INFO:teuthology.orchestra.run.smithi192.stdout:(26/156): rbd-fuse-19.0.0-3538.gbee299f4.el9.x8 1.7 MB/s | 85 kB 00:00 2024-05-08T05:53:35.176 INFO:teuthology.orchestra.run.smithi163.stdout:(152/156): NetworkManager-tui-1.47.5-1.el9.x86_ 4.2 MB/s | 248 kB 00:00 2024-05-08T05:53:35.217 INFO:teuthology.orchestra.run.smithi192.stdout:(27/156): rbd-nbd-19.0.0-3538.gbee299f4.el9.x86 2.5 MB/s | 171 kB 00:00 2024-05-08T05:53:35.276 INFO:teuthology.orchestra.run.smithi192.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3538.g 590 kB/s | 34 kB 00:00 2024-05-08T05:53:35.334 INFO:teuthology.orchestra.run.smithi192.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3538.gbee299f 2.4 MB/s | 142 kB 00:00 2024-05-08T05:53:35.351 INFO:teuthology.orchestra.run.smithi163.stdout:(153/156): openssl-3.2.1-1.el9.x86_64.rpm 7.7 MB/s | 1.3 MB 00:00 2024-05-08T05:53:35.402 INFO:teuthology.orchestra.run.smithi163.stdout:(154/156): librados2-19.0.0-3538.gbee299f4.el9. 2.9 MB/s | 3.4 MB 00:01 2024-05-08T05:53:35.452 INFO:teuthology.orchestra.run.smithi163.stdout:(155/156): librbd1-19.0.0-3538.gbee299f4.el9.x8 2.7 MB/s | 3.1 MB 00:01 2024-05-08T05:53:35.602 INFO:teuthology.orchestra.run.smithi163.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 9.6 MB/s | 2.4 MB 00:00 2024-05-08T05:53:35.606 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2024-05-08T05:53:35.606 INFO:teuthology.orchestra.run.smithi163.stdout:Total 15 MB/s | 226 MB 00:15 2024-05-08T05:53:35.610 INFO:teuthology.orchestra.run.smithi123.stdout:(26/156): ceph-test-19.0.0-3538.gbee299f4.el9.x 10 MB/s | 49 MB 00:04 2024-05-08T05:53:35.677 INFO:teuthology.orchestra.run.smithi192.stdout:(30/156): ceph-mgr-dashboard-19.0.0-3538.gbee29 10 MB/s | 3.5 MB 00:00 2024-05-08T05:53:35.685 INFO:teuthology.orchestra.run.smithi123.stdout:(27/156): rbd-mirror-19.0.0-3538.gbee299f4.el9. 2.0 MB/s | 3.1 MB 00:01 2024-05-08T05:53:35.718 INFO:teuthology.orchestra.run.smithi123.stdout:(28/156): rbd-nbd-19.0.0-3538.gbee299f4.el9.x86 1.6 MB/s | 171 kB 00:00 2024-05-08T05:53:35.777 INFO:teuthology.orchestra.run.smithi123.stdout:(29/156): ceph-grafana-dashboards-19.0.0-3538.g 376 kB/s | 34 kB 00:00 2024-05-08T05:53:35.803 INFO:teuthology.orchestra.run.smithi123.stdout:(30/156): ceph-mgr-cephadm-19.0.0-3538.gbee299f 1.7 MB/s | 142 kB 00:00 2024-05-08T05:53:35.852 INFO:teuthology.orchestra.run.smithi192.stdout:(31/156): rbd-mirror-19.0.0-3538.gbee299f4.el9. 4.3 MB/s | 3.1 MB 00:00 2024-05-08T05:53:35.994 INFO:teuthology.orchestra.run.smithi192.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3538.gbe 1.9 MB/s | 274 kB 00:00 2024-05-08T05:53:36.077 INFO:teuthology.orchestra.run.smithi192.stdout:(33/156): ceph-mgr-rook-19.0.0-3538.gbee299f4.e 594 kB/s | 49 kB 00:00 2024-05-08T05:53:36.161 INFO:teuthology.orchestra.run.smithi192.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3538.gb 198 kB/s | 16 kB 00:00 2024-05-08T05:53:36.328 INFO:teuthology.orchestra.run.smithi192.stdout:(35/156): ceph-volume-19.0.0-3538.gbee299f4.el9 1.6 MB/s | 273 kB 00:00 2024-05-08T05:53:36.736 INFO:teuthology.orchestra.run.smithi192.stdout:(36/156): ceph-mgr-diskprediction-local-19.0.0- 7.0 MB/s | 7.4 MB 00:01 2024-05-08T05:53:36.762 INFO:teuthology.orchestra.run.smithi192.stdout:(37/156): cephadm-19.0.0-3538.gbee299f4.el9.noa 1.7 MB/s | 763 kB 00:00 2024-05-08T05:53:36.775 INFO:teuthology.orchestra.run.smithi123.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3538.gbee29 3.6 MB/s | 3.5 MB 00:00 2024-05-08T05:53:36.787 INFO:teuthology.orchestra.run.smithi192.stdout:(38/156): cephfs-top-19.0.0-3538.gbee299f4.el9. 364 kB/s | 18 kB 00:00 2024-05-08T05:53:36.806 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2024-05-08T05:53:36.945 INFO:teuthology.orchestra.run.smithi123.stdout:(32/156): ceph-mgr-diskprediction-local-19.0.0- 6.5 MB/s | 7.4 MB 00:01 2024-05-08T05:53:36.959 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction check succeeded. 2024-05-08T05:53:36.959 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2024-05-08T05:53:36.995 INFO:teuthology.orchestra.run.smithi123.stdout:(33/156): ceph-mgr-modules-core-19.0.0-3538.gbe 1.2 MB/s | 274 kB 00:00 2024-05-08T05:53:37.037 INFO:teuthology.orchestra.run.smithi123.stdout:(34/156): ceph-mgr-rook-19.0.0-3538.gbee299f4.e 542 kB/s | 49 kB 00:00 2024-05-08T05:53:37.104 INFO:teuthology.orchestra.run.smithi123.stdout:(35/156): ceph-prometheus-alerts-19.0.0-3538.gb 153 kB/s | 16 kB 00:00 2024-05-08T05:53:37.138 INFO:teuthology.orchestra.run.smithi123.stdout:(36/156): ceph-volume-19.0.0-3538.gbee299f4.el9 2.7 MB/s | 273 kB 00:00 2024-05-08T05:53:37.212 INFO:teuthology.orchestra.run.smithi192.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 178 kB/s | 80 kB 00:00 2024-05-08T05:53:37.230 INFO:teuthology.orchestra.run.smithi123.stdout:(37/156): cephfs-top-19.0.0-3538.gbee299f4.el9. 200 kB/s | 18 kB 00:00 2024-05-08T05:53:37.272 INFO:teuthology.orchestra.run.smithi123.stdout:(38/156): cephadm-19.0.0-3538.gbee299f4.el9.noa 4.5 MB/s | 763 kB 00:00 2024-05-08T05:53:37.496 INFO:teuthology.orchestra.run.smithi192.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 690 kB/s | 195 kB 00:00 2024-05-08T05:53:37.522 INFO:teuthology.orchestra.run.smithi123.stdout:(39/156): ceph-osd-19.0.0-3538.gbee299f4.el9.x8 2.0 MB/s | 17 MB 00:08 2024-05-08T05:53:37.547 INFO:teuthology.orchestra.run.smithi123.stdout:(40/156): fuse-2.9.9-15.el9.x86_64.rpm 254 kB/s | 80 kB 00:00 2024-05-08T05:53:37.613 INFO:teuthology.orchestra.run.smithi192.stdout:(41/156): libgfortran-11.4.1-3.el9.x86_64.rpm 973 kB/s | 803 kB 00:00 2024-05-08T05:53:37.681 INFO:teuthology.orchestra.run.smithi123.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.9 MB/s | 253 kB 00:00 2024-05-08T05:53:37.714 INFO:teuthology.orchestra.run.smithi123.stdout:(42/156): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-05-08T05:53:37.721 INFO:teuthology.orchestra.run.smithi192.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-05-08T05:53:37.756 INFO:teuthology.orchestra.run.smithi123.stdout:(43/156): libgfortran-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 803 kB 00:00 2024-05-08T05:53:37.782 INFO:teuthology.orchestra.run.smithi123.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-05-08T05:53:37.815 INFO:teuthology.orchestra.run.smithi123.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-05-08T05:53:37.830 INFO:teuthology.orchestra.run.smithi192.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 986 kB/s | 106 kB 00:00 2024-05-08T05:53:37.849 INFO:teuthology.orchestra.run.smithi123.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-05-08T05:53:37.883 INFO:teuthology.orchestra.run.smithi123.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 215 kB 00:00 2024-05-08T05:53:37.924 INFO:teuthology.orchestra.run.smithi123.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 5.1 MB/s | 1.2 MB 00:00 2024-05-08T05:53:38.000 INFO:teuthology.orchestra.run.smithi123.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 4.5 MB/s | 689 kB 00:00 2024-05-08T05:53:38.025 INFO:teuthology.orchestra.run.smithi123.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 107 kB 00:00 2024-05-08T05:53:38.064 INFO:teuthology.orchestra.run.smithi192.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 2.7 MB/s | 1.2 MB 00:00 2024-05-08T05:53:38.075 INFO:teuthology.orchestra.run.smithi123.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 4.8 MB/s | 943 kB 00:00 2024-05-08T05:53:38.089 INFO:teuthology.orchestra.run.smithi192.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 522 kB/s | 135 kB 00:00 2024-05-08T05:53:38.173 INFO:teuthology.orchestra.run.smithi192.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-05-08T05:53:38.197 INFO:teuthology.orchestra.run.smithi123.stdout:(52/156): elfutils-libelf-devel-0.190-2.el9.x86 110 kB/s | 21 kB 00:00 2024-05-08T05:53:38.198 INFO:teuthology.orchestra.run.smithi192.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-05-08T05:53:38.284 INFO:teuthology.orchestra.run.smithi123.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 157 kB/s | 33 kB 00:00 2024-05-08T05:53:38.318 INFO:teuthology.orchestra.run.smithi123.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 1.0 MB/s | 313 kB 00:00 2024-05-08T05:53:38.357 INFO:teuthology.orchestra.run.smithi192.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 689 kB 00:00 2024-05-08T05:53:38.385 INFO:teuthology.orchestra.run.smithi123.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 183 kB/s | 18 kB 00:00 2024-05-08T05:53:38.407 INFO:teuthology.orchestra.run.smithi192.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-05-08T05:53:38.427 INFO:teuthology.orchestra.run.smithi123.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 103 kB/s | 11 kB 00:00 2024-05-08T05:53:38.440 INFO:teuthology.orchestra.run.smithi192.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 3.8 MB/s | 943 kB 00:00 2024-05-08T05:53:38.466 INFO:teuthology.orchestra.run.smithi192.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 367 kB/s | 21 kB 00:00 2024-05-08T05:53:38.491 INFO:teuthology.orchestra.run.smithi192.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-05-08T05:53:38.502 INFO:teuthology.orchestra.run.smithi123.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 610 kB/s | 45 kB 00:00 2024-05-08T05:53:38.516 INFO:teuthology.orchestra.run.smithi192.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 653 kB/s | 33 kB 00:00 2024-05-08T05:53:38.528 INFO:teuthology.orchestra.run.smithi123.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.1 MB/s | 160 kB 00:00 2024-05-08T05:53:38.567 INFO:teuthology.orchestra.run.smithi192.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 366 kB/s | 18 kB 00:00 2024-05-08T05:53:38.617 INFO:teuthology.orchestra.run.smithi192.stdout:(55/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-05-08T05:53:38.619 INFO:teuthology.orchestra.run.smithi123.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.1 MB/s | 3.0 MB 00:00 2024-05-08T05:53:38.642 INFO:teuthology.orchestra.run.smithi192.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 148 kB/s | 11 kB 00:00 2024-05-08T05:53:38.668 INFO:teuthology.orchestra.run.smithi192.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-05-08T05:53:38.686 INFO:teuthology.orchestra.run.smithi123.stdout:(60/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 387 kB/s | 25 kB 00:00 2024-05-08T05:53:38.693 INFO:teuthology.orchestra.run.smithi192.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 905 kB/s | 45 kB 00:00 2024-05-08T05:53:38.727 INFO:teuthology.orchestra.run.smithi192.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-05-08T05:53:38.728 INFO:teuthology.orchestra.run.smithi123.stdout:(61/156): librdkafka-1.6.1-102.el9.x86_64.rpm 2.9 MB/s | 662 kB 00:00 2024-05-08T05:53:38.760 INFO:teuthology.orchestra.run.smithi192.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 582 kB 00:00 2024-05-08T05:53:38.780 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded. 2024-05-08T05:53:38.781 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2024-05-08T05:53:38.785 INFO:teuthology.orchestra.run.smithi192.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 437 kB/s | 25 kB 00:00 2024-05-08T05:53:38.787 INFO:teuthology.orchestra.run.smithi123.stdout:(62/156): libtool-2.4.6-45.el9.x86_64.rpm 2.2 MB/s | 582 kB 00:00 2024-05-08T05:53:38.811 INFO:teuthology.orchestra.run.smithi192.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-05-08T05:53:38.829 INFO:teuthology.orchestra.run.smithi123.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 336 kB/s | 47 kB 00:00 2024-05-08T05:53:38.836 INFO:teuthology.orchestra.run.smithi192.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-05-08T05:53:38.861 INFO:teuthology.orchestra.run.smithi192.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-05-08T05:53:38.886 INFO:teuthology.orchestra.run.smithi192.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 702 kB/s | 35 kB 00:00 2024-05-08T05:53:38.896 INFO:teuthology.orchestra.run.smithi123.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.7 MB/s | 292 kB 00:00 2024-05-08T05:53:38.938 INFO:teuthology.orchestra.run.smithi123.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 327 kB/s | 35 kB 00:00 2024-05-08T05:53:38.971 INFO:teuthology.orchestra.run.smithi123.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.0 MB/s | 188 kB 00:00 2024-05-08T05:53:39.012 INFO:teuthology.orchestra.run.smithi192.stdout:(66/156): openblas-openmp-0.3.21-2.el9.x86_64.r 32 MB/s | 4.7 MB 00:00 2024-05-08T05:53:39.087 INFO:teuthology.orchestra.run.smithi192.stdout:(67/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 22 MB/s | 4.4 MB 00:00 2024-05-08T05:53:39.129 INFO:teuthology.orchestra.run.smithi192.stdout:(68/156): postgresql-13.14-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-05-08T05:53:39.168 INFO:teuthology.orchestra.run.smithi123.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 8.1 MB/s | 1.6 MB 00:00 2024-05-08T05:53:39.171 INFO:teuthology.orchestra.run.smithi192.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 911 kB 00:00 2024-05-08T05:53:39.196 INFO:teuthology.orchestra.run.smithi192.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-05-08T05:53:39.255 INFO:teuthology.orchestra.run.smithi192.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-05-08T05:53:39.297 INFO:teuthology.orchestra.run.smithi123.stdout:(68/156): openblas-openmp-0.3.21-2.el9.x86_64.r 12 MB/s | 4.7 MB 00:00 2024-05-08T05:53:39.297 INFO:teuthology.orchestra.run.smithi192.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 2.1 MB/s | 87 kB 00:00 2024-05-08T05:53:39.397 INFO:teuthology.orchestra.run.smithi123.stdout:(69/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 9.7 MB/s | 4.4 MB 00:00 2024-05-08T05:53:39.422 INFO:teuthology.orchestra.run.smithi192.stdout:(73/156): postgresql-server-13.14-1.el9.x86_64. 23 MB/s | 5.8 MB 00:00 2024-05-08T05:53:39.489 INFO:teuthology.orchestra.run.smithi123.stdout:(70/156): postgresql-contrib-13.14-1.el9.x86_64 2.8 MB/s | 911 kB 00:00 2024-05-08T05:53:39.506 INFO:teuthology.orchestra.run.smithi192.stdout:(74/156): python3-babel-2.9.1-2.el9.noarch.rpm 29 MB/s | 6.0 MB 00:00 2024-05-08T05:53:39.514 INFO:teuthology.orchestra.run.smithi123.stdout:(71/156): postgresql-private-libs-13.14-1.el9.x 616 kB/s | 133 kB 00:00 2024-05-08T05:53:39.531 INFO:teuthology.orchestra.run.smithi192.stdout:(75/156): python3-devel-3.9.18-3.el9.x86_64.rpm 2.2 MB/s | 244 kB 00:00 2024-05-08T05:53:39.557 INFO:teuthology.orchestra.run.smithi192.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-05-08T05:53:39.573 INFO:teuthology.orchestra.run.smithi123.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-05-08T05:53:39.574 INFO:teuthology.orchestra.run.smithi192.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 614 kB/s | 26 kB 00:00 2024-05-08T05:53:39.591 INFO:teuthology.orchestra.run.smithi192.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 571 kB/s | 19 kB 00:00 2024-05-08T05:53:39.616 INFO:teuthology.orchestra.run.smithi192.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-05-08T05:53:39.633 INFO:teuthology.orchestra.run.smithi192.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 831 kB/s | 35 kB 00:00 2024-05-08T05:53:39.640 INFO:teuthology.orchestra.run.smithi123.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 6.7 MB/s | 1.0 MB 00:00 2024-05-08T05:53:39.683 INFO:teuthology.orchestra.run.smithi192.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-05-08T05:53:39.733 INFO:teuthology.orchestra.run.smithi192.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-05-08T05:53:39.784 INFO:teuthology.orchestra.run.smithi192.stdout:(83/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-08T05:53:39.815 INFO:teuthology.orchestra.run.smithi123.stdout:(74/156): postgresql-server-13.14-1.el9.x86_64. 14 MB/s | 5.8 MB 00:00 2024-05-08T05:53:39.840 INFO:teuthology.orchestra.run.smithi123.stdout:(75/156): python3-devel-3.9.18-3.el9.x86_64.rpm 1.2 MB/s | 244 kB 00:00 2024-05-08T05:53:39.851 INFO:teuthology.orchestra.run.smithi192.stdout:(84/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 22 MB/s | 5.1 MB 00:00 2024-05-08T05:53:39.876 INFO:teuthology.orchestra.run.smithi192.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 460 kB/s | 42 kB 00:00 2024-05-08T05:53:39.901 INFO:teuthology.orchestra.run.smithi192.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-05-08T05:53:39.916 INFO:teuthology.orchestra.run.smithi123.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 345 kB/s | 26 kB 00:00 2024-05-08T05:53:39.918 INFO:teuthology.orchestra.run.smithi192.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.7 MB/s | 159 kB 00:00 2024-05-08T05:53:39.958 INFO:teuthology.orchestra.run.smithi123.stdout:(77/156): python3-jinja2-2.11.3-5.el9.noarch.rp 1.7 MB/s | 248 kB 00:00 2024-05-08T05:53:39.960 INFO:teuthology.orchestra.run.smithi192.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.6 MB/s | 279 kB 00:00 2024-05-08T05:53:39.985 INFO:teuthology.orchestra.run.smithi192.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 763 kB/s | 51 kB 00:00 2024-05-08T05:53:40.000 INFO:teuthology.orchestra.run.smithi123.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 232 kB/s | 19 kB 00:00 2024-05-08T05:53:40.019 INFO:teuthology.orchestra.run.smithi192.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 921 kB/s | 54 kB 00:00 2024-05-08T05:53:40.070 INFO:teuthology.orchestra.run.smithi192.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 832 kB/s | 42 kB 00:00 2024-05-08T05:53:40.111 INFO:teuthology.orchestra.run.smithi192.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-05-08T05:53:40.133 INFO:teuthology.orchestra.run.smithi123.stdout:(79/156): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-05-08T05:53:40.175 INFO:teuthology.orchestra.run.smithi123.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 199 kB/s | 35 kB 00:00 2024-05-08T05:53:40.187 INFO:teuthology.orchestra.run.smithi192.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-05-08T05:53:40.200 INFO:teuthology.orchestra.run.smithi123.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 713 kB/s | 172 kB 00:00 2024-05-08T05:53:40.254 INFO:teuthology.orchestra.run.smithi192.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 539 kB/s | 36 kB 00:00 2024-05-08T05:53:40.284 INFO:teuthology.orchestra.run.smithi123.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-05-08T05:53:40.304 INFO:teuthology.orchestra.run.smithi192.stdout:(95/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 951 kB/s | 47 kB 00:00 2024-05-08T05:53:40.317 INFO:teuthology.orchestra.run.smithi123.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.9 MB/s | 222 kB 00:00 2024-05-08T05:53:40.393 INFO:teuthology.orchestra.run.smithi123.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 716 kB/s | 77 kB 00:00 2024-05-08T05:53:40.443 INFO:teuthology.orchestra.run.smithi123.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 339 kB/s | 42 kB 00:00 2024-05-08T05:53:40.463 INFO:teuthology.orchestra.run.smithi192.stdout:(96/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-05-08T05:53:40.496 INFO:teuthology.orchestra.run.smithi192.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 1.6 MB/s | 305 kB 00:00 2024-05-08T05:53:40.522 INFO:teuthology.orchestra.run.smithi192.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 992 kB/s | 58 kB 00:00 2024-05-08T05:53:40.547 INFO:teuthology.orchestra.run.smithi192.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-05-08T05:53:40.572 INFO:teuthology.orchestra.run.smithi192.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-08T05:53:40.577 INFO:teuthology.orchestra.run.smithi123.stdout:(86/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-05-08T05:53:40.598 INFO:teuthology.orchestra.run.smithi192.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-05-08T05:53:40.603 INFO:teuthology.orchestra.run.smithi123.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-05-08T05:53:40.623 INFO:teuthology.orchestra.run.smithi192.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 442 kB/s | 22 kB 00:00 2024-05-08T05:53:40.657 INFO:teuthology.orchestra.run.smithi192.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 14 MB/s | 863 kB 00:00 2024-05-08T05:53:40.665 INFO:teuthology.orchestra.run.smithi123.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 994 kB/s | 269 kB 00:00 2024-05-08T05:53:40.746 INFO:teuthology.orchestra.run.smithi123.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 361 kB/s | 51 kB 00:00 2024-05-08T05:53:40.795 INFO:teuthology.orchestra.run.smithi123.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 417 kB/s | 54 kB 00:00 2024-05-08T05:53:40.829 INFO:teuthology.orchestra.run.smithi123.stdout:(91/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.1 MB/s | 279 kB 00:00 2024-05-08T05:53:40.904 INFO:teuthology.orchestra.run.smithi123.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 386 kB/s | 42 kB 00:00 2024-05-08T05:53:40.965 INFO:teuthology.orchestra.run.smithi192.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.6 MB/s | 548 kB 00:00 2024-05-08T05:53:40.976 INFO:teuthology.orchestra.run.smithi123.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 302 kB/s | 44 kB 00:00 2024-05-08T05:53:40.991 INFO:teuthology.orchestra.run.smithi192.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 1.0 MB/s | 309 kB 00:00 2024-05-08T05:53:41.024 INFO:teuthology.orchestra.run.smithi192.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 334 kB/s | 19 kB 00:00 2024-05-08T05:53:41.063 INFO:teuthology.orchestra.run.smithi123.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 420 kB/s | 36 kB 00:00 2024-05-08T05:53:41.091 INFO:teuthology.orchestra.run.smithi192.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 389 kB/s | 26 kB 00:00 2024-05-08T05:53:41.096 INFO:teuthology.orchestra.run.smithi123.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 1.1 MB/s | 221 kB 00:00 2024-05-08T05:53:41.150 INFO:teuthology.orchestra.run.smithi192.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 836 kB/s | 49 kB 00:00 2024-05-08T05:53:41.180 INFO:teuthology.orchestra.run.smithi123.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 408 kB/s | 47 kB 00:00 2024-05-08T05:53:41.225 INFO:teuthology.orchestra.run.smithi192.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 899 kB/s | 67 kB 00:00 2024-05-08T05:53:41.271 INFO:teuthology.orchestra.run.smithi123.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 1.7 MB/s | 305 kB 00:00 2024-05-08T05:53:41.317 INFO:teuthology.orchestra.run.smithi192.stdout:(110/156): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-05-08T05:53:41.331 INFO:teuthology.orchestra.run.smithi123.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 387 kB/s | 58 kB 00:00 2024-05-08T05:53:41.389 INFO:teuthology.orchestra.run.smithi123.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 800 kB/s | 94 kB 00:00 2024-05-08T05:53:41.440 INFO:teuthology.orchestra.run.smithi123.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 591 kB/s | 64 kB 00:00 2024-05-08T05:53:41.501 INFO:teuthology.orchestra.run.smithi192.stdout:(111/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.5 MB/s | 839 kB 00:00 2024-05-08T05:53:41.523 INFO:teuthology.orchestra.run.smithi123.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 336 kB/s | 45 kB 00:00 2024-05-08T05:53:41.593 INFO:teuthology.orchestra.run.smithi192.stdout:(112/156): libarrow-9.0.0-11.el9.x86_64.rpm 7.3 MB/s | 4.4 MB 00:00 2024-05-08T05:53:41.676 INFO:teuthology.orchestra.run.smithi192.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 522 kB/s | 43 kB 00:00 2024-05-08T05:53:41.702 INFO:teuthology.orchestra.run.smithi192.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 2.7 MB/s | 548 kB 00:00 2024-05-08T05:53:41.760 INFO:teuthology.orchestra.run.smithi192.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 386 kB/s | 32 kB 00:00 2024-05-08T05:53:41.774 INFO:teuthology.orchestra.run.smithi123.stdout:(102/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 16 MB/s | 16 MB 00:01 2024-05-08T05:53:41.785 INFO:teuthology.orchestra.run.smithi192.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 180 kB/s | 15 kB 00:00 2024-05-08T05:53:41.844 INFO:teuthology.orchestra.run.smithi192.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-05-08T05:53:41.928 INFO:teuthology.orchestra.run.smithi192.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 2.5 MB/s | 358 kB 00:00 2024-05-08T05:53:41.953 INFO:teuthology.orchestra.run.smithi192.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-05-08T05:53:42.003 INFO:teuthology.orchestra.run.smithi192.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 470 kB/s | 35 kB 00:00 2024-05-08T05:53:42.446 INFO:teuthology.orchestra.run.smithi192.stdout:(121/156): ceph-test-19.0.0-3538.gbee299f4.el9. 4.5 MB/s | 49 MB 00:10 2024-05-08T05:53:42.451 INFO:teuthology.orchestra.run.smithi123.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 22 kB/s | 22 kB 00:01 2024-05-08T05:53:42.471 INFO:teuthology.orchestra.run.smithi192.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 397 kB/s | 206 kB 00:00 2024-05-08T05:53:42.521 INFO:teuthology.orchestra.run.smithi192.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 215 kB/s | 11 kB 00:00 2024-05-08T05:53:42.596 INFO:teuthology.orchestra.run.smithi192.stdout:(124/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.4 MB/s | 2.0 MB 00:00 2024-05-08T05:53:42.621 INFO:teuthology.orchestra.run.smithi192.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-05-08T05:53:42.655 INFO:teuthology.orchestra.run.smithi192.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 397 kB/s | 23 kB 00:00 2024-05-08T05:53:42.680 INFO:teuthology.orchestra.run.smithi192.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 331 kB/s | 19 kB 00:00 2024-05-08T05:53:42.714 INFO:teuthology.orchestra.run.smithi192.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 335 kB/s | 20 kB 00:00 2024-05-08T05:53:42.748 INFO:teuthology.orchestra.run.smithi192.stdout:(129/156): python3-grpcio-tools-1.46.7-10.el9.x 479 kB/s | 144 kB 00:00 2024-05-08T05:53:42.773 INFO:teuthology.orchestra.run.smithi192.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 788 kB/s | 46 kB 00:00 2024-05-08T05:53:42.807 INFO:teuthology.orchestra.run.smithi192.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-05-08T05:53:42.832 INFO:teuthology.orchestra.run.smithi192.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 989 kB/s | 58 kB 00:00 2024-05-08T05:53:42.882 INFO:teuthology.orchestra.run.smithi192.stdout:(133/156): python3-kubernetes-26.1.0-2.el9.noar 5.1 MB/s | 1.0 MB 00:00 2024-05-08T05:53:42.907 INFO:teuthology.orchestra.run.smithi192.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-05-08T05:53:42.932 INFO:teuthology.orchestra.run.smithi123.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 613 kB/s | 863 kB 00:01 2024-05-08T05:53:42.933 INFO:teuthology.orchestra.run.smithi192.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-05-08T05:53:42.958 INFO:teuthology.orchestra.run.smithi192.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-05-08T05:53:42.966 INFO:teuthology.orchestra.run.smithi123.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 460 kB/s | 548 kB 00:01 2024-05-08T05:53:42.983 INFO:teuthology.orchestra.run.smithi192.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 373 kB/s | 28 kB 00:00 2024-05-08T05:53:43.009 INFO:teuthology.orchestra.run.smithi192.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-05-08T05:53:43.034 INFO:teuthology.orchestra.run.smithi192.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-05-08T05:53:43.076 INFO:teuthology.orchestra.run.smithi192.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 642 kB/s | 59 kB 00:00 2024-05-08T05:53:43.101 INFO:teuthology.orchestra.run.smithi192.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 388 kB/s | 36 kB 00:00 2024-05-08T05:53:43.127 INFO:teuthology.orchestra.run.smithi192.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 553 kB/s | 51 kB 00:00 2024-05-08T05:53:43.152 INFO:teuthology.orchestra.run.smithi192.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-05-08T05:53:43.177 INFO:teuthology.orchestra.run.smithi192.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-05-08T05:53:43.211 INFO:teuthology.orchestra.run.smithi192.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 341 kB/s | 20 kB 00:00 2024-05-08T05:53:43.236 INFO:teuthology.orchestra.run.smithi192.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-05-08T05:53:43.261 INFO:teuthology.orchestra.run.smithi192.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-05-08T05:53:43.383 INFO:teuthology.orchestra.run.smithi123.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 332 kB/s | 309 kB 00:00 2024-05-08T05:53:43.466 INFO:teuthology.orchestra.run.smithi123.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 36 kB/s | 19 kB 00:00 2024-05-08T05:53:43.470 INFO:teuthology.orchestra.run.smithi192.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 6.1 MB/s | 1.6 MB 00:00 2024-05-08T05:53:43.508 INFO:teuthology.orchestra.run.smithi123.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 208 kB/s | 26 kB 00:00 2024-05-08T05:53:43.628 INFO:teuthology.orchestra.run.smithi192.stdout:(149/156): librados2-19.0.0-3538.gbee299f4.el9. 8.7 MB/s | 3.4 MB 00:00 2024-05-08T05:53:43.675 INFO:teuthology.orchestra.run.smithi123.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 233 kB/s | 49 kB 00:00 2024-05-08T05:53:43.701 INFO:teuthology.orchestra.run.smithi123.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 352 kB/s | 67 kB 00:00 2024-05-08T05:53:43.879 INFO:teuthology.orchestra.run.smithi192.stdout:(150/156): librbd1-19.0.0-3538.gbee299f4.el9.x8 5.1 MB/s | 3.1 MB 00:00 2024-05-08T05:53:43.951 INFO:teuthology.orchestra.run.smithi123.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 550 kB/s | 151 kB 00:00 2024-05-08T05:53:44.229 INFO:teuthology.orchestra.run.smithi192.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 119 kB/s | 42 kB 00:00 2024-05-08T05:53:44.496 INFO:teuthology.orchestra.run.smithi192.stdout:(152/156): NetworkManager-tui-1.47.5-1.el9.x86_ 932 kB/s | 248 kB 00:00 2024-05-08T05:53:44.588 INFO:teuthology.orchestra.run.smithi192.stdout:(153/156): NetworkManager-1.47.5-1.el9.x86_64.r 2.1 MB/s | 2.3 MB 00:01 2024-05-08T05:53:44.709 INFO:teuthology.orchestra.run.smithi123.stdout:(112/156): python3-asyncssh-2.13.2-5.el9.noarch 724 kB/s | 548 kB 00:00 2024-05-08T05:53:44.713 INFO:teuthology.orchestra.run.smithi192.stdout:(154/156): NetworkManager-libnm-1.47.5-1.el9.x8 1.7 MB/s | 1.8 MB 00:01 2024-05-08T05:53:44.813 INFO:teuthology.orchestra.run.smithi192.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-05-08T05:53:44.851 INFO:teuthology.orchestra.run.smithi123.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 729 kB/s | 839 kB 00:01 2024-05-08T05:53:44.890 INFO:teuthology.orchestra.run.smithi163.stdout: Preparing : 1/1 2024-05-08T05:53:44.893 INFO:teuthology.orchestra.run.smithi123.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 238 kB/s | 43 kB 00:00 2024-05-08T05:53:44.990 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-08T05:53:45.019 INFO:teuthology.orchestra.run.smithi123.stdout:(115/156): python3-certifi-2021.10.8-2.el9.noar 121 kB/s | 15 kB 00:00 2024-05-08T05:53:45.028 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-08T05:53:45.061 INFO:teuthology.orchestra.run.smithi123.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 194 kB/s | 32 kB 00:00 2024-05-08T05:53:45.082 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-08T05:53:45.115 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-08T05:53:45.141 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-08T05:53:45.192 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-08T05:53:45.278 INFO:teuthology.orchestra.run.smithi123.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 666 kB/s | 172 kB 00:00 2024-05-08T05:53:45.330 INFO:teuthology.orchestra.run.smithi192.stdout:(156/156): openssl-3.2.1-1.el9.x86_64.rpm 1.6 MB/s | 1.3 MB 00:00 2024-05-08T05:53:45.337 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-05-08T05:53:45.337 INFO:teuthology.orchestra.run.smithi192.stdout:Total 10 MB/s | 226 MB 00:22 2024-05-08T05:53:45.522 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-08T05:53:45.578 INFO:teuthology.orchestra.run.smithi123.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 694 kB/s | 358 kB 00:00 2024-05-08T05:53:45.634 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 8/164 2024-05-08T05:53:45.678 INFO:teuthology.orchestra.run.smithi123.stdout:(119/156): python3-colorama-0.4.4-12.el9.noarch 355 kB/s | 35 kB 00:00 2024-05-08T05:53:45.737 INFO:teuthology.orchestra.run.smithi123.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 661 kB/s | 303 kB 00:00 2024-05-08T05:53:45.755 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 8/164 2024-05-08T05:53:45.776 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 9/164 2024-05-08T05:53:45.867 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 9/164 2024-05-08T05:53:45.898 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rados-2:19.0.0-3538.gbee299f4.el9.x86_64 10/164 2024-05-08T05:53:46.045 INFO:teuthology.orchestra.run.smithi123.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 561 kB/s | 206 kB 00:00 2024-05-08T05:53:46.047 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-08T05:53:46.111 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-08T05:53:46.139 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 13/164 2024-05-08T05:53:46.241 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 13/164 2024-05-08T05:53:46.265 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86 14/164 2024-05-08T05:53:46.311 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86 14/164 2024-05-08T05:53:46.333 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-08T05:53:46.345 INFO:teuthology.orchestra.run.smithi123.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 483 kB/s | 144 kB 00:00 2024-05-08T05:53:46.357 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-08T05:53:46.454 INFO:teuthology.orchestra.run.smithi123.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 99 kB/s | 11 kB 00:00 2024-05-08T05:53:46.525 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-05-08T05:53:46.563 INFO:teuthology.orchestra.run.smithi123.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 164 kB/s | 18 kB 00:00 2024-05-08T05:53:46.576 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-08T05:53:46.672 INFO:teuthology.orchestra.run.smithi123.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 215 kB/s | 23 kB 00:00 2024-05-08T05:53:46.676 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-05-08T05:53:46.676 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-05-08T05:53:46.721 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-08T05:53:46.780 INFO:teuthology.orchestra.run.smithi123.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 181 kB/s | 19 kB 00:00 2024-05-08T05:53:46.875 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-08T05:53:46.889 INFO:teuthology.orchestra.run.smithi123.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 183 kB/s | 20 kB 00:00 2024-05-08T05:53:46.941 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-08T05:53:47.051 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:53:47.125 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:53:47.319 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:53:47.431 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-08T05:53:47.462 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-08T05:53:47.497 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-08T05:53:47.561 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-08T05:53:47.596 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-08T05:53:47.640 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-08T05:53:47.673 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-08T05:53:47.701 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-08T05:53:47.769 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-08T05:53:47.822 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-08T05:53:47.853 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-08T05:53:47.897 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-08T05:53:47.973 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-08T05:53:48.008 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-ceph-common-2:19.0.0-3538.gbee299f4.el9. 32/164 2024-05-08T05:53:48.051 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-ceph-argparse-2:19.0.0-3538.gbee299f4.el 33/164 2024-05-08T05:53:48.088 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cephfs-2:19.0.0-3538.gbee299f4.el9.x86_6 34/164 2024-05-08T05:53:48.158 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-08T05:53:48.203 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-08T05:53:48.247 INFO:teuthology.orchestra.run.smithi123.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 831 kB/s | 2.0 MB 00:02 2024-05-08T05:53:48.279 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-08T05:53:48.289 INFO:teuthology.orchestra.run.smithi123.stdout:(129/156): python3-kubernetes-26.1.0-2.el9.noar 747 kB/s | 1.0 MB 00:01 2024-05-08T05:53:48.410 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-08T05:53:48.464 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-08T05:53:48.499 INFO:teuthology.orchestra.run.smithi123.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 185 kB/s | 46 kB 00:00 2024-05-08T05:53:48.502 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-05-08T05:53:48.503 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-05-08T05:53:48.532 INFO:teuthology.orchestra.run.smithi123.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 326 kB/s | 79 kB 00:00 2024-05-08T05:53:48.649 INFO:teuthology.orchestra.run.smithi123.stdout:(132/156): libarrow-9.0.0-11.el9.x86_64.rpm 795 kB/s | 4.4 MB 00:05 2024-05-08T05:53:48.724 INFO:teuthology.orchestra.run.smithi123.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 258 kB/s | 58 kB 00:00 2024-05-08T05:53:48.758 INFO:teuthology.orchestra.run.smithi123.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 152 kB/s | 16 kB 00:00 2024-05-08T05:53:48.866 INFO:teuthology.orchestra.run.smithi123.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 262 kB/s | 28 kB 00:00 2024-05-08T05:53:48.917 INFO:teuthology.orchestra.run.smithi123.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 469 kB/s | 90 kB 00:00 2024-05-08T05:53:48.975 INFO:teuthology.orchestra.run.smithi123.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 285 kB/s | 31 kB 00:00 2024-05-08T05:53:48.983 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-08T05:53:49.000 INFO:teuthology.orchestra.run.smithi123.stdout:(138/156): python3-pecan-1.4.2-3.el9.noarch.rpm 582 kB/s | 272 kB 00:00 2024-05-08T05:53:49.060 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-08T05:53:49.089 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-08T05:53:49.117 INFO:teuthology.orchestra.run.smithi123.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 308 kB/s | 36 kB 00:00 2024-05-08T05:53:49.131 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-08T05:53:49.159 INFO:teuthology.orchestra.run.smithi123.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 323 kB/s | 59 kB 00:00 2024-05-08T05:53:49.169 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librados-devel-2:19.0.0-3538.gbee299f4.el9.x86_6 44/164 2024-05-08T05:53:49.211 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-08T05:53:49.226 INFO:teuthology.orchestra.run.smithi123.stdout:(141/156): python3-typing-extensions-4.4.0-2.el 473 kB/s | 51 kB 00:00 2024-05-08T05:53:49.252 INFO:teuthology.orchestra.run.smithi123.stdout:(142/156): python3-routes-2.5.1-5.el9.noarch.rp 564 kB/s | 188 kB 00:00 2024-05-08T05:53:49.365 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-08T05:53:49.418 INFO:teuthology.orchestra.run.smithi123.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 468 kB/s | 90 kB 00:00 2024-05-08T05:53:49.422 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-08T05:53:49.514 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-08T05:53:49.527 INFO:teuthology.orchestra.run.smithi123.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 185 kB/s | 20 kB 00:00 2024-05-08T05:53:49.561 INFO:teuthology.orchestra.run.smithi123.stdout:(145/156): python3-webob-1.8.7-6.el9.noarch.rpm 573 kB/s | 230 kB 00:00 2024-05-08T05:53:49.578 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-08T05:53:49.651 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-08T05:53:49.682 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-08T05:53:49.719 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-08T05:53:49.765 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-08T05:53:49.801 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-08T05:53:49.803 INFO:teuthology.orchestra.run.smithi123.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 698 kB/s | 192 kB 00:00 2024-05-08T05:53:49.845 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-08T05:53:49.887 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-08T05:53:49.911 INFO:teuthology.orchestra.run.smithi123.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 648 kB/s | 427 kB 00:00 2024-05-08T05:53:50.023 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-08T05:53:50.195 INFO:teuthology.orchestra.run.smithi123.stdout:(148/156): librados2-19.0.0-3538.gbee299f4.el9. 8.7 MB/s | 3.4 MB 00:00 2024-05-08T05:53:50.269 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-08T05:53:50.323 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-08T05:53:50.366 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-08T05:53:50.420 INFO:teuthology.orchestra.run.smithi123.stdout:(149/156): librbd1-19.0.0-3538.gbee299f4.el9.x8 6.2 MB/s | 3.1 MB 00:00 2024-05-08T05:53:50.463 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-08T05:53:50.487 INFO:teuthology.orchestra.run.smithi123.stdout:(150/156): NetworkManager-1.47.5-1.el9.x86_64.r 7.9 MB/s | 2.3 MB 00:00 2024-05-08T05:53:50.497 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-08T05:53:50.537 INFO:teuthology.orchestra.run.smithi123.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 847 kB/s | 42 kB 00:00 2024-05-08T05:53:50.546 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-08T05:53:50.571 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-08T05:53:50.596 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-08T05:53:50.596 INFO:teuthology.orchestra.run.smithi123.stdout:(152/156): NetworkManager-tui-1.47.5-1.el9.x86_ 4.2 MB/s | 248 kB 00:00 2024-05-08T05:53:50.671 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-08T05:53:50.688 INFO:teuthology.orchestra.run.smithi123.stdout:(153/156): openssl-3.2.1-1.el9.x86_64.rpm 15 MB/s | 1.3 MB 00:00 2024-05-08T05:53:50.746 INFO:teuthology.orchestra.run.smithi123.stdout:(154/156): NetworkManager-libnm-1.47.5-1.el9.x8 5.6 MB/s | 1.8 MB 00:00 2024-05-08T05:53:50.798 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-08T05:53:50.828 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-08T05:53:50.830 INFO:teuthology.orchestra.run.smithi123.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 17 MB/s | 2.4 MB 00:00 2024-05-08T05:53:50.856 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-08T05:53:50.890 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-08T05:53:50.938 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-08T05:53:50.969 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-08T05:53:50.999 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-08T05:53:51.039 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-08T05:53:51.211 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-08T05:53:51.314 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-08T05:53:51.435 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-08T05:53:51.521 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-08T05:53:51.565 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-08T05:53:51.596 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-08T05:53:51.622 INFO:teuthology.orchestra.run.smithi123.stdout:(156/156): thrift-0.15.0-2.el9.x86_64.rpm 786 kB/s | 1.6 MB 00:02 2024-05-08T05:53:51.635 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-05-08T05:53:51.635 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-08T05:53:51.636 INFO:teuthology.orchestra.run.smithi123.stdout:Total 8.8 MB/s | 226 MB 00:25 2024-05-08T05:53:51.734 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-08T05:53:51.790 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-08T05:53:52.512 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-08T05:53:52.561 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-08T05:53:52.595 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-08T05:53:52.693 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-08T05:53:52.752 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-08T05:53:52.831 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-08T05:53:52.864 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-08T05:53:53.041 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-08T05:53:53.069 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-08T05:53:53.166 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-08T05:53:53.817 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-08T05:53:53.935 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-08T05:53:54.620 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-05-08T05:53:54.710 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-08T05:53:54.725 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-05-08T05:53:54.747 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-08T05:53:54.801 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-08T05:53:54.835 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-08T05:53:54.861 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-08T05:53:54.903 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-08T05:53:54.967 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-05-08T05:53:54.968 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-05-08T05:53:55.158 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-08T05:53:55.230 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-08T05:53:55.249 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-08T05:53:55.277 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-08T05:53:55.314 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-08T05:53:55.347 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 8/164 2024-05-08T05:53:55.478 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 8/164 2024-05-08T05:53:55.513 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 9/164 2024-05-08T05:53:55.606 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 9/164 2024-05-08T05:53:55.621 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-08T05:53:55.634 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rados-2:19.0.0-3538.gbee299f4.el9.x86_64 10/164 2024-05-08T05:53:55.677 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 101/164 2024-05-08T05:53:55.772 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 101/164 2024-05-08T05:53:55.788 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-08T05:53:55.803 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rbd-2:19.0.0-3538.gbee299f4.el9.x86_64 102/164 2024-05-08T05:53:55.840 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-08T05:53:55.840 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-08T05:53:55.867 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 13/164 2024-05-08T05:53:55.978 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 13/164 2024-05-08T05:53:56.002 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86 14/164 2024-05-08T05:53:56.064 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86 14/164 2024-05-08T05:53:56.086 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-08T05:53:56.105 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-08T05:53:56.286 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-08T05:53:56.316 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-08T05:53:56.360 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 105/164 2024-05-08T05:53:56.413 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 105/164 2024-05-08T05:53:56.414 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-08T05:53:56.436 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rgw-2:19.0.0-3538.gbee299f4.el9.x86_64 106/164 2024-05-08T05:53:56.559 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-08T05:53:56.611 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-08T05:53:56.724 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:53:56.795 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:53:57.171 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:53:57.400 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-08T05:53:57.441 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-08T05:53:57.476 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-08T05:53:57.539 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-08T05:53:57.583 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-08T05:53:57.623 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-05-08T05:53:57.624 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-05-08T05:53:57.627 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-08T05:53:57.660 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-08T05:53:57.688 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-08T05:53:57.756 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-08T05:53:57.818 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-08T05:53:57.849 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-08T05:53:57.884 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-08T05:53:57.960 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-08T05:53:57.995 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-common-2:19.0.0-3538.gbee299f4.el9. 32/164 2024-05-08T05:53:58.039 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-argparse-2:19.0.0-3538.gbee299f4.el 33/164 2024-05-08T05:53:58.067 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cephfs-2:19.0.0-3538.gbee299f4.el9.x86_6 34/164 2024-05-08T05:53:58.136 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-08T05:53:58.182 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-08T05:53:58.256 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-08T05:53:58.397 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-08T05:53:58.450 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-08T05:53:58.650 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 107/164 2024-05-08T05:53:58.699 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 107/164 2024-05-08T05:53:58.968 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-08T05:53:59.055 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-08T05:53:59.093 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-08T05:53:59.135 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-08T05:53:59.174 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librados-devel-2:19.0.0-3538.gbee299f4.el9.x86_6 44/164 2024-05-08T05:53:59.215 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-08T05:53:59.238 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 107/164 2024-05-08T05:53:59.277 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 108/164 2024-05-08T05:53:59.366 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-08T05:53:59.368 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 108/164 2024-05-08T05:53:59.368 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-08T05:53:59.368 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-08T05:53:59.368 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T05:53:59.403 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 109/164 2024-05-08T05:53:59.418 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-08T05:53:59.509 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-08T05:53:59.565 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-08T05:53:59.639 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-08T05:53:59.669 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-08T05:53:59.715 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-08T05:53:59.761 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-08T05:53:59.798 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-08T05:53:59.840 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-08T05:53:59.883 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-08T05:54:00.017 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-08T05:54:00.054 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-08T05:54:00.101 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-08T05:54:00.143 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-08T05:54:00.237 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-08T05:54:00.268 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-08T05:54:00.308 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-08T05:54:00.334 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-08T05:54:00.359 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-08T05:54:00.432 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-08T05:54:00.576 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-08T05:54:00.608 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-08T05:54:00.636 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-08T05:54:00.679 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-08T05:54:00.718 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-08T05:54:00.749 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-08T05:54:00.779 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-08T05:54:00.814 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-08T05:54:00.847 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-08T05:54:00.889 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-08T05:54:00.923 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-08T05:54:01.009 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-08T05:54:01.053 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-08T05:54:01.085 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-08T05:54:01.132 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-08T05:54:01.229 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-08T05:54:01.288 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-08T05:54:02.005 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-08T05:54:02.061 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-08T05:54:02.095 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-08T05:54:02.192 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-08T05:54:02.244 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-08T05:54:02.331 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-08T05:54:02.389 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-08T05:54:02.567 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-08T05:54:02.595 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-08T05:54:02.673 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-08T05:54:03.316 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-08T05:54:03.435 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-08T05:54:04.322 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-05-08T05:54:04.455 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-08T05:54:04.542 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-08T05:54:04.647 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-08T05:54:04.667 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-08T05:54:04.726 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-08T05:54:04.727 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-08T05:54:04.753 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-08T05:54:04.764 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-08T05:54:04.787 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-08T05:54:04.861 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-08T05:54:05.098 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-08T05:54:05.142 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 101/164 2024-05-08T05:54:05.216 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 101/164 2024-05-08T05:54:05.247 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rbd-2:19.0.0-3538.gbee299f4.el9.x86_64 102/164 2024-05-08T05:54:05.283 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-08T05:54:05.286 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-08T05:54:05.387 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 8/164 2024-05-08T05:54:05.566 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 8/164 2024-05-08T05:54:05.605 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 9/164 2024-05-08T05:54:05.732 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 9/164 2024-05-08T05:54:05.761 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-08T05:54:05.779 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rados-2:19.0.0-3538.gbee299f4.el9.x86_64 10/164 2024-05-08T05:54:05.787 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 105/164 2024-05-08T05:54:05.857 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 105/164 2024-05-08T05:54:05.880 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rgw-2:19.0.0-3538.gbee299f4.el9.x86_64 106/164 2024-05-08T05:54:06.105 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-08T05:54:06.176 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-08T05:54:06.200 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 13/164 2024-05-08T05:54:06.331 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 13/164 2024-05-08T05:54:06.368 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86 14/164 2024-05-08T05:54:06.462 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86 14/164 2024-05-08T05:54:06.494 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-08T05:54:06.568 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-08T05:54:06.885 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-08T05:54:07.244 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-08T05:54:07.456 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-08T05:54:07.531 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-08T05:54:07.707 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:54:07.845 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:54:08.177 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 107/164 2024-05-08T05:54:08.180 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:54:08.218 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 107/164 2024-05-08T05:54:08.395 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-08T05:54:08.434 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-08T05:54:08.503 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-08T05:54:08.611 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-08T05:54:08.682 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-08T05:54:08.755 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-08T05:54:08.757 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 107/164 2024-05-08T05:54:08.797 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 108/164 2024-05-08T05:54:08.801 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-08T05:54:08.841 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-08T05:54:08.881 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 108/164 2024-05-08T05:54:08.881 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-08T05:54:08.881 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-08T05:54:08.881 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T05:54:08.915 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 109/164 2024-05-08T05:54:08.954 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-08T05:54:09.029 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-08T05:54:09.068 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-08T05:54:09.123 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-08T05:54:09.232 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-08T05:54:09.305 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-common-2:19.0.0-3538.gbee299f4.el9. 32/164 2024-05-08T05:54:09.357 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-argparse-2:19.0.0-3538.gbee299f4.el 33/164 2024-05-08T05:54:09.395 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cephfs-2:19.0.0-3538.gbee299f4.el9.x86_6 34/164 2024-05-08T05:54:09.514 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-08T05:54:09.600 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-08T05:54:09.707 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-08T05:54:09.882 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-08T05:54:09.951 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-08T05:54:10.628 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-08T05:54:10.731 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-08T05:54:10.792 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-08T05:54:10.844 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-08T05:54:10.889 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librados-devel-2:19.0.0-3538.gbee299f4.el9.x86_6 44/164 2024-05-08T05:54:10.949 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-08T05:54:11.006 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 109/164 2024-05-08T05:54:11.006 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /sys 2024-05-08T05:54:11.006 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /proc 2024-05-08T05:54:11.006 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /mnt 2024-05-08T05:54:11.006 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /var/tmp 2024-05-08T05:54:11.006 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /home 2024-05-08T05:54:11.006 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /root 2024-05-08T05:54:11.006 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /tmp 2024-05-08T05:54:11.006 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T05:54:11.152 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-08T05:54:11.182 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 110/164 2024-05-08T05:54:11.198 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 110/164 2024-05-08T05:54:11.198 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:54:11.198 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-08T05:54:11.198 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T05:54:11.219 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-08T05:54:11.334 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-08T05:54:11.421 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-08T05:54:11.505 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-08T05:54:11.553 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-08T05:54:11.591 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-08T05:54:11.685 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-08T05:54:11.731 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-08T05:54:11.788 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-08T05:54:11.844 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-08T05:54:12.039 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-08T05:54:12.100 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-08T05:54:12.137 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-08T05:54:12.193 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-08T05:54:12.353 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-08T05:54:12.394 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-08T05:54:12.450 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-08T05:54:12.493 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-08T05:54:12.519 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-08T05:54:12.689 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 111/164 2024-05-08T05:54:12.715 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 111/164 2024-05-08T05:54:12.715 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:54:12.715 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-08T05:54:12.716 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T05:54:12.716 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T05:54:12.716 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T05:54:12.722 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-08T05:54:12.907 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-08T05:54:12.944 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 112/164 2024-05-08T05:54:12.957 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-08T05:54:12.967 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 112/164 2024-05-08T05:54:12.967 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:54:12.967 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-08T05:54:12.967 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T05:54:12.967 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T05:54:12.967 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T05:54:12.996 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-08T05:54:13.043 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-08T05:54:13.101 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-08T05:54:13.179 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-08T05:54:13.229 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-08T05:54:13.274 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-08T05:54:13.324 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-08T05:54:13.399 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-08T05:54:13.411 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 113/164 2024-05-08T05:54:13.434 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 113/164 2024-05-08T05:54:13.434 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:54:13.434 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-08T05:54:13.434 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T05:54:13.434 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T05:54:13.434 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T05:54:13.473 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-08T05:54:13.550 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-08T05:54:13.607 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-08T05:54:13.620 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-08T05:54:13.662 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-08T05:54:13.684 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-08T05:54:13.757 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-08T05:54:13.837 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-08T05:54:13.850 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-08T05:54:13.903 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-08T05:54:13.982 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-08T05:54:13.991 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-08T05:54:14.070 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-08T05:54:14.088 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-08T05:54:14.209 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-08T05:54:14.253 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-08T05:54:14.285 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-08T05:54:14.313 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-08T05:54:14.690 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 125/164 2024-05-08T05:54:14.711 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 125/164 2024-05-08T05:54:14.743 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3538.gbee299f4.e 126/164 2024-05-08T05:54:14.805 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3538.gbee299f4. 127/164 2024-05-08T05:54:14.834 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noa 128/164 2024-05-08T05:54:15.050 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-08T05:54:15.107 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-08T05:54:15.140 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-08T05:54:15.261 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-08T05:54:15.326 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-08T05:54:15.425 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-08T05:54:15.467 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-08T05:54:15.699 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-08T05:54:15.739 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-08T05:54:15.846 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-08T05:54:15.879 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noa 128/164 2024-05-08T05:54:15.919 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.n 129/164 2024-05-08T05:54:16.687 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-08T05:54:16.845 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-08T05:54:16.886 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.n 129/164 2024-05-08T05:54:16.911 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee 130/164 2024-05-08T05:54:17.023 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee 130/164 2024-05-08T05:54:17.200 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3538.gbee299f4.el 131/164 2024-05-08T05:54:17.230 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 132/164 2024-05-08T05:54:17.254 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 132/164 2024-05-08T05:54:17.342 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:54:17.343 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-08T05:54:17.343 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T05:54:17.343 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T05:54:17.343 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T05:54:17.344 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 133/164 2024-05-08T05:54:17.344 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 133/164 2024-05-08T05:54:17.414 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-2:19.0.0-3538.gbee299f4.el9.x86_64 134/164 2024-05-08T05:54:17.462 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 135/164 2024-05-08T05:54:18.381 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-08T05:54:18.416 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 137/164 2024-05-08T05:54:18.439 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 137/164 2024-05-08T05:54:18.439 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:54:18.440 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-08T05:54:18.440 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T05:54:18.440 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T05:54:18.440 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T05:54:18.466 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-08T05:54:18.483 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3538.gbee29 138/164 2024-05-08T05:54:18.505 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3538.gbee29 138/164 2024-05-08T05:54:18.505 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:54:18.506 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-08T05:54:18.506 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T05:54:18.550 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 139/164 2024-05-08T05:54:18.561 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-08T05:54:18.573 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 139/164 2024-05-08T05:54:18.573 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:54:18.573 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-08T05:54:18.573 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-08T05:54:18.573 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-08T05:54:18.574 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T05:54:18.589 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-08T05:54:18.624 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-08T05:54:18.876 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 140/164 2024-05-08T05:54:18.899 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 140/164 2024-05-08T05:54:18.899 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:54:18.899 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-08T05:54:18.899 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T05:54:18.899 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T05:54:18.900 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T05:54:18.992 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-08T05:54:19.041 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 101/164 2024-05-08T05:54:19.180 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 101/164 2024-05-08T05:54:19.214 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rbd-2:19.0.0-3538.gbee299f4.el9.x86_64 102/164 2024-05-08T05:54:19.271 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-08T05:54:19.811 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-08T05:54:19.844 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 105/164 2024-05-08T05:54:19.935 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 105/164 2024-05-08T05:54:19.962 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rgw-2:19.0.0-3538.gbee299f4.el9.x86_64 106/164 2024-05-08T05:54:20.332 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 109/164 2024-05-08T05:54:20.332 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /sys 2024-05-08T05:54:20.332 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /proc 2024-05-08T05:54:20.332 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /mnt 2024-05-08T05:54:20.332 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /var/tmp 2024-05-08T05:54:20.332 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /home 2024-05-08T05:54:20.332 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /root 2024-05-08T05:54:20.332 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /tmp 2024-05-08T05:54:20.332 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T05:54:20.533 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 110/164 2024-05-08T05:54:20.549 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 110/164 2024-05-08T05:54:20.549 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:54:20.549 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-08T05:54:20.550 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T05:54:22.045 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 111/164 2024-05-08T05:54:22.071 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 111/164 2024-05-08T05:54:22.071 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:54:22.071 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-08T05:54:22.071 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T05:54:22.071 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T05:54:22.071 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T05:54:22.295 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 112/164 2024-05-08T05:54:22.318 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 112/164 2024-05-08T05:54:22.318 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:54:22.318 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-08T05:54:22.318 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T05:54:22.318 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T05:54:22.318 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T05:54:22.702 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 107/164 2024-05-08T05:54:22.742 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 107/164 2024-05-08T05:54:22.771 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 113/164 2024-05-08T05:54:22.793 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 113/164 2024-05-08T05:54:22.793 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:54:22.793 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-08T05:54:22.794 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T05:54:22.794 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T05:54:22.794 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T05:54:22.893 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-08T05:54:22.941 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-08T05:54:22.996 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-08T05:54:23.182 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-08T05:54:23.231 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-08T05:54:23.308 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-08T05:54:23.397 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-08T05:54:23.398 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 107/164 2024-05-08T05:54:23.445 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 108/164 2024-05-08T05:54:23.521 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-test-2:19.0.0-3538.gbee299f4.el9.x86_64 141/164 2024-05-08T05:54:23.536 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-08T05:54:23.562 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : rbd-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 142/164 2024-05-08T05:54:23.589 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-08T05:54:23.603 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : rbd-nbd-2:19.0.0-3538.gbee299f4.el9.x86_64 143/164 2024-05-08T05:54:23.626 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 108/164 2024-05-08T05:54:23.627 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-08T05:54:23.627 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-08T05:54:23.627 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T05:54:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-08T05:54:23.635 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-08T05:54:23.657 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-08T05:54:23.663 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-08T05:54:23.671 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 109/164 2024-05-08T05:54:23.846 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephfs-shell-2:19.0.0-3538.gbee299f4.el9.x86_64 146/164 2024-05-08T05:54:23.892 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-08T05:54:23.922 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcephfs-devel-2:19.0.0-3538.gbee299f4.el9.x86_ 148/164 2024-05-08T05:54:23.951 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-08T05:54:23.990 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-08T05:54:24.068 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 125/164 2024-05-08T05:54:24.098 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 125/164 2024-05-08T05:54:24.130 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3538.gbee299f4.e 126/164 2024-05-08T05:54:24.192 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3538.gbee299f4. 127/164 2024-05-08T05:54:24.221 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noa 128/164 2024-05-08T05:54:24.608 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:54:24.645 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:54:24.666 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:54:25.255 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noa 128/164 2024-05-08T05:54:25.290 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.n 129/164 2024-05-08T05:54:25.513 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephfs-top-2:19.0.0-3538.gbee299f4.el9.noarch 152/164 2024-05-08T05:54:25.583 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-08T05:54:25.624 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-08T05:54:25.707 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-08T05:54:25.754 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-08T05:54:25.787 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-08T05:54:25.787 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-08T05:54:25.871 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-08T05:54:25.896 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-08T05:54:25.896 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-08T05:54:25.980 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-08T05:54:26.005 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-08T05:54:26.011 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:54:26.011 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:54:26.190 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:54:26.190 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-08T05:54:26.249 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.n 129/164 2024-05-08T05:54:26.266 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-08T05:54:26.282 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee 130/164 2024-05-08T05:54:26.335 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:54:26.390 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee 130/164 2024-05-08T05:54:26.551 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3538.gbee299f4.el 131/164 2024-05-08T05:54:26.576 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 132/164 2024-05-08T05:54:26.599 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 132/164 2024-05-08T05:54:26.599 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:54:26.599 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-08T05:54:26.599 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T05:54:26.599 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T05:54:26.599 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T05:54:26.643 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 133/164 2024-05-08T05:54:26.654 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 133/164 2024-05-08T05:54:26.752 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-2:19.0.0-3538.gbee299f4.el9.x86_64 134/164 2024-05-08T05:54:26.808 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 135/164 2024-05-08T05:54:27.728 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-08T05:54:27.754 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 137/164 2024-05-08T05:54:27.777 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 137/164 2024-05-08T05:54:27.778 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:54:27.778 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-08T05:54:27.778 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T05:54:27.778 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T05:54:27.778 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T05:54:27.822 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3538.gbee29 138/164 2024-05-08T05:54:27.844 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3538.gbee29 138/164 2024-05-08T05:54:27.844 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:54:27.844 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-08T05:54:27.844 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T05:54:27.897 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 139/164 2024-05-08T05:54:27.919 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 139/164 2024-05-08T05:54:27.920 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:54:27.920 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-08T05:54:27.920 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-08T05:54:27.920 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-08T05:54:27.920 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T05:54:28.223 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 140/164 2024-05-08T05:54:28.246 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 140/164 2024-05-08T05:54:28.246 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:54:28.246 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-08T05:54:28.246 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T05:54:28.246 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T05:54:28.247 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T05:54:31.982 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:54:31.983 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-2:19.0.0-3538.gbee299f4.el9.x86_64 1/164 2024-05-08T05:54:31.983 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 2/164 2024-05-08T05:54:31.983 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 3/164 2024-05-08T05:54:31.983 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 4/164 2024-05-08T05:54:31.983 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3538.gbee29 5/164 2024-05-08T05:54:31.983 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 6/164 2024-05-08T05:54:31.983 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 7/164 2024-05-08T05:54:31.983 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 8/164 2024-05-08T05:54:31.983 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 9/164 2024-05-08T05:54:31.983 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 10/164 2024-05-08T05:54:31.983 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 11/164 2024-05-08T05:54:31.983 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-test-2:19.0.0-3538.gbee299f4.el9.x86_64 12/164 2024-05-08T05:54:31.984 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 13/164 2024-05-08T05:54:31.984 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephfs-shell-2:19.0.0-3538.gbee299f4.el9.x86_64 14/164 2024-05-08T05:54:31.984 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcephfs-devel-2:19.0.0-3538.gbee299f4.el9.x86_ 15/164 2024-05-08T05:54:31.984 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 16/164 2024-05-08T05:54:31.984 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 17/164 2024-05-08T05:54:31.984 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librados-devel-2:19.0.0-3538.gbee299f4.el9.x86_6 18/164 2024-05-08T05:54:31.984 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86 19/164 2024-05-08T05:54:31.984 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 20/164 2024-05-08T05:54:31.984 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3538.gbee299f4.el 21/164 2024-05-08T05:54:31.984 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-ceph-common-2:19.0.0-3538.gbee299f4.el9. 22/164 2024-05-08T05:54:31.984 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cephfs-2:19.0.0-3538.gbee299f4.el9.x86_6 23/164 2024-05-08T05:54:31.984 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rados-2:19.0.0-3538.gbee299f4.el9.x86_64 24/164 2024-05-08T05:54:31.984 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rbd-2:19.0.0-3538.gbee299f4.el9.x86_64 25/164 2024-05-08T05:54:31.984 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rgw-2:19.0.0-3538.gbee299f4.el9.x86_64 26/164 2024-05-08T05:54:31.985 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : rbd-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 27/164 2024-05-08T05:54:31.985 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 28/164 2024-05-08T05:54:31.985 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : rbd-nbd-2:19.0.0-3538.gbee299f4.el9.x86_64 29/164 2024-05-08T05:54:31.985 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3538.gbee299f4. 30/164 2024-05-08T05:54:31.997 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noa 31/164 2024-05-08T05:54:31.997 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.n 32/164 2024-05-08T05:54:31.998 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee 33/164 2024-05-08T05:54:31.998 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3538.gbee299f4.el 34/164 2024-05-08T05:54:31.998 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 35/164 2024-05-08T05:54:31.998 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3538.gbee299f4.e 36/164 2024-05-08T05:54:31.998 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 37/164 2024-05-08T05:54:31.998 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 38/164 2024-05-08T05:54:31.998 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephfs-top-2:19.0.0-3538.gbee299f4.el9.noarch 39/164 2024-05-08T05:54:31.998 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-08T05:54:31.998 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-08T05:54:31.998 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-08T05:54:31.998 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-08T05:54:31.998 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-08T05:54:31.999 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-08T05:54:31.999 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-08T05:54:31.999 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-08T05:54:31.999 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-08T05:54:31.999 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-08T05:54:31.999 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-08T05:54:31.999 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-08T05:54:31.999 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-08T05:54:31.999 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-08T05:54:31.999 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-08T05:54:31.999 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-08T05:54:31.999 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-08T05:54:31.999 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-08T05:54:32.000 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-08T05:54:32.000 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-08T05:54:32.000 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-08T05:54:32.000 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-08T05:54:32.000 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-08T05:54:32.000 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-08T05:54:32.000 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-08T05:54:32.000 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-08T05:54:32.000 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-08T05:54:32.000 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-08T05:54:32.000 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-08T05:54:32.000 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-08T05:54:32.000 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-08T05:54:32.000 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-08T05:54:32.001 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-08T05:54:32.001 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-08T05:54:32.001 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-08T05:54:32.001 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-08T05:54:32.001 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-08T05:54:32.001 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-08T05:54:32.001 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-08T05:54:32.001 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-08T05:54:32.001 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-08T05:54:32.001 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-08T05:54:32.001 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-08T05:54:32.001 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-08T05:54:32.001 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-08T05:54:32.001 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-08T05:54:32.002 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-08T05:54:32.002 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-08T05:54:32.002 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-08T05:54:32.002 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-08T05:54:32.002 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-08T05:54:32.002 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-08T05:54:32.002 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-08T05:54:32.002 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-08T05:54:32.002 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-08T05:54:32.002 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-08T05:54:32.002 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-08T05:54:32.002 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-08T05:54:32.003 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-08T05:54:32.003 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-08T05:54:32.003 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-08T05:54:32.003 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-08T05:54:32.003 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-08T05:54:32.003 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-08T05:54:32.003 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-08T05:54:32.003 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-08T05:54:32.003 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-08T05:54:32.003 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-08T05:54:32.003 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-08T05:54:32.003 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-08T05:54:32.003 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-08T05:54:32.003 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-08T05:54:32.004 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-08T05:54:32.004 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-08T05:54:32.004 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-08T05:54:32.004 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-08T05:54:32.004 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-08T05:54:32.004 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-08T05:54:32.004 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-08T05:54:32.004 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-08T05:54:32.004 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-08T05:54:32.004 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-08T05:54:32.004 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-08T05:54:32.004 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-08T05:54:32.004 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-08T05:54:32.005 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-08T05:54:32.005 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-08T05:54:32.005 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-08T05:54:32.005 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-08T05:54:32.005 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-08T05:54:32.005 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-08T05:54:32.005 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-08T05:54:32.005 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-08T05:54:32.005 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-08T05:54:32.005 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-08T05:54:32.005 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-08T05:54:32.005 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-08T05:54:32.005 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-08T05:54:32.006 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-08T05:54:32.006 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-08T05:54:32.006 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-08T05:54:32.006 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-08T05:54:32.006 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-08T05:54:32.006 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-08T05:54:32.006 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-08T05:54:32.006 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-08T05:54:32.006 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-08T05:54:32.006 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-08T05:54:32.006 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-08T05:54:32.006 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 149/164 2024-05-08T05:54:32.007 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-08T05:54:32.007 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 151/164 2024-05-08T05:54:32.007 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-08T05:54:32.007 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-08T05:54:32.007 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-08T05:54:32.007 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-08T05:54:32.007 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-08T05:54:32.007 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-08T05:54:32.007 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-08T05:54:32.007 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-08T05:54:32.007 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-08T05:54:32.007 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-08T05:54:32.007 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-08T05:54:32.008 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-08T05:54:32.819 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-test-2:19.0.0-3538.gbee299f4.el9.x86_64 141/164 2024-05-08T05:54:32.860 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 142/164 2024-05-08T05:54:32.901 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-nbd-2:19.0.0-3538.gbee299f4.el9.x86_64 143/164 2024-05-08T05:54:32.933 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-08T05:54:32.961 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-08T05:54:33.133 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephfs-shell-2:19.0.0-3538.gbee299f4.el9.x86_64 146/164 2024-05-08T05:54:33.174 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-08T05:54:33.204 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs-devel-2:19.0.0-3538.gbee299f4.el9.x86_ 148/164 2024-05-08T05:54:33.234 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-08T05:54:33.272 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-08T05:54:33.884 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:54:33.920 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:54:33.939 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:54:34.501 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:54:34.501 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T05:54:34.501 INFO:teuthology.orchestra.run.smithi163.stdout:Upgraded: 2024-05-08T05:54:34.501 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-08T05:54:34.501 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-08T05:54:34.501 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-08T05:54:34.501 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-08T05:54:34.501 INFO:teuthology.orchestra.run.smithi163.stdout: librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:34.501 INFO:teuthology.orchestra.run.smithi163.stdout: librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:34.501 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-08T05:54:34.502 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-08T05:54:34.502 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2024-05-08T05:54:34.502 INFO:teuthology.orchestra.run.smithi163.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-08T05:54:34.502 INFO:teuthology.orchestra.run.smithi163.stdout: automake-1.16.2-8.el9.noarch 2024-05-08T05:54:34.502 INFO:teuthology.orchestra.run.smithi163.stdout: bison-3.7.4-5.el9.x86_64 2024-05-08T05:54:34.502 INFO:teuthology.orchestra.run.smithi163.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-08T05:54:34.502 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:34.502 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:34.502 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:34.502 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:34.502 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-grafana-dashboards-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:54:34.503 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-immutable-object-cache-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:34.503 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:34.503 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:34.503 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:54:34.503 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:54:34.503 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:54:34.503 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-modules-core-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:54:34.503 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:54:34.503 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:34.503 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:34.503 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-prometheus-alerts-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:54:34.503 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:34.504 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:34.504 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-test-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:34.504 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:54:34.504 INFO:teuthology.orchestra.run.smithi163.stdout: cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:54:34.504 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:34.504 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-shell-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:34.504 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-top-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:54:34.504 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-08T05:54:34.504 INFO:teuthology.orchestra.run.smithi163.stdout: flex-2.6.4-9.el9.x86_64 2024-05-08T05:54:34.504 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-08T05:54:34.504 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-08T05:54:34.504 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-08T05:54:34.505 INFO:teuthology.orchestra.run.smithi163.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-08T05:54:34.505 INFO:teuthology.orchestra.run.smithi163.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-08T05:54:34.505 INFO:teuthology.orchestra.run.smithi163.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-08T05:54:34.505 INFO:teuthology.orchestra.run.smithi163.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-08T05:54:34.505 INFO:teuthology.orchestra.run.smithi163.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-08T05:54:34.505 INFO:teuthology.orchestra.run.smithi163.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-08T05:54:34.505 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs-devel-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:34.505 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:34.505 INFO:teuthology.orchestra.run.smithi163.stdout: libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:34.505 INFO:teuthology.orchestra.run.smithi163.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-08T05:54:34.505 INFO:teuthology.orchestra.run.smithi163.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-08T05:54:34.505 INFO:teuthology.orchestra.run.smithi163.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-08T05:54:34.506 INFO:teuthology.orchestra.run.smithi163.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-08T05:54:34.506 INFO:teuthology.orchestra.run.smithi163.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-08T05:54:34.506 INFO:teuthology.orchestra.run.smithi163.stdout: librados-devel-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:34.506 INFO:teuthology.orchestra.run.smithi163.stdout: libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:34.506 INFO:teuthology.orchestra.run.smithi163.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-08T05:54:34.506 INFO:teuthology.orchestra.run.smithi163.stdout: librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:34.506 INFO:teuthology.orchestra.run.smithi163.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-08T05:54:34.506 INFO:teuthology.orchestra.run.smithi163.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-08T05:54:34.506 INFO:teuthology.orchestra.run.smithi163.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-08T05:54:34.506 INFO:teuthology.orchestra.run.smithi163.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-08T05:54:34.506 INFO:teuthology.orchestra.run.smithi163.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-08T05:54:34.507 INFO:teuthology.orchestra.run.smithi163.stdout: lua-5.4.4-4.el9.x86_64 2024-05-08T05:54:34.507 INFO:teuthology.orchestra.run.smithi163.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-08T05:54:34.507 INFO:teuthology.orchestra.run.smithi163.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-08T05:54:34.507 INFO:teuthology.orchestra.run.smithi163.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-08T05:54:34.507 INFO:teuthology.orchestra.run.smithi163.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-08T05:54:34.507 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-08T05:54:34.507 INFO:teuthology.orchestra.run.smithi163.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-08T05:54:34.507 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-13.14-1.el9.x86_64 2024-05-08T05:54:34.507 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-08T05:54:34.507 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-08T05:54:34.507 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-08T05:54:34.507 INFO:teuthology.orchestra.run.smithi163.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-08T05:54:34.508 INFO:teuthology.orchestra.run.smithi163.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-08T05:54:34.508 INFO:teuthology.orchestra.run.smithi163.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-08T05:54:34.508 INFO:teuthology.orchestra.run.smithi163.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-08T05:54:34.508 INFO:teuthology.orchestra.run.smithi163.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-08T05:54:34.508 INFO:teuthology.orchestra.run.smithi163.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-08T05:54:34.508 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-08T05:54:34.508 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-argparse-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:34.508 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:34.508 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cephfs-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:34.508 INFO:teuthology.orchestra.run.smithi163.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-08T05:54:34.508 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-08T05:54:34.508 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-08T05:54:34.509 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-08T05:54:34.509 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-08T05:54:34.509 INFO:teuthology.orchestra.run.smithi163.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-08T05:54:34.509 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-08T05:54:34.509 INFO:teuthology.orchestra.run.smithi163.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-08T05:54:34.509 INFO:teuthology.orchestra.run.smithi163.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-08T05:54:34.509 INFO:teuthology.orchestra.run.smithi163.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-08T05:54:34.509 INFO:teuthology.orchestra.run.smithi163.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-08T05:54:34.509 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-08T05:54:34.509 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-08T05:54:34.509 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-08T05:54:34.509 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-08T05:54:34.510 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-08T05:54:34.510 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-08T05:54:34.510 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-08T05:54:34.510 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-08T05:54:34.510 INFO:teuthology.orchestra.run.smithi163.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-08T05:54:34.510 INFO:teuthology.orchestra.run.smithi163.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-08T05:54:34.510 INFO:teuthology.orchestra.run.smithi163.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-08T05:54:34.510 INFO:teuthology.orchestra.run.smithi163.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-08T05:54:34.510 INFO:teuthology.orchestra.run.smithi163.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-08T05:54:34.510 INFO:teuthology.orchestra.run.smithi163.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-08T05:54:34.510 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-08T05:54:34.511 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-08T05:54:34.511 INFO:teuthology.orchestra.run.smithi163.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-08T05:54:34.511 INFO:teuthology.orchestra.run.smithi163.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-08T05:54:34.511 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-08T05:54:34.511 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ply-3.11-14.el9.noarch 2024-05-08T05:54:34.511 INFO:teuthology.orchestra.run.smithi163.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-08T05:54:34.511 INFO:teuthology.orchestra.run.smithi163.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-08T05:54:34.511 INFO:teuthology.orchestra.run.smithi163.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-08T05:54:34.511 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-08T05:54:34.511 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-08T05:54:34.511 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-08T05:54:34.511 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-08T05:54:34.512 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-08T05:54:34.512 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-08T05:54:34.512 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rados-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:34.512 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rbd-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:34.512 INFO:teuthology.orchestra.run.smithi163.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-08T05:54:34.512 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-08T05:54:34.512 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-08T05:54:34.512 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rgw-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:34.512 INFO:teuthology.orchestra.run.smithi163.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-08T05:54:34.512 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-08T05:54:34.512 INFO:teuthology.orchestra.run.smithi163.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-08T05:54:34.512 INFO:teuthology.orchestra.run.smithi163.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-08T05:54:34.513 INFO:teuthology.orchestra.run.smithi163.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-08T05:54:34.513 INFO:teuthology.orchestra.run.smithi163.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-08T05:54:34.513 INFO:teuthology.orchestra.run.smithi163.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-08T05:54:34.513 INFO:teuthology.orchestra.run.smithi163.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-08T05:54:34.513 INFO:teuthology.orchestra.run.smithi163.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-08T05:54:34.513 INFO:teuthology.orchestra.run.smithi163.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-08T05:54:34.513 INFO:teuthology.orchestra.run.smithi163.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-08T05:54:34.513 INFO:teuthology.orchestra.run.smithi163.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-08T05:54:34.513 INFO:teuthology.orchestra.run.smithi163.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-08T05:54:34.513 INFO:teuthology.orchestra.run.smithi163.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-08T05:54:34.513 INFO:teuthology.orchestra.run.smithi163.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-08T05:54:34.514 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:34.514 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:34.514 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-nbd-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:34.514 INFO:teuthology.orchestra.run.smithi163.stdout: re2-1:20211101-3.el9.x86_64 2024-05-08T05:54:34.514 INFO:teuthology.orchestra.run.smithi163.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-08T05:54:34.514 INFO:teuthology.orchestra.run.smithi163.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-08T05:54:34.514 INFO:teuthology.orchestra.run.smithi163.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-08T05:54:34.514 INFO:teuthology.orchestra.run.smithi163.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-08T05:54:34.514 INFO:teuthology.orchestra.run.smithi163.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-08T05:54:34.515 INFO:teuthology.orchestra.run.smithi163.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-08T05:54:34.515 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-08T05:54:34.515 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2024-05-08T05:54:34.775 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephfs-top-2:19.0.0-3538.gbee299f4.el9.noarch 152/164 2024-05-08T05:54:34.867 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-08T05:54:34.908 DEBUG:teuthology.parallel:result is None 2024-05-08T05:54:34.909 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-08T05:54:34.989 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-08T05:54:35.021 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-08T05:54:35.047 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-08T05:54:35.047 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-08T05:54:35.131 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-08T05:54:35.156 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-08T05:54:35.156 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-08T05:54:35.240 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-08T05:54:35.265 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-08T05:54:35.270 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:54:35.270 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:54:35.451 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:54:35.451 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-08T05:54:35.544 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-08T05:54:35.607 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:54:36.278 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 109/164 2024-05-08T05:54:36.278 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /sys 2024-05-08T05:54:36.278 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /proc 2024-05-08T05:54:36.278 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /mnt 2024-05-08T05:54:36.278 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /var/tmp 2024-05-08T05:54:36.278 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /home 2024-05-08T05:54:36.278 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /root 2024-05-08T05:54:36.279 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /tmp 2024-05-08T05:54:36.279 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T05:54:36.512 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 110/164 2024-05-08T05:54:36.556 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 110/164 2024-05-08T05:54:36.556 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:54:36.556 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-08T05:54:36.556 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T05:54:38.165 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 111/164 2024-05-08T05:54:38.246 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 111/164 2024-05-08T05:54:38.247 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:54:38.247 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-08T05:54:38.247 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T05:54:38.247 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T05:54:38.247 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T05:54:38.524 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 112/164 2024-05-08T05:54:38.621 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 112/164 2024-05-08T05:54:38.621 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:54:38.621 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-08T05:54:38.622 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T05:54:38.622 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T05:54:38.622 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T05:54:39.124 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 113/164 2024-05-08T05:54:39.217 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 113/164 2024-05-08T05:54:39.217 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:54:39.217 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-08T05:54:39.217 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T05:54:39.217 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T05:54:39.218 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T05:54:39.349 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-08T05:54:39.440 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-08T05:54:39.522 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-08T05:54:39.765 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-08T05:54:39.829 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-08T05:54:39.915 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-08T05:54:40.023 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-08T05:54:40.190 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-08T05:54:40.236 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-08T05:54:40.313 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-08T05:54:40.352 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-08T05:54:40.968 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 125/164 2024-05-08T05:54:41.004 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 125/164 2024-05-08T05:54:41.057 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3538.gbee299f4.e 126/164 2024-05-08T05:54:41.168 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3538.gbee299f4. 127/164 2024-05-08T05:54:41.178 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:54:41.179 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-2:19.0.0-3538.gbee299f4.el9.x86_64 1/164 2024-05-08T05:54:41.179 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 2/164 2024-05-08T05:54:41.179 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 3/164 2024-05-08T05:54:41.179 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 4/164 2024-05-08T05:54:41.179 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3538.gbee29 5/164 2024-05-08T05:54:41.179 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 6/164 2024-05-08T05:54:41.179 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 7/164 2024-05-08T05:54:41.179 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 8/164 2024-05-08T05:54:41.179 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 9/164 2024-05-08T05:54:41.179 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 10/164 2024-05-08T05:54:41.179 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 11/164 2024-05-08T05:54:41.179 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-test-2:19.0.0-3538.gbee299f4.el9.x86_64 12/164 2024-05-08T05:54:41.179 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 13/164 2024-05-08T05:54:41.180 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephfs-shell-2:19.0.0-3538.gbee299f4.el9.x86_64 14/164 2024-05-08T05:54:41.180 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs-devel-2:19.0.0-3538.gbee299f4.el9.x86_ 15/164 2024-05-08T05:54:41.180 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 16/164 2024-05-08T05:54:41.180 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 17/164 2024-05-08T05:54:41.180 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados-devel-2:19.0.0-3538.gbee299f4.el9.x86_6 18/164 2024-05-08T05:54:41.180 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86 19/164 2024-05-08T05:54:41.180 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 20/164 2024-05-08T05:54:41.180 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3538.gbee299f4.el 21/164 2024-05-08T05:54:41.180 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-common-2:19.0.0-3538.gbee299f4.el9. 22/164 2024-05-08T05:54:41.180 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cephfs-2:19.0.0-3538.gbee299f4.el9.x86_6 23/164 2024-05-08T05:54:41.180 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rados-2:19.0.0-3538.gbee299f4.el9.x86_64 24/164 2024-05-08T05:54:41.180 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rbd-2:19.0.0-3538.gbee299f4.el9.x86_64 25/164 2024-05-08T05:54:41.180 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rgw-2:19.0.0-3538.gbee299f4.el9.x86_64 26/164 2024-05-08T05:54:41.180 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 27/164 2024-05-08T05:54:41.181 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 28/164 2024-05-08T05:54:41.181 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-nbd-2:19.0.0-3538.gbee299f4.el9.x86_64 29/164 2024-05-08T05:54:41.181 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3538.gbee299f4. 30/164 2024-05-08T05:54:41.181 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noa 31/164 2024-05-08T05:54:41.181 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.n 32/164 2024-05-08T05:54:41.181 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee 33/164 2024-05-08T05:54:41.181 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3538.gbee299f4.el 34/164 2024-05-08T05:54:41.181 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 35/164 2024-05-08T05:54:41.181 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3538.gbee299f4.e 36/164 2024-05-08T05:54:41.181 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 37/164 2024-05-08T05:54:41.181 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 38/164 2024-05-08T05:54:41.181 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephfs-top-2:19.0.0-3538.gbee299f4.el9.noarch 39/164 2024-05-08T05:54:41.181 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-08T05:54:41.182 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-08T05:54:41.182 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-08T05:54:41.182 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-08T05:54:41.182 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-08T05:54:41.182 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-08T05:54:41.182 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-08T05:54:41.182 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-08T05:54:41.182 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-08T05:54:41.182 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-08T05:54:41.182 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-08T05:54:41.182 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-08T05:54:41.182 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-08T05:54:41.182 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-08T05:54:41.182 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-08T05:54:41.183 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-08T05:54:41.183 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-08T05:54:41.183 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-08T05:54:41.183 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-08T05:54:41.183 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-08T05:54:41.183 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-08T05:54:41.183 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-08T05:54:41.183 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-08T05:54:41.183 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-08T05:54:41.183 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-08T05:54:41.183 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-08T05:54:41.183 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-08T05:54:41.183 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-08T05:54:41.183 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-08T05:54:41.183 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-08T05:54:41.183 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-08T05:54:41.184 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-08T05:54:41.184 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-08T05:54:41.184 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-08T05:54:41.184 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-08T05:54:41.184 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-08T05:54:41.184 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-08T05:54:41.184 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-08T05:54:41.184 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-08T05:54:41.184 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-08T05:54:41.184 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-08T05:54:41.184 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-08T05:54:41.184 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-08T05:54:41.184 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-08T05:54:41.184 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-08T05:54:41.184 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-08T05:54:41.185 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-08T05:54:41.185 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-08T05:54:41.185 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-08T05:54:41.185 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-08T05:54:41.185 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-08T05:54:41.185 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-08T05:54:41.185 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-08T05:54:41.185 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-08T05:54:41.185 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-08T05:54:41.185 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-08T05:54:41.185 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-08T05:54:41.185 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-08T05:54:41.185 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-08T05:54:41.185 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-08T05:54:41.185 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-08T05:54:41.185 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-08T05:54:41.186 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-08T05:54:41.186 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-08T05:54:41.186 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-08T05:54:41.186 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-08T05:54:41.186 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-08T05:54:41.186 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-08T05:54:41.186 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-08T05:54:41.186 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-08T05:54:41.186 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-08T05:54:41.186 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-08T05:54:41.186 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-08T05:54:41.186 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-08T05:54:41.187 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-08T05:54:41.187 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-08T05:54:41.187 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-08T05:54:41.187 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-08T05:54:41.187 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-08T05:54:41.187 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-08T05:54:41.187 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-08T05:54:41.187 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-08T05:54:41.187 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-08T05:54:41.187 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-08T05:54:41.187 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-08T05:54:41.187 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-08T05:54:41.187 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-08T05:54:41.188 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-08T05:54:41.188 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-08T05:54:41.188 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-08T05:54:41.188 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-08T05:54:41.188 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-08T05:54:41.188 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-08T05:54:41.188 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-08T05:54:41.188 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-08T05:54:41.188 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-08T05:54:41.188 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-08T05:54:41.188 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-08T05:54:41.188 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-08T05:54:41.188 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-08T05:54:41.188 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-08T05:54:41.188 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-08T05:54:41.189 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-08T05:54:41.189 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-08T05:54:41.189 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-08T05:54:41.189 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-08T05:54:41.189 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-08T05:54:41.189 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-08T05:54:41.189 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-08T05:54:41.189 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 149/164 2024-05-08T05:54:41.189 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-08T05:54:41.189 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 151/164 2024-05-08T05:54:41.189 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-08T05:54:41.189 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-08T05:54:41.189 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-08T05:54:41.189 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-08T05:54:41.189 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-08T05:54:41.189 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-08T05:54:41.190 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-08T05:54:41.190 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-08T05:54:41.190 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-08T05:54:41.190 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-08T05:54:41.190 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-08T05:54:41.190 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-08T05:54:41.215 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noa 128/164 2024-05-08T05:54:42.526 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noa 128/164 2024-05-08T05:54:42.575 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.n 129/164 2024-05-08T05:54:43.674 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.n 129/164 2024-05-08T05:54:43.717 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee 130/164 2024-05-08T05:54:43.824 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:54:43.824 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T05:54:43.824 INFO:teuthology.orchestra.run.smithi192.stdout:Upgraded: 2024-05-08T05:54:43.824 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-08T05:54:43.824 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-08T05:54:43.824 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-08T05:54:43.824 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-08T05:54:43.824 INFO:teuthology.orchestra.run.smithi192.stdout: librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:43.824 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:43.824 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-08T05:54:43.824 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-08T05:54:43.825 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-05-08T05:54:43.825 INFO:teuthology.orchestra.run.smithi192.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-08T05:54:43.825 INFO:teuthology.orchestra.run.smithi192.stdout: automake-1.16.2-8.el9.noarch 2024-05-08T05:54:43.825 INFO:teuthology.orchestra.run.smithi192.stdout: bison-3.7.4-5.el9.x86_64 2024-05-08T05:54:43.825 INFO:teuthology.orchestra.run.smithi192.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-08T05:54:43.825 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:43.825 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:43.825 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:43.825 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:43.825 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:54:43.825 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:43.825 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:43.825 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:43.826 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:54:43.826 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:54:43.826 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:54:43.826 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:54:43.826 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:54:43.826 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:43.826 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:43.826 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:54:43.826 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:43.826 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:43.826 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:43.826 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:54:43.826 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:54:43.827 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:43.827 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-shell-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:43.827 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-top-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:54:43.827 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-08T05:54:43.827 INFO:teuthology.orchestra.run.smithi192.stdout: flex-2.6.4-9.el9.x86_64 2024-05-08T05:54:43.827 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-08T05:54:43.827 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-08T05:54:43.827 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-08T05:54:43.827 INFO:teuthology.orchestra.run.smithi192.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-08T05:54:43.827 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-08T05:54:43.827 INFO:teuthology.orchestra.run.smithi192.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-08T05:54:43.827 INFO:teuthology.orchestra.run.smithi192.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-08T05:54:43.827 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-08T05:54:43.827 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-08T05:54:43.827 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:43.827 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:43.828 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:43.828 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-08T05:54:43.828 INFO:teuthology.orchestra.run.smithi192.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-08T05:54:43.828 INFO:teuthology.orchestra.run.smithi192.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-08T05:54:43.828 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-08T05:54:43.828 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-08T05:54:43.828 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:43.828 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:43.828 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-08T05:54:43.828 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:43.828 INFO:teuthology.orchestra.run.smithi192.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-08T05:54:43.828 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-08T05:54:43.828 INFO:teuthology.orchestra.run.smithi192.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-08T05:54:43.828 INFO:teuthology.orchestra.run.smithi192.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-08T05:54:43.828 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-08T05:54:43.829 INFO:teuthology.orchestra.run.smithi192.stdout: lua-5.4.4-4.el9.x86_64 2024-05-08T05:54:43.829 INFO:teuthology.orchestra.run.smithi192.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-08T05:54:43.829 INFO:teuthology.orchestra.run.smithi192.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-08T05:54:43.829 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-08T05:54:43.829 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-08T05:54:43.829 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-08T05:54:43.829 INFO:teuthology.orchestra.run.smithi192.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-08T05:54:43.829 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-13.14-1.el9.x86_64 2024-05-08T05:54:43.829 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-08T05:54:43.829 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-08T05:54:43.829 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-08T05:54:43.829 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-08T05:54:43.829 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-08T05:54:43.829 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-08T05:54:43.829 INFO:teuthology.orchestra.run.smithi192.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-08T05:54:43.830 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-08T05:54:43.830 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-08T05:54:43.830 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-08T05:54:43.830 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:43.830 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:43.830 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:43.830 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-08T05:54:43.830 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-08T05:54:43.830 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-08T05:54:43.830 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-08T05:54:43.830 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-08T05:54:43.830 INFO:teuthology.orchestra.run.smithi192.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-08T05:54:43.830 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-08T05:54:43.830 INFO:teuthology.orchestra.run.smithi192.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-08T05:54:43.831 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-08T05:54:43.831 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-08T05:54:43.831 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-08T05:54:43.831 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-08T05:54:43.831 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-08T05:54:43.831 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-08T05:54:43.831 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-08T05:54:43.831 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-08T05:54:43.831 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-08T05:54:43.831 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-08T05:54:43.831 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-08T05:54:43.831 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-08T05:54:43.831 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-08T05:54:43.831 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-08T05:54:43.831 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-08T05:54:43.832 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-08T05:54:43.832 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-08T05:54:43.832 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-08T05:54:43.832 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-08T05:54:43.832 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-08T05:54:43.832 INFO:teuthology.orchestra.run.smithi192.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-08T05:54:43.832 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-08T05:54:43.832 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ply-3.11-14.el9.noarch 2024-05-08T05:54:43.832 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-08T05:54:43.832 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-08T05:54:43.832 INFO:teuthology.orchestra.run.smithi192.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-08T05:54:43.832 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-08T05:54:43.832 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-08T05:54:43.832 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-08T05:54:43.832 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-08T05:54:43.832 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-08T05:54:43.833 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-08T05:54:43.833 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:43.833 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:43.833 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-08T05:54:43.833 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-08T05:54:43.833 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-08T05:54:43.833 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:43.833 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-08T05:54:43.833 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-08T05:54:43.833 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-08T05:54:43.833 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-08T05:54:43.833 INFO:teuthology.orchestra.run.smithi192.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-08T05:54:43.833 INFO:teuthology.orchestra.run.smithi192.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-08T05:54:43.833 INFO:teuthology.orchestra.run.smithi192.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-08T05:54:43.833 INFO:teuthology.orchestra.run.smithi192.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-08T05:54:43.833 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-08T05:54:43.834 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-08T05:54:43.834 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-08T05:54:43.834 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-08T05:54:43.834 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-08T05:54:43.834 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-08T05:54:43.834 INFO:teuthology.orchestra.run.smithi192.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-08T05:54:43.834 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:43.834 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:43.834 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:54:43.834 INFO:teuthology.orchestra.run.smithi192.stdout: re2-1:20211101-3.el9.x86_64 2024-05-08T05:54:43.834 INFO:teuthology.orchestra.run.smithi192.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-08T05:54:43.834 INFO:teuthology.orchestra.run.smithi192.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-08T05:54:43.834 INFO:teuthology.orchestra.run.smithi192.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-08T05:54:43.834 INFO:teuthology.orchestra.run.smithi192.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-08T05:54:43.835 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-08T05:54:43.835 INFO:teuthology.orchestra.run.smithi192.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-08T05:54:43.835 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T05:54:43.835 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-05-08T05:54:43.874 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee 130/164 2024-05-08T05:54:44.055 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3538.gbee299f4.el 131/164 2024-05-08T05:54:44.086 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 132/164 2024-05-08T05:54:44.162 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 132/164 2024-05-08T05:54:44.162 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:54:44.162 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-08T05:54:44.162 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T05:54:44.162 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T05:54:44.163 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T05:54:44.228 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 133/164 2024-05-08T05:54:44.241 DEBUG:teuthology.parallel:result is None 2024-05-08T05:54:44.267 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 133/164 2024-05-08T05:54:44.408 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-2:19.0.0-3538.gbee299f4.el9.x86_64 134/164 2024-05-08T05:54:44.511 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 135/164 2024-05-08T05:54:45.577 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-08T05:54:45.614 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 137/164 2024-05-08T05:54:45.693 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 137/164 2024-05-08T05:54:45.693 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:54:45.693 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-08T05:54:45.693 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T05:54:45.693 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T05:54:45.694 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T05:54:45.748 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3538.gbee29 138/164 2024-05-08T05:54:45.837 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3538.gbee29 138/164 2024-05-08T05:54:45.837 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:54:45.837 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-08T05:54:45.837 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T05:54:45.915 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 139/164 2024-05-08T05:54:46.017 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 139/164 2024-05-08T05:54:46.017 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:54:46.018 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-08T05:54:46.018 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-08T05:54:46.018 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-08T05:54:46.018 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T05:54:46.407 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 140/164 2024-05-08T05:54:46.488 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 140/164 2024-05-08T05:54:46.488 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:54:46.488 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-08T05:54:46.488 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T05:54:46.488 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T05:54:46.488 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T05:54:51.534 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-test-2:19.0.0-3538.gbee299f4.el9.x86_64 141/164 2024-05-08T05:54:51.624 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 142/164 2024-05-08T05:54:51.693 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-nbd-2:19.0.0-3538.gbee299f4.el9.x86_64 143/164 2024-05-08T05:54:51.734 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-08T05:54:51.781 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-08T05:54:52.108 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-shell-2:19.0.0-3538.gbee299f4.el9.x86_64 146/164 2024-05-08T05:54:52.171 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-08T05:54:52.241 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs-devel-2:19.0.0-3538.gbee299f4.el9.x86_ 148/164 2024-05-08T05:54:52.287 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-08T05:54:52.335 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-08T05:54:53.329 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:54:53.404 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:54:53.460 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:54:54.782 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-top-2:19.0.0-3538.gbee299f4.el9.noarch 152/164 2024-05-08T05:54:54.905 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-08T05:54:54.989 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-08T05:54:55.130 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-08T05:54:55.172 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-08T05:54:55.227 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-08T05:54:55.227 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-08T05:54:55.354 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-08T05:54:55.401 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-08T05:54:55.401 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-08T05:54:55.564 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-08T05:54:55.593 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-08T05:54:55.627 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:54:55.627 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:54:55.973 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:54:55.973 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-08T05:54:56.151 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-08T05:54:56.284 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:54:56.575 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:54:56.575 INFO:teuthology.orchestra.run.smithi123.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-08T05:54:56.575 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T05:55:02.368 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-2:19.0.0-3538.gbee299f4.el9.x86_64 1/164 2024-05-08T05:55:02.368 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 2/164 2024-05-08T05:55:02.368 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 3/164 2024-05-08T05:55:02.368 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 4/164 2024-05-08T05:55:02.369 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3538.gbee29 5/164 2024-05-08T05:55:02.369 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 6/164 2024-05-08T05:55:02.369 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 7/164 2024-05-08T05:55:02.369 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 8/164 2024-05-08T05:55:02.369 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 9/164 2024-05-08T05:55:02.369 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 10/164 2024-05-08T05:55:02.369 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 11/164 2024-05-08T05:55:02.369 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-test-2:19.0.0-3538.gbee299f4.el9.x86_64 12/164 2024-05-08T05:55:02.369 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 13/164 2024-05-08T05:55:02.369 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-shell-2:19.0.0-3538.gbee299f4.el9.x86_64 14/164 2024-05-08T05:55:02.369 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs-devel-2:19.0.0-3538.gbee299f4.el9.x86_ 15/164 2024-05-08T05:55:02.369 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 16/164 2024-05-08T05:55:02.369 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 17/164 2024-05-08T05:55:02.370 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados-devel-2:19.0.0-3538.gbee299f4.el9.x86_6 18/164 2024-05-08T05:55:02.370 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86 19/164 2024-05-08T05:55:02.370 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 20/164 2024-05-08T05:55:02.370 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3538.gbee299f4.el 21/164 2024-05-08T05:55:02.370 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-common-2:19.0.0-3538.gbee299f4.el9. 22/164 2024-05-08T05:55:02.370 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cephfs-2:19.0.0-3538.gbee299f4.el9.x86_6 23/164 2024-05-08T05:55:02.370 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rados-2:19.0.0-3538.gbee299f4.el9.x86_64 24/164 2024-05-08T05:55:02.370 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rbd-2:19.0.0-3538.gbee299f4.el9.x86_64 25/164 2024-05-08T05:55:02.370 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rgw-2:19.0.0-3538.gbee299f4.el9.x86_64 26/164 2024-05-08T05:55:02.370 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 27/164 2024-05-08T05:55:02.370 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 28/164 2024-05-08T05:55:02.370 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-nbd-2:19.0.0-3538.gbee299f4.el9.x86_64 29/164 2024-05-08T05:55:02.370 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3538.gbee299f4. 30/164 2024-05-08T05:55:02.383 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noa 31/164 2024-05-08T05:55:02.383 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.n 32/164 2024-05-08T05:55:02.383 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee 33/164 2024-05-08T05:55:02.383 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3538.gbee299f4.el 34/164 2024-05-08T05:55:02.383 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 35/164 2024-05-08T05:55:02.384 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3538.gbee299f4.e 36/164 2024-05-08T05:55:02.384 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 37/164 2024-05-08T05:55:02.384 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 38/164 2024-05-08T05:55:02.384 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-top-2:19.0.0-3538.gbee299f4.el9.noarch 39/164 2024-05-08T05:55:02.384 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-08T05:55:02.384 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-08T05:55:02.384 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-08T05:55:02.384 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-08T05:55:02.384 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-08T05:55:02.384 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-08T05:55:02.384 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-08T05:55:02.384 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-08T05:55:02.384 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-08T05:55:02.385 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-08T05:55:02.385 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-08T05:55:02.385 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-08T05:55:02.385 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-08T05:55:02.385 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-08T05:55:02.385 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-08T05:55:02.385 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-08T05:55:02.385 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-08T05:55:02.385 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-08T05:55:02.385 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-08T05:55:02.385 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-08T05:55:02.385 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-08T05:55:02.385 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-08T05:55:02.385 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-08T05:55:02.386 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-08T05:55:02.386 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-08T05:55:02.386 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-08T05:55:02.386 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-08T05:55:02.386 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-08T05:55:02.386 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-08T05:55:02.386 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-08T05:55:02.386 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-08T05:55:02.386 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-08T05:55:02.386 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-08T05:55:02.386 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-08T05:55:02.386 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-08T05:55:02.387 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-08T05:55:02.387 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-08T05:55:02.387 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-08T05:55:02.387 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-08T05:55:02.387 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-08T05:55:02.387 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-08T05:55:02.387 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-08T05:55:02.387 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-08T05:55:02.387 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-08T05:55:02.387 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-08T05:55:02.387 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-08T05:55:02.387 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-08T05:55:02.387 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-08T05:55:02.388 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-08T05:55:02.388 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-08T05:55:02.388 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-08T05:55:02.388 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-08T05:55:02.388 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-08T05:55:02.388 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-08T05:55:02.388 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-08T05:55:02.388 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-08T05:55:02.388 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-08T05:55:02.388 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-08T05:55:02.388 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-08T05:55:02.388 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-08T05:55:02.388 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-08T05:55:02.389 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-08T05:55:02.389 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-08T05:55:02.389 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-08T05:55:02.389 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-08T05:55:02.389 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-08T05:55:02.389 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-08T05:55:02.389 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-08T05:55:02.389 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-08T05:55:02.389 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-08T05:55:02.389 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-08T05:55:02.389 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-08T05:55:02.389 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-08T05:55:02.389 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-08T05:55:02.389 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-08T05:55:02.390 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-08T05:55:02.390 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-08T05:55:02.390 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-08T05:55:02.390 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-08T05:55:02.390 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-08T05:55:02.390 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-08T05:55:02.390 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-08T05:55:02.390 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-08T05:55:02.390 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-08T05:55:02.390 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-08T05:55:02.390 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-08T05:55:02.390 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-08T05:55:02.390 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-08T05:55:02.390 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-08T05:55:02.391 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-08T05:55:02.391 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-08T05:55:02.391 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-08T05:55:02.391 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-08T05:55:02.391 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-08T05:55:02.391 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-08T05:55:02.391 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-08T05:55:02.391 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-08T05:55:02.391 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-08T05:55:02.391 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-08T05:55:02.391 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-08T05:55:02.391 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-08T05:55:02.391 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-08T05:55:02.391 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-08T05:55:02.392 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-08T05:55:02.392 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-08T05:55:02.392 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-08T05:55:02.392 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-08T05:55:02.392 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-08T05:55:02.392 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-08T05:55:02.392 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 149/164 2024-05-08T05:55:02.392 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-08T05:55:02.392 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 151/164 2024-05-08T05:55:02.392 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-08T05:55:02.392 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-08T05:55:02.392 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-08T05:55:02.392 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-08T05:55:02.392 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-08T05:55:02.393 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-08T05:55:02.393 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-08T05:55:02.393 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-08T05:55:02.393 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-08T05:55:02.393 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-08T05:55:02.393 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-08T05:55:02.393 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-08T05:55:04.688 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:55:04.688 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T05:55:04.688 INFO:teuthology.orchestra.run.smithi123.stdout:Upgraded: 2024-05-08T05:55:04.688 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-08T05:55:04.688 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-08T05:55:04.688 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-08T05:55:04.688 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-08T05:55:04.689 INFO:teuthology.orchestra.run.smithi123.stdout: librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:55:04.689 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:55:04.689 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-08T05:55:04.689 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-08T05:55:04.689 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-05-08T05:55:04.689 INFO:teuthology.orchestra.run.smithi123.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-08T05:55:04.689 INFO:teuthology.orchestra.run.smithi123.stdout: automake-1.16.2-8.el9.noarch 2024-05-08T05:55:04.689 INFO:teuthology.orchestra.run.smithi123.stdout: bison-3.7.4-5.el9.x86_64 2024-05-08T05:55:04.689 INFO:teuthology.orchestra.run.smithi123.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-08T05:55:04.689 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:55:04.689 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:55:04.689 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:55:04.689 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:55:04.690 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:55:04.690 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:55:04.690 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:55:04.690 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:55:04.690 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:55:04.690 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:55:04.690 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:55:04.690 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:55:04.690 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:55:04.690 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:55:04.690 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:55:04.690 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:55:04.690 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:55:04.691 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:55:04.691 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:55:04.691 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:55:04.691 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:55:04.691 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:55:04.691 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-shell-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:55:04.691 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:55:04.691 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-08T05:55:04.691 INFO:teuthology.orchestra.run.smithi123.stdout: flex-2.6.4-9.el9.x86_64 2024-05-08T05:55:04.691 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-08T05:55:04.691 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-08T05:55:04.691 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-08T05:55:04.691 INFO:teuthology.orchestra.run.smithi123.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-08T05:55:04.691 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-08T05:55:04.692 INFO:teuthology.orchestra.run.smithi123.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-08T05:55:04.692 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-08T05:55:04.692 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-08T05:55:04.692 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-08T05:55:04.692 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:55:04.692 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:55:04.692 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:55:04.692 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-08T05:55:04.692 INFO:teuthology.orchestra.run.smithi123.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-08T05:55:04.692 INFO:teuthology.orchestra.run.smithi123.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-08T05:55:04.692 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-08T05:55:04.692 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-08T05:55:04.692 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:55:04.693 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:55:04.693 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-08T05:55:04.693 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:55:04.693 INFO:teuthology.orchestra.run.smithi123.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-08T05:55:04.693 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-08T05:55:04.693 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-08T05:55:04.693 INFO:teuthology.orchestra.run.smithi123.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-08T05:55:04.693 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-08T05:55:04.693 INFO:teuthology.orchestra.run.smithi123.stdout: lua-5.4.4-4.el9.x86_64 2024-05-08T05:55:04.693 INFO:teuthology.orchestra.run.smithi123.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-08T05:55:04.693 INFO:teuthology.orchestra.run.smithi123.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-08T05:55:04.693 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-08T05:55:04.693 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-08T05:55:04.694 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-08T05:55:04.694 INFO:teuthology.orchestra.run.smithi123.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-08T05:55:04.694 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-13.14-1.el9.x86_64 2024-05-08T05:55:04.694 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-08T05:55:04.694 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-08T05:55:04.694 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-08T05:55:04.694 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-08T05:55:04.694 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-08T05:55:04.694 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-08T05:55:04.694 INFO:teuthology.orchestra.run.smithi123.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-08T05:55:04.694 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-08T05:55:04.694 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-08T05:55:04.694 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-08T05:55:04.694 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:55:04.694 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:55:04.695 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:55:04.695 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-08T05:55:04.695 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-08T05:55:04.695 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-08T05:55:04.695 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-08T05:55:04.695 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-08T05:55:04.695 INFO:teuthology.orchestra.run.smithi123.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-08T05:55:04.695 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-08T05:55:04.695 INFO:teuthology.orchestra.run.smithi123.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-08T05:55:04.696 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-08T05:55:04.696 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-08T05:55:04.696 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-08T05:55:04.696 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-08T05:55:04.696 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-08T05:55:04.696 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-08T05:55:04.696 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-08T05:55:04.696 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-08T05:55:04.696 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-08T05:55:04.696 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-08T05:55:04.696 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-08T05:55:04.697 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-08T05:55:04.697 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-08T05:55:04.697 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-08T05:55:04.697 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-08T05:55:04.697 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-08T05:55:04.697 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-08T05:55:04.697 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-08T05:55:04.697 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-08T05:55:04.697 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-08T05:55:04.697 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-08T05:55:04.697 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-08T05:55:04.697 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ply-3.11-14.el9.noarch 2024-05-08T05:55:04.697 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-08T05:55:04.697 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-08T05:55:04.698 INFO:teuthology.orchestra.run.smithi123.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-08T05:55:04.698 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-08T05:55:04.698 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-08T05:55:04.698 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-08T05:55:04.698 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-08T05:55:04.698 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-08T05:55:04.698 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-08T05:55:04.698 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:55:04.699 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:55:04.699 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-08T05:55:04.699 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-08T05:55:04.699 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-08T05:55:04.699 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:55:04.699 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-08T05:55:04.700 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-08T05:55:04.700 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-08T05:55:04.700 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-08T05:55:04.700 INFO:teuthology.orchestra.run.smithi123.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-08T05:55:04.700 INFO:teuthology.orchestra.run.smithi123.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-08T05:55:04.700 INFO:teuthology.orchestra.run.smithi123.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-08T05:55:04.700 INFO:teuthology.orchestra.run.smithi123.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-08T05:55:04.700 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-08T05:55:04.700 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-08T05:55:04.700 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-08T05:55:04.700 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-08T05:55:04.700 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-08T05:55:04.700 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-08T05:55:04.700 INFO:teuthology.orchestra.run.smithi123.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-08T05:55:04.701 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:55:04.701 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:55:04.701 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:55:04.701 INFO:teuthology.orchestra.run.smithi123.stdout: re2-1:20211101-3.el9.x86_64 2024-05-08T05:55:04.701 INFO:teuthology.orchestra.run.smithi123.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-08T05:55:04.701 INFO:teuthology.orchestra.run.smithi123.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-08T05:55:04.701 INFO:teuthology.orchestra.run.smithi123.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-08T05:55:04.701 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-08T05:55:04.701 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-08T05:55:04.701 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-08T05:55:04.702 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T05:55:04.702 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-05-08T05:55:05.313 DEBUG:teuthology.parallel:result is None 2024-05-08T05:55:05.314 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:55:05.535 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T05:55:05.589 INFO:teuthology.orchestra.run.smithi123.stdout:19.0.0-3538.gbee299f4.el9 2024-05-08T05:55:05.590 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3538.gbee299f4.el9 2024-05-08T05:55:05.590 INFO:teuthology.task.install:The correct ceph version 19.0.0-3538.gbee299f4 is installed. 2024-05-08T05:55:05.592 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:55:05.738 DEBUG:teuthology.orchestra.run.smithi163:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T05:55:05.764 INFO:teuthology.orchestra.run.smithi163.stdout:19.0.0-3538.gbee299f4.el9 2024-05-08T05:55:05.765 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3538.gbee299f4.el9 2024-05-08T05:55:05.765 INFO:teuthology.task.install:The correct ceph version 19.0.0-3538.gbee299f4 is installed. 2024-05-08T05:55:05.767 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:55:06.010 DEBUG:teuthology.orchestra.run.smithi192:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T05:55:06.035 INFO:teuthology.orchestra.run.smithi192.stdout:19.0.0-3538.gbee299f4.el9 2024-05-08T05:55:06.035 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3538.gbee299f4.el9 2024-05-08T05:55:06.036 INFO:teuthology.task.install:The correct ceph version 19.0.0-3538.gbee299f4 is installed. 2024-05-08T05:55:06.037 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-08T05:55:06.038 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-05-08T05:55:06.038 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T05:55:06.102 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T05:55:06.102 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T05:55:06.134 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T05:55:06.134 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T05:55:06.166 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-08T05:55:06.166 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-05-08T05:55:06.166 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T05:55:06.237 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T05:55:06.339 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T05:55:06.339 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T05:55:06.368 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T05:55:06.436 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T05:55:06.436 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T05:55:06.461 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T05:55:06.524 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-08T05:55:06.525 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-05-08T05:55:06.525 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T05:55:06.585 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T05:55:06.685 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T05:55:06.686 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T05:55:06.711 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T05:55:06.774 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T05:55:06.775 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T05:55:06.799 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T05:55:06.863 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-08T05:55:06.863 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-05-08T05:55:06.863 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T05:55:06.922 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T05:55:07.022 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T05:55:07.022 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T05:55:07.046 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T05:55:07.110 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T05:55:07.110 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T05:55:07.133 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T05:55:07.197 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-08T05:55:07.294 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'bee299f4ac344e4c45e550d7a80abcadd7bc3d5a', 'subvols': {'create': 2}} 2024-05-08T05:55:07.294 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:55:07.295 INFO:tasks.cephadm:Cluster fsid is 85cfb2e6-0cff-11ef-bc97-c7b262605968 2024-05-08T05:55:07.295 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-08T05:55:07.295 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.123', 'mon.b': '172.21.15.163', 'mon.c': '172.21.15.192'} 2024-05-08T05:55:07.295 INFO:tasks.cephadm:First mon is mon.a on smithi123 2024-05-08T05:55:07.295 INFO:tasks.cephadm:First mgr is x 2024-05-08T05:55:07.295 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-08T05:55:07.295 DEBUG:teuthology.orchestra.run.smithi123:> sudo hostname $(hostname -s) 2024-05-08T05:55:07.357 DEBUG:teuthology.orchestra.run.smithi163:> sudo hostname $(hostname -s) 2024-05-08T05:55:07.381 DEBUG:teuthology.orchestra.run.smithi192:> sudo hostname $(hostname -s) 2024-05-08T05:55:07.405 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-08T05:55:07.405 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:55:07.544 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'bee299f4ac344e4c45e550d7a80abcadd7bc3d5a', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79166/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3538-gbee299f4', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3538.gbee299f4'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-07 17:50:48.409840', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-57676-3', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-05-08T05:55:07.690 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-57676-3, sha1 bee299f4ac344e4c45e550d7a80abcadd7bc3d5a from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:55:07.692 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/x86_64/flavors/default/cephadm 2024-05-08T05:55:07.692 DEBUG:teuthology.orchestra.run.smithi123:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-08T05:55:09.857 INFO:teuthology.orchestra.run.smithi123.stdout:-rw-r--r--. 1 ubuntu ubuntu 782795 May 8 05:55 /home/ubuntu/cephtest/cephadm 2024-05-08T05:55:09.858 DEBUG:teuthology.orchestra.run.smithi163:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-08T05:55:11.641 INFO:teuthology.orchestra.run.smithi163.stdout:-rw-r--r--. 1 ubuntu ubuntu 782795 May 8 05:55 /home/ubuntu/cephtest/cephadm 2024-05-08T05:55:11.642 DEBUG:teuthology.orchestra.run.smithi192:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-08T05:55:13.757 INFO:teuthology.orchestra.run.smithi192.stdout:-rw-r--r--. 1 ubuntu ubuntu 782795 May 8 05:55 /home/ubuntu/cephtest/cephadm 2024-05-08T05:55:13.757 DEBUG:teuthology.orchestra.run.smithi123:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-08T05:55:13.817 DEBUG:teuthology.orchestra.run.smithi163:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-08T05:55:13.836 DEBUG:teuthology.orchestra.run.smithi192:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-08T05:55:13.871 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a on all hosts... 2024-05-08T05:55:13.872 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a pull 2024-05-08T05:55:13.879 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a pull 2024-05-08T05:55:13.881 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a pull 2024-05-08T05:55:15.431 INFO:teuthology.orchestra.run.smithi163.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a... 2024-05-08T05:55:15.451 INFO:teuthology.orchestra.run.smithi192.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a... 2024-05-08T05:55:16.022 INFO:teuthology.orchestra.run.smithi123.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a... 2024-05-08T05:55:45.752 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T05:55:45.752 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 19.0.0-3538-gbee299f4 (bee299f4ac344e4c45e550d7a80abcadd7bc3d5a) squid (dev)", 2024-05-08T05:55:45.753 INFO:teuthology.orchestra.run.smithi192.stdout: "image_id": "352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce", 2024-05-08T05:55:45.753 INFO:teuthology.orchestra.run.smithi192.stdout: "repo_digests": [ 2024-05-08T05:55:45.753 INFO:teuthology.orchestra.run.smithi192.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c" 2024-05-08T05:55:45.753 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-05-08T05:55:45.753 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T05:55:49.670 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2024-05-08T05:55:49.671 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_version": "ceph version 19.0.0-3538-gbee299f4 (bee299f4ac344e4c45e550d7a80abcadd7bc3d5a) squid (dev)", 2024-05-08T05:55:49.671 INFO:teuthology.orchestra.run.smithi163.stdout: "image_id": "352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce", 2024-05-08T05:55:49.671 INFO:teuthology.orchestra.run.smithi163.stdout: "repo_digests": [ 2024-05-08T05:55:49.671 INFO:teuthology.orchestra.run.smithi163.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c" 2024-05-08T05:55:49.671 INFO:teuthology.orchestra.run.smithi163.stdout: ] 2024-05-08T05:55:49.671 INFO:teuthology.orchestra.run.smithi163.stdout:} 2024-05-08T05:55:59.634 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-05-08T05:55:59.634 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_version": "ceph version 19.0.0-3538-gbee299f4 (bee299f4ac344e4c45e550d7a80abcadd7bc3d5a) squid (dev)", 2024-05-08T05:55:59.634 INFO:teuthology.orchestra.run.smithi123.stdout: "image_id": "352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce", 2024-05-08T05:55:59.634 INFO:teuthology.orchestra.run.smithi123.stdout: "repo_digests": [ 2024-05-08T05:55:59.634 INFO:teuthology.orchestra.run.smithi123.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c" 2024-05-08T05:55:59.635 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-05-08T05:55:59.635 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-05-08T05:55:59.686 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /etc/ceph 2024-05-08T05:55:59.788 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /etc/ceph 2024-05-08T05:55:59.822 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /etc/ceph 2024-05-08T05:55:59.854 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 777 /etc/ceph 2024-05-08T05:55:59.950 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 777 /etc/ceph 2024-05-08T05:55:59.980 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 777 /etc/ceph 2024-05-08T05:56:00.009 INFO:tasks.cephadm:Writing seed config... 2024-05-08T05:56:00.010 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-08T05:56:00.011 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-08T05:56:00.011 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-08T05:56:00.011 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-08T05:56:00.011 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-08T05:56:00.011 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-08T05:56:00.011 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-08T05:56:00.011 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-08T05:56:00.011 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-08T05:56:00.011 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-08T05:56:00.011 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-08T05:56:00.011 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-08T05:56:00.011 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-08T05:56:00.012 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-08T05:56:00.012 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-08T05:56:00.012 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-08T05:56:00.012 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-08T05:56:00.012 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-05-08T05:56:00.012 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-05-08T05:56:00.012 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-08T05:56:00.012 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-08T05:56:00.012 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-08T05:56:00.012 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-08T05:56:00.012 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-08T05:56:00.012 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-08T05:56:00.012 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-08T05:56:00.012 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-08T05:56:00.012 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-08T05:56:00.013 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-08T05:56:00.013 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-08T05:56:00.013 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-08T05:56:00.013 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-08T05:56:00.013 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-05-08T05:56:00.013 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-08T05:56:00.013 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-08T05:56:00.013 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-08T05:56:00.013 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-08T05:56:00.013 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-08T05:56:00.013 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-08T05:56:00.013 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-08T05:56:00.013 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-08T05:56:00.013 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-08T05:56:00.013 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-08T05:56:00.014 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-08T05:56:00.014 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-08T05:56:00.014 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-08T05:56:00.014 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-05-08T05:56:00.015 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-05-08T05:56:00.015 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-08T05:56:00.072 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 85cfb2e6-0cff-11ef-bc97-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-08T05:56:00.072 DEBUG:teuthology.orchestra.run.smithi123:mon.a> sudo journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mon.a.service 2024-05-08T05:56:00.125 DEBUG:teuthology.orchestra.run.smithi123:mgr.x> sudo journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mgr.x.service 2024-05-08T05:56:00.175 INFO:tasks.cephadm:Bootstrapping... 2024-05-08T05:56:00.175 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a -v bootstrap --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.123 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-08T05:56:00.442 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-05-08T05:56:00.443 INFO:teuthology.orchestra.run.smithi123.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a', '-v', 'bootstrap', '--fsid', '85cfb2e6-0cff-11ef-bc97-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.123', '--skip-admin-label'] 2024-05-08T05:56:00.443 INFO:teuthology.orchestra.run.smithi123.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-08T05:56:00.443 INFO:teuthology.orchestra.run.smithi123.stdout:Verifying podman|docker is present... 2024-05-08T05:56:00.512 INFO:teuthology.orchestra.run.smithi123.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-08T05:56:00.512 INFO:teuthology.orchestra.run.smithi123.stdout:Verifying lvm2 is present... 2024-05-08T05:56:00.512 INFO:teuthology.orchestra.run.smithi123.stdout:Verifying time synchronization is in place... 2024-05-08T05:56:00.528 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-08T05:56:00.528 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-08T05:56:00.545 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-08T05:56:00.545 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout inactive 2024-05-08T05:56:00.564 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout enabled 2024-05-08T05:56:00.584 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout active 2024-05-08T05:56:00.584 INFO:teuthology.orchestra.run.smithi123.stdout:Unit chronyd.service is enabled and running 2024-05-08T05:56:00.584 INFO:teuthology.orchestra.run.smithi123.stdout:Repeating the final host check... 2024-05-08T05:56:00.639 INFO:teuthology.orchestra.run.smithi123.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-08T05:56:00.640 INFO:teuthology.orchestra.run.smithi123.stdout:podman (/bin/podman) version 4.9.4 is present 2024-05-08T05:56:00.640 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl is present 2024-05-08T05:56:00.640 INFO:teuthology.orchestra.run.smithi123.stdout:lvcreate is present 2024-05-08T05:56:00.654 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-08T05:56:00.655 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-08T05:56:00.671 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-08T05:56:00.672 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout inactive 2024-05-08T05:56:00.696 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout enabled 2024-05-08T05:56:00.717 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout active 2024-05-08T05:56:00.718 INFO:teuthology.orchestra.run.smithi123.stdout:Unit chronyd.service is enabled and running 2024-05-08T05:56:00.718 INFO:teuthology.orchestra.run.smithi123.stdout:Host looks OK 2024-05-08T05:56:00.718 INFO:teuthology.orchestra.run.smithi123.stdout:Cluster fsid: 85cfb2e6-0cff-11ef-bc97-c7b262605968 2024-05-08T05:56:00.719 INFO:teuthology.orchestra.run.smithi123.stdout:Acquiring lock 140241046535616 on /run/cephadm/85cfb2e6-0cff-11ef-bc97-c7b262605968.lock 2024-05-08T05:56:00.719 INFO:teuthology.orchestra.run.smithi123.stdout:Lock 140241046535616 acquired on /run/cephadm/85cfb2e6-0cff-11ef-bc97-c7b262605968.lock 2024-05-08T05:56:00.721 INFO:teuthology.orchestra.run.smithi123.stdout:Verifying IP 172.21.15.123 port 3300 ... 2024-05-08T05:56:00.721 INFO:teuthology.orchestra.run.smithi123.stdout:Verifying IP 172.21.15.123 port 6789 ... 2024-05-08T05:56:00.722 INFO:teuthology.orchestra.run.smithi123.stdout:Base mon IP(s) is [172.21.15.123:3300, 172.21.15.123:6789], mon addrv is [v2:172.21.15.123:3300,v1:172.21.15.123:6789] 2024-05-08T05:56:00.736 INFO:teuthology.orchestra.run.smithi123.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.123 metric 100 2024-05-08T05:56:00.737 INFO:teuthology.orchestra.run.smithi123.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.123 metric 100 2024-05-08T05:56:00.749 INFO:teuthology.orchestra.run.smithi123.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-08T05:56:00.749 INFO:teuthology.orchestra.run.smithi123.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-05-08T05:56:00.749 INFO:teuthology.orchestra.run.smithi123.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1796sec hoplimit 64 pref medium 2024-05-08T05:56:00.760 INFO:teuthology.orchestra.run.smithi123.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-08T05:56:00.761 INFO:teuthology.orchestra.run.smithi123.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-08T05:56:00.761 INFO:teuthology.orchestra.run.smithi123.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-08T05:56:00.761 INFO:teuthology.orchestra.run.smithi123.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-05-08T05:56:00.761 INFO:teuthology.orchestra.run.smithi123.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d3f1/64 scope link 2024-05-08T05:56:00.761 INFO:teuthology.orchestra.run.smithi123.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-08T05:56:00.764 INFO:teuthology.orchestra.run.smithi123.stdout:Mon IP `172.21.15.123` is in CIDR network `172.21.0.0/20` 2024-05-08T05:56:00.764 INFO:teuthology.orchestra.run.smithi123.stdout:Mon IP `172.21.15.123` is in CIDR network `172.21.0.0/20` 2024-05-08T05:56:00.764 INFO:teuthology.orchestra.run.smithi123.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-08T05:56:00.770 INFO:teuthology.orchestra.run.smithi123.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-08T05:56:00.770 INFO:teuthology.orchestra.run.smithi123.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a... 2024-05-08T05:56:01.467 INFO:teuthology.orchestra.run.smithi123.stdout:/bin/podman: stdout 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce 2024-05-08T05:56:01.467 INFO:teuthology.orchestra.run.smithi123.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a... 2024-05-08T05:56:01.467 INFO:teuthology.orchestra.run.smithi123.stdout:/bin/podman: stderr Getting image source signatures 2024-05-08T05:56:01.467 INFO:teuthology.orchestra.run.smithi123.stdout:/bin/podman: stderr Copying blob sha256:bf5b95bd68251da39c6cd2eb96c6ba4138bb37a62bdc39287908b25e92047d7e 2024-05-08T05:56:01.467 INFO:teuthology.orchestra.run.smithi123.stdout:/bin/podman: stderr Copying blob sha256:f08a3efca46759c0efc03dda6bb4c9adf7daf77004822005f2f0da3e8d1877bb 2024-05-08T05:56:01.467 INFO:teuthology.orchestra.run.smithi123.stdout:/bin/podman: stderr Copying config sha256:352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce 2024-05-08T05:56:01.468 INFO:teuthology.orchestra.run.smithi123.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-08T05:56:02.841 INFO:teuthology.orchestra.run.smithi123.stdout:ceph: stdout ceph version 19.0.0-3538-gbee299f4 (bee299f4ac344e4c45e550d7a80abcadd7bc3d5a) squid (dev) 2024-05-08T05:56:02.841 INFO:teuthology.orchestra.run.smithi123.stdout:Ceph version: ceph version 19.0.0-3538-gbee299f4 (bee299f4ac344e4c45e550d7a80abcadd7bc3d5a) squid (dev) 2024-05-08T05:56:02.841 INFO:teuthology.orchestra.run.smithi123.stdout:Extracting ceph user uid/gid from container image... 2024-05-08T05:56:04.078 INFO:teuthology.orchestra.run.smithi123.stdout:stat: stdout 167 167 2024-05-08T05:56:04.078 INFO:teuthology.orchestra.run.smithi123.stdout:Creating initial keys... 2024-05-08T05:56:05.325 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph-authtool: stdout AQD0EztmgH/cIRAA/OW9WvQvU/Hxh8RGEto/Vg== 2024-05-08T05:56:06.496 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph-authtool: stdout AQD1Eztm/dPGMxAAY+mDbiJhg3NuC3dlNIGonw== 2024-05-08T05:56:07.692 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph-authtool: stdout AQD3EztmD1U+AhAAWM6LhTJENNzuPTP0MpzCGg== 2024-05-08T05:56:07.694 INFO:teuthology.orchestra.run.smithi123.stdout:Creating initial monmap... 2024-05-08T05:56:08.880 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-08T05:56:08.880 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-08T05:56:08.880 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 85cfb2e6-0cff-11ef-bc97-c7b262605968 2024-05-08T05:56:08.881 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-08T05:56:08.881 INFO:teuthology.orchestra.run.smithi123.stdout:monmaptool for a [v2:172.21.15.123:3300,v1:172.21.15.123:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-08T05:56:08.881 INFO:teuthology.orchestra.run.smithi123.stdout:setting min_mon_release = quincy 2024-05-08T05:56:08.881 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/monmaptool: set fsid to 85cfb2e6-0cff-11ef-bc97-c7b262605968 2024-05-08T05:56:08.881 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-08T05:56:08.881 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T05:56:08.881 INFO:teuthology.orchestra.run.smithi123.stdout:Creating mon... 2024-05-08T05:56:10.086 INFO:teuthology.orchestra.run.smithi123.stdout:create mon.a on 2024-05-08T05:56:10.495 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-08T05:56:10.742 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-08T05:56:11.033 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968.target. 2024-05-08T05:56:11.033 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968.target. 2024-05-08T05:56:11.410 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mon.a 2024-05-08T05:56:11.410 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stderr Failed to reset failed state of unit ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mon.a.service: Unit ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mon.a.service not loaded. 2024-05-08T05:56:11.703 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968.target.wants/ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mon.a.service → /etc/systemd/system/ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@.service. 2024-05-08T05:56:12.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:12 smithi123 podman[29404]: 2024-05-08 05:56:12.128464414 +0000 UTC m=+0.160016661 container create 582ef545622ce2e06e39ac9d304ec499192393a2b29faf068afac1b28afa5e35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-08T05:56:12.575 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T05:56:12.575 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout disabled 2024-05-08T05:56:12.590 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T05:56:12.590 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout inactive 2024-05-08T05:56:12.590 INFO:teuthology.orchestra.run.smithi123.stdout:firewalld.service is not enabled 2024-05-08T05:56:12.590 INFO:teuthology.orchestra.run.smithi123.stdout:Not possible to enable service . firewalld.service is not available 2024-05-08T05:56:12.591 INFO:teuthology.orchestra.run.smithi123.stdout:Waiting for mon to start... 2024-05-08T05:56:12.591 INFO:teuthology.orchestra.run.smithi123.stdout:Waiting for mon... 2024-05-08T05:56:12.739 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:12 smithi123 podman[29404]: 2024-05-08 05:56:12.456757969 +0000 UTC m=+0.488310236 container init 582ef545622ce2e06e39ac9d304ec499192393a2b29faf068afac1b28afa5e35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mon-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0) 2024-05-08T05:56:12.739 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:12 smithi123 podman[29404]: 2024-05-08 05:56:12.46706902 +0000 UTC m=+0.498621269 container start 582ef545622ce2e06e39ac9d304ec499192393a2b29faf068afac1b28afa5e35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mon-a, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-08T05:56:12.739 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:12 smithi123 bash[29404]: 582ef545622ce2e06e39ac9d304ec499192393a2b29faf068afac1b28afa5e35 2024-05-08T05:56:12.739 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:12 smithi123 ceph-mon[29423]: mkfs 85cfb2e6-0cff-11ef-bc97-c7b262605968 2024-05-08T05:56:12.739 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:12 smithi123 ceph-mon[29423]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-08T05:56:12.739 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:12 smithi123 systemd[1]: Started Ceph mon.a for 85cfb2e6-0cff-11ef-bc97-c7b262605968. 2024-05-08T05:56:13.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:13 smithi123 ceph-mon[29423]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-08T05:56:13.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:13 smithi123 ceph-mon[29423]: monmap epoch 1 2024-05-08T05:56:13.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:13 smithi123 ceph-mon[29423]: fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 2024-05-08T05:56:13.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:13 smithi123 ceph-mon[29423]: last_changed 2024-05-08T05:56:08.254675+0000 2024-05-08T05:56:13.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:13 smithi123 ceph-mon[29423]: created 2024-05-08T05:56:08.254675+0000 2024-05-08T05:56:13.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:13 smithi123 ceph-mon[29423]: min_mon_release 19 (squid) 2024-05-08T05:56:13.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:13 smithi123 ceph-mon[29423]: election_strategy: 1 2024-05-08T05:56:13.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:13 smithi123 ceph-mon[29423]: 0: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.a 2024-05-08T05:56:13.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:13 smithi123 ceph-mon[29423]: fsmap 2024-05-08T05:56:13.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:13 smithi123 ceph-mon[29423]: osdmap e1: 0 total, 0 up, 0 in 2024-05-08T05:56:13.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:13 smithi123 ceph-mon[29423]: mgrmap e1: no daemons active 2024-05-08T05:56:13.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:13 smithi123 ceph-mon[29423]: from='client.? 172.21.15.123:0/2344671904' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-08T05:56:13.988 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout cluster: 2024-05-08T05:56:13.989 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout id: 85cfb2e6-0cff-11ef-bc97-c7b262605968 2024-05-08T05:56:13.989 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-08T05:56:13.989 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 2024-05-08T05:56:13.989 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout services: 2024-05-08T05:56:13.989 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.747689s) 2024-05-08T05:56:13.991 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-08T05:56:13.991 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-08T05:56:13.991 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 2024-05-08T05:56:13.991 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout data: 2024-05-08T05:56:13.992 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-08T05:56:13.992 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-08T05:56:13.992 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-08T05:56:13.992 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout pgs: 2024-05-08T05:56:13.992 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 2024-05-08T05:56:13.992 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.200+0000 7f3eff000700 1 Processor -- start 2024-05-08T05:56:13.992 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.200+0000 7f3eff000700 1 -- start start 2024-05-08T05:56:13.992 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.200+0000 7f3eff000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef81077a0 0x7f3ef8107bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:13.992 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.200+0000 7f3eff000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef807ce50 con 0x7f3ef81077a0 2024-05-08T05:56:13.992 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.201+0000 7f3ef7e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef81077a0 0x7f3ef8107bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:13.992 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.201+0000 7f3ef7e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef81077a0 0x7f3ef8107bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60150/0 (socket says 172.21.15.123:60150) 2024-05-08T05:56:13.997 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.201+0000 7f3ef7e00700 1 -- 172.21.15.123:0/934224291 learned_addr learned my addr 172.21.15.123:0/934224291 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:56:13.997 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.202+0000 7f3ef7e00700 1 -- 172.21.15.123:0/934224291 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef8108100 con 0x7f3ef81077a0 2024-05-08T05:56:13.997 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.202+0000 7f3ef7e00700 1 --2- 172.21.15.123:0/934224291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef81077a0 0x7f3ef8107bc0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3ee80098a0 tx=0x7f3ee8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9f0ca9f57700e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:13.997 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.202+0000 7f3ef6a00700 1 -- 172.21.15.123:0/934224291 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ee8004160 con 0x7f3ef81077a0 2024-05-08T05:56:13.997 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.202+0000 7f3ef6a00700 1 -- 172.21.15.123:0/934224291 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ee8004300 con 0x7f3ef81077a0 2024-05-08T05:56:13.997 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.202+0000 7f3ef6a00700 1 -- 172.21.15.123:0/934224291 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ee80036a0 con 0x7f3ef81077a0 2024-05-08T05:56:13.997 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.203+0000 7f3eff000700 1 -- 172.21.15.123:0/934224291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef81077a0 msgr2=0x7f3ef8107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:13.997 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.203+0000 7f3eff000700 1 --2- 172.21.15.123:0/934224291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef81077a0 0x7f3ef8107bc0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3ee80098a0 tx=0x7f3ee8009bb0 comp rx=0 tx=0).stop 2024-05-08T05:56:13.997 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.203+0000 7f3eff000700 1 -- 172.21.15.123:0/934224291 shutdown_connections 2024-05-08T05:56:13.997 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.203+0000 7f3eff000700 1 --2- 172.21.15.123:0/934224291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef81077a0 0x7f3ef8107bc0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:13.998 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.203+0000 7f3eff000700 1 -- 172.21.15.123:0/934224291 >> 172.21.15.123:0/934224291 conn(0x7f3ef8103380 msgr2=0x7f3ef81057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:13.998 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.203+0000 7f3eff000700 1 -- 172.21.15.123:0/934224291 shutdown_connections 2024-05-08T05:56:13.998 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.203+0000 7f3eff000700 1 -- 172.21.15.123:0/934224291 wait complete. 2024-05-08T05:56:13.998 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.204+0000 7f3eff000700 1 Processor -- start 2024-05-08T05:56:13.998 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.205+0000 7f3eff000700 1 -- start start 2024-05-08T05:56:13.998 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.205+0000 7f3eff000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef81077a0 0x7f3ef819e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:13.998 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.205+0000 7f3eff000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef8108630 con 0x7f3ef81077a0 2024-05-08T05:56:13.998 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.205+0000 7f3ef7e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef81077a0 0x7f3ef819e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:13.998 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.205+0000 7f3ef7e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef81077a0 0x7f3ef819e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60158/0 (socket says 172.21.15.123:60158) 2024-05-08T05:56:13.998 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.205+0000 7f3ef7e00700 1 -- 172.21.15.123:0/2344671904 learned_addr learned my addr 172.21.15.123:0/2344671904 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:56:13.998 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.206+0000 7f3ef7e00700 1 -- 172.21.15.123:0/2344671904 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef819ec80 con 0x7f3ef81077a0 2024-05-08T05:56:13.998 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.206+0000 7f3ef7e00700 1 --2- 172.21.15.123:0/2344671904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef81077a0 0x7f3ef819e740 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8000c00 tx=0x7f3ee8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:13.998 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.206+0000 7f3ef4c00700 1 -- 172.21.15.123:0/2344671904 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ee8003db0 con 0x7f3ef81077a0 2024-05-08T05:56:13.999 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.206+0000 7f3ef4c00700 1 -- 172.21.15.123:0/2344671904 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ee8011630 con 0x7f3ef81077a0 2024-05-08T05:56:13.999 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.206+0000 7f3ef4c00700 1 -- 172.21.15.123:0/2344671904 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ee8011950 con 0x7f3ef81077a0 2024-05-08T05:56:13.999 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.206+0000 7f3eff000700 1 -- 172.21.15.123:0/2344671904 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ef819ef10 con 0x7f3ef81077a0 2024-05-08T05:56:13.999 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.206+0000 7f3eff000700 1 -- 172.21.15.123:0/2344671904 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ef81a1c00 con 0x7f3ef81077a0 2024-05-08T05:56:13.999 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.207+0000 7f3ef4c00700 1 -- 172.21.15.123:0/2344671904 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3ee8011af0 con 0x7f3ef81077a0 2024-05-08T05:56:13.999 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.207+0000 7f3ef4c00700 1 -- 172.21.15.123:0/2344671904 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3ee801c450 con 0x7f3ef81077a0 2024-05-08T05:56:13.999 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.207+0000 7f3eff000700 1 -- 172.21.15.123:0/2344671904 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ef8064b00 con 0x7f3ef81077a0 2024-05-08T05:56:13.999 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.211+0000 7f3ef4c00700 1 -- 172.21.15.123:0/2344671904 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f3ee8032020 con 0x7f3ef81077a0 2024-05-08T05:56:13.999 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.289+0000 7f3eff000700 1 -- 172.21.15.123:0/2344671904 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f3ef81a20f0 con 0x7f3ef81077a0 2024-05-08T05:56:13.999 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.290+0000 7f3ef4c00700 1 -- 172.21.15.123:0/2344671904 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f3ee8004260 con 0x7f3ef81077a0 2024-05-08T05:56:13.999 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.291+0000 7f3eff000700 1 -- 172.21.15.123:0/2344671904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef81077a0 msgr2=0x7f3ef819e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:13.999 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.291+0000 7f3eff000700 1 --2- 172.21.15.123:0/2344671904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef81077a0 0x7f3ef819e740 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8000c00 tx=0x7f3ee8004750 comp rx=0 tx=0).stop 2024-05-08T05:56:13.999 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.291+0000 7f3eff000700 1 -- 172.21.15.123:0/2344671904 shutdown_connections 2024-05-08T05:56:13.999 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.291+0000 7f3eff000700 1 --2- 172.21.15.123:0/2344671904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ef81077a0 0x7f3ef819e740 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.000 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.291+0000 7f3eff000700 1 -- 172.21.15.123:0/2344671904 >> 172.21.15.123:0/2344671904 conn(0x7f3ef8103380 msgr2=0x7f3ef8194d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:14.000 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.292+0000 7f3eff000700 1 -- 172.21.15.123:0/2344671904 shutdown_connections 2024-05-08T05:56:14.000 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:13.292+0000 7f3eff000700 1 -- 172.21.15.123:0/2344671904 wait complete. 2024-05-08T05:56:14.000 INFO:teuthology.orchestra.run.smithi123.stdout:mon is available 2024-05-08T05:56:14.000 INFO:teuthology.orchestra.run.smithi123.stdout:Assimilating anything we can from ceph.conf... 2024-05-08T05:56:14.874 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:14 smithi123 ceph-mon[29423]: from='client.? 172.21.15.123:0/2794400679' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-08T05:56:14.874 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:14 smithi123 ceph-mon[29423]: from='client.? 172.21.15.123:0/2794400679' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-08T05:56:15.446 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 2024-05-08T05:56:15.446 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout [global] 2024-05-08T05:56:15.446 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout fsid = 85cfb2e6-0cff-11ef-bc97-c7b262605968 2024-05-08T05:56:15.446 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-08T05:56:15.446 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.123:3300,v1:172.21.15.123:6789] 2024-05-08T05:56:15.446 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-08T05:56:15.446 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-08T05:56:15.446 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-08T05:56:15.447 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-08T05:56:15.447 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 2024-05-08T05:56:15.447 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-08T05:56:15.447 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-08T05:56:15.447 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 2024-05-08T05:56:15.447 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout [osd] 2024-05-08T05:56:15.447 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-08T05:56:15.447 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-08T05:56:15.447 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-08T05:56:15.447 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-08T05:56:15.447 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-08T05:56:15.447 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.634+0000 7f7922000700 1 Processor -- start 2024-05-08T05:56:15.447 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.634+0000 7f7922000700 1 -- start start 2024-05-08T05:56:15.447 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.634+0000 7f7922000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f791c07ff60 0x7f791c080380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:15.447 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.634+0000 7f7922000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f791c080950 con 0x7f791c07ff60 2024-05-08T05:56:15.448 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.635+0000 7f791b400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f791c07ff60 0x7f791c080380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:15.448 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.635+0000 7f791b400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f791c07ff60 0x7f791c080380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60170/0 (socket says 172.21.15.123:60170) 2024-05-08T05:56:15.448 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.635+0000 7f791b400700 1 -- 172.21.15.123:0/3620645239 learned_addr learned my addr 172.21.15.123:0/3620645239 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:56:15.448 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.635+0000 7f791b400700 1 -- 172.21.15.123:0/3620645239 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f791c081100 con 0x7f791c07ff60 2024-05-08T05:56:15.448 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.636+0000 7f791b400700 1 --2- 172.21.15.123:0/3620645239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f791c07ff60 0x7f791c080380 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f790c0098a0 tx=0x7f790c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8e679fd631322b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:15.448 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.636+0000 7f791a000700 1 -- 172.21.15.123:0/3620645239 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f790c004160 con 0x7f791c07ff60 2024-05-08T05:56:15.448 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.636+0000 7f791a000700 1 -- 172.21.15.123:0/3620645239 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f790c004300 con 0x7f791c07ff60 2024-05-08T05:56:15.448 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.636+0000 7f791a000700 1 -- 172.21.15.123:0/3620645239 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f790c0036a0 con 0x7f791c07ff60 2024-05-08T05:56:15.448 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.637+0000 7f7922000700 1 -- 172.21.15.123:0/3620645239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f791c07ff60 msgr2=0x7f791c080380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:15.448 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.637+0000 7f7922000700 1 --2- 172.21.15.123:0/3620645239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f791c07ff60 0x7f791c080380 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f790c0098a0 tx=0x7f790c009bb0 comp rx=0 tx=0).stop 2024-05-08T05:56:15.448 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.637+0000 7f7922000700 1 -- 172.21.15.123:0/3620645239 shutdown_connections 2024-05-08T05:56:15.448 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.637+0000 7f7922000700 1 --2- 172.21.15.123:0/3620645239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f791c07ff60 0x7f791c080380 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:15.448 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.637+0000 7f7922000700 1 -- 172.21.15.123:0/3620645239 >> 172.21.15.123:0/3620645239 conn(0x7f791c07e6e0 msgr2=0x7f791c07eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:15.449 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.637+0000 7f7922000700 1 -- 172.21.15.123:0/3620645239 shutdown_connections 2024-05-08T05:56:15.449 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.637+0000 7f7922000700 1 -- 172.21.15.123:0/3620645239 wait complete. 2024-05-08T05:56:15.449 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.638+0000 7f7922000700 1 Processor -- start 2024-05-08T05:56:15.449 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.638+0000 7f7922000700 1 -- start start 2024-05-08T05:56:15.449 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.639+0000 7f7922000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f791c07ff60 0x7f791c1a2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:15.449 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.639+0000 7f7922000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f791c0815d0 con 0x7f791c07ff60 2024-05-08T05:56:15.449 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.639+0000 7f791b400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f791c07ff60 0x7f791c1a2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:15.449 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.639+0000 7f791b400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f791c07ff60 0x7f791c1a2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60180/0 (socket says 172.21.15.123:60180) 2024-05-08T05:56:15.449 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.639+0000 7f791b400700 1 -- 172.21.15.123:0/2794400679 learned_addr learned my addr 172.21.15.123:0/2794400679 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:56:15.449 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.639+0000 7f791b400700 1 -- 172.21.15.123:0/2794400679 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f791c1a3120 con 0x7f791c07ff60 2024-05-08T05:56:15.449 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.639+0000 7f791b400700 1 --2- 172.21.15.123:0/2794400679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f791c07ff60 0x7f791c1a2be0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f790c000c00 tx=0x7f790c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:15.449 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.640+0000 7f7913e00700 1 -- 172.21.15.123:0/2794400679 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f790c003db0 con 0x7f791c07ff60 2024-05-08T05:56:15.449 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.640+0000 7f7913e00700 1 -- 172.21.15.123:0/2794400679 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f790c011630 con 0x7f791c07ff60 2024-05-08T05:56:15.449 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.640+0000 7f7913e00700 1 -- 172.21.15.123:0/2794400679 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f790c011950 con 0x7f791c07ff60 2024-05-08T05:56:15.450 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.640+0000 7f7922000700 1 -- 172.21.15.123:0/2794400679 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f791c1a33b0 con 0x7f791c07ff60 2024-05-08T05:56:15.450 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.640+0000 7f7922000700 1 -- 172.21.15.123:0/2794400679 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f791c1a60a0 con 0x7f791c07ff60 2024-05-08T05:56:15.450 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.640+0000 7f7913e00700 1 -- 172.21.15.123:0/2794400679 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f790c01a460 con 0x7f791c07ff60 2024-05-08T05:56:15.450 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.640+0000 7f7913e00700 1 -- 172.21.15.123:0/2794400679 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f790c01b780 con 0x7f791c07ff60 2024-05-08T05:56:15.450 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.641+0000 7f7922000700 1 -- 172.21.15.123:0/2794400679 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f791c064b00 con 0x7f791c07ff60 2024-05-08T05:56:15.450 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.644+0000 7f7913e00700 1 -- 172.21.15.123:0/2794400679 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f790c031020 con 0x7f791c07ff60 2024-05-08T05:56:15.450 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.719+0000 7f7922000700 1 -- 172.21.15.123:0/2794400679 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f791c1a6270 con 0x7f791c07ff60 2024-05-08T05:56:15.450 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.725+0000 7f7913e00700 1 -- 172.21.15.123:0/2794400679 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+467 (secure 0 0 0) 0x7f790c01b9d0 con 0x7f791c07ff60 2024-05-08T05:56:15.450 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.726+0000 7f7913e00700 1 -- 172.21.15.123:0/2794400679 <== mon.0 v2:172.21.15.123:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f790c01ad20 con 0x7f791c07ff60 2024-05-08T05:56:15.450 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.727+0000 7f7922000700 1 -- 172.21.15.123:0/2794400679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f791c07ff60 msgr2=0x7f791c1a2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:15.450 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.727+0000 7f7922000700 1 --2- 172.21.15.123:0/2794400679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f791c07ff60 0x7f791c1a2be0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f790c000c00 tx=0x7f790c004750 comp rx=0 tx=0).stop 2024-05-08T05:56:15.450 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.728+0000 7f7922000700 1 -- 172.21.15.123:0/2794400679 shutdown_connections 2024-05-08T05:56:15.450 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.728+0000 7f7922000700 1 --2- 172.21.15.123:0/2794400679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f791c07ff60 0x7f791c1a2be0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:15.450 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.728+0000 7f7922000700 1 -- 172.21.15.123:0/2794400679 >> 172.21.15.123:0/2794400679 conn(0x7f791c07e6e0 msgr2=0x7f791c108e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:15.450 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.728+0000 7f7922000700 1 -- 172.21.15.123:0/2794400679 shutdown_connections 2024-05-08T05:56:15.451 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:14.728+0000 7f7922000700 1 -- 172.21.15.123:0/2794400679 wait complete. 2024-05-08T05:56:15.451 INFO:teuthology.orchestra.run.smithi123.stdout:Generating new minimal ceph.conf... 2024-05-08T05:56:16.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:16 smithi123 ceph-mon[29423]: from='client.? 172.21.15.123:0/3915025417' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:56:16.869 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.061+0000 7f0b67600700 1 Processor -- start 2024-05-08T05:56:16.870 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.062+0000 7f0b67600700 1 -- start start 2024-05-08T05:56:16.870 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.062+0000 7f0b67600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b60107760 0x7f0b60107b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:16.870 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.062+0000 7f0b67600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6007d760 con 0x7f0b60107760 2024-05-08T05:56:16.870 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.062+0000 7f0b64c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b60107760 0x7f0b60107b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:16.870 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.062+0000 7f0b64c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b60107760 0x7f0b60107b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56766/0 (socket says 172.21.15.123:56766) 2024-05-08T05:56:16.870 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.062+0000 7f0b64c00700 1 -- 172.21.15.123:0/1824374773 learned_addr learned my addr 172.21.15.123:0/1824374773 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:56:16.870 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.063+0000 7f0b64c00700 1 -- 172.21.15.123:0/1824374773 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b601080c0 con 0x7f0b60107760 2024-05-08T05:56:16.870 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.063+0000 7f0b64c00700 1 --2- 172.21.15.123:0/1824374773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b60107760 0x7f0b60107b80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0b540098a0 tx=0x7f0b54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90616faa1850a4c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:16.870 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.063+0000 7f0b5f400700 1 -- 172.21.15.123:0/1824374773 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b54004160 con 0x7f0b60107760 2024-05-08T05:56:16.870 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.063+0000 7f0b5f400700 1 -- 172.21.15.123:0/1824374773 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0b54004300 con 0x7f0b60107760 2024-05-08T05:56:16.870 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.063+0000 7f0b5f400700 1 -- 172.21.15.123:0/1824374773 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b54011af0 con 0x7f0b60107760 2024-05-08T05:56:16.871 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.064+0000 7f0b67600700 1 -- 172.21.15.123:0/1824374773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b60107760 msgr2=0x7f0b60107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:16.871 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.064+0000 7f0b67600700 1 --2- 172.21.15.123:0/1824374773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b60107760 0x7f0b60107b80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0b540098a0 tx=0x7f0b54009bb0 comp rx=0 tx=0).stop 2024-05-08T05:56:16.871 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.064+0000 7f0b67600700 1 -- 172.21.15.123:0/1824374773 shutdown_connections 2024-05-08T05:56:16.871 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.064+0000 7f0b67600700 1 --2- 172.21.15.123:0/1824374773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b60107760 0x7f0b60107b80 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:16.871 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.064+0000 7f0b67600700 1 -- 172.21.15.123:0/1824374773 >> 172.21.15.123:0/1824374773 conn(0x7f0b60103440 msgr2=0x7f0b60105860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:16.871 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.064+0000 7f0b67600700 1 -- 172.21.15.123:0/1824374773 shutdown_connections 2024-05-08T05:56:16.871 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.064+0000 7f0b67600700 1 -- 172.21.15.123:0/1824374773 wait complete. 2024-05-08T05:56:16.871 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.065+0000 7f0b67600700 1 Processor -- start 2024-05-08T05:56:16.871 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.066+0000 7f0b67600700 1 -- start start 2024-05-08T05:56:16.871 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.066+0000 7f0b67600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b60107760 0x7f0b6019f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:16.871 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.066+0000 7f0b67600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0b60109080 con 0x7f0b60107760 2024-05-08T05:56:16.871 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.066+0000 7f0b64c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b60107760 0x7f0b6019f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:16.871 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.066+0000 7f0b64c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b60107760 0x7f0b6019f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56776/0 (socket says 172.21.15.123:56776) 2024-05-08T05:56:16.871 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.066+0000 7f0b64c00700 1 -- 172.21.15.123:0/3915025417 learned_addr learned my addr 172.21.15.123:0/3915025417 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:56:16.871 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.066+0000 7f0b64c00700 1 -- 172.21.15.123:0/3915025417 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b6019f540 con 0x7f0b60107760 2024-05-08T05:56:16.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.067+0000 7f0b64c00700 1 --2- 172.21.15.123:0/3915025417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b60107760 0x7f0b6019f000 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0b54011ac0 tx=0x7f0b54011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:16.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.067+0000 7f0b5d600700 1 -- 172.21.15.123:0/3915025417 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b54003970 con 0x7f0b60107760 2024-05-08T05:56:16.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.067+0000 7f0b5d600700 1 -- 172.21.15.123:0/3915025417 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0b54003b10 con 0x7f0b60107760 2024-05-08T05:56:16.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.067+0000 7f0b67600700 1 -- 172.21.15.123:0/3915025417 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b6019f7d0 con 0x7f0b60107760 2024-05-08T05:56:16.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.067+0000 7f0b5d600700 1 -- 172.21.15.123:0/3915025417 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b54024610 con 0x7f0b60107760 2024-05-08T05:56:16.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.067+0000 7f0b67600700 1 -- 172.21.15.123:0/3915025417 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b601a24c0 con 0x7f0b60107760 2024-05-08T05:56:16.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.067+0000 7f0b5d600700 1 -- 172.21.15.123:0/3915025417 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0b54024bd0 con 0x7f0b60107760 2024-05-08T05:56:16.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.068+0000 7f0b5d600700 1 -- 172.21.15.123:0/3915025417 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0b54019700 con 0x7f0b60107760 2024-05-08T05:56:16.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.069+0000 7f0b67600700 1 -- 172.21.15.123:0/3915025417 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b60064b00 con 0x7f0b60107760 2024-05-08T05:56:16.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.071+0000 7f0b5d600700 1 -- 172.21.15.123:0/3915025417 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f0b54043ec0 con 0x7f0b60107760 2024-05-08T05:56:16.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.147+0000 7f0b67600700 1 -- 172.21.15.123:0/3915025417 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f0b600008d0 con 0x7f0b60107760 2024-05-08T05:56:16.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.148+0000 7f0b5d600700 1 -- 172.21.15.123:0/3915025417 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7f0b54032030 con 0x7f0b60107760 2024-05-08T05:56:16.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.150+0000 7f0b67600700 1 -- 172.21.15.123:0/3915025417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b60107760 msgr2=0x7f0b6019f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:16.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.150+0000 7f0b67600700 1 --2- 172.21.15.123:0/3915025417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b60107760 0x7f0b6019f000 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0b54011ac0 tx=0x7f0b54011ba0 comp rx=0 tx=0).stop 2024-05-08T05:56:16.872 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.150+0000 7f0b67600700 1 -- 172.21.15.123:0/3915025417 shutdown_connections 2024-05-08T05:56:16.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.150+0000 7f0b67600700 1 --2- 172.21.15.123:0/3915025417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b60107760 0x7f0b6019f000 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:16.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.150+0000 7f0b67600700 1 -- 172.21.15.123:0/3915025417 >> 172.21.15.123:0/3915025417 conn(0x7f0b60103440 msgr2=0x7f0b60195c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:16.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.150+0000 7f0b67600700 1 -- 172.21.15.123:0/3915025417 shutdown_connections 2024-05-08T05:56:16.873 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:16.150+0000 7f0b67600700 1 -- 172.21.15.123:0/3915025417 wait complete. 2024-05-08T05:56:16.873 INFO:teuthology.orchestra.run.smithi123.stdout:Restarting the monitor... 2024-05-08T05:56:17.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:16 smithi123 systemd[1]: Stopping Ceph mon.a for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T05:56:17.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:17 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mon-a[29419]: 2024-05-08T05:56:17.389+0000 7fdc48e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T05:56:17.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:17 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mon-a[29419]: 2024-05-08T05:56:17.389+0000 7fdc48e00700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-08T05:56:17.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:17 smithi123 podman[29665]: 2024-05-08 05:56:17.606138183 +0000 UTC m=+0.379708112 container died 582ef545622ce2e06e39ac9d304ec499192393a2b29faf068afac1b28afa5e35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-08T05:56:26.865 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:26 smithi123 podman[29665]: 2024-05-08 05:56:26.68924802 +0000 UTC m=+9.462817939 container cleanup 582ef545622ce2e06e39ac9d304ec499192393a2b29faf068afac1b28afa5e35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mon-a, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-08T05:56:26.865 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:26 smithi123 bash[29665]: ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mon-a 2024-05-08T05:56:27.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:26 smithi123 podman[29679]: 2024-05-08 05:56:26.865049029 +0000 UTC m=+9.238613459 container remove 582ef545622ce2e06e39ac9d304ec499192393a2b29faf068afac1b28afa5e35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mon-a, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-08T05:56:27.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:27 smithi123 systemd[1]: ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-08T05:56:27.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:27 smithi123 systemd[1]: Stopped Ceph mon.a for 85cfb2e6-0cff-11ef-bc97-c7b262605968. 2024-05-08T05:56:27.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:27 smithi123 systemd[1]: ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mon.a.service: Consumed 1.917s CPU time. 2024-05-08T05:56:27.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:27 smithi123 systemd[1]: Starting Ceph mon.a for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T05:56:28.095 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:27 smithi123 podman[29769]: 2024-05-08 05:56:27.657752709 +0000 UTC m=+0.033931682 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:56:28.095 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:27 smithi123 podman[29769]: 2024-05-08 05:56:27.790818978 +0000 UTC m=+0.166997984 container create f0e5e4ca808065702ad1f6b524525c89e14c8ee4d9920e81144d7e309f8874d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mon-a, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-05-08T05:56:28.095 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 podman[29769]: 2024-05-08 05:56:28.094500037 +0000 UTC m=+0.470679063 container init f0e5e4ca808065702ad1f6b524525c89e14c8ee4d9920e81144d7e309f8874d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mon-a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS) 2024-05-08T05:56:28.204 INFO:teuthology.orchestra.run.smithi123.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-08T05:56:28.355 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 podman[29769]: 2024-05-08 05:56:28.105298177 +0000 UTC m=+0.481477156 container start f0e5e4ca808065702ad1f6b524525c89e14c8ee4d9920e81144d7e309f8874d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD) 2024-05-08T05:56:28.355 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: set uid:gid to 167:167 (ceph:ceph) 2024-05-08T05:56:28.355 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: ceph version 19.0.0-3538-gbee299f4 (bee299f4ac344e4c45e550d7a80abcadd7bc3d5a) squid (dev), process ceph-mon, pid 2 2024-05-08T05:56:28.355 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: pidfile_write: ignore empty --pid-file 2024-05-08T05:56:28.355 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: load: jerasure load: lrc 2024-05-08T05:56:28.355 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: RocksDB version: 7.9.2 2024-05-08T05:56:28.356 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Git sha 0 2024-05-08T05:56:28.356 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Compile date 2024-05-07 11:18:56 2024-05-08T05:56:28.356 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: DB SUMMARY 2024-05-08T05:56:28.356 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: DB Session ID: IJ5UZQ6YUQ0A6KFI279J 2024-05-08T05:56:28.356 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: CURRENT file: CURRENT 2024-05-08T05:56:28.356 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: IDENTITY file: IDENTITY 2024-05-08T05:56:28.356 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-08T05:56:28.356 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-08T05:56:28.356 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105270 ; 2024-05-08T05:56:28.356 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.error_if_exists: 0 2024-05-08T05:56:28.356 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.create_if_missing: 0 2024-05-08T05:56:28.356 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.paranoid_checks: 1 2024-05-08T05:56:28.356 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-08T05:56:28.357 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-08T05:56:28.357 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-08T05:56:28.357 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.env: 0x563c1fc05cc0 2024-05-08T05:56:28.357 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.fs: PosixFileSystem 2024-05-08T05:56:28.357 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.info_log: 0x563c3e867de0 2024-05-08T05:56:28.357 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.max_file_opening_threads: 16 2024-05-08T05:56:28.357 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.statistics: (nil) 2024-05-08T05:56:28.357 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.use_fsync: 0 2024-05-08T05:56:28.357 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.max_log_file_size: 0 2024-05-08T05:56:28.357 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-08T05:56:28.357 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-08T05:56:28.357 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.keep_log_file_num: 1000 2024-05-08T05:56:28.357 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.recycle_log_file_num: 0 2024-05-08T05:56:28.357 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.allow_fallocate: 1 2024-05-08T05:56:28.358 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.allow_mmap_reads: 0 2024-05-08T05:56:28.358 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.allow_mmap_writes: 0 2024-05-08T05:56:28.359 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.use_direct_reads: 0 2024-05-08T05:56:28.359 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-08T05:56:28.359 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.create_missing_column_families: 0 2024-05-08T05:56:28.359 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.db_log_dir: 2024-05-08T05:56:28.359 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.wal_dir: 2024-05-08T05:56:28.359 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-08T05:56:28.359 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-08T05:56:28.359 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-08T05:56:28.359 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-08T05:56:28.359 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-08T05:56:28.359 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-08T05:56:28.359 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.advise_random_on_open: 1 2024-05-08T05:56:28.359 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.db_write_buffer_size: 0 2024-05-08T05:56:28.360 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.write_buffer_manager: 0x563c3e8390e0 2024-05-08T05:56:28.360 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-08T05:56:28.360 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-08T05:56:28.360 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-08T05:56:28.360 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.rate_limiter: (nil) 2024-05-08T05:56:28.360 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-08T05:56:28.360 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.wal_recovery_mode: 2 2024-05-08T05:56:28.360 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.enable_thread_tracking: 0 2024-05-08T05:56:28.360 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.enable_pipelined_write: 0 2024-05-08T05:56:28.360 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.unordered_write: 0 2024-05-08T05:56:28.360 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-08T05:56:28.360 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-08T05:56:28.360 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-08T05:56:28.360 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-08T05:56:28.361 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.row_cache: None 2024-05-08T05:56:28.361 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.wal_filter: None 2024-05-08T05:56:28.361 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-08T05:56:28.361 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.allow_ingest_behind: 0 2024-05-08T05:56:28.361 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.two_write_queues: 0 2024-05-08T05:56:28.361 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.manual_wal_flush: 0 2024-05-08T05:56:28.361 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.wal_compression: 0 2024-05-08T05:56:28.361 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.atomic_flush: 0 2024-05-08T05:56:28.361 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-08T05:56:28.361 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-08T05:56:28.361 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-08T05:56:28.361 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.log_readahead_size: 0 2024-05-08T05:56:28.361 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-08T05:56:28.361 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.best_efforts_recovery: 0 2024-05-08T05:56:28.361 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-08T05:56:28.362 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-08T05:56:28.362 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.allow_data_in_errors: 0 2024-05-08T05:56:28.362 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.db_host_id: __hostname__ 2024-05-08T05:56:28.362 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-08T05:56:28.362 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.max_background_jobs: 2 2024-05-08T05:56:28.362 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.max_background_compactions: -1 2024-05-08T05:56:28.362 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.max_subcompactions: 1 2024-05-08T05:56:28.362 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-08T05:56:28.362 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-08T05:56:28.362 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-08T05:56:28.362 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.max_total_wal_size: 0 2024-05-08T05:56:28.362 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-08T05:56:28.362 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-08T05:56:28.362 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-08T05:56:28.363 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-08T05:56:28.363 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.max_open_files: -1 2024-05-08T05:56:28.363 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.bytes_per_sync: 0 2024-05-08T05:56:28.363 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-08T05:56:28.363 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-08T05:56:28.363 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.compaction_readahead_size: 0 2024-05-08T05:56:28.363 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.max_background_flushes: -1 2024-05-08T05:56:28.363 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Compression algorithms supported: 2024-05-08T05:56:28.363 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-08T05:56:28.363 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: kZSTD supported: 0 2024-05-08T05:56:28.363 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: kXpressCompression supported: 0 2024-05-08T05:56:28.363 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: kLZ4HCCompression supported: 1 2024-05-08T05:56:28.363 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: kZlibCompression supported: 1 2024-05-08T05:56:28.363 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: kSnappyCompression supported: 1 2024-05-08T05:56:28.363 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: kLZ4Compression supported: 1 2024-05-08T05:56:28.364 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: kBZip2Compression supported: 0 2024-05-08T05:56:28.364 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-08T05:56:28.364 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-08T05:56:28.364 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-08T05:56:28.364 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-08T05:56:28.364 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-08T05:56:28.364 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.merge_operator: 2024-05-08T05:56:28.364 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.compaction_filter: None 2024-05-08T05:56:28.364 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.compaction_filter_factory: None 2024-05-08T05:56:28.364 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.sst_partitioner_factory: None 2024-05-08T05:56:28.364 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-08T05:56:28.364 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-08T05:56:28.364 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x563c3e867fa0) 2024-05-08T05:56:28.364 INFO:journalctl@ceph.mon.a.smithi123.stdout: cache_index_and_filter_blocks: 1 2024-05-08T05:56:28.365 INFO:journalctl@ceph.mon.a.smithi123.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-08T05:56:28.365 INFO:journalctl@ceph.mon.a.smithi123.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-08T05:56:28.365 INFO:journalctl@ceph.mon.a.smithi123.stdout: pin_top_level_index_and_filter: 1 2024-05-08T05:56:28.365 INFO:journalctl@ceph.mon.a.smithi123.stdout: index_type: 0 2024-05-08T05:56:28.365 INFO:journalctl@ceph.mon.a.smithi123.stdout: data_block_index_type: 0 2024-05-08T05:56:28.365 INFO:journalctl@ceph.mon.a.smithi123.stdout: index_shortening: 1 2024-05-08T05:56:28.365 INFO:journalctl@ceph.mon.a.smithi123.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-08T05:56:28.365 INFO:journalctl@ceph.mon.a.smithi123.stdout: checksum: 4 2024-05-08T05:56:28.365 INFO:journalctl@ceph.mon.a.smithi123.stdout: no_block_cache: 0 2024-05-08T05:56:28.365 INFO:journalctl@ceph.mon.a.smithi123.stdout: block_cache: 0x563c3e86b1f0 2024-05-08T05:56:28.365 INFO:journalctl@ceph.mon.a.smithi123.stdout: block_cache_name: BinnedLRUCache 2024-05-08T05:56:28.365 INFO:journalctl@ceph.mon.a.smithi123.stdout: block_cache_options: 2024-05-08T05:56:28.365 INFO:journalctl@ceph.mon.a.smithi123.stdout: capacity : 536870912 2024-05-08T05:56:28.365 INFO:journalctl@ceph.mon.a.smithi123.stdout: num_shard_bits : 4 2024-05-08T05:56:28.365 INFO:journalctl@ceph.mon.a.smithi123.stdout: strict_capacity_limit : 0 2024-05-08T05:56:28.366 INFO:journalctl@ceph.mon.a.smithi123.stdout: high_pri_pool_ratio: 0.000 2024-05-08T05:56:28.366 INFO:journalctl@ceph.mon.a.smithi123.stdout: block_cache_compressed: (nil) 2024-05-08T05:56:28.366 INFO:journalctl@ceph.mon.a.smithi123.stdout: persistent_cache: (nil) 2024-05-08T05:56:28.366 INFO:journalctl@ceph.mon.a.smithi123.stdout: block_size: 4096 2024-05-08T05:56:28.366 INFO:journalctl@ceph.mon.a.smithi123.stdout: block_size_deviation: 10 2024-05-08T05:56:28.366 INFO:journalctl@ceph.mon.a.smithi123.stdout: block_restart_interval: 16 2024-05-08T05:56:28.366 INFO:journalctl@ceph.mon.a.smithi123.stdout: index_block_restart_interval: 1 2024-05-08T05:56:28.366 INFO:journalctl@ceph.mon.a.smithi123.stdout: metadata_block_size: 4096 2024-05-08T05:56:28.366 INFO:journalctl@ceph.mon.a.smithi123.stdout: partition_filters: 0 2024-05-08T05:56:28.366 INFO:journalctl@ceph.mon.a.smithi123.stdout: use_delta_encoding: 1 2024-05-08T05:56:28.366 INFO:journalctl@ceph.mon.a.smithi123.stdout: filter_policy: bloomfilter 2024-05-08T05:56:28.366 INFO:journalctl@ceph.mon.a.smithi123.stdout: whole_key_filtering: 1 2024-05-08T05:56:28.366 INFO:journalctl@ceph.mon.a.smithi123.stdout: verify_compression: 0 2024-05-08T05:56:28.366 INFO:journalctl@ceph.mon.a.smithi123.stdout: read_amp_bytes_per_bit: 0 2024-05-08T05:56:28.367 INFO:journalctl@ceph.mon.a.smithi123.stdout: format_version: 5 2024-05-08T05:56:28.367 INFO:journalctl@ceph.mon.a.smithi123.stdout: enable_index_compression: 1 2024-05-08T05:56:28.367 INFO:journalctl@ceph.mon.a.smithi123.stdout: block_align: 0 2024-05-08T05:56:28.367 INFO:journalctl@ceph.mon.a.smithi123.stdout: max_auto_readahead_size: 262144 2024-05-08T05:56:28.367 INFO:journalctl@ceph.mon.a.smithi123.stdout: prepopulate_block_cache: 0 2024-05-08T05:56:28.367 INFO:journalctl@ceph.mon.a.smithi123.stdout: initial_auto_readahead_size: 8192 2024-05-08T05:56:28.367 INFO:journalctl@ceph.mon.a.smithi123.stdout: num_file_reads_for_auto_readahead: 2 2024-05-08T05:56:28.367 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.write_buffer_size: 33554432 2024-05-08T05:56:28.367 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.max_write_buffer_number: 2 2024-05-08T05:56:28.367 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.compression: NoCompression 2024-05-08T05:56:28.367 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.bottommost_compression: Disabled 2024-05-08T05:56:28.367 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.prefix_extractor: nullptr 2024-05-08T05:56:28.367 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-08T05:56:28.367 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.num_levels: 7 2024-05-08T05:56:28.368 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-08T05:56:28.368 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-08T05:56:28.368 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-08T05:56:28.368 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-08T05:56:28.368 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-08T05:56:28.368 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-08T05:56:28.368 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-08T05:56:28.368 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-08T05:56:28.368 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-08T05:56:28.368 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-08T05:56:28.368 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-08T05:56:28.368 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-08T05:56:28.368 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-08T05:56:28.368 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.compression_opts.level: 32767 2024-05-08T05:56:28.368 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.compression_opts.strategy: 0 2024-05-08T05:56:28.369 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-08T05:56:28.369 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-08T05:56:28.369 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-08T05:56:28.369 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-08T05:56:28.369 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.compression_opts.enabled: false 2024-05-08T05:56:28.369 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-08T05:56:28.369 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-08T05:56:28.369 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-08T05:56:28.369 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-08T05:56:28.369 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.target_file_size_base: 67108864 2024-05-08T05:56:28.369 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-08T05:56:28.369 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-08T05:56:28.369 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-08T05:56:28.369 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-08T05:56:28.370 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-08T05:56:28.370 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-08T05:56:28.370 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-08T05:56:28.370 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-08T05:56:28.370 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-08T05:56:28.370 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-08T05:56:28.370 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-08T05:56:28.370 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-08T05:56:28.370 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-08T05:56:28.370 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-08T05:56:28.371 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.arena_block_size: 1048576 2024-05-08T05:56:28.371 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-08T05:56:28.371 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-08T05:56:28.371 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.disable_auto_compactions: 0 2024-05-08T05:56:28.372 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-08T05:56:28.372 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-08T05:56:28.372 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-08T05:56:28.372 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-08T05:56:28.372 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-08T05:56:28.372 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-08T05:56:28.372 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-08T05:56:28.372 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-08T05:56:28.372 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-08T05:56:28.373 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-08T05:56:28.373 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-08T05:56:28.373 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.inplace_update_support: 0 2024-05-08T05:56:28.373 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-08T05:56:28.373 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-08T05:56:28.373 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-08T05:56:28.373 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-08T05:56:28.373 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.bloom_locality: 0 2024-05-08T05:56:28.373 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.max_successive_merges: 0 2024-05-08T05:56:28.373 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-08T05:56:28.373 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.paranoid_file_checks: 0 2024-05-08T05:56:28.373 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.force_consistency_checks: 1 2024-05-08T05:56:28.373 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.report_bg_io_stats: 0 2024-05-08T05:56:28.373 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.ttl: 2592000 2024-05-08T05:56:28.374 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-08T05:56:28.374 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-08T05:56:28.374 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-08T05:56:28.374 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.enable_blob_files: false 2024-05-08T05:56:28.374 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.min_blob_size: 0 2024-05-08T05:56:28.374 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.blob_file_size: 268435456 2024-05-08T05:56:28.374 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-08T05:56:28.374 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-08T05:56:28.374 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-08T05:56:28.374 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-08T05:56:28.374 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-08T05:56:28.374 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.blob_file_starting_level: 0 2024-05-08T05:56:28.374 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-08T05:56:28.374 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-08T05:56:28.375 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-08T05:56:28.375 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 87a06216-cc93-4092-bfcc-8d3ad693e8e9 2024-05-08T05:56:28.375 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715147788162195, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-08T05:56:28.375 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-08T05:56:28.375 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715147788163653, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100421, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 98315, "index_size": 298, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14191, "raw_average_key_size": 49, "raw_value_size": 90584, "raw_average_value_size": 317, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715147788, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "87a06216-cc93-4092-bfcc-8d3ad693e8e9", "db_session_id": "IJ5UZQ6YUQ0A6KFI279J", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-08T05:56:28.375 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715147788163926, "job": 1, "event": "recovery_finished"} 2024-05-08T05:56:28.375 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-08T05:56:28.375 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-08T05:56:28.375 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x563c3e988000 2024-05-08T05:56:28.375 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: DB pointer 0x563c3e974000 2024-05-08T05:56:28.375 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-08T05:56:28.375 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-08T05:56:28.375 INFO:journalctl@ceph.mon.a.smithi123.stdout: ** DB Stats ** 2024-05-08T05:56:28.376 INFO:journalctl@ceph.mon.a.smithi123.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T05:56:28.376 INFO:journalctl@ceph.mon.a.smithi123.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-08T05:56:28.376 INFO:journalctl@ceph.mon.a.smithi123.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T05:56:28.376 INFO:journalctl@ceph.mon.a.smithi123.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T05:56:28.376 INFO:journalctl@ceph.mon.a.smithi123.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-08T05:56:28.376 INFO:journalctl@ceph.mon.a.smithi123.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T05:56:28.376 INFO:journalctl@ceph.mon.a.smithi123.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T05:56:28.376 INFO:journalctl@ceph.mon.a.smithi123.stdout: 2024-05-08T05:56:28.376 INFO:journalctl@ceph.mon.a.smithi123.stdout: ** Compaction Stats [default] ** 2024-05-08T05:56:28.376 INFO:journalctl@ceph.mon.a.smithi123.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T05:56:28.376 INFO:journalctl@ceph.mon.a.smithi123.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-08T05:56:28.376 INFO:journalctl@ceph.mon.a.smithi123.stdout: L0 2/0 99.93 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 109.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T05:56:28.376 INFO:journalctl@ceph.mon.a.smithi123.stdout: Sum 2/0 99.93 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 109.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T05:56:28.376 INFO:journalctl@ceph.mon.a.smithi123.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 109.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T05:56:28.377 INFO:journalctl@ceph.mon.a.smithi123.stdout: 2024-05-08T05:56:28.377 INFO:journalctl@ceph.mon.a.smithi123.stdout: ** Compaction Stats [default] ** 2024-05-08T05:56:28.377 INFO:journalctl@ceph.mon.a.smithi123.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T05:56:28.377 INFO:journalctl@ceph.mon.a.smithi123.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-08T05:56:28.377 INFO:journalctl@ceph.mon.a.smithi123.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 109.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T05:56:28.377 INFO:journalctl@ceph.mon.a.smithi123.stdout: 2024-05-08T05:56:28.377 INFO:journalctl@ceph.mon.a.smithi123.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-08T05:56:28.377 INFO:journalctl@ceph.mon.a.smithi123.stdout: 2024-05-08T05:56:28.377 INFO:journalctl@ceph.mon.a.smithi123.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T05:56:28.377 INFO:journalctl@ceph.mon.a.smithi123.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-08T05:56:28.377 INFO:journalctl@ceph.mon.a.smithi123.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-08T05:56:28.377 INFO:journalctl@ceph.mon.a.smithi123.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-08T05:56:28.377 INFO:journalctl@ceph.mon.a.smithi123.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-08T05:56:28.377 INFO:journalctl@ceph.mon.a.smithi123.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-08T05:56:28.378 INFO:journalctl@ceph.mon.a.smithi123.stdout: Cumulative compaction: 0.00 GB write, 14.07 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T05:56:28.378 INFO:journalctl@ceph.mon.a.smithi123.stdout: Interval compaction: 0.00 GB write, 14.07 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T05:56:28.378 INFO:journalctl@ceph.mon.a.smithi123.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-08T05:56:28.378 INFO:journalctl@ceph.mon.a.smithi123.stdout: Block cache BinnedLRUCache@0x563c3e86b1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2024-05-08T05:56:28.378 INFO:journalctl@ceph.mon.a.smithi123.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-08T05:56:28.378 INFO:journalctl@ceph.mon.a.smithi123.stdout: 2024-05-08T05:56:28.378 INFO:journalctl@ceph.mon.a.smithi123.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-08T05:56:28.378 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: starting mon.a rank 0 at public addrs [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] at bind addrs [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 2024-05-08T05:56:28.378 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: mon.a@-1(???) e1 preinit fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 2024-05-08T05:56:28.378 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: mon.a@-1(???).mds e1 new map 2024-05-08T05:56:28.378 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: mon.a@-1(???).mds e1 print_map 2024-05-08T05:56:28.378 INFO:journalctl@ceph.mon.a.smithi123.stdout: e1 2024-05-08T05:56:28.378 INFO:journalctl@ceph.mon.a.smithi123.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T05:56:28.378 INFO:journalctl@ceph.mon.a.smithi123.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-08T05:56:28.379 INFO:journalctl@ceph.mon.a.smithi123.stdout: legacy client fscid: -1 2024-05-08T05:56:28.379 INFO:journalctl@ceph.mon.a.smithi123.stdout: 2024-05-08T05:56:28.379 INFO:journalctl@ceph.mon.a.smithi123.stdout: No filesystems configured 2024-05-08T05:56:28.379 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-08T05:56:28.379 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T05:56:28.379 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T05:56:28.379 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T05:56:28.379 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-08T05:56:28.379 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 bash[29769]: f0e5e4ca808065702ad1f6b524525c89e14c8ee4d9920e81144d7e309f8874d5 2024-05-08T05:56:28.379 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 systemd[1]: Started Ceph mon.a for 85cfb2e6-0cff-11ef-bc97-c7b262605968. 2024-05-08T05:56:28.379 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-08T05:56:28.379 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: monmap epoch 1 2024-05-08T05:56:28.379 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 2024-05-08T05:56:28.379 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: last_changed 2024-05-08T05:56:08.254675+0000 2024-05-08T05:56:28.379 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: created 2024-05-08T05:56:08.254675+0000 2024-05-08T05:56:28.380 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: min_mon_release 19 (squid) 2024-05-08T05:56:28.380 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: election_strategy: 1 2024-05-08T05:56:28.380 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: 0: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.a 2024-05-08T05:56:28.380 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: fsmap 2024-05-08T05:56:28.380 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: osdmap e1: 0 total, 0 up, 0 in 2024-05-08T05:56:28.380 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:28 smithi123 ceph-mon[29787]: mgrmap e1: no daemons active 2024-05-08T05:56:29.744 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:28.959+0000 7f3a8aa00700 1 Processor -- start 2024-05-08T05:56:29.744 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:28.959+0000 7f3a8aa00700 1 -- start start 2024-05-08T05:56:29.744 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:28.960+0000 7f3a8aa00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a84108ed0 0x7f3a841092f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:29.745 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:28.960+0000 7f3a8aa00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3a84109830 con 0x7f3a84108ed0 2024-05-08T05:56:29.745 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:28.961+0000 7f3a83e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a84108ed0 0x7f3a841092f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:29.745 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:28.961+0000 7f3a83e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a84108ed0 0x7f3a841092f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48642/0 (socket says 172.21.15.123:48642) 2024-05-08T05:56:29.745 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:28.961+0000 7f3a83e00700 1 -- 172.21.15.123:0/1758707207 learned_addr learned my addr 172.21.15.123:0/1758707207 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:56:29.745 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:28.961+0000 7f3a83e00700 1 -- 172.21.15.123:0/1758707207 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a8410a070 con 0x7f3a84108ed0 2024-05-08T05:56:29.745 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:28.961+0000 7f3a83e00700 1 --2- 172.21.15.123:0/1758707207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a84108ed0 0x7f3a841092f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3a740098a0 tx=0x7f3a74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d62f80b038aaa28 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:29.745 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:28.962+0000 7f3a82a00700 1 -- 172.21.15.123:0/1758707207 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a74004160 con 0x7f3a84108ed0 2024-05-08T05:56:29.745 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:28.962+0000 7f3a82a00700 1 -- 172.21.15.123:0/1758707207 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3a74004300 con 0x7f3a84108ed0 2024-05-08T05:56:29.745 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:28.962+0000 7f3a82a00700 1 -- 172.21.15.123:0/1758707207 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a74011af0 con 0x7f3a84108ed0 2024-05-08T05:56:29.745 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:28.963+0000 7f3a8aa00700 1 -- 172.21.15.123:0/1758707207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a84108ed0 msgr2=0x7f3a841092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:29.745 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:28.963+0000 7f3a8aa00700 1 --2- 172.21.15.123:0/1758707207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a84108ed0 0x7f3a841092f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3a740098a0 tx=0x7f3a74009bb0 comp rx=0 tx=0).stop 2024-05-08T05:56:29.745 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:28.963+0000 7f3a8aa00700 1 -- 172.21.15.123:0/1758707207 shutdown_connections 2024-05-08T05:56:29.745 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:28.963+0000 7f3a8aa00700 1 --2- 172.21.15.123:0/1758707207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a84108ed0 0x7f3a841092f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:29.746 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:28.963+0000 7f3a8aa00700 1 -- 172.21.15.123:0/1758707207 >> 172.21.15.123:0/1758707207 conn(0x7f3a841044a0 msgr2=0x7f3a84106900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:29.746 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:28.963+0000 7f3a8aa00700 1 -- 172.21.15.123:0/1758707207 shutdown_connections 2024-05-08T05:56:29.746 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:28.963+0000 7f3a8aa00700 1 -- 172.21.15.123:0/1758707207 wait complete. 2024-05-08T05:56:29.746 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:28.964+0000 7f3a8aa00700 1 Processor -- start 2024-05-08T05:56:29.746 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:28.964+0000 7f3a8aa00700 1 -- start start 2024-05-08T05:56:29.746 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:28.965+0000 7f3a8aa00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a84108ed0 0x7f3a8419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:29.746 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:28.965+0000 7f3a8aa00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3a841099b0 con 0x7f3a84108ed0 2024-05-08T05:56:29.746 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:28.965+0000 7f3a83e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a84108ed0 0x7f3a8419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:29.746 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:28.965+0000 7f3a83e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a84108ed0 0x7f3a8419f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48658/0 (socket says 172.21.15.123:48658) 2024-05-08T05:56:29.746 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:28.965+0000 7f3a83e00700 1 -- 172.21.15.123:0/3344694629 learned_addr learned my addr 172.21.15.123:0/3344694629 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:56:29.746 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:28.965+0000 7f3a83e00700 1 -- 172.21.15.123:0/3344694629 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a8419f5c0 con 0x7f3a84108ed0 2024-05-08T05:56:29.746 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:28.966+0000 7f3a83e00700 1 --2- 172.21.15.123:0/3344694629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a84108ed0 0x7f3a8419f080 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3a740117b0 tx=0x7f3a74011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:29.746 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:28.966+0000 7f3a80c00700 1 -- 172.21.15.123:0/3344694629 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a74003af0 con 0x7f3a84108ed0 2024-05-08T05:56:29.747 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:28.966+0000 7f3a80c00700 1 -- 172.21.15.123:0/3344694629 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3a74020070 con 0x7f3a84108ed0 2024-05-08T05:56:29.747 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:28.966+0000 7f3a80c00700 1 -- 172.21.15.123:0/3344694629 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a74019550 con 0x7f3a84108ed0 2024-05-08T05:56:29.747 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:28.966+0000 7f3a8aa00700 1 -- 172.21.15.123:0/3344694629 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a8419f850 con 0x7f3a84108ed0 2024-05-08T05:56:29.747 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:28.966+0000 7f3a8aa00700 1 -- 172.21.15.123:0/3344694629 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a841a2540 con 0x7f3a84108ed0 2024-05-08T05:56:29.747 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:28.966+0000 7f3a80c00700 1 -- 172.21.15.123:0/3344694629 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3a74019b50 con 0x7f3a84108ed0 2024-05-08T05:56:29.747 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:28.966+0000 7f3a80c00700 1 -- 172.21.15.123:0/3344694629 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3a7400b720 con 0x7f3a84108ed0 2024-05-08T05:56:29.747 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:28.967+0000 7f3a8aa00700 1 -- 172.21.15.123:0/3344694629 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a84064b00 con 0x7f3a84108ed0 2024-05-08T05:56:29.747 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:28.973+0000 7f3a80c00700 1 -- 172.21.15.123:0/3344694629 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f3a740042c0 con 0x7f3a84108ed0 2024-05-08T05:56:29.747 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:29.050+0000 7f3a8aa00700 1 -- 172.21.15.123:0/3344694629 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f3a841a2760 con 0x7f3a84108ed0 2024-05-08T05:56:29.747 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:29.053+0000 7f3a80c00700 1 -- 172.21.15.123:0/3344694629 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f3a7400b920 con 0x7f3a84108ed0 2024-05-08T05:56:29.747 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:29.054+0000 7f3a8aa00700 1 -- 172.21.15.123:0/3344694629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a84108ed0 msgr2=0x7f3a8419f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:29.747 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:29.054+0000 7f3a8aa00700 1 --2- 172.21.15.123:0/3344694629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a84108ed0 0x7f3a8419f080 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3a740117b0 tx=0x7f3a74011b70 comp rx=0 tx=0).stop 2024-05-08T05:56:29.747 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:29.054+0000 7f3a8aa00700 1 -- 172.21.15.123:0/3344694629 shutdown_connections 2024-05-08T05:56:29.747 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:29.054+0000 7f3a8aa00700 1 --2- 172.21.15.123:0/3344694629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a84108ed0 0x7f3a8419f080 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:29.747 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:29.054+0000 7f3a8aa00700 1 -- 172.21.15.123:0/3344694629 >> 172.21.15.123:0/3344694629 conn(0x7f3a841044a0 msgr2=0x7f3a84105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:29.748 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:29.055+0000 7f3a8aa00700 1 -- 172.21.15.123:0/3344694629 shutdown_connections 2024-05-08T05:56:29.748 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:29.055+0000 7f3a8aa00700 1 -- 172.21.15.123:0/3344694629 wait complete. 2024-05-08T05:56:29.748 INFO:teuthology.orchestra.run.smithi123.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-08T05:56:29.774 INFO:teuthology.orchestra.run.smithi123.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-08T05:56:29.774 INFO:teuthology.orchestra.run.smithi123.stdout:Creating mgr... 2024-05-08T05:56:29.775 INFO:teuthology.orchestra.run.smithi123.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-08T05:56:29.776 INFO:teuthology.orchestra.run.smithi123.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-08T05:56:30.222 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mgr.x 2024-05-08T05:56:30.222 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stderr Failed to reset failed state of unit ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mgr.x.service: Unit ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mgr.x.service not loaded. 2024-05-08T05:56:30.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:30 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/3344694629' entity='client.admin' 2024-05-08T05:56:30.483 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968.target.wants/ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@.service. 2024-05-08T05:56:30.883 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:30 smithi123 systemd[1]: Starting Ceph mgr.x for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T05:56:31.206 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:30 smithi123 podman[30012]: 2024-05-08 05:56:30.787113783 +0000 UTC m=+0.032293532 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:56:31.206 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:30 smithi123 podman[30012]: 2024-05-08 05:56:30.885173973 +0000 UTC m=+0.130353742 container create 1df64aa6e04a75c924ebcf1acc4968a5d0a94432f61ef7eeae365e7eba7ec580 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T05:56:31.206 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:31 smithi123 podman[30012]: 2024-05-08 05:56:31.121825511 +0000 UTC m=+0.367005339 container init 1df64aa6e04a75c924ebcf1acc4968a5d0a94432f61ef7eeae365e7eba7ec580 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux ) 2024-05-08T05:56:31.206 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:31 smithi123 podman[30012]: 2024-05-08 05:56:31.131515701 +0000 UTC m=+0.376695493 container start 1df64aa6e04a75c924ebcf1acc4968a5d0a94432f61ef7eeae365e7eba7ec580 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-08T05:56:31.955 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:31 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:31.695+0000 7ff568401200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T05:56:31.955 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:31 smithi123 bash[30012]: 1df64aa6e04a75c924ebcf1acc4968a5d0a94432f61ef7eeae365e7eba7ec580 2024-05-08T05:56:31.955 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:31 smithi123 systemd[1]: Started Ceph mgr.x for 85cfb2e6-0cff-11ef-bc97-c7b262605968. 2024-05-08T05:56:31.974 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T05:56:31.974 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout disabled 2024-05-08T05:56:31.992 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T05:56:31.992 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout inactive 2024-05-08T05:56:31.992 INFO:teuthology.orchestra.run.smithi123.stdout:firewalld.service is not enabled 2024-05-08T05:56:31.992 INFO:teuthology.orchestra.run.smithi123.stdout:Not possible to enable service . firewalld.service is not available 2024-05-08T05:56:32.021 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T05:56:32.021 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout disabled 2024-05-08T05:56:32.042 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T05:56:32.043 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout inactive 2024-05-08T05:56:32.043 INFO:teuthology.orchestra.run.smithi123.stdout:firewalld.service is not enabled 2024-05-08T05:56:32.043 INFO:teuthology.orchestra.run.smithi123.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-08T05:56:32.044 INFO:teuthology.orchestra.run.smithi123.stdout:Waiting for mgr to start... 2024-05-08T05:56:32.044 INFO:teuthology.orchestra.run.smithi123.stdout:Waiting for mgr... 2024-05-08T05:56:32.222 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:31 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:31.960+0000 7ff568401200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:56:32.222 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:32 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:32.108+0000 7ff568401200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T05:56:32.730 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:32 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:32.533+0000 7ff568401200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T05:56:32.730 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:32 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:32.648+0000 7ff568401200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T05:56:33.157 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:32 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/2368927650' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T05:56:33.509 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 2024-05-08T05:56:33.510 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout { 2024-05-08T05:56:33.510 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "fsid": "85cfb2e6-0cff-11ef-bc97-c7b262605968", 2024-05-08T05:56:33.510 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T05:56:33.510 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T05:56:33.511 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T05:56:33.511 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T05:56:33.511 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:56:33.511 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T05:56:33.512 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T05:56:33.512 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 0 2024-05-08T05:56:33.513 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:56:33.513 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T05:56:33.513 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T05:56:33.513 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:56:33.513 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "quorum_age": 4, 2024-05-08T05:56:33.513 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T05:56:33.513 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:56:33.514 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-08T05:56:33.514 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T05:56:33.514 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:56:33.514 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T05:56:33.515 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:56:33.515 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T05:56:33.515 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T05:56:33.515 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T05:56:33.515 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T05:56:33.515 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T05:56:33.515 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T05:56:33.515 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:56:33.515 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T05:56:33.515 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T05:56:33.517 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T05:56:33.517 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T05:56:33.517 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T05:56:33.517 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T05:56:33.517 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T05:56:33.518 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T05:56:33.518 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T05:56:33.518 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:56:33.518 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T05:56:33.518 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:56:33.518 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T05:56:33.518 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T05:56:33.518 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:56:33.518 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T05:56:33.518 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-08T05:56:33.518 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T05:56:33.520 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T05:56:33.520 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T05:56:33.520 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T05:56:33.520 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T05:56:33.521 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:56:33.521 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T05:56:33.521 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:56:33.521 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T05:56:33.521 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:56:33.521 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T05:56:12.545340+0000", 2024-05-08T05:56:33.521 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T05:56:33.521 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:56:33.521 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T05:56:33.521 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout } 2024-05-08T05:56:33.521 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.621+0000 7f1544400700 1 Processor -- start 2024-05-08T05:56:33.521 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.621+0000 7f1544400700 1 -- start start 2024-05-08T05:56:33.521 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.622+0000 7f1544400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f153c07ef30 0x7f153c07d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:33.521 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.622+0000 7f1544400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f153c07d8b0 con 0x7f153c07ef30 2024-05-08T05:56:33.521 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.622+0000 7f1541a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f153c07ef30 0x7f153c07d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:33.524 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.622+0000 7f1541a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f153c07ef30 0x7f153c07d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48672/0 (socket says 172.21.15.123:48672) 2024-05-08T05:56:33.524 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.622+0000 7f1541a00700 1 -- 172.21.15.123:0/3026423473 learned_addr learned my addr 172.21.15.123:0/3026423473 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:56:33.524 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.623+0000 7f1541a00700 1 -- 172.21.15.123:0/3026423473 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f153c07da30 con 0x7f153c07ef30 2024-05-08T05:56:33.524 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.623+0000 7f1541a00700 1 --2- 172.21.15.123:0/3026423473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f153c07ef30 0x7f153c07d2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f15300098a0 tx=0x7f1530009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d8efa9ab5b64a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:33.524 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.623+0000 7f153be00700 1 -- 172.21.15.123:0/3026423473 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1530004160 con 0x7f153c07ef30 2024-05-08T05:56:33.524 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.623+0000 7f153be00700 1 -- 172.21.15.123:0/3026423473 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1530004300 con 0x7f153c07ef30 2024-05-08T05:56:33.525 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.623+0000 7f153be00700 1 -- 172.21.15.123:0/3026423473 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1530011af0 con 0x7f153c07ef30 2024-05-08T05:56:33.525 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.624+0000 7f1544400700 1 -- 172.21.15.123:0/3026423473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f153c07ef30 msgr2=0x7f153c07d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:33.525 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.624+0000 7f1544400700 1 --2- 172.21.15.123:0/3026423473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f153c07ef30 0x7f153c07d2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f15300098a0 tx=0x7f1530009bb0 comp rx=0 tx=0).stop 2024-05-08T05:56:33.525 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.624+0000 7f1544400700 1 -- 172.21.15.123:0/3026423473 shutdown_connections 2024-05-08T05:56:33.525 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.624+0000 7f1544400700 1 --2- 172.21.15.123:0/3026423473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f153c07ef30 0x7f153c07d2e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:33.525 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.624+0000 7f1544400700 1 -- 172.21.15.123:0/3026423473 >> 172.21.15.123:0/3026423473 conn(0x7f153c1045b0 msgr2=0x7f153c1069d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:33.525 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.624+0000 7f1544400700 1 -- 172.21.15.123:0/3026423473 shutdown_connections 2024-05-08T05:56:33.525 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.625+0000 7f1544400700 1 -- 172.21.15.123:0/3026423473 wait complete. 2024-05-08T05:56:33.525 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.625+0000 7f1544400700 1 Processor -- start 2024-05-08T05:56:33.525 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.626+0000 7f1544400700 1 -- start start 2024-05-08T05:56:33.525 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.626+0000 7f1544400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f153c07ef30 0x7f153c19f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:33.525 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.626+0000 7f1544400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f153c10f540 con 0x7f153c07ef30 2024-05-08T05:56:33.525 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.627+0000 7f1541a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f153c07ef30 0x7f153c19f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:33.525 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.627+0000 7f1541a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f153c07ef30 0x7f153c19f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48686/0 (socket says 172.21.15.123:48686) 2024-05-08T05:56:33.525 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.627+0000 7f1541a00700 1 -- 172.21.15.123:0/2368927650 learned_addr learned my addr 172.21.15.123:0/2368927650 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:56:33.526 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.627+0000 7f1541a00700 1 -- 172.21.15.123:0/2368927650 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f153c19f650 con 0x7f153c07ef30 2024-05-08T05:56:33.526 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.627+0000 7f1541a00700 1 --2- 172.21.15.123:0/2368927650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f153c07ef30 0x7f153c19f110 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1530011ac0 tx=0x7f1530011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:33.528 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.628+0000 7f153a000700 1 -- 172.21.15.123:0/2368927650 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1530003970 con 0x7f153c07ef30 2024-05-08T05:56:33.528 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.628+0000 7f153a000700 1 -- 172.21.15.123:0/2368927650 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1530003b10 con 0x7f153c07ef30 2024-05-08T05:56:33.528 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.628+0000 7f1544400700 1 -- 172.21.15.123:0/2368927650 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f153c19f8e0 con 0x7f153c07ef30 2024-05-08T05:56:33.528 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.628+0000 7f1544400700 1 -- 172.21.15.123:0/2368927650 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f153c1a25d0 con 0x7f153c07ef30 2024-05-08T05:56:33.528 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.628+0000 7f153a000700 1 -- 172.21.15.123:0/2368927650 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1530024610 con 0x7f153c07ef30 2024-05-08T05:56:33.528 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.628+0000 7f153a000700 1 -- 172.21.15.123:0/2368927650 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1530024c40 con 0x7f153c07ef30 2024-05-08T05:56:33.528 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.628+0000 7f153a000700 1 -- 172.21.15.123:0/2368927650 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1530019700 con 0x7f153c07ef30 2024-05-08T05:56:33.528 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.629+0000 7f1544400700 1 -- 172.21.15.123:0/2368927650 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f153c064b00 con 0x7f153c07ef30 2024-05-08T05:56:33.528 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.632+0000 7f153a000700 1 -- 172.21.15.123:0/2368927650 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f1530043ec0 con 0x7f153c07ef30 2024-05-08T05:56:33.528 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.710+0000 7f1544400700 1 -- 172.21.15.123:0/2368927650 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f153c1a2b10 con 0x7f153c07ef30 2024-05-08T05:56:33.528 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.711+0000 7f153a000700 1 -- 172.21.15.123:0/2368927650 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f1530032030 con 0x7f153c07ef30 2024-05-08T05:56:33.529 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.712+0000 7f1544400700 1 -- 172.21.15.123:0/2368927650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f153c07ef30 msgr2=0x7f153c19f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:33.529 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.712+0000 7f1544400700 1 --2- 172.21.15.123:0/2368927650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f153c07ef30 0x7f153c19f110 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1530011ac0 tx=0x7f1530011ba0 comp rx=0 tx=0).stop 2024-05-08T05:56:33.529 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.712+0000 7f1544400700 1 -- 172.21.15.123:0/2368927650 shutdown_connections 2024-05-08T05:56:33.529 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.712+0000 7f1544400700 1 --2- 172.21.15.123:0/2368927650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f153c07ef30 0x7f153c19f110 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:33.529 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.712+0000 7f1544400700 1 -- 172.21.15.123:0/2368927650 >> 172.21.15.123:0/2368927650 conn(0x7f153c1045b0 msgr2=0x7f153c1050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:33.529 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.713+0000 7f1544400700 1 -- 172.21.15.123:0/2368927650 shutdown_connections 2024-05-08T05:56:33.529 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:32.713+0000 7f1544400700 1 -- 172.21.15.123:0/2368927650 wait complete. 2024-05-08T05:56:33.529 INFO:teuthology.orchestra.run.smithi123.stdout:mgr not available, waiting (1/15)... 2024-05-08T05:56:34.696 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:34 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:34.244+0000 7ff568401200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T05:56:34.696 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:34 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:34.469+0000 7ff568401200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T05:56:34.955 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:34 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:34.694+0000 7ff568401200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T05:56:35.441 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:35 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:35.068+0000 7ff568401200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T05:56:35.441 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:35 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:35.188+0000 7ff568401200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T05:56:35.700 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:35 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:35.440+0000 7ff568401200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T05:56:35.700 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:35 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:35.698+0000 7ff568401200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:56:36.515 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:36 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/2875151933' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T05:56:36.804 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:36 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:36.513+0000 7ff568401200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T05:56:36.940 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 2024-05-08T05:56:36.941 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout { 2024-05-08T05:56:36.941 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "fsid": "85cfb2e6-0cff-11ef-bc97-c7b262605968", 2024-05-08T05:56:36.941 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T05:56:36.941 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T05:56:36.941 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T05:56:36.943 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T05:56:36.944 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:56:36.944 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T05:56:36.944 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T05:56:36.944 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 0 2024-05-08T05:56:36.944 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:56:36.944 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T05:56:36.944 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T05:56:36.944 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:56:36.944 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-05-08T05:56:36.944 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T05:56:36.944 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:56:36.944 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-08T05:56:36.944 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T05:56:36.944 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:56:36.944 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T05:56:36.945 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:56:36.945 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T05:56:36.945 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T05:56:36.954 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T05:56:36.954 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T05:56:36.954 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T05:56:36.954 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T05:56:36.954 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:56:36.954 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T05:56:36.954 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T05:56:36.954 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T05:56:36.954 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T05:56:36.954 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T05:56:36.954 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T05:56:36.955 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T05:56:36.955 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T05:56:36.955 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T05:56:36.955 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:56:36.955 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T05:56:36.955 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:56:36.955 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T05:56:36.955 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T05:56:36.955 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:56:36.955 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T05:56:36.955 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-08T05:56:36.955 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T05:56:36.955 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T05:56:36.955 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T05:56:36.956 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T05:56:36.956 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T05:56:36.956 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:56:36.956 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T05:56:36.956 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:56:36.956 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T05:56:36.956 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:56:36.956 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T05:56:12.545340+0000", 2024-05-08T05:56:36.956 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T05:56:36.956 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:56:36.956 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T05:56:36.956 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout } 2024-05-08T05:56:36.957 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.147+0000 7f9842600700 1 Processor -- start 2024-05-08T05:56:36.957 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.147+0000 7f9842600700 1 -- start start 2024-05-08T05:56:36.957 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.148+0000 7f9842600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f983c07eea0 0x7f983c07d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:36.957 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.148+0000 7f9842600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f983c07d8d0 con 0x7f983c07eea0 2024-05-08T05:56:36.957 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.148+0000 7f983b400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f983c07eea0 0x7f983c07d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:36.957 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.148+0000 7f983b400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f983c07eea0 0x7f983c07d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52516/0 (socket says 172.21.15.123:52516) 2024-05-08T05:56:36.957 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.148+0000 7f983b400700 1 -- 172.21.15.123:0/1812130596 learned_addr learned my addr 172.21.15.123:0/1812130596 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:56:36.957 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.148+0000 7f983b400700 1 -- 172.21.15.123:0/1812130596 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f983c07da50 con 0x7f983c07eea0 2024-05-08T05:56:36.957 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.149+0000 7f983b400700 1 --2- 172.21.15.123:0/1812130596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f983c07eea0 0x7f983c07d300 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f982c0098a0 tx=0x7f982c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc7d3b7e11a8c332 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:36.957 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.149+0000 7f983a000700 1 -- 172.21.15.123:0/1812130596 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f982c004160 con 0x7f983c07eea0 2024-05-08T05:56:36.957 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.149+0000 7f983a000700 1 -- 172.21.15.123:0/1812130596 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f982c004300 con 0x7f983c07eea0 2024-05-08T05:56:36.957 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.149+0000 7f983a000700 1 -- 172.21.15.123:0/1812130596 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f982c011af0 con 0x7f983c07eea0 2024-05-08T05:56:36.957 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.150+0000 7f9842600700 1 -- 172.21.15.123:0/1812130596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f983c07eea0 msgr2=0x7f983c07d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:36.958 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.150+0000 7f9842600700 1 --2- 172.21.15.123:0/1812130596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f983c07eea0 0x7f983c07d300 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f982c0098a0 tx=0x7f982c009bb0 comp rx=0 tx=0).stop 2024-05-08T05:56:36.958 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.150+0000 7f9842600700 1 -- 172.21.15.123:0/1812130596 shutdown_connections 2024-05-08T05:56:36.958 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.150+0000 7f9842600700 1 --2- 172.21.15.123:0/1812130596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f983c07eea0 0x7f983c07d300 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:36.958 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.150+0000 7f9842600700 1 -- 172.21.15.123:0/1812130596 >> 172.21.15.123:0/1812130596 conn(0x7f983c104440 msgr2=0x7f983c1068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:36.958 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.150+0000 7f9842600700 1 -- 172.21.15.123:0/1812130596 shutdown_connections 2024-05-08T05:56:36.958 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.150+0000 7f9842600700 1 -- 172.21.15.123:0/1812130596 wait complete. 2024-05-08T05:56:36.958 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.151+0000 7f9842600700 1 Processor -- start 2024-05-08T05:56:36.958 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.151+0000 7f9842600700 1 -- start start 2024-05-08T05:56:36.958 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.152+0000 7f9842600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f983c07eea0 0x7f983c19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:36.958 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.152+0000 7f9842600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f983c10f430 con 0x7f983c07eea0 2024-05-08T05:56:36.958 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.152+0000 7f983b400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f983c07eea0 0x7f983c19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:36.958 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.152+0000 7f983b400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f983c07eea0 0x7f983c19efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52530/0 (socket says 172.21.15.123:52530) 2024-05-08T05:56:36.958 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.152+0000 7f983b400700 1 -- 172.21.15.123:0/2875151933 learned_addr learned my addr 172.21.15.123:0/2875151933 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:56:36.958 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.152+0000 7f983b400700 1 -- 172.21.15.123:0/2875151933 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f983c19f520 con 0x7f983c07eea0 2024-05-08T05:56:36.959 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.153+0000 7f983b400700 1 --2- 172.21.15.123:0/2875151933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f983c07eea0 0x7f983c19efe0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f982c011ac0 tx=0x7f982c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:36.959 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.153+0000 7f9833e00700 1 -- 172.21.15.123:0/2875151933 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f982c003970 con 0x7f983c07eea0 2024-05-08T05:56:36.959 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.153+0000 7f9833e00700 1 -- 172.21.15.123:0/2875151933 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f982c003b10 con 0x7f983c07eea0 2024-05-08T05:56:36.959 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.153+0000 7f9842600700 1 -- 172.21.15.123:0/2875151933 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f983c19f7b0 con 0x7f983c07eea0 2024-05-08T05:56:36.959 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.153+0000 7f9833e00700 1 -- 172.21.15.123:0/2875151933 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f982c024610 con 0x7f983c07eea0 2024-05-08T05:56:36.959 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.153+0000 7f9842600700 1 -- 172.21.15.123:0/2875151933 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f983c1a24a0 con 0x7f983c07eea0 2024-05-08T05:56:36.959 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.153+0000 7f9833e00700 1 -- 172.21.15.123:0/2875151933 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f982c024bd0 con 0x7f983c07eea0 2024-05-08T05:56:36.959 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.153+0000 7f9833e00700 1 -- 172.21.15.123:0/2875151933 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f982c019700 con 0x7f983c07eea0 2024-05-08T05:56:36.959 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.154+0000 7f9842600700 1 -- 172.21.15.123:0/2875151933 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f983c064b00 con 0x7f983c07eea0 2024-05-08T05:56:36.959 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.157+0000 7f9833e00700 1 -- 172.21.15.123:0/2875151933 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f982c043ec0 con 0x7f983c07eea0 2024-05-08T05:56:36.959 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.236+0000 7f9842600700 1 -- 172.21.15.123:0/2875151933 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f983c1a2670 con 0x7f983c07eea0 2024-05-08T05:56:36.959 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.237+0000 7f9833e00700 1 -- 172.21.15.123:0/2875151933 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f982c032030 con 0x7f983c07eea0 2024-05-08T05:56:36.959 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.238+0000 7f9842600700 1 -- 172.21.15.123:0/2875151933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f983c07eea0 msgr2=0x7f983c19efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:36.959 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.238+0000 7f9842600700 1 --2- 172.21.15.123:0/2875151933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f983c07eea0 0x7f983c19efe0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f982c011ac0 tx=0x7f982c011ba0 comp rx=0 tx=0).stop 2024-05-08T05:56:36.960 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.239+0000 7f9842600700 1 -- 172.21.15.123:0/2875151933 shutdown_connections 2024-05-08T05:56:36.960 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.239+0000 7f9842600700 1 --2- 172.21.15.123:0/2875151933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f983c07eea0 0x7f983c19efe0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:36.960 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.239+0000 7f9842600700 1 -- 172.21.15.123:0/2875151933 >> 172.21.15.123:0/2875151933 conn(0x7f983c104440 msgr2=0x7f983c105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:36.960 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.239+0000 7f9842600700 1 -- 172.21.15.123:0/2875151933 shutdown_connections 2024-05-08T05:56:36.960 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:36.239+0000 7f9842600700 1 -- 172.21.15.123:0/2875151933 wait complete. 2024-05-08T05:56:36.960 INFO:teuthology.orchestra.run.smithi123.stdout:mgr not available, waiting (2/15)... 2024-05-08T05:56:37.205 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:36 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:36.802+0000 7ff568401200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-08T05:56:37.705 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:37 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:37.357+0000 7ff568401200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T05:56:37.705 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:37 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:37.484+0000 7ff568401200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T05:56:38.205 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:37 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:37.940+0000 7ff568401200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T05:56:38.705 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:38 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:38.384+0000 7ff568401200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T05:56:38.705 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:38 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:38.507+0000 7ff568401200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T05:56:39.109 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:38 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:38.826+0000 7ff568401200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T05:56:39.109 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:39 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:39.008+0000 7ff568401200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T05:56:39.445 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:39 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:39.136+0000 7ff568401200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T05:56:39.705 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:39 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:39.540+0000 7ff568401200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T05:56:39.705 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:39 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:39.665+0000 7ff568401200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T05:56:40.058 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:39 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/1879008525' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T05:56:40.334 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 2024-05-08T05:56:40.334 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout { 2024-05-08T05:56:40.334 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "fsid": "85cfb2e6-0cff-11ef-bc97-c7b262605968", 2024-05-08T05:56:40.334 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T05:56:40.334 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T05:56:40.334 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T05:56:40.334 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T05:56:40.335 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:56:40.335 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T05:56:40.335 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T05:56:40.335 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 0 2024-05-08T05:56:40.335 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:56:40.335 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T05:56:40.335 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T05:56:40.335 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:56:40.335 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-05-08T05:56:40.335 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T05:56:40.335 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:56:40.335 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-08T05:56:40.335 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T05:56:40.336 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:56:40.336 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T05:56:40.336 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:56:40.336 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T05:56:40.336 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T05:56:40.336 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T05:56:40.336 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T05:56:40.336 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T05:56:40.336 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T05:56:40.336 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:56:40.336 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T05:56:40.336 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T05:56:40.336 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T05:56:40.336 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T05:56:40.337 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T05:56:40.337 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T05:56:40.337 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T05:56:40.337 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T05:56:40.337 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T05:56:40.337 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:56:40.337 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T05:56:40.337 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:56:40.337 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T05:56:40.337 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T05:56:40.337 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:56:40.337 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T05:56:40.337 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-08T05:56:40.337 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T05:56:40.337 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T05:56:40.338 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T05:56:40.338 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T05:56:40.338 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T05:56:40.338 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:56:40.338 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T05:56:40.338 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:56:40.338 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T05:56:40.338 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:56:40.338 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T05:56:12.545340+0000", 2024-05-08T05:56:40.338 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T05:56:40.338 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:56:40.338 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T05:56:40.339 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout } 2024-05-08T05:56:40.339 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.598+0000 7fa2cc000700 1 Processor -- start 2024-05-08T05:56:40.339 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.598+0000 7fa2cc000700 1 -- start start 2024-05-08T05:56:40.339 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.598+0000 7fa2cc000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2c4108e70 0x7fa2c4109290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:40.339 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.598+0000 7fa2cc000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c41097d0 con 0x7fa2c4108e70 2024-05-08T05:56:40.339 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.599+0000 7fa2c9600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2c4108e70 0x7fa2c4109290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:40.339 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.599+0000 7fa2c9600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2c4108e70 0x7fa2c4109290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52538/0 (socket says 172.21.15.123:52538) 2024-05-08T05:56:40.339 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.599+0000 7fa2c9600700 1 -- 172.21.15.123:0/3876799345 learned_addr learned my addr 172.21.15.123:0/3876799345 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:56:40.339 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.599+0000 7fa2c9600700 1 -- 172.21.15.123:0/3876799345 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c410a060 con 0x7fa2c4108e70 2024-05-08T05:56:40.339 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.599+0000 7fa2c9600700 1 --2- 172.21.15.123:0/3876799345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2c4108e70 0x7fa2c4109290 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa2b80098a0 tx=0x7fa2b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6958225a8d48dff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:40.339 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.600+0000 7fa2c3e00700 1 -- 172.21.15.123:0/3876799345 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2b8004160 con 0x7fa2c4108e70 2024-05-08T05:56:40.339 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.600+0000 7fa2c3e00700 1 -- 172.21.15.123:0/3876799345 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa2b8004300 con 0x7fa2c4108e70 2024-05-08T05:56:40.339 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.600+0000 7fa2c3e00700 1 -- 172.21.15.123:0/3876799345 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2b8011af0 con 0x7fa2c4108e70 2024-05-08T05:56:40.340 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.600+0000 7fa2cc000700 1 -- 172.21.15.123:0/3876799345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2c4108e70 msgr2=0x7fa2c4109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:40.340 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.600+0000 7fa2cc000700 1 --2- 172.21.15.123:0/3876799345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2c4108e70 0x7fa2c4109290 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa2b80098a0 tx=0x7fa2b8009bb0 comp rx=0 tx=0).stop 2024-05-08T05:56:40.340 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.601+0000 7fa2cc000700 1 -- 172.21.15.123:0/3876799345 shutdown_connections 2024-05-08T05:56:40.340 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.601+0000 7fa2cc000700 1 --2- 172.21.15.123:0/3876799345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2c4108e70 0x7fa2c4109290 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:40.340 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.601+0000 7fa2cc000700 1 -- 172.21.15.123:0/3876799345 >> 172.21.15.123:0/3876799345 conn(0x7fa2c4104440 msgr2=0x7fa2c41068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:40.340 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.601+0000 7fa2cc000700 1 -- 172.21.15.123:0/3876799345 shutdown_connections 2024-05-08T05:56:40.340 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.601+0000 7fa2cc000700 1 -- 172.21.15.123:0/3876799345 wait complete. 2024-05-08T05:56:40.340 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.602+0000 7fa2cc000700 1 Processor -- start 2024-05-08T05:56:40.340 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.603+0000 7fa2cc000700 1 -- start start 2024-05-08T05:56:40.340 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.603+0000 7fa2cc000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2c4108e70 0x7fa2c407eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:40.340 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.603+0000 7fa2cc000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c4109950 con 0x7fa2c4108e70 2024-05-08T05:56:40.340 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.603+0000 7fa2c9600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2c4108e70 0x7fa2c407eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:40.340 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.603+0000 7fa2c9600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2c4108e70 0x7fa2c407eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52540/0 (socket says 172.21.15.123:52540) 2024-05-08T05:56:40.341 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.603+0000 7fa2c9600700 1 -- 172.21.15.123:0/1879008525 learned_addr learned my addr 172.21.15.123:0/1879008525 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:56:40.341 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.604+0000 7fa2c9600700 1 -- 172.21.15.123:0/1879008525 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c407f0d0 con 0x7fa2c4108e70 2024-05-08T05:56:40.341 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.604+0000 7fa2c9600700 1 --2- 172.21.15.123:0/1879008525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2c4108e70 0x7fa2c407eb90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa2b80117b0 tx=0x7fa2b8011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:40.341 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.604+0000 7fa2c2000700 1 -- 172.21.15.123:0/1879008525 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2b8003af0 con 0x7fa2c4108e70 2024-05-08T05:56:40.341 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.604+0000 7fa2c2000700 1 -- 172.21.15.123:0/1879008525 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa2b8020070 con 0x7fa2c4108e70 2024-05-08T05:56:40.341 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.604+0000 7fa2cc000700 1 -- 172.21.15.123:0/1879008525 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2c407d2c0 con 0x7fa2c4108e70 2024-05-08T05:56:40.341 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.604+0000 7fa2c2000700 1 -- 172.21.15.123:0/1879008525 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2b8019550 con 0x7fa2c4108e70 2024-05-08T05:56:40.341 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.604+0000 7fa2cc000700 1 -- 172.21.15.123:0/1879008525 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2c407d800 con 0x7fa2c4108e70 2024-05-08T05:56:40.341 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.605+0000 7fa2c2000700 1 -- 172.21.15.123:0/1879008525 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa2b8019ba0 con 0x7fa2c4108e70 2024-05-08T05:56:40.341 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.605+0000 7fa2c2000700 1 -- 172.21.15.123:0/1879008525 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa2b800b720 con 0x7fa2c4108e70 2024-05-08T05:56:40.341 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.606+0000 7fa2cc000700 1 -- 172.21.15.123:0/1879008525 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2c4064b00 con 0x7fa2c4108e70 2024-05-08T05:56:40.341 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.610+0000 7fa2c2000700 1 -- 172.21.15.123:0/1879008525 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fa2b80042a0 con 0x7fa2c4108e70 2024-05-08T05:56:40.341 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.689+0000 7fa2cc000700 1 -- 172.21.15.123:0/1879008525 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa2c407db40 con 0x7fa2c4108e70 2024-05-08T05:56:40.341 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.689+0000 7fa2c2000700 1 -- 172.21.15.123:0/1879008525 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7fa2b800b970 con 0x7fa2c4108e70 2024-05-08T05:56:40.342 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.691+0000 7fa2cc000700 1 -- 172.21.15.123:0/1879008525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2c4108e70 msgr2=0x7fa2c407eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:40.342 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.691+0000 7fa2cc000700 1 --2- 172.21.15.123:0/1879008525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2c4108e70 0x7fa2c407eb90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa2b80117b0 tx=0x7fa2b8011b70 comp rx=0 tx=0).stop 2024-05-08T05:56:40.342 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.691+0000 7fa2cc000700 1 -- 172.21.15.123:0/1879008525 shutdown_connections 2024-05-08T05:56:40.342 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.691+0000 7fa2cc000700 1 --2- 172.21.15.123:0/1879008525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2c4108e70 0x7fa2c407eb90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:40.342 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.691+0000 7fa2cc000700 1 -- 172.21.15.123:0/1879008525 >> 172.21.15.123:0/1879008525 conn(0x7fa2c4104440 msgr2=0x7fa2c4105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:40.342 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.691+0000 7fa2cc000700 1 -- 172.21.15.123:0/1879008525 shutdown_connections 2024-05-08T05:56:40.342 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:39.691+0000 7fa2cc000700 1 -- 172.21.15.123:0/1879008525 wait complete. 2024-05-08T05:56:40.342 INFO:teuthology.orchestra.run.smithi123.stdout:mgr not available, waiting (3/15)... 2024-05-08T05:56:40.705 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:40 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:40.447+0000 7ff568401200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T05:56:41.087 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:40 smithi123 ceph-mon[29787]: Activating manager daemon x 2024-05-08T05:56:41.087 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:40 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:40.730+0000 7ff568401200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T05:56:42.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:41 smithi123 ceph-mon[29787]: mgrmap e2: x(active, starting, since 0.0112085s) 2024-05-08T05:56:42.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:41 smithi123 ceph-mon[29787]: from='mgr.14100 172.21.15.123:0/593221181' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T05:56:42.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:41 smithi123 ceph-mon[29787]: from='mgr.14100 172.21.15.123:0/593221181' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T05:56:42.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:41 smithi123 ceph-mon[29787]: from='mgr.14100 172.21.15.123:0/593221181' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T05:56:42.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:41 smithi123 ceph-mon[29787]: from='mgr.14100 172.21.15.123:0/593221181' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:56:42.207 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:41 smithi123 ceph-mon[29787]: from='mgr.14100 172.21.15.123:0/593221181' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T05:56:42.207 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:41 smithi123 ceph-mon[29787]: Manager daemon x is now available 2024-05-08T05:56:42.207 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:41 smithi123 ceph-mon[29787]: from='mgr.14100 172.21.15.123:0/593221181' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T05:56:42.207 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:41 smithi123 ceph-mon[29787]: from='mgr.14100 172.21.15.123:0/593221181' entity='mgr.x' 2024-05-08T05:56:42.207 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:41 smithi123 ceph-mon[29787]: from='mgr.14100 172.21.15.123:0/593221181' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T05:56:42.207 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:41 smithi123 ceph-mon[29787]: from='mgr.14100 172.21.15.123:0/593221181' entity='mgr.x' 2024-05-08T05:56:42.207 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:41 smithi123 ceph-mon[29787]: from='mgr.14100 172.21.15.123:0/593221181' entity='mgr.x' 2024-05-08T05:56:43.017 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:42 smithi123 ceph-mon[29787]: mgrmap e3: x(active, since 1.04095s) 2024-05-08T05:56:44.046 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:43 smithi123 ceph-mon[29787]: mgrmap e4: x(active, since 2s) 2024-05-08T05:56:44.046 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:43 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/1420934840' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T05:56:44.059 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 2024-05-08T05:56:44.059 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout { 2024-05-08T05:56:44.059 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "fsid": "85cfb2e6-0cff-11ef-bc97-c7b262605968", 2024-05-08T05:56:44.059 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T05:56:44.059 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T05:56:44.059 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T05:56:44.060 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T05:56:44.060 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:56:44.060 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T05:56:44.060 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T05:56:44.060 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 0 2024-05-08T05:56:44.060 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:56:44.060 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T05:56:44.060 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T05:56:44.060 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:56:44.060 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2024-05-08T05:56:44.060 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T05:56:44.060 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:56:44.060 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-08T05:56:44.060 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T05:56:44.060 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:56:44.061 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T05:56:44.061 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:56:44.061 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T05:56:44.061 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T05:56:44.061 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T05:56:44.070 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T05:56:44.070 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T05:56:44.070 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T05:56:44.070 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:56:44.070 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T05:56:44.071 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T05:56:44.071 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T05:56:44.071 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T05:56:44.071 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T05:56:44.071 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T05:56:44.071 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T05:56:44.071 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T05:56:44.071 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T05:56:44.071 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:56:44.071 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T05:56:44.071 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:56:44.071 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T05:56:44.071 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T05:56:44.072 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:56:44.072 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T05:56:44.072 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-08T05:56:44.072 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T05:56:44.072 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T05:56:44.072 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T05:56:44.072 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T05:56:44.072 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T05:56:44.072 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:56:44.073 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T05:56:44.073 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:56:44.073 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T05:56:44.073 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:56:44.073 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T05:56:12.545340+0000", 2024-05-08T05:56:44.073 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T05:56:44.073 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:56:44.073 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T05:56:44.073 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout } 2024-05-08T05:56:44.073 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.002+0000 7f69be600700 1 Processor -- start 2024-05-08T05:56:44.073 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.003+0000 7f69be600700 1 -- start start 2024-05-08T05:56:44.073 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.003+0000 7f69be600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69b810b120 0x7f69b810b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:44.073 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.003+0000 7f69be600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f69b810bb30 con 0x7f69b810b120 2024-05-08T05:56:44.073 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.003+0000 7f69bd200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69b810b120 0x7f69b810b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:44.074 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.003+0000 7f69bd200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69b810b120 0x7f69b810b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52598/0 (socket says 172.21.15.123:52598) 2024-05-08T05:56:44.074 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.004+0000 7f69bd200700 1 -- 172.21.15.123:0/2584763099 learned_addr learned my addr 172.21.15.123:0/2584763099 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:56:44.074 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.004+0000 7f69bd200700 1 -- 172.21.15.123:0/2584763099 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69b810bcb0 con 0x7f69b810b120 2024-05-08T05:56:44.074 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.004+0000 7f69bd200700 1 --2- 172.21.15.123:0/2584763099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69b810b120 0x7f69b810b560 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f69ac0098a0 tx=0x7f69ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d7b6eafef53d7d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:44.074 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.005+0000 7f69b7400700 1 -- 172.21.15.123:0/2584763099 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69ac004160 con 0x7f69b810b120 2024-05-08T05:56:44.074 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.005+0000 7f69b7400700 1 -- 172.21.15.123:0/2584763099 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f69ac004300 con 0x7f69b810b120 2024-05-08T05:56:44.074 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.005+0000 7f69b7400700 1 -- 172.21.15.123:0/2584763099 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69ac011af0 con 0x7f69b810b120 2024-05-08T05:56:44.074 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.005+0000 7f69be600700 1 -- 172.21.15.123:0/2584763099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69b810b120 msgr2=0x7f69b810b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:44.074 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.005+0000 7f69be600700 1 --2- 172.21.15.123:0/2584763099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69b810b120 0x7f69b810b560 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f69ac0098a0 tx=0x7f69ac009bb0 comp rx=0 tx=0).stop 2024-05-08T05:56:44.074 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.005+0000 7f69be600700 1 -- 172.21.15.123:0/2584763099 shutdown_connections 2024-05-08T05:56:44.074 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.005+0000 7f69be600700 1 --2- 172.21.15.123:0/2584763099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69b810b120 0x7f69b810b560 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:44.074 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.005+0000 7f69be600700 1 -- 172.21.15.123:0/2584763099 >> 172.21.15.123:0/2584763099 conn(0x7f69b807e350 msgr2=0x7f69b807e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:44.074 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.005+0000 7f69be600700 1 -- 172.21.15.123:0/2584763099 shutdown_connections 2024-05-08T05:56:44.075 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.006+0000 7f69be600700 1 -- 172.21.15.123:0/2584763099 wait complete. 2024-05-08T05:56:44.075 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.007+0000 7f69be600700 1 Processor -- start 2024-05-08T05:56:44.075 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.007+0000 7f69be600700 1 -- start start 2024-05-08T05:56:44.075 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.007+0000 7f69be600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69b810b120 0x7f69b819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:44.075 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.007+0000 7f69be600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f69b810cf10 con 0x7f69b810b120 2024-05-08T05:56:44.075 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.007+0000 7f69bd200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69b810b120 0x7f69b819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:44.075 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.007+0000 7f69bd200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69b810b120 0x7f69b819eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52610/0 (socket says 172.21.15.123:52610) 2024-05-08T05:56:44.075 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.007+0000 7f69bd200700 1 -- 172.21.15.123:0/1420934840 learned_addr learned my addr 172.21.15.123:0/1420934840 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:56:44.075 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.008+0000 7f69bd200700 1 -- 172.21.15.123:0/1420934840 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69b819f3f0 con 0x7f69b810b120 2024-05-08T05:56:44.075 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.008+0000 7f69bd200700 1 --2- 172.21.15.123:0/1420934840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69b810b120 0x7f69b819eeb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f69ac0117b0 tx=0x7f69ac011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:44.075 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.008+0000 7f69b5600700 1 -- 172.21.15.123:0/1420934840 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69ac003af0 con 0x7f69b810b120 2024-05-08T05:56:44.075 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.008+0000 7f69b5600700 1 -- 172.21.15.123:0/1420934840 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f69ac020070 con 0x7f69b810b120 2024-05-08T05:56:44.075 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.008+0000 7f69b5600700 1 -- 172.21.15.123:0/1420934840 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69ac019630 con 0x7f69b810b120 2024-05-08T05:56:44.076 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.008+0000 7f69be600700 1 -- 172.21.15.123:0/1420934840 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69b819f680 con 0x7f69b810b120 2024-05-08T05:56:44.076 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.009+0000 7f69be600700 1 -- 172.21.15.123:0/1420934840 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69b81a2370 con 0x7f69b810b120 2024-05-08T05:56:44.076 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.010+0000 7f69b5600700 1 -- 172.21.15.123:0/1420934840 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f69ac01a9c0 con 0x7f69b810b120 2024-05-08T05:56:44.076 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.010+0000 7f69be600700 1 -- 172.21.15.123:0/1420934840 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69b806a5e0 con 0x7f69b810b120 2024-05-08T05:56:44.076 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.010+0000 7f69b5600700 1 --2- 172.21.15.123:0/1420934840 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7f69a003e470 0x7f69a0040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:44.076 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.010+0000 7f69b5600700 1 -- 172.21.15.123:0/1420934840 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f69ac053a60 con 0x7f69b810b120 2024-05-08T05:56:44.076 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.010+0000 7f69b7e00700 1 --2- 172.21.15.123:0/1420934840 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7f69a003e470 0x7f69a0040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:44.076 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.011+0000 7f69b7e00700 1 --2- 172.21.15.123:0/1420934840 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7f69a003e470 0x7f69a0040930 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f69a4006fd0 tx=0x7f69a4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:44.076 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.017+0000 7f69b5600700 1 -- 172.21.15.123:0/1420934840 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f69b810bcb0 con 0x7f69b810b120 2024-05-08T05:56:44.076 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.326+0000 7f69be600700 1 -- 172.21.15.123:0/1420934840 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f69b81a2540 con 0x7f69b810b120 2024-05-08T05:56:44.076 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.327+0000 7f69b5600700 1 -- 172.21.15.123:0/1420934840 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f69b81a2540 con 0x7f69b810b120 2024-05-08T05:56:44.076 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.329+0000 7f69be600700 1 -- 172.21.15.123:0/1420934840 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7f69a003e470 msgr2=0x7f69a0040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:44.076 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.329+0000 7f69be600700 1 --2- 172.21.15.123:0/1420934840 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7f69a003e470 0x7f69a0040930 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f69a4006fd0 tx=0x7f69a4006e40 comp rx=0 tx=0).stop 2024-05-08T05:56:44.076 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.329+0000 7f69be600700 1 -- 172.21.15.123:0/1420934840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69b810b120 msgr2=0x7f69b819eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:44.077 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.329+0000 7f69be600700 1 --2- 172.21.15.123:0/1420934840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69b810b120 0x7f69b819eeb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f69ac0117b0 tx=0x7f69ac011b70 comp rx=0 tx=0).stop 2024-05-08T05:56:44.077 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.330+0000 7f69be600700 1 -- 172.21.15.123:0/1420934840 shutdown_connections 2024-05-08T05:56:44.077 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.330+0000 7f69be600700 1 --2- 172.21.15.123:0/1420934840 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7f69a003e470 0x7f69a0040930 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:44.077 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.330+0000 7f69be600700 1 --2- 172.21.15.123:0/1420934840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69b810b120 0x7f69b819eeb0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:44.077 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.330+0000 7f69be600700 1 -- 172.21.15.123:0/1420934840 >> 172.21.15.123:0/1420934840 conn(0x7f69b807e350 msgr2=0x7f69b8108dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:44.077 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.330+0000 7f69be600700 1 -- 172.21.15.123:0/1420934840 shutdown_connections 2024-05-08T05:56:44.077 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:43.330+0000 7f69be600700 1 -- 172.21.15.123:0/1420934840 wait complete. 2024-05-08T05:56:44.077 INFO:teuthology.orchestra.run.smithi123.stdout:mgr is available 2024-05-08T05:56:45.372 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:45 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/126385477' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-08T05:56:45.695 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 2024-05-08T05:56:45.695 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout [global] 2024-05-08T05:56:45.695 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout fsid = 85cfb2e6-0cff-11ef-bc97-c7b262605968 2024-05-08T05:56:45.695 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-08T05:56:45.695 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-08T05:56:45.695 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-08T05:56:45.695 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-08T05:56:45.696 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-08T05:56:45.696 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 2024-05-08T05:56:45.696 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-08T05:56:45.696 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-08T05:56:45.696 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 2024-05-08T05:56:45.696 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout [osd] 2024-05-08T05:56:45.696 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-08T05:56:45.696 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-08T05:56:45.696 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-08T05:56:45.696 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-08T05:56:45.696 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-08T05:56:45.696 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.761+0000 7fba2fe00700 1 Processor -- start 2024-05-08T05:56:45.696 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.762+0000 7fba2fe00700 1 -- start start 2024-05-08T05:56:45.696 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.762+0000 7fba2fe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba28108ed0 0x7fba281092f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:45.697 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.762+0000 7fba2fe00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fba28109830 con 0x7fba28108ed0 2024-05-08T05:56:45.697 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.762+0000 7fba2d400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba28108ed0 0x7fba281092f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:45.697 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.763+0000 7fba2d400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba28108ed0 0x7fba281092f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52624/0 (socket says 172.21.15.123:52624) 2024-05-08T05:56:45.697 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.763+0000 7fba2d400700 1 -- 172.21.15.123:0/3453241294 learned_addr learned my addr 172.21.15.123:0/3453241294 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:56:45.697 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.763+0000 7fba2d400700 1 -- 172.21.15.123:0/3453241294 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba2810a070 con 0x7fba28108ed0 2024-05-08T05:56:45.697 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.763+0000 7fba2d400700 1 --2- 172.21.15.123:0/3453241294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba28108ed0 0x7fba281092f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fba1c0098a0 tx=0x7fba1c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42639a1028a20352 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:45.697 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.763+0000 7fba27e00700 1 -- 172.21.15.123:0/3453241294 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba1c004160 con 0x7fba28108ed0 2024-05-08T05:56:45.697 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.764+0000 7fba27e00700 1 -- 172.21.15.123:0/3453241294 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fba1c004300 con 0x7fba28108ed0 2024-05-08T05:56:45.697 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.764+0000 7fba27e00700 1 -- 172.21.15.123:0/3453241294 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba1c011af0 con 0x7fba28108ed0 2024-05-08T05:56:45.697 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.764+0000 7fba2fe00700 1 -- 172.21.15.123:0/3453241294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba28108ed0 msgr2=0x7fba281092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:45.697 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.764+0000 7fba2fe00700 1 --2- 172.21.15.123:0/3453241294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba28108ed0 0x7fba281092f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fba1c0098a0 tx=0x7fba1c009bb0 comp rx=0 tx=0).stop 2024-05-08T05:56:45.697 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.764+0000 7fba2fe00700 1 -- 172.21.15.123:0/3453241294 shutdown_connections 2024-05-08T05:56:45.697 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.764+0000 7fba2fe00700 1 --2- 172.21.15.123:0/3453241294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba28108ed0 0x7fba281092f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:45.697 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.764+0000 7fba2fe00700 1 -- 172.21.15.123:0/3453241294 >> 172.21.15.123:0/3453241294 conn(0x7fba281044e0 msgr2=0x7fba28106900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:45.698 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.764+0000 7fba2fe00700 1 -- 172.21.15.123:0/3453241294 shutdown_connections 2024-05-08T05:56:45.698 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.764+0000 7fba2fe00700 1 -- 172.21.15.123:0/3453241294 wait complete. 2024-05-08T05:56:45.698 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.765+0000 7fba2fe00700 1 Processor -- start 2024-05-08T05:56:45.698 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.766+0000 7fba2fe00700 1 -- start start 2024-05-08T05:56:45.698 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.766+0000 7fba2fe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba28108ed0 0x7fba2807f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:45.698 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.766+0000 7fba2fe00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fba281099b0 con 0x7fba28108ed0 2024-05-08T05:56:45.698 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.766+0000 7fba2d400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba28108ed0 0x7fba2807f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:45.698 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.766+0000 7fba2d400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba28108ed0 0x7fba2807f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52632/0 (socket says 172.21.15.123:52632) 2024-05-08T05:56:45.698 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.766+0000 7fba2d400700 1 -- 172.21.15.123:0/126385477 learned_addr learned my addr 172.21.15.123:0/126385477 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:56:45.698 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.766+0000 7fba2d400700 1 -- 172.21.15.123:0/126385477 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba2807f960 con 0x7fba28108ed0 2024-05-08T05:56:45.698 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.767+0000 7fba2d400700 1 --2- 172.21.15.123:0/126385477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba28108ed0 0x7fba2807f420 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fba1c0117b0 tx=0x7fba1c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:45.698 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.767+0000 7fba26000700 1 -- 172.21.15.123:0/126385477 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba1c003af0 con 0x7fba28108ed0 2024-05-08T05:56:45.698 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.767+0000 7fba26000700 1 -- 172.21.15.123:0/126385477 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fba1c020070 con 0x7fba28108ed0 2024-05-08T05:56:45.698 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.767+0000 7fba2fe00700 1 -- 172.21.15.123:0/126385477 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba2807db50 con 0x7fba28108ed0 2024-05-08T05:56:45.699 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.767+0000 7fba26000700 1 -- 172.21.15.123:0/126385477 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba1c019550 con 0x7fba28108ed0 2024-05-08T05:56:45.699 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.767+0000 7fba2fe00700 1 -- 172.21.15.123:0/126385477 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba2807e090 con 0x7fba28108ed0 2024-05-08T05:56:45.699 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.768+0000 7fba26000700 1 -- 172.21.15.123:0/126385477 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fba1c01a430 con 0x7fba28108ed0 2024-05-08T05:56:45.699 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.768+0000 7fba2fe00700 1 -- 172.21.15.123:0/126385477 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba28064b00 con 0x7fba28108ed0 2024-05-08T05:56:45.699 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.769+0000 7fba26000700 1 --2- 172.21.15.123:0/126385477 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7fba1003e420 0x7fba100408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:45.699 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.769+0000 7fba26000700 1 -- 172.21.15.123:0/126385477 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fba1c053990 con 0x7fba28108ed0 2024-05-08T05:56:45.699 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.769+0000 7fba2ca00700 1 --2- 172.21.15.123:0/126385477 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7fba1003e420 0x7fba100408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:45.699 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.770+0000 7fba2ca00700 1 --2- 172.21.15.123:0/126385477 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7fba1003e420 0x7fba100408e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fba14006fd0 tx=0x7fba14006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:45.699 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:44.774+0000 7fba26000700 1 -- 172.21.15.123:0/126385477 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fba1c0172b0 con 0x7fba28108ed0 2024-05-08T05:56:45.699 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:45.010+0000 7fba2fe00700 1 -- 172.21.15.123:0/126385477 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fba2807e310 con 0x7fba28108ed0 2024-05-08T05:56:45.699 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:45.013+0000 7fba26000700 1 -- 172.21.15.123:0/126385477 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7fba1c00be30 con 0x7fba28108ed0 2024-05-08T05:56:45.699 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:45.016+0000 7fba2fe00700 1 -- 172.21.15.123:0/126385477 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7fba1003e420 msgr2=0x7fba100408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:45.699 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:45.016+0000 7fba2fe00700 1 --2- 172.21.15.123:0/126385477 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7fba1003e420 0x7fba100408e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fba14006fd0 tx=0x7fba14006e40 comp rx=0 tx=0).stop 2024-05-08T05:56:45.699 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:45.016+0000 7fba2fe00700 1 -- 172.21.15.123:0/126385477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba28108ed0 msgr2=0x7fba2807f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:45.700 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:45.016+0000 7fba2fe00700 1 --2- 172.21.15.123:0/126385477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba28108ed0 0x7fba2807f420 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fba1c0117b0 tx=0x7fba1c011b70 comp rx=0 tx=0).stop 2024-05-08T05:56:45.700 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:45.016+0000 7fba2fe00700 1 -- 172.21.15.123:0/126385477 shutdown_connections 2024-05-08T05:56:45.700 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:45.016+0000 7fba2fe00700 1 --2- 172.21.15.123:0/126385477 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7fba1003e420 0x7fba100408e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:45.700 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:45.016+0000 7fba2fe00700 1 --2- 172.21.15.123:0/126385477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba28108ed0 0x7fba2807f420 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:45.700 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:45.016+0000 7fba2fe00700 1 -- 172.21.15.123:0/126385477 >> 172.21.15.123:0/126385477 conn(0x7fba281044e0 msgr2=0x7fba281050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:45.700 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:45.016+0000 7fba2fe00700 1 -- 172.21.15.123:0/126385477 shutdown_connections 2024-05-08T05:56:45.700 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:45.016+0000 7fba2fe00700 1 -- 172.21.15.123:0/126385477 wait complete. 2024-05-08T05:56:45.700 INFO:teuthology.orchestra.run.smithi123.stdout:Enabling cephadm module... 2024-05-08T05:56:46.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:46 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/3585497550' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-08T05:56:46.955 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:46 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: ignoring --setuser ceph since I am not root 2024-05-08T05:56:46.955 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:46 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: ignoring --setgroup ceph since I am not root 2024-05-08T05:56:47.342 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:46 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:46.972+0000 7f2a2e0a8200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T05:56:47.342 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:47 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:47.212+0000 7f2a2e0a8200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:56:47.342 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:47 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:47.341+0000 7f2a2e0a8200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T05:56:47.412 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.323+0000 7fa7f5600700 1 Processor -- start 2024-05-08T05:56:47.413 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.323+0000 7fa7f5600700 1 -- start start 2024-05-08T05:56:47.413 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.323+0000 7fa7f5600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7f01077a0 0x7fa7f0107bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:47.413 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.324+0000 7fa7f5600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f007ce50 con 0x7fa7f01077a0 2024-05-08T05:56:47.413 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.324+0000 7fa7eea00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7f01077a0 0x7fa7f0107bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:47.413 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.324+0000 7fa7eea00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7f01077a0 0x7fa7f0107bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41804/0 (socket says 172.21.15.123:41804) 2024-05-08T05:56:47.413 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.324+0000 7fa7eea00700 1 -- 172.21.15.123:0/477371169 learned_addr learned my addr 172.21.15.123:0/477371169 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:56:47.413 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.325+0000 7fa7eea00700 1 -- 172.21.15.123:0/477371169 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7f0108100 con 0x7fa7f01077a0 2024-05-08T05:56:47.413 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.325+0000 7fa7eea00700 1 --2- 172.21.15.123:0/477371169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7f01077a0 0x7fa7f0107bc0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa7e00098a0 tx=0x7fa7e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e3fe7fff2337dde server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:47.413 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.325+0000 7fa7ed600700 1 -- 172.21.15.123:0/477371169 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7e0004160 con 0x7fa7f01077a0 2024-05-08T05:56:47.413 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.325+0000 7fa7ed600700 1 -- 172.21.15.123:0/477371169 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa7e0004300 con 0x7fa7f01077a0 2024-05-08T05:56:47.413 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.325+0000 7fa7ed600700 1 -- 172.21.15.123:0/477371169 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7e0011af0 con 0x7fa7f01077a0 2024-05-08T05:56:47.413 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.326+0000 7fa7f5600700 1 -- 172.21.15.123:0/477371169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7f01077a0 msgr2=0x7fa7f0107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:47.413 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.326+0000 7fa7f5600700 1 --2- 172.21.15.123:0/477371169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7f01077a0 0x7fa7f0107bc0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa7e00098a0 tx=0x7fa7e0009bb0 comp rx=0 tx=0).stop 2024-05-08T05:56:47.414 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.326+0000 7fa7f5600700 1 -- 172.21.15.123:0/477371169 shutdown_connections 2024-05-08T05:56:47.414 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.326+0000 7fa7f5600700 1 --2- 172.21.15.123:0/477371169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7f01077a0 0x7fa7f0107bc0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:47.414 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.326+0000 7fa7f5600700 1 -- 172.21.15.123:0/477371169 >> 172.21.15.123:0/477371169 conn(0x7fa7f0103380 msgr2=0x7fa7f01057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:47.414 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.326+0000 7fa7f5600700 1 -- 172.21.15.123:0/477371169 shutdown_connections 2024-05-08T05:56:47.414 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.326+0000 7fa7f5600700 1 -- 172.21.15.123:0/477371169 wait complete. 2024-05-08T05:56:47.414 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.327+0000 7fa7f5600700 1 Processor -- start 2024-05-08T05:56:47.414 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.327+0000 7fa7f5600700 1 -- start start 2024-05-08T05:56:47.414 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.328+0000 7fa7f5600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7f01077a0 0x7fa7f019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:47.414 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.328+0000 7fa7f5600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f01090c0 con 0x7fa7f01077a0 2024-05-08T05:56:47.414 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.328+0000 7fa7eea00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7f01077a0 0x7fa7f019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:47.414 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.328+0000 7fa7eea00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7f01077a0 0x7fa7f019ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41820/0 (socket says 172.21.15.123:41820) 2024-05-08T05:56:47.414 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.328+0000 7fa7eea00700 1 -- 172.21.15.123:0/3585497550 learned_addr learned my addr 172.21.15.123:0/3585497550 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:56:47.414 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.328+0000 7fa7eea00700 1 -- 172.21.15.123:0/3585497550 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7f019f490 con 0x7fa7f01077a0 2024-05-08T05:56:47.414 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.329+0000 7fa7eea00700 1 --2- 172.21.15.123:0/3585497550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7f01077a0 0x7fa7f019ef50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0003f40 tx=0x7fa7e0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:47.415 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.329+0000 7fa7e7400700 1 -- 172.21.15.123:0/3585497550 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7e00038b0 con 0x7fa7f01077a0 2024-05-08T05:56:47.415 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.329+0000 7fa7e7400700 1 -- 172.21.15.123:0/3585497550 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa7e0003a50 con 0x7fa7f01077a0 2024-05-08T05:56:47.415 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.329+0000 7fa7e7400700 1 -- 172.21.15.123:0/3585497550 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7e001a5c0 con 0x7fa7f01077a0 2024-05-08T05:56:47.415 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.329+0000 7fa7f5600700 1 -- 172.21.15.123:0/3585497550 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7f019f720 con 0x7fa7f01077a0 2024-05-08T05:56:47.415 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.329+0000 7fa7f5600700 1 -- 172.21.15.123:0/3585497550 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7f01a2410 con 0x7fa7f01077a0 2024-05-08T05:56:47.415 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.330+0000 7fa7f5600700 1 -- 172.21.15.123:0/3585497550 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7f0064b00 con 0x7fa7f01077a0 2024-05-08T05:56:47.415 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.330+0000 7fa7e7400700 1 -- 172.21.15.123:0/3585497550 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fa7e0024430 con 0x7fa7f01077a0 2024-05-08T05:56:47.415 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.331+0000 7fa7e7400700 1 --2- 172.21.15.123:0/3585497550 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7fa7d403e3d0 0x7fa7d4040890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:47.415 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.331+0000 7fa7e7400700 1 -- 172.21.15.123:0/3585497550 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa7e0053a50 con 0x7fa7f01077a0 2024-05-08T05:56:47.415 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.331+0000 7fa7ee000700 1 --2- 172.21.15.123:0/3585497550 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7fa7d403e3d0 0x7fa7d4040890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:47.415 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.332+0000 7fa7ee000700 1 --2- 172.21.15.123:0/3585497550 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7fa7d403e3d0 0x7fa7d4040890 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc006fd0 tx=0x7fa7dc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:47.415 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.336+0000 7fa7e7400700 1 -- 172.21.15.123:0/3585497550 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa7e000b3e0 con 0x7fa7f01077a0 2024-05-08T05:56:47.415 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.630+0000 7fa7f5600700 1 -- 172.21.15.123:0/3585497550 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fa7f00008d0 con 0x7fa7f01077a0 2024-05-08T05:56:47.416 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.683+0000 7fa7e7400700 1 -- 172.21.15.123:0/3585497550 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fa7e000e260 con 0x7fa7f01077a0 2024-05-08T05:56:47.416 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.687+0000 7fa7e7400700 1 -- 172.21.15.123:0/3585497550 <== mon.0 v2:172.21.15.123:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fa7e002d3f0 con 0x7fa7f01077a0 2024-05-08T05:56:47.416 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.688+0000 7fa7f5600700 1 -- 172.21.15.123:0/3585497550 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7fa7d403e3d0 msgr2=0x7fa7d4040890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:47.416 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.688+0000 7fa7f5600700 1 --2- 172.21.15.123:0/3585497550 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7fa7d403e3d0 0x7fa7d4040890 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc006fd0 tx=0x7fa7dc006e40 comp rx=0 tx=0).stop 2024-05-08T05:56:47.416 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.688+0000 7fa7f5600700 1 -- 172.21.15.123:0/3585497550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7f01077a0 msgr2=0x7fa7f019ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:47.416 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.689+0000 7fa7f5600700 1 --2- 172.21.15.123:0/3585497550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7f01077a0 0x7fa7f019ef50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0003f40 tx=0x7fa7e0011890 comp rx=0 tx=0).stop 2024-05-08T05:56:47.416 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.689+0000 7fa7f5600700 1 -- 172.21.15.123:0/3585497550 shutdown_connections 2024-05-08T05:56:47.416 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.689+0000 7fa7f5600700 1 --2- 172.21.15.123:0/3585497550 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7fa7d403e3d0 0x7fa7d4040890 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:47.416 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.689+0000 7fa7f5600700 1 --2- 172.21.15.123:0/3585497550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7f01077a0 0x7fa7f019ef50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:47.416 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.689+0000 7fa7f5600700 1 -- 172.21.15.123:0/3585497550 >> 172.21.15.123:0/3585497550 conn(0x7fa7f0103380 msgr2=0x7fa7f0195bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:47.416 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.689+0000 7fa7f5600700 1 -- 172.21.15.123:0/3585497550 shutdown_connections 2024-05-08T05:56:47.416 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:46.689+0000 7fa7f5600700 1 -- 172.21.15.123:0/3585497550 wait complete. 2024-05-08T05:56:47.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:47 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/3585497550' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-08T05:56:47.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:47 smithi123 ceph-mon[29787]: mgrmap e5: x(active, since 5s) 2024-05-08T05:56:47.955 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:47 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:47.752+0000 7f2a2e0a8200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T05:56:47.955 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:47 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:47.867+0000 7f2a2e0a8200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T05:56:48.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:48 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/1156907241' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-08T05:56:48.984 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout { 2024-05-08T05:56:48.985 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-08T05:56:48.985 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-08T05:56:48.985 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-08T05:56:48.985 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-08T05:56:48.985 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout } 2024-05-08T05:56:48.985 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.053+0000 7f925d600700 1 Processor -- start 2024-05-08T05:56:48.985 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.053+0000 7f925d600700 1 -- start start 2024-05-08T05:56:48.985 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.053+0000 7f925d600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9258108e70 0x7f9258109290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:48.985 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.053+0000 7f925d600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f92581097d0 con 0x7f9258108e70 2024-05-08T05:56:48.985 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.054+0000 7f9256a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9258108e70 0x7f9258109290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:48.985 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.054+0000 7f9256a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9258108e70 0x7f9258109290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41856/0 (socket says 172.21.15.123:41856) 2024-05-08T05:56:48.986 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.054+0000 7f9256a00700 1 -- 172.21.15.123:0/713218108 learned_addr learned my addr 172.21.15.123:0/713218108 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:56:48.986 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.054+0000 7f9256a00700 1 -- 172.21.15.123:0/713218108 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f925810a060 con 0x7f9258108e70 2024-05-08T05:56:48.986 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.054+0000 7f9256a00700 1 --2- 172.21.15.123:0/713218108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9258108e70 0x7f9258109290 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f92480098a0 tx=0x7f9248009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e39029148cd8ffd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:48.986 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.055+0000 7f9255600700 1 -- 172.21.15.123:0/713218108 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9248004160 con 0x7f9258108e70 2024-05-08T05:56:48.986 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.055+0000 7f9255600700 1 -- 172.21.15.123:0/713218108 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9248004300 con 0x7f9258108e70 2024-05-08T05:56:48.986 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.055+0000 7f9255600700 1 -- 172.21.15.123:0/713218108 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9248011af0 con 0x7f9258108e70 2024-05-08T05:56:48.986 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.055+0000 7f925d600700 1 -- 172.21.15.123:0/713218108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9258108e70 msgr2=0x7f9258109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:48.986 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.055+0000 7f925d600700 1 --2- 172.21.15.123:0/713218108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9258108e70 0x7f9258109290 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f92480098a0 tx=0x7f9248009bb0 comp rx=0 tx=0).stop 2024-05-08T05:56:48.986 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.056+0000 7f925d600700 1 -- 172.21.15.123:0/713218108 shutdown_connections 2024-05-08T05:56:48.986 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.056+0000 7f925d600700 1 --2- 172.21.15.123:0/713218108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9258108e70 0x7f9258109290 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:48.986 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.056+0000 7f925d600700 1 -- 172.21.15.123:0/713218108 >> 172.21.15.123:0/713218108 conn(0x7f9258104440 msgr2=0x7f92581068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:48.986 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.056+0000 7f925d600700 1 -- 172.21.15.123:0/713218108 shutdown_connections 2024-05-08T05:56:48.986 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.056+0000 7f925d600700 1 -- 172.21.15.123:0/713218108 wait complete. 2024-05-08T05:56:48.987 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.057+0000 7f925d600700 1 Processor -- start 2024-05-08T05:56:48.987 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.057+0000 7f925d600700 1 -- start start 2024-05-08T05:56:48.987 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.057+0000 7f925d600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9258108e70 0x7f925819f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:48.987 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.057+0000 7f925d600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9258109950 con 0x7f9258108e70 2024-05-08T05:56:48.987 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.057+0000 7f9256a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9258108e70 0x7f925819f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:48.987 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.057+0000 7f9256a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9258108e70 0x7f925819f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41862/0 (socket says 172.21.15.123:41862) 2024-05-08T05:56:48.987 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.058+0000 7f9256a00700 1 -- 172.21.15.123:0/1156907241 learned_addr learned my addr 172.21.15.123:0/1156907241 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:56:48.987 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.058+0000 7f9256a00700 1 -- 172.21.15.123:0/1156907241 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f925819f580 con 0x7f9258108e70 2024-05-08T05:56:48.987 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.058+0000 7f9256a00700 1 --2- 172.21.15.123:0/1156907241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9258108e70 0x7f925819f040 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f92480117b0 tx=0x7f9248011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:48.987 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.058+0000 7f924f400700 1 -- 172.21.15.123:0/1156907241 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9248003af0 con 0x7f9258108e70 2024-05-08T05:56:48.987 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.058+0000 7f924f400700 1 -- 172.21.15.123:0/1156907241 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9248020070 con 0x7f9258108e70 2024-05-08T05:56:48.987 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.058+0000 7f925d600700 1 -- 172.21.15.123:0/1156907241 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f925819f810 con 0x7f9258108e70 2024-05-08T05:56:48.987 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.058+0000 7f924f400700 1 -- 172.21.15.123:0/1156907241 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9248019550 con 0x7f9258108e70 2024-05-08T05:56:48.987 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.059+0000 7f925d600700 1 -- 172.21.15.123:0/1156907241 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92581a2500 con 0x7f9258108e70 2024-05-08T05:56:48.988 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.060+0000 7f924cc00700 1 -- 172.21.15.123:0/1156907241 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9230005330 con 0x7f9258108e70 2024-05-08T05:56:48.988 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.060+0000 7f924f400700 1 -- 172.21.15.123:0/1156907241 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f924801a430 con 0x7f9258108e70 2024-05-08T05:56:48.988 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.061+0000 7f924f400700 1 --2- 172.21.15.123:0/1156907241 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7f923c03e4c0 0x7f923c040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:48.988 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.061+0000 7f924f400700 1 -- 172.21.15.123:0/1156907241 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9248053960 con 0x7f9258108e70 2024-05-08T05:56:48.988 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.061+0000 7f9256000700 1 -- 172.21.15.123:0/1156907241 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7f923c03e4c0 msgr2=0x7f923c040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/2769724312 2024-05-08T05:56:48.988 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.061+0000 7f9256000700 1 --2- 172.21.15.123:0/1156907241 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7f923c03e4c0 0x7f923c040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T05:56:48.988 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.066+0000 7f924f400700 1 -- 172.21.15.123:0/1156907241 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f924802ade0 con 0x7f9258108e70 2024-05-08T05:56:48.988 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.262+0000 7f9256000700 1 -- 172.21.15.123:0/1156907241 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7f923c03e4c0 msgr2=0x7f923c040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/2769724312 2024-05-08T05:56:48.988 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.262+0000 7f9256000700 1 --2- 172.21.15.123:0/1156907241 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7f923c03e4c0 0x7f923c040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T05:56:48.988 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.360+0000 7f924cc00700 1 -- 172.21.15.123:0/1156907241 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f9230005e30 con 0x7f9258108e70 2024-05-08T05:56:48.988 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.361+0000 7f924f400700 1 -- 172.21.15.123:0/1156907241 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f9248053380 con 0x7f9258108e70 2024-05-08T05:56:48.988 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.363+0000 7f924cc00700 1 -- 172.21.15.123:0/1156907241 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7f923c03e4c0 msgr2=0x7f923c040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T05:56:48.988 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.363+0000 7f924cc00700 1 --2- 172.21.15.123:0/1156907241 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7f923c03e4c0 0x7f923c040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:48.988 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.364+0000 7f924cc00700 1 -- 172.21.15.123:0/1156907241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9258108e70 msgr2=0x7f925819f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:48.989 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.364+0000 7f924cc00700 1 --2- 172.21.15.123:0/1156907241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9258108e70 0x7f925819f040 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f92480117b0 tx=0x7f9248011b70 comp rx=0 tx=0).stop 2024-05-08T05:56:48.989 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.364+0000 7f924cc00700 1 -- 172.21.15.123:0/1156907241 shutdown_connections 2024-05-08T05:56:48.989 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.364+0000 7f924cc00700 1 --2- 172.21.15.123:0/1156907241 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7f923c03e4c0 0x7f923c040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:48.989 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.364+0000 7f924cc00700 1 --2- 172.21.15.123:0/1156907241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9258108e70 0x7f925819f040 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:48.989 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.364+0000 7f924cc00700 1 -- 172.21.15.123:0/1156907241 >> 172.21.15.123:0/1156907241 conn(0x7f9258104440 msgr2=0x7f9258104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:48.989 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.364+0000 7f924cc00700 1 -- 172.21.15.123:0/1156907241 shutdown_connections 2024-05-08T05:56:48.989 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:48.364+0000 7f924cc00700 1 -- 172.21.15.123:0/1156907241 wait complete. 2024-05-08T05:56:48.989 INFO:teuthology.orchestra.run.smithi123.stdout:Waiting for the mgr to restart... 2024-05-08T05:56:48.989 INFO:teuthology.orchestra.run.smithi123.stdout:Waiting for mgr epoch 5... 2024-05-08T05:56:49.543 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:49 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:49.442+0000 7f2a2e0a8200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T05:56:49.835 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:49 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:49.677+0000 7f2a2e0a8200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T05:56:50.205 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:49 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:49.893+0000 7f2a2e0a8200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T05:56:50.654 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:50 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:50.268+0000 7f2a2e0a8200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T05:56:50.655 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:50 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:50.390+0000 7f2a2e0a8200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T05:56:50.955 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:50 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:50.654+0000 7f2a2e0a8200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T05:56:50.955 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:50 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:50.902+0000 7f2a2e0a8200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:56:51.955 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:51 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:51.682+0000 7f2a2e0a8200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T05:56:52.455 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:51 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:51.966+0000 7f2a2e0a8200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-08T05:56:52.955 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:52 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:52.510+0000 7f2a2e0a8200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T05:56:52.956 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:52 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:52.636+0000 7f2a2e0a8200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T05:56:53.455 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:53 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:53.095+0000 7f2a2e0a8200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T05:56:53.955 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:53 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:53.540+0000 7f2a2e0a8200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T05:56:53.955 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:53 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:53.663+0000 7f2a2e0a8200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T05:56:54.293 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:53 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:53.985+0000 7f2a2e0a8200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T05:56:54.293 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:54 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:54.163+0000 7f2a2e0a8200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T05:56:54.293 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:54 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:54.292+0000 7f2a2e0a8200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T05:56:54.955 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:54 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:54.687+0000 7f2a2e0a8200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T05:56:54.955 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:54 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:54.811+0000 7f2a2e0a8200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T05:56:55.842 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:55 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:55.561+0000 7f2a2e0a8200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T05:56:56.205 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:56:55 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:56:55.845+0000 7f2a2e0a8200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T05:56:56.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:55 smithi123 ceph-mon[29787]: Active manager daemon x restarted 2024-05-08T05:56:56.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:55 smithi123 ceph-mon[29787]: Activating manager daemon x 2024-05-08T05:56:56.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:55 smithi123 ceph-mon[29787]: osdmap e2: 0 total, 0 up, 0 in 2024-05-08T05:56:56.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:55 smithi123 ceph-mon[29787]: mgrmap e6: x(active, starting, since 0.00468181s) 2024-05-08T05:56:56.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:55 smithi123 ceph-mon[29787]: from='mgr.14120 172.21.15.123:0/1121855348' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:56:56.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:55 smithi123 ceph-mon[29787]: from='mgr.14120 172.21.15.123:0/1121855348' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T05:56:56.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:55 smithi123 ceph-mon[29787]: from='mgr.14120 172.21.15.123:0/1121855348' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T05:56:56.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:55 smithi123 ceph-mon[29787]: from='mgr.14120 172.21.15.123:0/1121855348' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T05:56:56.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:55 smithi123 ceph-mon[29787]: from='mgr.14120 172.21.15.123:0/1121855348' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T05:56:56.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:55 smithi123 ceph-mon[29787]: Manager daemon x is now available 2024-05-08T05:56:56.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:55 smithi123 ceph-mon[29787]: from='mgr.14120 172.21.15.123:0/1121855348' entity='mgr.x' 2024-05-08T05:56:57.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:56 smithi123 ceph-mon[29787]: Found migration_current of "None". Setting to last migration. 2024-05-08T05:56:57.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:56 smithi123 ceph-mon[29787]: from='mgr.14120 172.21.15.123:0/1121855348' entity='mgr.x' 2024-05-08T05:56:57.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:56 smithi123 ceph-mon[29787]: from='mgr.14120 172.21.15.123:0/1121855348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:56:57.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:56 smithi123 ceph-mon[29787]: from='mgr.14120 172.21.15.123:0/1121855348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:56:57.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:56 smithi123 ceph-mon[29787]: from='mgr.14120 172.21.15.123:0/1121855348' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T05:56:57.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:56 smithi123 ceph-mon[29787]: from='mgr.14120 172.21.15.123:0/1121855348' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T05:56:57.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:56 smithi123 ceph-mon[29787]: mgrmap e7: x(active, since 1.01834s) 2024-05-08T05:56:57.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:56 smithi123 ceph-mon[29787]: from='mgr.14120 172.21.15.123:0/1121855348' entity='mgr.x' 2024-05-08T05:56:57.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:56 smithi123 ceph-mon[29787]: from='mgr.14120 172.21.15.123:0/1121855348' entity='mgr.x' 2024-05-08T05:56:57.595 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout { 2024-05-08T05:56:57.595 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-08T05:56:57.595 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-08T05:56:57.595 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout } 2024-05-08T05:56:57.595 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.625+0000 7fe480e00700 1 Processor -- start 2024-05-08T05:56:57.596 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.625+0000 7fe480e00700 1 -- start start 2024-05-08T05:56:57.596 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.626+0000 7fe480e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe47c07eea0 0x7fe47c07d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:57.596 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.626+0000 7fe480e00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe47c07d8d0 con 0x7fe47c07eea0 2024-05-08T05:56:57.596 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.626+0000 7fe479e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe47c07eea0 0x7fe47c07d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:57.596 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.626+0000 7fe479e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe47c07eea0 0x7fe47c07d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41870/0 (socket says 172.21.15.123:41870) 2024-05-08T05:56:57.596 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.626+0000 7fe479e00700 1 -- 172.21.15.123:0/2038400847 learned_addr learned my addr 172.21.15.123:0/2038400847 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:56:57.596 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.627+0000 7fe479e00700 1 -- 172.21.15.123:0/2038400847 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe47c07da50 con 0x7fe47c07eea0 2024-05-08T05:56:57.596 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.627+0000 7fe479e00700 1 --2- 172.21.15.123:0/2038400847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe47c07eea0 0x7fe47c07d300 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe46c0098a0 tx=0x7fe46c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=352e4b5f8ff461f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:57.596 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.627+0000 7fe478a00700 1 -- 172.21.15.123:0/2038400847 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe46c004160 con 0x7fe47c07eea0 2024-05-08T05:56:57.596 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.627+0000 7fe478a00700 1 -- 172.21.15.123:0/2038400847 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe46c004300 con 0x7fe47c07eea0 2024-05-08T05:56:57.596 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.627+0000 7fe478a00700 1 -- 172.21.15.123:0/2038400847 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe46c011af0 con 0x7fe47c07eea0 2024-05-08T05:56:57.596 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.628+0000 7fe480e00700 1 -- 172.21.15.123:0/2038400847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe47c07eea0 msgr2=0x7fe47c07d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:57.596 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.628+0000 7fe480e00700 1 --2- 172.21.15.123:0/2038400847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe47c07eea0 0x7fe47c07d300 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe46c0098a0 tx=0x7fe46c009bb0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.596 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.628+0000 7fe480e00700 1 -- 172.21.15.123:0/2038400847 shutdown_connections 2024-05-08T05:56:57.597 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.628+0000 7fe480e00700 1 --2- 172.21.15.123:0/2038400847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe47c07eea0 0x7fe47c07d300 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.597 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.628+0000 7fe480e00700 1 -- 172.21.15.123:0/2038400847 >> 172.21.15.123:0/2038400847 conn(0x7fe47c104440 msgr2=0x7fe47c1068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:57.597 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.628+0000 7fe480e00700 1 -- 172.21.15.123:0/2038400847 shutdown_connections 2024-05-08T05:56:57.597 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.628+0000 7fe480e00700 1 -- 172.21.15.123:0/2038400847 wait complete. 2024-05-08T05:56:57.597 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.629+0000 7fe480e00700 1 Processor -- start 2024-05-08T05:56:57.597 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.629+0000 7fe480e00700 1 -- start start 2024-05-08T05:56:57.597 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.630+0000 7fe480e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe47c07eea0 0x7fe47c19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:57.597 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.630+0000 7fe480e00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe47c10f430 con 0x7fe47c07eea0 2024-05-08T05:56:57.597 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.630+0000 7fe479e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe47c07eea0 0x7fe47c19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:57.597 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.630+0000 7fe479e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe47c07eea0 0x7fe47c19f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41880/0 (socket says 172.21.15.123:41880) 2024-05-08T05:56:57.597 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.630+0000 7fe479e00700 1 -- 172.21.15.123:0/2012609743 learned_addr learned my addr 172.21.15.123:0/2012609743 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:56:57.597 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.630+0000 7fe479e00700 1 -- 172.21.15.123:0/2012609743 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe47c19f540 con 0x7fe47c07eea0 2024-05-08T05:56:57.597 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.631+0000 7fe479e00700 1 --2- 172.21.15.123:0/2012609743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe47c07eea0 0x7fe47c19f000 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe46c011ac0 tx=0x7fe46c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:57.597 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.631+0000 7fe472a00700 1 -- 172.21.15.123:0/2012609743 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe46c003970 con 0x7fe47c07eea0 2024-05-08T05:56:57.598 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.631+0000 7fe472a00700 1 -- 172.21.15.123:0/2012609743 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe46c003b10 con 0x7fe47c07eea0 2024-05-08T05:56:57.598 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.631+0000 7fe472a00700 1 -- 172.21.15.123:0/2012609743 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe46c024610 con 0x7fe47c07eea0 2024-05-08T05:56:57.598 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.631+0000 7fe480e00700 1 -- 172.21.15.123:0/2012609743 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe47c19f7d0 con 0x7fe47c07eea0 2024-05-08T05:56:57.598 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.631+0000 7fe480e00700 1 -- 172.21.15.123:0/2012609743 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe47c1a24c0 con 0x7fe47c07eea0 2024-05-08T05:56:57.598 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.632+0000 7fe472a00700 1 -- 172.21.15.123:0/2012609743 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fe46c01a9e0 con 0x7fe47c07eea0 2024-05-08T05:56:57.598 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.633+0000 7fe472a00700 1 --2- 172.21.15.123:0/2012609743 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7fe46003e4c0 0x7fe460040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:57.598 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.633+0000 7fe472a00700 1 -- 172.21.15.123:0/2012609743 --> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe460041090 con 0x7fe46003e4c0 2024-05-08T05:56:57.598 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.633+0000 7fe472a00700 1 -- 172.21.15.123:0/2012609743 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe46c053840 con 0x7fe47c07eea0 2024-05-08T05:56:57.598 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.633+0000 7fe479400700 1 -- 172.21.15.123:0/2012609743 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7fe46003e4c0 msgr2=0x7fe460040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/2769724312 2024-05-08T05:56:57.598 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.633+0000 7fe479400700 1 --2- 172.21.15.123:0/2012609743 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7fe46003e4c0 0x7fe460040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T05:56:57.598 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.834+0000 7fe479400700 1 -- 172.21.15.123:0/2012609743 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7fe46003e4c0 msgr2=0x7fe460040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/2769724312 2024-05-08T05:56:57.598 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:49.834+0000 7fe479400700 1 --2- 172.21.15.123:0/2012609743 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7fe46003e4c0 0x7fe460040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T05:56:57.598 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:50.235+0000 7fe479400700 1 -- 172.21.15.123:0/2012609743 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7fe46003e4c0 msgr2=0x7fe460040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/2769724312 2024-05-08T05:56:57.598 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:50.235+0000 7fe479400700 1 --2- 172.21.15.123:0/2012609743 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7fe46003e4c0 0x7fe460040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T05:56:57.599 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:51.036+0000 7fe479400700 1 -- 172.21.15.123:0/2012609743 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7fe46003e4c0 msgr2=0x7fe460040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/2769724312 2024-05-08T05:56:57.599 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:51.036+0000 7fe479400700 1 --2- 172.21.15.123:0/2012609743 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7fe46003e4c0 0x7fe460040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T05:56:57.599 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:52.638+0000 7fe479400700 1 -- 172.21.15.123:0/2012609743 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7fe46003e4c0 msgr2=0x7fe460040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/2769724312 2024-05-08T05:56:57.599 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:52.638+0000 7fe479400700 1 --2- 172.21.15.123:0/2012609743 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7fe46003e4c0 0x7fe460040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-08T05:56:57.599 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:55.841+0000 7fe479400700 1 -- 172.21.15.123:0/2012609743 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7fe46003e4c0 msgr2=0x7fe460040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/2769724312 2024-05-08T05:56:57.599 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:55.842+0000 7fe479400700 1 --2- 172.21.15.123:0/2012609743 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7fe46003e4c0 0x7fe460040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-05-08T05:56:57.599 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:55.853+0000 7fe472a00700 1 -- 172.21.15.123:0/2012609743 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7fe46c01a430 con 0x7fe47c07eea0 2024-05-08T05:56:57.599 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:55.854+0000 7fe472a00700 1 -- 172.21.15.123:0/2012609743 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7fe46003e4c0 msgr2=0x7fe460040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T05:56:57.599 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:55.854+0000 7fe472a00700 1 --2- 172.21.15.123:0/2012609743 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7fe46003e4c0 0x7fe460040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.599 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:56.860+0000 7fe472a00700 1 -- 172.21.15.123:0/2012609743 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7fe46c054790 con 0x7fe47c07eea0 2024-05-08T05:56:57.599 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:56.860+0000 7fe472a00700 1 --2- 172.21.15.123:0/2012609743 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7fe460041db0 0x7fe4600441a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:57.599 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:56.860+0000 7fe472a00700 1 -- 172.21.15.123:0/2012609743 --> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe46c025e70 con 0x7fe460041db0 2024-05-08T05:56:57.599 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:56.863+0000 7fe479400700 1 --2- 172.21.15.123:0/2012609743 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7fe460041db0 0x7fe4600441a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:57.599 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:56.866+0000 7fe479400700 1 --2- 172.21.15.123:0/2012609743 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7fe460041db0 0x7fe4600441a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe468003a10 tx=0x7fe4680092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:57.599 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:56.867+0000 7fe472a00700 1 -- 172.21.15.123:0/2012609743 <== mgr.14120 v2:172.21.15.123:6800/3057688649 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fe46c025e70 con 0x7fe460041db0 2024-05-08T05:56:57.600 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:56.876+0000 7fe480e00700 1 -- 172.21.15.123:0/2012609743 --> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fe47c06a5e0 con 0x7fe460041db0 2024-05-08T05:56:57.600 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:56.877+0000 7fe472a00700 1 -- 172.21.15.123:0/2012609743 <== mgr.14120 v2:172.21.15.123:6800/3057688649 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fe47c06a5e0 con 0x7fe460041db0 2024-05-08T05:56:57.600 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:56.877+0000 7fe480e00700 1 -- 172.21.15.123:0/2012609743 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7fe460041db0 msgr2=0x7fe4600441a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:57.600 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:56.877+0000 7fe480e00700 1 --2- 172.21.15.123:0/2012609743 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7fe460041db0 0x7fe4600441a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe468003a10 tx=0x7fe4680092b0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.600 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:56.877+0000 7fe480e00700 1 -- 172.21.15.123:0/2012609743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe47c07eea0 msgr2=0x7fe47c19f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:57.600 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:56.877+0000 7fe480e00700 1 --2- 172.21.15.123:0/2012609743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe47c07eea0 0x7fe47c19f000 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe46c011ac0 tx=0x7fe46c011ba0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.600 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:56.878+0000 7fe480e00700 1 -- 172.21.15.123:0/2012609743 shutdown_connections 2024-05-08T05:56:57.600 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:56.878+0000 7fe480e00700 1 --2- 172.21.15.123:0/2012609743 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7fe460041db0 0x7fe4600441a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.600 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:56.878+0000 7fe480e00700 1 --2- 172.21.15.123:0/2012609743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe47c07eea0 0x7fe47c19f000 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.600 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:56.878+0000 7fe480e00700 1 --2- 172.21.15.123:0/2012609743 >> [v2:172.21.15.123:6800/2769724312,v1:172.21.15.123:6801/2769724312] conn(0x7fe46003e4c0 0x7fe460040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:57.600 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:56.878+0000 7fe480e00700 1 -- 172.21.15.123:0/2012609743 >> 172.21.15.123:0/2012609743 conn(0x7fe47c104440 msgr2=0x7fe47c105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:57.600 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:56.878+0000 7fe480e00700 1 -- 172.21.15.123:0/2012609743 shutdown_connections 2024-05-08T05:56:57.600 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:56.878+0000 7fe480e00700 1 -- 172.21.15.123:0/2012609743 wait complete. 2024-05-08T05:56:57.600 INFO:teuthology.orchestra.run.smithi123.stdout:mgr epoch 5 is available 2024-05-08T05:56:57.600 INFO:teuthology.orchestra.run.smithi123.stdout:Setting orchestrator backend to cephadm... 2024-05-08T05:56:58.165 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:57 smithi123 ceph-mon[29787]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-08T05:56:58.165 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:57 smithi123 ceph-mon[29787]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-08T05:56:58.165 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:57 smithi123 ceph-mon[29787]: [08/May/2024:05:56:57] ENGINE Bus STARTING 2024-05-08T05:56:58.165 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:57 smithi123 ceph-mon[29787]: [08/May/2024:05:56:57] ENGINE Serving on http://172.21.15.123:8765 2024-05-08T05:56:58.166 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:57 smithi123 ceph-mon[29787]: [08/May/2024:05:56:57] ENGINE Serving on https://172.21.15.123:7150 2024-05-08T05:56:58.166 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:57 smithi123 ceph-mon[29787]: [08/May/2024:05:56:57] ENGINE Bus STARTED 2024-05-08T05:56:58.166 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:57 smithi123 ceph-mon[29787]: from='mgr.14120 172.21.15.123:0/1121855348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:56:59.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:58 smithi123 ceph-mon[29787]: mgrmap e8: x(active, since 2s) 2024-05-08T05:56:59.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:58 smithi123 ceph-mon[29787]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:56:59.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:58 smithi123 ceph-mon[29787]: from='mgr.14120 172.21.15.123:0/1121855348' entity='mgr.x' 2024-05-08T05:56:59.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:56:58 smithi123 ceph-mon[29787]: from='mgr.14120 172.21.15.123:0/1121855348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:56:59.289 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.235+0000 7fb471c00700 1 Processor -- start 2024-05-08T05:56:59.289 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.236+0000 7fb471c00700 1 -- start start 2024-05-08T05:56:59.289 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.236+0000 7fb471c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb46c1097a0 0x7fb46c10bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:59.289 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.236+0000 7fb471c00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c07ced0 con 0x7fb46c1097a0 2024-05-08T05:56:59.289 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.236+0000 7fb46be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb46c1097a0 0x7fb46c10bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:59.289 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.236+0000 7fb46be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb46c1097a0 0x7fb46c10bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47078/0 (socket says 172.21.15.123:47078) 2024-05-08T05:56:59.289 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.236+0000 7fb46be00700 1 -- 172.21.15.123:0/1985919296 learned_addr learned my addr 172.21.15.123:0/1985919296 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:56:59.289 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.237+0000 7fb46be00700 1 -- 172.21.15.123:0/1985919296 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb46c10c0d0 con 0x7fb46c1097a0 2024-05-08T05:56:59.290 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.237+0000 7fb46be00700 1 --2- 172.21.15.123:0/1985919296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb46c1097a0 0x7fb46c10bb90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0098a0 tx=0x7fb45c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4157b61145869d89 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:59.290 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.237+0000 7fb46aa00700 1 -- 172.21.15.123:0/1985919296 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb45c004160 con 0x7fb46c1097a0 2024-05-08T05:56:59.290 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.237+0000 7fb46aa00700 1 -- 172.21.15.123:0/1985919296 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb45c004300 con 0x7fb46c1097a0 2024-05-08T05:56:59.290 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.237+0000 7fb46aa00700 1 -- 172.21.15.123:0/1985919296 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb45c011af0 con 0x7fb46c1097a0 2024-05-08T05:56:59.290 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.238+0000 7fb471c00700 1 -- 172.21.15.123:0/1985919296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb46c1097a0 msgr2=0x7fb46c10bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:59.290 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.238+0000 7fb471c00700 1 --2- 172.21.15.123:0/1985919296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb46c1097a0 0x7fb46c10bb90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0098a0 tx=0x7fb45c009bb0 comp rx=0 tx=0).stop 2024-05-08T05:56:59.290 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.238+0000 7fb471c00700 1 -- 172.21.15.123:0/1985919296 shutdown_connections 2024-05-08T05:56:59.290 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.238+0000 7fb471c00700 1 --2- 172.21.15.123:0/1985919296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb46c1097a0 0x7fb46c10bb90 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:59.290 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.238+0000 7fb471c00700 1 -- 172.21.15.123:0/1985919296 >> 172.21.15.123:0/1985919296 conn(0x7fb46c1031b0 msgr2=0x7fb46c105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:59.290 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.239+0000 7fb471c00700 1 -- 172.21.15.123:0/1985919296 shutdown_connections 2024-05-08T05:56:59.290 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.239+0000 7fb471c00700 1 -- 172.21.15.123:0/1985919296 wait complete. 2024-05-08T05:56:59.290 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.239+0000 7fb471c00700 1 Processor -- start 2024-05-08T05:56:59.290 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.240+0000 7fb471c00700 1 -- start start 2024-05-08T05:56:59.291 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.240+0000 7fb471c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb46c1097a0 0x7fb46c19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:59.291 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.240+0000 7fb471c00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c10d030 con 0x7fb46c1097a0 2024-05-08T05:56:59.291 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.240+0000 7fb46be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb46c1097a0 0x7fb46c19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:59.291 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.240+0000 7fb46be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb46c1097a0 0x7fb46c19ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47090/0 (socket says 172.21.15.123:47090) 2024-05-08T05:56:59.291 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.240+0000 7fb46be00700 1 -- 172.21.15.123:0/3761979557 learned_addr learned my addr 172.21.15.123:0/3761979557 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:56:59.291 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.241+0000 7fb46be00700 1 -- 172.21.15.123:0/3761979557 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb46c19f3a0 con 0x7fb46c1097a0 2024-05-08T05:56:59.291 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.241+0000 7fb46be00700 1 --2- 172.21.15.123:0/3761979557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb46c1097a0 0x7fb46c19ee60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb45c011ac0 tx=0x7fb45c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:59.291 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.241+0000 7fb468c00700 1 -- 172.21.15.123:0/3761979557 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb45c003970 con 0x7fb46c1097a0 2024-05-08T05:56:59.291 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.241+0000 7fb468c00700 1 -- 172.21.15.123:0/3761979557 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb45c003b10 con 0x7fb46c1097a0 2024-05-08T05:56:59.291 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.241+0000 7fb468c00700 1 -- 172.21.15.123:0/3761979557 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb45c024610 con 0x7fb46c1097a0 2024-05-08T05:56:59.291 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.241+0000 7fb471c00700 1 -- 172.21.15.123:0/3761979557 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb46c19f630 con 0x7fb46c1097a0 2024-05-08T05:56:59.291 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.241+0000 7fb471c00700 1 -- 172.21.15.123:0/3761979557 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb46c1a2320 con 0x7fb46c1097a0 2024-05-08T05:56:59.291 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.243+0000 7fb468c00700 1 -- 172.21.15.123:0/3761979557 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fb45c01a430 con 0x7fb46c1097a0 2024-05-08T05:56:59.291 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.243+0000 7fb471c00700 1 -- 172.21.15.123:0/3761979557 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb46c06a5e0 con 0x7fb46c1097a0 2024-05-08T05:56:59.291 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.243+0000 7fb468c00700 1 --2- 172.21.15.123:0/3761979557 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7fb45003e470 0x7fb450040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:59.292 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.243+0000 7fb468c00700 1 -- 172.21.15.123:0/3761979557 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb45c053870 con 0x7fb46c1097a0 2024-05-08T05:56:59.292 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.243+0000 7fb46b400700 1 --2- 172.21.15.123:0/3761979557 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7fb45003e470 0x7fb450040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:59.292 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.244+0000 7fb46b400700 1 --2- 172.21.15.123:0/3761979557 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7fb45003e470 0x7fb450040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb458006fd0 tx=0x7fb458006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:59.292 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.249+0000 7fb468c00700 1 -- 172.21.15.123:0/3761979557 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb45c00b3e0 con 0x7fb46c1097a0 2024-05-08T05:56:59.292 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.498+0000 7fb471c00700 1 -- 172.21.15.123:0/3761979557 --> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fb46c1043e0 con 0x7fb45003e470 2024-05-08T05:56:59.292 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.506+0000 7fb468c00700 1 -- 172.21.15.123:0/3761979557 <== mgr.14120 v2:172.21.15.123:6800/3057688649 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb46c1043e0 con 0x7fb45003e470 2024-05-08T05:56:59.292 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.510+0000 7fb471c00700 1 -- 172.21.15.123:0/3761979557 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7fb45003e470 msgr2=0x7fb450040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:59.292 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.510+0000 7fb471c00700 1 --2- 172.21.15.123:0/3761979557 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7fb45003e470 0x7fb450040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb458006fd0 tx=0x7fb458006e40 comp rx=0 tx=0).stop 2024-05-08T05:56:59.292 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.510+0000 7fb471c00700 1 -- 172.21.15.123:0/3761979557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb46c1097a0 msgr2=0x7fb46c19ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:59.292 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.510+0000 7fb471c00700 1 --2- 172.21.15.123:0/3761979557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb46c1097a0 0x7fb46c19ee60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb45c011ac0 tx=0x7fb45c011ba0 comp rx=0 tx=0).stop 2024-05-08T05:56:59.292 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.510+0000 7fb471c00700 1 -- 172.21.15.123:0/3761979557 shutdown_connections 2024-05-08T05:56:59.292 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.510+0000 7fb471c00700 1 --2- 172.21.15.123:0/3761979557 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7fb45003e470 0x7fb450040930 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:59.292 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.510+0000 7fb471c00700 1 --2- 172.21.15.123:0/3761979557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb46c1097a0 0x7fb46c19ee60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:59.292 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.510+0000 7fb471c00700 1 -- 172.21.15.123:0/3761979557 >> 172.21.15.123:0/3761979557 conn(0x7fb46c1031b0 msgr2=0x7fb46c103cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:59.293 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.510+0000 7fb471c00700 1 -- 172.21.15.123:0/3761979557 shutdown_connections 2024-05-08T05:56:59.293 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:58.510+0000 7fb471c00700 1 -- 172.21.15.123:0/3761979557 wait complete. 2024-05-08T05:57:00.845 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-08T05:57:00.845 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.911+0000 7f3ecb200700 1 Processor -- start 2024-05-08T05:57:00.845 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.911+0000 7f3ecb200700 1 -- start start 2024-05-08T05:57:00.845 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.911+0000 7f3ecb200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ec410b220 0x7f3ec410b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:00.847 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.912+0000 7f3ecb200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec410bc10 con 0x7f3ec410b220 2024-05-08T05:57:00.847 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.912+0000 7f3ec3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ec410b220 0x7f3ec410b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:00.847 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.913+0000 7f3ec3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ec410b220 0x7f3ec410b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47092/0 (socket says 172.21.15.123:47092) 2024-05-08T05:57:00.847 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.913+0000 7f3ec3e00700 1 -- 172.21.15.123:0/3205727963 learned_addr learned my addr 172.21.15.123:0/3205727963 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:57:00.847 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.913+0000 7f3ec3e00700 1 -- 172.21.15.123:0/3205727963 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec410c300 con 0x7f3ec410b220 2024-05-08T05:57:00.847 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.913+0000 7f3ec3e00700 1 --2- 172.21.15.123:0/3205727963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ec410b220 0x7f3ec410b640 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3eb40098a0 tx=0x7f3eb4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd5a7bfb499cfcbe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:00.847 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.914+0000 7f3ec2a00700 1 -- 172.21.15.123:0/3205727963 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3eb4004160 con 0x7f3ec410b220 2024-05-08T05:57:00.847 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.914+0000 7f3ec2a00700 1 -- 172.21.15.123:0/3205727963 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3eb4004300 con 0x7f3ec410b220 2024-05-08T05:57:00.847 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.914+0000 7f3ec2a00700 1 -- 172.21.15.123:0/3205727963 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3eb4011af0 con 0x7f3ec410b220 2024-05-08T05:57:00.847 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.915+0000 7f3ecb200700 1 -- 172.21.15.123:0/3205727963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ec410b220 msgr2=0x7f3ec410b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:00.851 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.915+0000 7f3ecb200700 1 --2- 172.21.15.123:0/3205727963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ec410b220 0x7f3ec410b640 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3eb40098a0 tx=0x7f3eb4009bb0 comp rx=0 tx=0).stop 2024-05-08T05:57:00.851 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.915+0000 7f3ecb200700 1 -- 172.21.15.123:0/3205727963 shutdown_connections 2024-05-08T05:57:00.851 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.915+0000 7f3ecb200700 1 --2- 172.21.15.123:0/3205727963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ec410b220 0x7f3ec410b640 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:00.851 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.915+0000 7f3ecb200700 1 -- 172.21.15.123:0/3205727963 >> 172.21.15.123:0/3205727963 conn(0x7f3ec407e6e0 msgr2=0x7f3ec407eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:00.851 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.915+0000 7f3ecb200700 1 -- 172.21.15.123:0/3205727963 shutdown_connections 2024-05-08T05:57:00.851 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.915+0000 7f3ecb200700 1 -- 172.21.15.123:0/3205727963 wait complete. 2024-05-08T05:57:00.852 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.916+0000 7f3ecb200700 1 Processor -- start 2024-05-08T05:57:00.852 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.916+0000 7f3ecb200700 1 -- start start 2024-05-08T05:57:00.852 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.916+0000 7f3ecb200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ec410b220 0x7f3ec419f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:00.852 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.916+0000 7f3ecb200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec410f530 con 0x7f3ec410b220 2024-05-08T05:57:00.852 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.917+0000 7f3ec3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ec410b220 0x7f3ec419f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:00.852 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.917+0000 7f3ec3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ec410b220 0x7f3ec419f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47100/0 (socket says 172.21.15.123:47100) 2024-05-08T05:57:00.852 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.917+0000 7f3ec3e00700 1 -- 172.21.15.123:0/2330935858 learned_addr learned my addr 172.21.15.123:0/2330935858 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:57:00.852 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.917+0000 7f3ec3e00700 1 -- 172.21.15.123:0/2330935858 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec419f560 con 0x7f3ec410b220 2024-05-08T05:57:00.852 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.917+0000 7f3ec3e00700 1 --2- 172.21.15.123:0/2330935858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ec410b220 0x7f3ec419f020 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4009fb0 tx=0x7f3eb4011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:00.852 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.918+0000 7f3ec0c00700 1 -- 172.21.15.123:0/2330935858 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3eb40036a0 con 0x7f3ec410b220 2024-05-08T05:57:00.852 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.918+0000 7f3ec0c00700 1 -- 172.21.15.123:0/2330935858 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3eb401a430 con 0x7f3ec410b220 2024-05-08T05:57:00.852 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.918+0000 7f3ec0c00700 1 -- 172.21.15.123:0/2330935858 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3eb4019550 con 0x7f3ec410b220 2024-05-08T05:57:00.852 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.918+0000 7f3ecb200700 1 -- 172.21.15.123:0/2330935858 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ec419f7f0 con 0x7f3ec410b220 2024-05-08T05:57:00.852 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.918+0000 7f3ecb200700 1 -- 172.21.15.123:0/2330935858 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ec41a24e0 con 0x7f3ec410b220 2024-05-08T05:57:00.853 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.919+0000 7f3ecb200700 1 -- 172.21.15.123:0/2330935858 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ec410f6b0 con 0x7f3ec410b220 2024-05-08T05:57:00.853 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.919+0000 7f3ec0c00700 1 -- 172.21.15.123:0/2330935858 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f3eb401a8e0 con 0x7f3ec410b220 2024-05-08T05:57:00.853 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.920+0000 7f3ec0c00700 1 --2- 172.21.15.123:0/2330935858 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f3ea803e470 0x7f3ea8040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:00.853 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.920+0000 7f3ec0c00700 1 -- 172.21.15.123:0/2330935858 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3eb40538c0 con 0x7f3ec410b220 2024-05-08T05:57:00.853 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.920+0000 7f3ec3400700 1 --2- 172.21.15.123:0/2330935858 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f3ea803e470 0x7f3ea8040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:00.853 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.921+0000 7f3ec3400700 1 --2- 172.21.15.123:0/2330935858 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f3ea803e470 0x7f3ea8040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0006fd0 tx=0x7f3eb0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:00.853 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:56:59.925+0000 7f3ec0c00700 1 -- 172.21.15.123:0/2330935858 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3eb400b3e0 con 0x7f3ec410b220 2024-05-08T05:57:00.853 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:00.159+0000 7f3ecb200700 1 -- 172.21.15.123:0/2330935858 --> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f3ec4109590 con 0x7f3ea803e470 2024-05-08T05:57:00.853 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:00.160+0000 7f3ec0c00700 1 -- 172.21.15.123:0/2330935858 <== mgr.14120 v2:172.21.15.123:6800/3057688649 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f3ec4109590 con 0x7f3ea803e470 2024-05-08T05:57:00.853 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:00.164+0000 7f3ecb200700 1 -- 172.21.15.123:0/2330935858 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f3ea803e470 msgr2=0x7f3ea8040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:00.854 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:00.164+0000 7f3ecb200700 1 --2- 172.21.15.123:0/2330935858 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f3ea803e470 0x7f3ea8040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0006fd0 tx=0x7f3eb0006e40 comp rx=0 tx=0).stop 2024-05-08T05:57:00.855 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:00.164+0000 7f3ecb200700 1 -- 172.21.15.123:0/2330935858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ec410b220 msgr2=0x7f3ec419f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:00.855 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:00.164+0000 7f3ecb200700 1 --2- 172.21.15.123:0/2330935858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ec410b220 0x7f3ec419f020 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4009fb0 tx=0x7f3eb4011940 comp rx=0 tx=0).stop 2024-05-08T05:57:00.855 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:00.165+0000 7f3ecb200700 1 -- 172.21.15.123:0/2330935858 shutdown_connections 2024-05-08T05:57:00.855 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:00.165+0000 7f3ecb200700 1 --2- 172.21.15.123:0/2330935858 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f3ea803e470 0x7f3ea8040930 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:00.855 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:00.165+0000 7f3ecb200700 1 --2- 172.21.15.123:0/2330935858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ec410b220 0x7f3ec419f020 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:00.855 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:00.165+0000 7f3ecb200700 1 -- 172.21.15.123:0/2330935858 >> 172.21.15.123:0/2330935858 conn(0x7f3ec407e6e0 msgr2=0x7f3ec4108e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:00.855 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:00.165+0000 7f3ecb200700 1 -- 172.21.15.123:0/2330935858 shutdown_connections 2024-05-08T05:57:00.855 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:00.165+0000 7f3ecb200700 1 -- 172.21.15.123:0/2330935858 wait complete. 2024-05-08T05:57:00.855 INFO:teuthology.orchestra.run.smithi123.stdout:Generating ssh key... 2024-05-08T05:57:01.200 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:00 smithi123 ceph-mon[29787]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:57:02.601 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:02 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: Generating public/private rsa key pair. 2024-05-08T05:57:02.602 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:02 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: Your identification has been saved in /tmp/tmp_h21cakj/key. 2024-05-08T05:57:02.602 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:02 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: Your public key has been saved in /tmp/tmp_h21cakj/key.pub. 2024-05-08T05:57:02.602 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:02 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: The key fingerprint is: 2024-05-08T05:57:02.602 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:02 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: SHA256:inHKT2Tc2Pvo2ayZLHCTe3y5trZ6tVnQaD4AfkYISGs ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968 2024-05-08T05:57:02.602 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:02 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: The key's randomart image is: 2024-05-08T05:57:02.602 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:02 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: +---[RSA 3072]----+ 2024-05-08T05:57:02.602 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:02 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: | ..... . | 2024-05-08T05:57:02.602 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:02 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: | .. o . | 2024-05-08T05:57:02.602 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:02 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: | E . o o | 2024-05-08T05:57:02.602 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:02 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: | . . +. + + . | 2024-05-08T05:57:02.602 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:02 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: | . *.So + . | 2024-05-08T05:57:02.602 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:02 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: | ..B+. . + . | 2024-05-08T05:57:02.603 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:02 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: | +oo+. o = | 2024-05-08T05:57:02.603 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:02 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: | oo.oO* o | 2024-05-08T05:57:02.603 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:02 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: | .+XBB+ | 2024-05-08T05:57:02.603 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:02 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: +----[SHA256]-----+ 2024-05-08T05:57:02.947 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.558+0000 7f62fec00700 1 Processor -- start 2024-05-08T05:57:02.947 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.559+0000 7f62fec00700 1 -- start start 2024-05-08T05:57:02.947 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.559+0000 7f62fec00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62f807f740 0x7f62f807dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:02.948 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.559+0000 7f62fec00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f62f807e1c0 con 0x7f62f807f740 2024-05-08T05:57:02.948 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.559+0000 7f62f7e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62f807f740 0x7f62f807dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:02.948 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.559+0000 7f62f7e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62f807f740 0x7f62f807dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47108/0 (socket says 172.21.15.123:47108) 2024-05-08T05:57:02.948 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.559+0000 7f62f7e00700 1 -- 172.21.15.123:0/2506563574 learned_addr learned my addr 172.21.15.123:0/2506563574 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:57:02.948 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.560+0000 7f62f7e00700 1 -- 172.21.15.123:0/2506563574 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f807e340 con 0x7f62f807f740 2024-05-08T05:57:02.948 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.560+0000 7f62f7e00700 1 --2- 172.21.15.123:0/2506563574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62f807f740 0x7f62f807dbf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f62e80098a0 tx=0x7f62e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4e66622f0e0d9fd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:02.948 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.560+0000 7f62f6a00700 1 -- 172.21.15.123:0/2506563574 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62e8004160 con 0x7f62f807f740 2024-05-08T05:57:02.948 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.560+0000 7f62f6a00700 1 -- 172.21.15.123:0/2506563574 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f62e8004300 con 0x7f62f807f740 2024-05-08T05:57:02.948 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.560+0000 7f62f6a00700 1 -- 172.21.15.123:0/2506563574 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62e8011af0 con 0x7f62f807f740 2024-05-08T05:57:02.948 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.561+0000 7f62fec00700 1 -- 172.21.15.123:0/2506563574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62f807f740 msgr2=0x7f62f807dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:02.948 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.561+0000 7f62fec00700 1 --2- 172.21.15.123:0/2506563574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62f807f740 0x7f62f807dbf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f62e80098a0 tx=0x7f62e8009bb0 comp rx=0 tx=0).stop 2024-05-08T05:57:02.948 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.561+0000 7f62fec00700 1 -- 172.21.15.123:0/2506563574 shutdown_connections 2024-05-08T05:57:02.948 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.561+0000 7f62fec00700 1 --2- 172.21.15.123:0/2506563574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62f807f740 0x7f62f807dbf0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:02.949 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.562+0000 7f62fec00700 1 -- 172.21.15.123:0/2506563574 >> 172.21.15.123:0/2506563574 conn(0x7f62f81044e0 msgr2=0x7f62f8106900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:02.949 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.562+0000 7f62fec00700 1 -- 172.21.15.123:0/2506563574 shutdown_connections 2024-05-08T05:57:02.949 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.562+0000 7f62fec00700 1 -- 172.21.15.123:0/2506563574 wait complete. 2024-05-08T05:57:02.949 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.563+0000 7f62fec00700 1 Processor -- start 2024-05-08T05:57:02.949 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.563+0000 7f62fec00700 1 -- start start 2024-05-08T05:57:02.949 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.563+0000 7f62fec00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62f807f740 0x7f62f81a33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:02.949 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.563+0000 7f62fec00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f62f810b020 con 0x7f62f807f740 2024-05-08T05:57:02.949 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.563+0000 7f62f7e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62f807f740 0x7f62f81a33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:02.949 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.563+0000 7f62f7e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62f807f740 0x7f62f81a33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47118/0 (socket says 172.21.15.123:47118) 2024-05-08T05:57:02.949 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.563+0000 7f62f7e00700 1 -- 172.21.15.123:0/4270448899 learned_addr learned my addr 172.21.15.123:0/4270448899 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:57:02.949 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.564+0000 7f62f7e00700 1 -- 172.21.15.123:0/4270448899 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f81a3930 con 0x7f62f807f740 2024-05-08T05:57:02.949 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.564+0000 7f62f7e00700 1 --2- 172.21.15.123:0/4270448899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62f807f740 0x7f62f81a33f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f62e8011ac0 tx=0x7f62e8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:02.949 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.564+0000 7f62f4c00700 1 -- 172.21.15.123:0/4270448899 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62e8003970 con 0x7f62f807f740 2024-05-08T05:57:02.949 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.564+0000 7f62f4c00700 1 -- 172.21.15.123:0/4270448899 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f62e8003b10 con 0x7f62f807f740 2024-05-08T05:57:02.950 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.564+0000 7f62fec00700 1 -- 172.21.15.123:0/4270448899 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62f81a3bc0 con 0x7f62f807f740 2024-05-08T05:57:02.950 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.564+0000 7f62f4c00700 1 -- 172.21.15.123:0/4270448899 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62e8024610 con 0x7f62f807f740 2024-05-08T05:57:02.950 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.564+0000 7f62fec00700 1 -- 172.21.15.123:0/4270448899 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62f81a68b0 con 0x7f62f807f740 2024-05-08T05:57:02.950 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.565+0000 7f62fec00700 1 -- 172.21.15.123:0/4270448899 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62f8064b00 con 0x7f62f807f740 2024-05-08T05:57:02.950 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.566+0000 7f62f4c00700 1 -- 172.21.15.123:0/4270448899 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f62e801a430 con 0x7f62f807f740 2024-05-08T05:57:02.950 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.566+0000 7f62f4c00700 1 --2- 172.21.15.123:0/4270448899 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f62dc03e4c0 0x7f62dc040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:02.950 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.566+0000 7f62f4c00700 1 -- 172.21.15.123:0/4270448899 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f62e8053820 con 0x7f62f807f740 2024-05-08T05:57:02.950 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.567+0000 7f62f7400700 1 --2- 172.21.15.123:0/4270448899 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f62dc03e4c0 0x7f62dc040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:02.950 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.567+0000 7f62f7400700 1 --2- 172.21.15.123:0/4270448899 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f62dc03e4c0 0x7f62dc040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f62e4006fd0 tx=0x7f62e4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:02.950 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.572+0000 7f62f4c00700 1 -- 172.21.15.123:0/4270448899 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f62e800b3e0 con 0x7f62f807f740 2024-05-08T05:57:02.950 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:01.803+0000 7f62fec00700 1 -- 172.21.15.123:0/4270448899 --> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f62f80008d0 con 0x7f62dc03e4c0 2024-05-08T05:57:02.950 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:02.255+0000 7f62f4c00700 1 -- 172.21.15.123:0/4270448899 <== mgr.14120 v2:172.21.15.123:6800/3057688649 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f62f80008d0 con 0x7f62dc03e4c0 2024-05-08T05:57:02.950 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:02.258+0000 7f62fec00700 1 -- 172.21.15.123:0/4270448899 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f62dc03e4c0 msgr2=0x7f62dc040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:02.951 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:02.258+0000 7f62fec00700 1 --2- 172.21.15.123:0/4270448899 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f62dc03e4c0 0x7f62dc040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f62e4006fd0 tx=0x7f62e4006e40 comp rx=0 tx=0).stop 2024-05-08T05:57:02.951 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:02.258+0000 7f62fec00700 1 -- 172.21.15.123:0/4270448899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62f807f740 msgr2=0x7f62f81a33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:02.951 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:02.258+0000 7f62fec00700 1 --2- 172.21.15.123:0/4270448899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62f807f740 0x7f62f81a33f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f62e8011ac0 tx=0x7f62e8011ba0 comp rx=0 tx=0).stop 2024-05-08T05:57:02.951 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:02.258+0000 7f62fec00700 1 -- 172.21.15.123:0/4270448899 shutdown_connections 2024-05-08T05:57:02.951 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:02.258+0000 7f62fec00700 1 --2- 172.21.15.123:0/4270448899 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f62dc03e4c0 0x7f62dc040980 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:02.951 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:02.258+0000 7f62fec00700 1 --2- 172.21.15.123:0/4270448899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62f807f740 0x7f62f81a33f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:02.951 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:02.258+0000 7f62fec00700 1 -- 172.21.15.123:0/4270448899 >> 172.21.15.123:0/4270448899 conn(0x7f62f81044e0 msgr2=0x7f62f81068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:02.951 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:02.258+0000 7f62fec00700 1 -- 172.21.15.123:0/4270448899 shutdown_connections 2024-05-08T05:57:02.951 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:02.258+0000 7f62fec00700 1 -- 172.21.15.123:0/4270448899 wait complete. 2024-05-08T05:57:03.525 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:03 smithi123 ceph-mon[29787]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:57:03.526 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:03 smithi123 ceph-mon[29787]: Generating ssh key... 2024-05-08T05:57:03.526 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:03 smithi123 ceph-mon[29787]: from='mgr.14120 172.21.15.123:0/1121855348' entity='mgr.x' 2024-05-08T05:57:03.526 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:03 smithi123 ceph-mon[29787]: from='mgr.14120 172.21.15.123:0/1121855348' entity='mgr.x' 2024-05-08T05:57:04.503 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCykLmj8Zu9o7AUQyt7sQ18YoTeooUkGU/LifWKlxlY36ROfvKzfhzppZ/GWMnMkB54BnuD0VuUZgUXwIFb3oTtRbr5BTxH1gDa6VI+rbD+n7lGO9ujf0QNU8sil0SKEucUdFASihyw8xbO9l/wpmBatDIPzIKtGp9R6Oh6pfBVZnL0r0/ZUn0IUe0j7Q0fXIaMKXNOAKS80V/Xr3dj9Hfj/j268iD2yK8BXVqPmbhU9drY7uz3C22+3upFmx3MfClBZcm3CDVYpRit3N2yD/jBMExbm6QqaOm55uE1DM1zx6RmuHWrs6ikO6hd6VDQVdBwimBdb5TGP+420KTZJkI3t/bd23RloeNWvx5uvCZ/li4TIZ2T3ObjNBhIRrcNnixLIXmNgZT2F5JDKjICvhAOvkdY/vM8yr3TL4a5Z/FEbjsbnxIiIGQiRDe+btrzDbOsqR35e36hYW1dRTJe5Bf1kQjcP24mplck1P2j68FBIlUG1af44F2i94YjkJSolM0= ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968 2024-05-08T05:57:04.503 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.597+0000 7f3fb0200700 1 Processor -- start 2024-05-08T05:57:04.504 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.597+0000 7f3fb0200700 1 -- start start 2024-05-08T05:57:04.504 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.598+0000 7f3fb0200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fa810b220 0x7f3fa810b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:04.505 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.598+0000 7f3fb0200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa810bc10 con 0x7f3fa810b220 2024-05-08T05:57:04.505 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.598+0000 7f3fad800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fa810b220 0x7f3fa810b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:04.505 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.598+0000 7f3fad800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fa810b220 0x7f3fa810b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47120/0 (socket says 172.21.15.123:47120) 2024-05-08T05:57:04.505 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.598+0000 7f3fad800700 1 -- 172.21.15.123:0/2293152535 learned_addr learned my addr 172.21.15.123:0/2293152535 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:57:04.505 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.599+0000 7f3fad800700 1 -- 172.21.15.123:0/2293152535 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fa810c300 con 0x7f3fa810b220 2024-05-08T05:57:04.506 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.599+0000 7f3fad800700 1 --2- 172.21.15.123:0/2293152535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fa810b220 0x7f3fa810b640 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0098a0 tx=0x7f3f9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b08d659d652e55 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:04.507 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.599+0000 7f3fa7e00700 1 -- 172.21.15.123:0/2293152535 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f9c004160 con 0x7f3fa810b220 2024-05-08T05:57:04.507 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.599+0000 7f3fa7e00700 1 -- 172.21.15.123:0/2293152535 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3f9c004300 con 0x7f3fa810b220 2024-05-08T05:57:04.507 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.600+0000 7f3fa7e00700 1 -- 172.21.15.123:0/2293152535 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f9c011af0 con 0x7f3fa810b220 2024-05-08T05:57:04.507 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.600+0000 7f3fb0200700 1 -- 172.21.15.123:0/2293152535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fa810b220 msgr2=0x7f3fa810b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:04.507 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.600+0000 7f3fb0200700 1 --2- 172.21.15.123:0/2293152535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fa810b220 0x7f3fa810b640 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0098a0 tx=0x7f3f9c009bb0 comp rx=0 tx=0).stop 2024-05-08T05:57:04.507 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.600+0000 7f3fb0200700 1 -- 172.21.15.123:0/2293152535 shutdown_connections 2024-05-08T05:57:04.507 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.600+0000 7f3fb0200700 1 --2- 172.21.15.123:0/2293152535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fa810b220 0x7f3fa810b640 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:04.507 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.600+0000 7f3fb0200700 1 -- 172.21.15.123:0/2293152535 >> 172.21.15.123:0/2293152535 conn(0x7f3fa807e6e0 msgr2=0x7f3fa807eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:04.507 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.600+0000 7f3fb0200700 1 -- 172.21.15.123:0/2293152535 shutdown_connections 2024-05-08T05:57:04.507 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.600+0000 7f3fb0200700 1 -- 172.21.15.123:0/2293152535 wait complete. 2024-05-08T05:57:04.507 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.601+0000 7f3fb0200700 1 Processor -- start 2024-05-08T05:57:04.507 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.602+0000 7f3fb0200700 1 -- start start 2024-05-08T05:57:04.510 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.602+0000 7f3fb0200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fa810b220 0x7f3fa819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:04.510 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.602+0000 7f3fb0200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa810f530 con 0x7f3fa810b220 2024-05-08T05:57:04.511 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.602+0000 7f3fad800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fa810b220 0x7f3fa819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:04.511 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.602+0000 7f3fad800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fa810b220 0x7f3fa819f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47132/0 (socket says 172.21.15.123:47132) 2024-05-08T05:57:04.511 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.602+0000 7f3fad800700 1 -- 172.21.15.123:0/3735276505 learned_addr learned my addr 172.21.15.123:0/3735276505 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:57:04.511 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.602+0000 7f3fad800700 1 -- 172.21.15.123:0/3735276505 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fa819f560 con 0x7f3fa810b220 2024-05-08T05:57:04.511 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.603+0000 7f3fad800700 1 --2- 172.21.15.123:0/3735276505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fa810b220 0x7f3fa819f020 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c011ac0 tx=0x7f3f9c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:04.511 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.603+0000 7f3fa6000700 1 -- 172.21.15.123:0/3735276505 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f9c003970 con 0x7f3fa810b220 2024-05-08T05:57:04.511 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.603+0000 7f3fa6000700 1 -- 172.21.15.123:0/3735276505 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3f9c003b10 con 0x7f3fa810b220 2024-05-08T05:57:04.511 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.603+0000 7f3fb0200700 1 -- 172.21.15.123:0/3735276505 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fa819f7f0 con 0x7f3fa810b220 2024-05-08T05:57:04.511 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.603+0000 7f3fa6000700 1 -- 172.21.15.123:0/3735276505 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f9c024610 con 0x7f3fa810b220 2024-05-08T05:57:04.511 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.603+0000 7f3fb0200700 1 -- 172.21.15.123:0/3735276505 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fa81a24e0 con 0x7f3fa810b220 2024-05-08T05:57:04.511 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.604+0000 7f3fb0200700 1 -- 172.21.15.123:0/3735276505 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fa810f6b0 con 0x7f3fa810b220 2024-05-08T05:57:04.511 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.605+0000 7f3fa6000700 1 -- 172.21.15.123:0/3735276505 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f3f9c01a9e0 con 0x7f3fa810b220 2024-05-08T05:57:04.511 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.605+0000 7f3fa6000700 1 --2- 172.21.15.123:0/3735276505 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f3f9003e4c0 0x7f3f90040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:04.512 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.605+0000 7f3fa6000700 1 -- 172.21.15.123:0/3735276505 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3f9c053990 con 0x7f3fa810b220 2024-05-08T05:57:04.512 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.606+0000 7f3face00700 1 --2- 172.21.15.123:0/3735276505 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f3f9003e4c0 0x7f3f90040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:04.512 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.606+0000 7f3face00700 1 --2- 172.21.15.123:0/3735276505 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f3f9003e4c0 0x7f3f90040980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3f94006fd0 tx=0x7f3f94006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:04.512 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.611+0000 7f3fa6000700 1 -- 172.21.15.123:0/3735276505 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3f9c0247f0 con 0x7f3fa810b220 2024-05-08T05:57:04.512 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.844+0000 7f3fb0200700 1 -- 172.21.15.123:0/3735276505 --> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f3fa8109550 con 0x7f3f9003e4c0 2024-05-08T05:57:04.512 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.846+0000 7f3fa6000700 1 -- 172.21.15.123:0/3735276505 <== mgr.14120 v2:172.21.15.123:6800/3057688649 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f3fa8109550 con 0x7f3f9003e4c0 2024-05-08T05:57:04.512 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.849+0000 7f3fb0200700 1 -- 172.21.15.123:0/3735276505 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f3f9003e4c0 msgr2=0x7f3f90040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:04.512 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.849+0000 7f3fb0200700 1 --2- 172.21.15.123:0/3735276505 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f3f9003e4c0 0x7f3f90040980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3f94006fd0 tx=0x7f3f94006e40 comp rx=0 tx=0).stop 2024-05-08T05:57:04.512 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.849+0000 7f3fb0200700 1 -- 172.21.15.123:0/3735276505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fa810b220 msgr2=0x7f3fa819f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:04.512 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.849+0000 7f3fb0200700 1 --2- 172.21.15.123:0/3735276505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fa810b220 0x7f3fa819f020 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c011ac0 tx=0x7f3f9c011ba0 comp rx=0 tx=0).stop 2024-05-08T05:57:04.512 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.849+0000 7f3fb0200700 1 -- 172.21.15.123:0/3735276505 shutdown_connections 2024-05-08T05:57:04.512 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.849+0000 7f3fb0200700 1 --2- 172.21.15.123:0/3735276505 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f3f9003e4c0 0x7f3f90040980 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:04.513 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.849+0000 7f3fb0200700 1 --2- 172.21.15.123:0/3735276505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fa810b220 0x7f3fa819f020 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:04.513 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.849+0000 7f3fb0200700 1 -- 172.21.15.123:0/3735276505 >> 172.21.15.123:0/3735276505 conn(0x7f3fa807e6e0 msgr2=0x7f3fa8108e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:04.513 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.849+0000 7f3fb0200700 1 -- 172.21.15.123:0/3735276505 shutdown_connections 2024-05-08T05:57:04.513 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:03.849+0000 7f3fb0200700 1 -- 172.21.15.123:0/3735276505 wait complete. 2024-05-08T05:57:04.513 INFO:teuthology.orchestra.run.smithi123.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-08T05:57:04.514 INFO:teuthology.orchestra.run.smithi123.stdout:Adding key to root@localhost authorized_keys... 2024-05-08T05:57:04.514 INFO:teuthology.orchestra.run.smithi123.stdout:Adding host smithi123... 2024-05-08T05:57:05.073 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:04 smithi123 ceph-mon[29787]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:57:06.053 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:05 smithi123 ceph-mon[29787]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi123", "addr": "172.21.15.123", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:57:06.939 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:06 smithi123 ceph-mon[29787]: Deploying cephadm binary to smithi123 2024-05-08T05:57:11.918 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout Added host 'smithi123' with addr '172.21.15.123' 2024-05-08T05:57:11.918 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.157+0000 7f9b56400700 1 Processor -- start 2024-05-08T05:57:11.918 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.157+0000 7f9b56400700 1 -- start start 2024-05-08T05:57:11.918 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.157+0000 7f9b56400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b50108ed0 0x7f9b501092f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:11.918 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.157+0000 7f9b56400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9b50109830 con 0x7f9b50108ed0 2024-05-08T05:57:11.918 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.158+0000 7f9b4f400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b50108ed0 0x7f9b501092f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:11.918 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.158+0000 7f9b4f400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b50108ed0 0x7f9b501092f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39396/0 (socket says 172.21.15.123:39396) 2024-05-08T05:57:11.918 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.158+0000 7f9b4f400700 1 -- 172.21.15.123:0/453976375 learned_addr learned my addr 172.21.15.123:0/453976375 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:57:11.918 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.158+0000 7f9b4f400700 1 -- 172.21.15.123:0/453976375 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b5010a070 con 0x7f9b50108ed0 2024-05-08T05:57:11.919 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.159+0000 7f9b4f400700 1 --2- 172.21.15.123:0/453976375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b50108ed0 0x7f9b501092f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9b400098a0 tx=0x7f9b40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89d2ab5ba497b3dd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:11.919 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.159+0000 7f9b4e000700 1 -- 172.21.15.123:0/453976375 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b40004160 con 0x7f9b50108ed0 2024-05-08T05:57:11.919 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.159+0000 7f9b4e000700 1 -- 172.21.15.123:0/453976375 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9b40004300 con 0x7f9b50108ed0 2024-05-08T05:57:11.919 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.159+0000 7f9b4e000700 1 -- 172.21.15.123:0/453976375 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b40011af0 con 0x7f9b50108ed0 2024-05-08T05:57:11.919 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.160+0000 7f9b56400700 1 -- 172.21.15.123:0/453976375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b50108ed0 msgr2=0x7f9b501092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:11.919 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.160+0000 7f9b56400700 1 --2- 172.21.15.123:0/453976375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b50108ed0 0x7f9b501092f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9b400098a0 tx=0x7f9b40009bb0 comp rx=0 tx=0).stop 2024-05-08T05:57:11.919 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.160+0000 7f9b56400700 1 -- 172.21.15.123:0/453976375 shutdown_connections 2024-05-08T05:57:11.919 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.160+0000 7f9b56400700 1 --2- 172.21.15.123:0/453976375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b50108ed0 0x7f9b501092f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:11.919 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.160+0000 7f9b56400700 1 -- 172.21.15.123:0/453976375 >> 172.21.15.123:0/453976375 conn(0x7f9b501044e0 msgr2=0x7f9b50106900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:11.919 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.160+0000 7f9b56400700 1 -- 172.21.15.123:0/453976375 shutdown_connections 2024-05-08T05:57:11.919 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.160+0000 7f9b56400700 1 -- 172.21.15.123:0/453976375 wait complete. 2024-05-08T05:57:11.919 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.161+0000 7f9b56400700 1 Processor -- start 2024-05-08T05:57:11.920 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.161+0000 7f9b56400700 1 -- start start 2024-05-08T05:57:11.920 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.162+0000 7f9b56400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b50108ed0 0x7f9b5019f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:11.920 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.162+0000 7f9b56400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9b501099b0 con 0x7f9b50108ed0 2024-05-08T05:57:11.920 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.162+0000 7f9b4f400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b50108ed0 0x7f9b5019f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:11.920 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.162+0000 7f9b4f400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b50108ed0 0x7f9b5019f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39398/0 (socket says 172.21.15.123:39398) 2024-05-08T05:57:11.920 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.162+0000 7f9b4f400700 1 -- 172.21.15.123:0/3269186778 learned_addr learned my addr 172.21.15.123:0/3269186778 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:57:11.920 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.162+0000 7f9b4f400700 1 -- 172.21.15.123:0/3269186778 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b5019f5b0 con 0x7f9b50108ed0 2024-05-08T05:57:11.920 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.163+0000 7f9b4f400700 1 --2- 172.21.15.123:0/3269186778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b50108ed0 0x7f9b5019f070 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9b400117b0 tx=0x7f9b40011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:11.920 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.163+0000 7f9b47e00700 1 -- 172.21.15.123:0/3269186778 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b40003af0 con 0x7f9b50108ed0 2024-05-08T05:57:11.920 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.163+0000 7f9b47e00700 1 -- 172.21.15.123:0/3269186778 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9b40020070 con 0x7f9b50108ed0 2024-05-08T05:57:11.920 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.163+0000 7f9b56400700 1 -- 172.21.15.123:0/3269186778 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b5019f840 con 0x7f9b50108ed0 2024-05-08T05:57:11.920 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.163+0000 7f9b47e00700 1 -- 172.21.15.123:0/3269186778 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b40019550 con 0x7f9b50108ed0 2024-05-08T05:57:11.920 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.163+0000 7f9b56400700 1 -- 172.21.15.123:0/3269186778 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b501a2530 con 0x7f9b50108ed0 2024-05-08T05:57:11.920 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.164+0000 7f9b47e00700 1 -- 172.21.15.123:0/3269186778 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f9b400197c0 con 0x7f9b50108ed0 2024-05-08T05:57:11.921 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.164+0000 7f9b56400700 1 -- 172.21.15.123:0/3269186778 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b50064b00 con 0x7f9b50108ed0 2024-05-08T05:57:11.921 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.165+0000 7f9b47e00700 1 --2- 172.21.15.123:0/3269186778 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f9b3403e4c0 0x7f9b34040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:11.921 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.165+0000 7f9b47e00700 1 -- 172.21.15.123:0/3269186778 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9b40053b60 con 0x7f9b50108ed0 2024-05-08T05:57:11.921 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.165+0000 7f9b4ea00700 1 --2- 172.21.15.123:0/3269186778 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f9b3403e4c0 0x7f9b34040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:11.921 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.166+0000 7f9b4ea00700 1 --2- 172.21.15.123:0/3269186778 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f9b3403e4c0 0x7f9b34040980 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c006fd0 tx=0x7f9b3c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:11.921 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.170+0000 7f9b47e00700 1 -- 172.21.15.123:0/3269186778 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9b40018d60 con 0x7f9b50108ed0 2024-05-08T05:57:11.921 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:05.407+0000 7f9b56400700 1 -- 172.21.15.123:0/3269186778 --> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi123", "addr": "172.21.15.123", "target": ["mon-mgr", ""]}) -- 0x7f9b501057f0 con 0x7f9b3403e4c0 2024-05-08T05:57:11.921 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:11.234+0000 7f9b47e00700 1 -- 172.21.15.123:0/3269186778 <== mgr.14120 v2:172.21.15.123:6800/3057688649 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f9b501057f0 con 0x7f9b3403e4c0 2024-05-08T05:57:11.921 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:11.238+0000 7f9b56400700 1 -- 172.21.15.123:0/3269186778 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f9b3403e4c0 msgr2=0x7f9b34040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:11.921 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:11.238+0000 7f9b56400700 1 --2- 172.21.15.123:0/3269186778 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f9b3403e4c0 0x7f9b34040980 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c006fd0 tx=0x7f9b3c006e40 comp rx=0 tx=0).stop 2024-05-08T05:57:11.922 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:11.238+0000 7f9b56400700 1 -- 172.21.15.123:0/3269186778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b50108ed0 msgr2=0x7f9b5019f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:11.922 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:11.238+0000 7f9b56400700 1 --2- 172.21.15.123:0/3269186778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b50108ed0 0x7f9b5019f070 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9b400117b0 tx=0x7f9b40011b70 comp rx=0 tx=0).stop 2024-05-08T05:57:11.922 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:11.239+0000 7f9b56400700 1 -- 172.21.15.123:0/3269186778 shutdown_connections 2024-05-08T05:57:11.922 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:11.239+0000 7f9b56400700 1 --2- 172.21.15.123:0/3269186778 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f9b3403e4c0 0x7f9b34040980 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:11.922 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:11.239+0000 7f9b56400700 1 --2- 172.21.15.123:0/3269186778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b50108ed0 0x7f9b5019f070 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:11.922 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:11.239+0000 7f9b56400700 1 -- 172.21.15.123:0/3269186778 >> 172.21.15.123:0/3269186778 conn(0x7f9b501044e0 msgr2=0x7f9b501050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:11.922 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:11.239+0000 7f9b56400700 1 -- 172.21.15.123:0/3269186778 shutdown_connections 2024-05-08T05:57:11.922 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:11.239+0000 7f9b56400700 1 -- 172.21.15.123:0/3269186778 wait complete. 2024-05-08T05:57:11.923 INFO:teuthology.orchestra.run.smithi123.stdout:Deploying unmanaged mon service... 2024-05-08T05:57:12.633 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:12 smithi123 ceph-mon[29787]: from='mgr.14120 172.21.15.123:0/1121855348' entity='mgr.x' 2024-05-08T05:57:12.633 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:12 smithi123 ceph-mon[29787]: Added host smithi123 2024-05-08T05:57:12.633 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:12 smithi123 ceph-mon[29787]: from='mgr.14120 172.21.15.123:0/1121855348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:57:14.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:13 smithi123 ceph-mon[29787]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:57:14.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:13 smithi123 ceph-mon[29787]: Saving service mon spec with placement count:5 2024-05-08T05:57:14.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:13 smithi123 ceph-mon[29787]: from='mgr.14120 172.21.15.123:0/1121855348' entity='mgr.x' 2024-05-08T05:57:14.355 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-08T05:57:14.355 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.632+0000 7f4d51000700 1 Processor -- start 2024-05-08T05:57:14.355 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.632+0000 7f4d51000700 1 -- start start 2024-05-08T05:57:14.355 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.632+0000 7f4d51000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d4c108ed0 0x7f4d4c1092f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:14.355 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.632+0000 7f4d51000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c109830 con 0x7f4d4c108ed0 2024-05-08T05:57:14.355 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.633+0000 7f4d49e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d4c108ed0 0x7f4d4c1092f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:14.355 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.633+0000 7f4d49e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d4c108ed0 0x7f4d4c1092f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39404/0 (socket says 172.21.15.123:39404) 2024-05-08T05:57:14.356 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.633+0000 7f4d49e00700 1 -- 172.21.15.123:0/3966502806 learned_addr learned my addr 172.21.15.123:0/3966502806 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:57:14.356 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.633+0000 7f4d49e00700 1 -- 172.21.15.123:0/3966502806 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d4c10a070 con 0x7f4d4c108ed0 2024-05-08T05:57:14.356 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.634+0000 7f4d49e00700 1 --2- 172.21.15.123:0/3966502806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d4c108ed0 0x7f4d4c1092f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c0098a0 tx=0x7f4d3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f0a3076cfbf9af3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:14.356 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.634+0000 7f4d48a00700 1 -- 172.21.15.123:0/3966502806 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d3c004160 con 0x7f4d4c108ed0 2024-05-08T05:57:14.356 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.634+0000 7f4d48a00700 1 -- 172.21.15.123:0/3966502806 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4d3c004300 con 0x7f4d4c108ed0 2024-05-08T05:57:14.356 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.634+0000 7f4d48a00700 1 -- 172.21.15.123:0/3966502806 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d3c011af0 con 0x7f4d4c108ed0 2024-05-08T05:57:14.356 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.634+0000 7f4d51000700 1 -- 172.21.15.123:0/3966502806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d4c108ed0 msgr2=0x7f4d4c1092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:14.356 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.634+0000 7f4d51000700 1 --2- 172.21.15.123:0/3966502806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d4c108ed0 0x7f4d4c1092f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c0098a0 tx=0x7f4d3c009bb0 comp rx=0 tx=0).stop 2024-05-08T05:57:14.356 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.635+0000 7f4d51000700 1 -- 172.21.15.123:0/3966502806 shutdown_connections 2024-05-08T05:57:14.356 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.635+0000 7f4d51000700 1 --2- 172.21.15.123:0/3966502806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d4c108ed0 0x7f4d4c1092f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:14.356 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.635+0000 7f4d51000700 1 -- 172.21.15.123:0/3966502806 >> 172.21.15.123:0/3966502806 conn(0x7f4d4c1044e0 msgr2=0x7f4d4c106900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:14.356 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.635+0000 7f4d51000700 1 -- 172.21.15.123:0/3966502806 shutdown_connections 2024-05-08T05:57:14.356 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.635+0000 7f4d51000700 1 -- 172.21.15.123:0/3966502806 wait complete. 2024-05-08T05:57:14.357 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.636+0000 7f4d51000700 1 Processor -- start 2024-05-08T05:57:14.357 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.636+0000 7f4d51000700 1 -- start start 2024-05-08T05:57:14.357 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.636+0000 7f4d51000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d4c108ed0 0x7f4d4c07f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:14.357 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.636+0000 7f4d51000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c1099b0 con 0x7f4d4c108ed0 2024-05-08T05:57:14.357 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.637+0000 7f4d49e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d4c108ed0 0x7f4d4c07f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:14.357 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.637+0000 7f4d49e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d4c108ed0 0x7f4d4c07f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39410/0 (socket says 172.21.15.123:39410) 2024-05-08T05:57:14.358 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.637+0000 7f4d49e00700 1 -- 172.21.15.123:0/1391289184 learned_addr learned my addr 172.21.15.123:0/1391289184 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:57:14.358 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.637+0000 7f4d49e00700 1 -- 172.21.15.123:0/1391289184 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d4c07f960 con 0x7f4d4c108ed0 2024-05-08T05:57:14.358 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.637+0000 7f4d49e00700 1 --2- 172.21.15.123:0/1391289184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d4c108ed0 0x7f4d4c07f420 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c0117b0 tx=0x7f4d3c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:14.358 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.638+0000 7f4d42a00700 1 -- 172.21.15.123:0/1391289184 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d3c003af0 con 0x7f4d4c108ed0 2024-05-08T05:57:14.358 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.638+0000 7f4d42a00700 1 -- 172.21.15.123:0/1391289184 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4d3c020070 con 0x7f4d4c108ed0 2024-05-08T05:57:14.358 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.638+0000 7f4d42a00700 1 -- 172.21.15.123:0/1391289184 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d3c019550 con 0x7f4d4c108ed0 2024-05-08T05:57:14.359 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.638+0000 7f4d51000700 1 -- 172.21.15.123:0/1391289184 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d4c07db50 con 0x7f4d4c108ed0 2024-05-08T05:57:14.359 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.638+0000 7f4d51000700 1 -- 172.21.15.123:0/1391289184 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d4c07e090 con 0x7f4d4c108ed0 2024-05-08T05:57:14.359 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.639+0000 7f4d42a00700 1 -- 172.21.15.123:0/1391289184 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f4d3c01a430 con 0x7f4d4c108ed0 2024-05-08T05:57:14.359 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.639+0000 7f4d51000700 1 -- 172.21.15.123:0/1391289184 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d4c064b00 con 0x7f4d4c108ed0 2024-05-08T05:57:14.359 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.639+0000 7f4d42a00700 1 --2- 172.21.15.123:0/1391289184 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f4d3003e4c0 0x7f4d30040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:14.360 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.639+0000 7f4d42a00700 1 -- 172.21.15.123:0/1391289184 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4d3c0539a0 con 0x7f4d4c108ed0 2024-05-08T05:57:14.360 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.640+0000 7f4d49400700 1 --2- 172.21.15.123:0/1391289184 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f4d3003e4c0 0x7f4d30040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:14.360 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.640+0000 7f4d49400700 1 --2- 172.21.15.123:0/1391289184 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f4d3003e4c0 0x7f4d30040980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4d38006fd0 tx=0x7f4d38006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:14.360 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.645+0000 7f4d42a00700 1 -- 172.21.15.123:0/1391289184 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4d3c0042a0 con 0x7f4d4c108ed0 2024-05-08T05:57:14.360 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.903+0000 7f4d51000700 1 -- 172.21.15.123:0/1391289184 --> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f4d4c1057e0 con 0x7f4d3003e4c0 2024-05-08T05:57:14.360 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.913+0000 7f4d42a00700 1 -- 172.21.15.123:0/1391289184 <== mgr.14120 v2:172.21.15.123:6800/3057688649 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4d4c1057e0 con 0x7f4d3003e4c0 2024-05-08T05:57:14.361 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.916+0000 7f4d51000700 1 -- 172.21.15.123:0/1391289184 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f4d3003e4c0 msgr2=0x7f4d30040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:14.361 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.916+0000 7f4d51000700 1 --2- 172.21.15.123:0/1391289184 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f4d3003e4c0 0x7f4d30040980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4d38006fd0 tx=0x7f4d38006e40 comp rx=0 tx=0).stop 2024-05-08T05:57:14.361 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.916+0000 7f4d51000700 1 -- 172.21.15.123:0/1391289184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d4c108ed0 msgr2=0x7f4d4c07f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:14.361 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.916+0000 7f4d51000700 1 --2- 172.21.15.123:0/1391289184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d4c108ed0 0x7f4d4c07f420 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c0117b0 tx=0x7f4d3c011b70 comp rx=0 tx=0).stop 2024-05-08T05:57:14.361 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.916+0000 7f4d51000700 1 -- 172.21.15.123:0/1391289184 shutdown_connections 2024-05-08T05:57:14.362 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.916+0000 7f4d51000700 1 --2- 172.21.15.123:0/1391289184 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f4d3003e4c0 0x7f4d30040980 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:14.362 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.916+0000 7f4d51000700 1 --2- 172.21.15.123:0/1391289184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d4c108ed0 0x7f4d4c07f420 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:14.362 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.916+0000 7f4d51000700 1 -- 172.21.15.123:0/1391289184 >> 172.21.15.123:0/1391289184 conn(0x7f4d4c1044e0 msgr2=0x7f4d4c1050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:14.362 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.916+0000 7f4d51000700 1 -- 172.21.15.123:0/1391289184 shutdown_connections 2024-05-08T05:57:14.362 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:12.916+0000 7f4d51000700 1 -- 172.21.15.123:0/1391289184 wait complete. 2024-05-08T05:57:14.362 INFO:teuthology.orchestra.run.smithi123.stdout:Deploying unmanaged mgr service... 2024-05-08T05:57:16.119 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:15 smithi123 ceph-mon[29787]: from='mgr.14120 172.21.15.123:0/1121855348' entity='mgr.x' 2024-05-08T05:57:16.119 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:15 smithi123 ceph-mon[29787]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:57:16.120 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:15 smithi123 ceph-mon[29787]: Saving service mgr spec with placement count:2 2024-05-08T05:57:16.120 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:15 smithi123 ceph-mon[29787]: from='mgr.14120 172.21.15.123:0/1121855348' entity='mgr.x' 2024-05-08T05:57:16.120 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:15 smithi123 ceph-mon[29787]: from='mgr.14120 172.21.15.123:0/1121855348' entity='mgr.x' 2024-05-08T05:57:16.425 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-08T05:57:16.426 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.433+0000 7f39f2a00700 1 Processor -- start 2024-05-08T05:57:16.426 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.433+0000 7f39f2a00700 1 -- start start 2024-05-08T05:57:16.426 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.434+0000 7f39f2a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39ec1097b0 0x7f39ec10bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:16.426 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.434+0000 7f39f2a00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f39ec07d760 con 0x7f39ec1097b0 2024-05-08T05:57:16.426 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.434+0000 7f39f1600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39ec1097b0 0x7f39ec10bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:16.426 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.434+0000 7f39f1600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39ec1097b0 0x7f39ec10bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55250/0 (socket says 172.21.15.123:55250) 2024-05-08T05:57:16.426 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.434+0000 7f39f1600700 1 -- 172.21.15.123:0/376986771 learned_addr learned my addr 172.21.15.123:0/376986771 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:57:16.426 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.435+0000 7f39f1600700 1 -- 172.21.15.123:0/376986771 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39ec10c0e0 con 0x7f39ec1097b0 2024-05-08T05:57:16.426 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.435+0000 7f39f1600700 1 --2- 172.21.15.123:0/376986771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39ec1097b0 0x7f39ec10bba0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f39e00098a0 tx=0x7f39e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89b4cc8b5318598 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:16.426 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.435+0000 7f39ebe00700 1 -- 172.21.15.123:0/376986771 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39e0004160 con 0x7f39ec1097b0 2024-05-08T05:57:16.426 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.435+0000 7f39ebe00700 1 -- 172.21.15.123:0/376986771 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39e0004300 con 0x7f39ec1097b0 2024-05-08T05:57:16.426 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.435+0000 7f39ebe00700 1 -- 172.21.15.123:0/376986771 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39e0011a20 con 0x7f39ec1097b0 2024-05-08T05:57:16.426 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.436+0000 7f39f2a00700 1 -- 172.21.15.123:0/376986771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39ec1097b0 msgr2=0x7f39ec10bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:16.427 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.436+0000 7f39f2a00700 1 --2- 172.21.15.123:0/376986771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39ec1097b0 0x7f39ec10bba0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f39e00098a0 tx=0x7f39e0009bb0 comp rx=0 tx=0).stop 2024-05-08T05:57:16.427 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.436+0000 7f39f2a00700 1 -- 172.21.15.123:0/376986771 shutdown_connections 2024-05-08T05:57:16.427 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.436+0000 7f39f2a00700 1 --2- 172.21.15.123:0/376986771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39ec1097b0 0x7f39ec10bba0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:16.427 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.436+0000 7f39f2a00700 1 -- 172.21.15.123:0/376986771 >> 172.21.15.123:0/376986771 conn(0x7f39ec103240 msgr2=0x7f39ec105660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:16.427 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.436+0000 7f39f2a00700 1 -- 172.21.15.123:0/376986771 shutdown_connections 2024-05-08T05:57:16.427 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.437+0000 7f39f2a00700 1 -- 172.21.15.123:0/376986771 wait complete. 2024-05-08T05:57:16.427 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.438+0000 7f39f2a00700 1 Processor -- start 2024-05-08T05:57:16.427 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.438+0000 7f39f2a00700 1 -- start start 2024-05-08T05:57:16.427 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.439+0000 7f39f2a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39ec1097b0 0x7f39ec19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:16.427 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.439+0000 7f39f2a00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f39ec10d0f0 con 0x7f39ec1097b0 2024-05-08T05:57:16.427 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.439+0000 7f39f1600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39ec1097b0 0x7f39ec19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:16.427 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.439+0000 7f39f1600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39ec1097b0 0x7f39ec19cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55266/0 (socket says 172.21.15.123:55266) 2024-05-08T05:57:16.427 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.439+0000 7f39f1600700 1 -- 172.21.15.123:0/1289843084 learned_addr learned my addr 172.21.15.123:0/1289843084 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:57:16.428 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.439+0000 7f39f1600700 1 -- 172.21.15.123:0/1289843084 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39ec19d180 con 0x7f39ec1097b0 2024-05-08T05:57:16.428 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.440+0000 7f39f1600700 1 --2- 172.21.15.123:0/1289843084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39ec1097b0 0x7f39ec19cc40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f39e0004660 tx=0x7f39e0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:16.428 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.440+0000 7f39ea000700 1 -- 172.21.15.123:0/1289843084 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39e0003aa0 con 0x7f39ec1097b0 2024-05-08T05:57:16.428 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.440+0000 7f39ea000700 1 -- 172.21.15.123:0/1289843084 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39e001b8f0 con 0x7f39ec1097b0 2024-05-08T05:57:16.428 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.440+0000 7f39f2a00700 1 -- 172.21.15.123:0/1289843084 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39ec19d410 con 0x7f39ec1097b0 2024-05-08T05:57:16.428 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.440+0000 7f39f2a00700 1 -- 172.21.15.123:0/1289843084 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39ec1a0100 con 0x7f39ec1097b0 2024-05-08T05:57:16.428 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.440+0000 7f39ea000700 1 -- 172.21.15.123:0/1289843084 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39e0023900 con 0x7f39ec1097b0 2024-05-08T05:57:16.428 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.441+0000 7f39ea000700 1 -- 172.21.15.123:0/1289843084 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f39e0023b70 con 0x7f39ec1097b0 2024-05-08T05:57:16.428 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.442+0000 7f39f2a00700 1 -- 172.21.15.123:0/1289843084 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39ec06a5e0 con 0x7f39ec1097b0 2024-05-08T05:57:16.428 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.442+0000 7f39ea000700 1 --2- 172.21.15.123:0/1289843084 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f39d403e4c0 0x7f39d4040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:16.428 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.442+0000 7f39ea000700 1 -- 172.21.15.123:0/1289843084 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f39e0054df0 con 0x7f39ec1097b0 2024-05-08T05:57:16.428 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.442+0000 7f39f0c00700 1 --2- 172.21.15.123:0/1289843084 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f39d403e4c0 0x7f39d4040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:16.428 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.443+0000 7f39f0c00700 1 --2- 172.21.15.123:0/1289843084 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f39d403e4c0 0x7f39d4040980 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f39d8006fd0 tx=0x7f39d8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:16.429 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.448+0000 7f39ea000700 1 -- 172.21.15.123:0/1289843084 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f39e001f030 con 0x7f39ec1097b0 2024-05-08T05:57:16.429 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.711+0000 7f39f2a00700 1 -- 172.21.15.123:0/1289843084 --> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f39ec104420 con 0x7f39d403e4c0 2024-05-08T05:57:16.429 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.718+0000 7f39ea000700 1 -- 172.21.15.123:0/1289843084 <== mgr.14120 v2:172.21.15.123:6800/3057688649 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f39ec104420 con 0x7f39d403e4c0 2024-05-08T05:57:16.429 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.720+0000 7f39f2a00700 1 -- 172.21.15.123:0/1289843084 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f39d403e4c0 msgr2=0x7f39d4040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:16.429 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.720+0000 7f39f2a00700 1 --2- 172.21.15.123:0/1289843084 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f39d403e4c0 0x7f39d4040980 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f39d8006fd0 tx=0x7f39d8006e40 comp rx=0 tx=0).stop 2024-05-08T05:57:16.429 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.720+0000 7f39f2a00700 1 -- 172.21.15.123:0/1289843084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39ec1097b0 msgr2=0x7f39ec19cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:16.429 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.720+0000 7f39f2a00700 1 --2- 172.21.15.123:0/1289843084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39ec1097b0 0x7f39ec19cc40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f39e0004660 tx=0x7f39e0011aa0 comp rx=0 tx=0).stop 2024-05-08T05:57:16.429 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.721+0000 7f39f2a00700 1 -- 172.21.15.123:0/1289843084 shutdown_connections 2024-05-08T05:57:16.429 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.721+0000 7f39f2a00700 1 --2- 172.21.15.123:0/1289843084 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f39d403e4c0 0x7f39d4040980 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:16.429 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.721+0000 7f39f2a00700 1 --2- 172.21.15.123:0/1289843084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39ec1097b0 0x7f39ec19cc40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:16.429 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.721+0000 7f39f2a00700 1 -- 172.21.15.123:0/1289843084 >> 172.21.15.123:0/1289843084 conn(0x7f39ec103240 msgr2=0x7f39ec103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:16.429 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.721+0000 7f39f2a00700 1 -- 172.21.15.123:0/1289843084 shutdown_connections 2024-05-08T05:57:16.429 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:15.721+0000 7f39f2a00700 1 -- 172.21.15.123:0/1289843084 wait complete. 2024-05-08T05:57:16.984 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:16 smithi123 ceph-mon[29787]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:57:18.464 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.068+0000 7f45cea00700 1 Processor -- start 2024-05-08T05:57:18.464 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.068+0000 7f45cea00700 1 -- start start 2024-05-08T05:57:18.464 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.068+0000 7f45cea00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45c81097c0 0x7f45c810bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:18.464 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.068+0000 7f45cea00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f45c807ceb0 con 0x7f45c81097c0 2024-05-08T05:57:18.464 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.069+0000 7f45cd600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45c81097c0 0x7f45c810bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:18.464 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.069+0000 7f45cd600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45c81097c0 0x7f45c810bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55280/0 (socket says 172.21.15.123:55280) 2024-05-08T05:57:18.464 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.069+0000 7f45cd600700 1 -- 172.21.15.123:0/1053309702 learned_addr learned my addr 172.21.15.123:0/1053309702 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:57:18.464 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.069+0000 7f45cd600700 1 -- 172.21.15.123:0/1053309702 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c810c0f0 con 0x7f45c81097c0 2024-05-08T05:57:18.464 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.070+0000 7f45cd600700 1 --2- 172.21.15.123:0/1053309702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45c81097c0 0x7f45c810bbb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f45bc0098a0 tx=0x7f45bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=510437291373468d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:18.464 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.070+0000 7f45c7e00700 1 -- 172.21.15.123:0/1053309702 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45bc004160 con 0x7f45c81097c0 2024-05-08T05:57:18.465 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.070+0000 7f45c7e00700 1 -- 172.21.15.123:0/1053309702 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45bc004300 con 0x7f45c81097c0 2024-05-08T05:57:18.465 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.070+0000 7f45c7e00700 1 -- 172.21.15.123:0/1053309702 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45bc011a20 con 0x7f45c81097c0 2024-05-08T05:57:18.465 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.071+0000 7f45cea00700 1 -- 172.21.15.123:0/1053309702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45c81097c0 msgr2=0x7f45c810bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:18.465 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.071+0000 7f45cea00700 1 --2- 172.21.15.123:0/1053309702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45c81097c0 0x7f45c810bbb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f45bc0098a0 tx=0x7f45bc009bb0 comp rx=0 tx=0).stop 2024-05-08T05:57:18.465 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.071+0000 7f45cea00700 1 -- 172.21.15.123:0/1053309702 shutdown_connections 2024-05-08T05:57:18.465 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.071+0000 7f45cea00700 1 --2- 172.21.15.123:0/1053309702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45c81097c0 0x7f45c810bbb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:18.465 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.071+0000 7f45cea00700 1 -- 172.21.15.123:0/1053309702 >> 172.21.15.123:0/1053309702 conn(0x7f45c81031b0 msgr2=0x7f45c8105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:18.465 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.071+0000 7f45cea00700 1 -- 172.21.15.123:0/1053309702 shutdown_connections 2024-05-08T05:57:18.465 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.071+0000 7f45cea00700 1 -- 172.21.15.123:0/1053309702 wait complete. 2024-05-08T05:57:18.465 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.072+0000 7f45cea00700 1 Processor -- start 2024-05-08T05:57:18.465 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.072+0000 7f45cea00700 1 -- start start 2024-05-08T05:57:18.465 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.072+0000 7f45cea00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45c81097c0 0x7f45c819edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:18.465 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.072+0000 7f45cea00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f45c810d0f0 con 0x7f45c81097c0 2024-05-08T05:57:18.465 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.073+0000 7f45cd600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45c81097c0 0x7f45c819edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:18.465 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.073+0000 7f45cd600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45c81097c0 0x7f45c819edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55296/0 (socket says 172.21.15.123:55296) 2024-05-08T05:57:18.466 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.073+0000 7f45cd600700 1 -- 172.21.15.123:0/3468688379 learned_addr learned my addr 172.21.15.123:0/3468688379 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:57:18.466 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.073+0000 7f45cd600700 1 -- 172.21.15.123:0/3468688379 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c819f300 con 0x7f45c81097c0 2024-05-08T05:57:18.466 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.073+0000 7f45cd600700 1 --2- 172.21.15.123:0/3468688379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45c81097c0 0x7f45c819edc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f45bc004660 tx=0x7f45bc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:18.466 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.074+0000 7f45c6000700 1 -- 172.21.15.123:0/3468688379 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45bc003aa0 con 0x7f45c81097c0 2024-05-08T05:57:18.466 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.074+0000 7f45c6000700 1 -- 172.21.15.123:0/3468688379 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45bc01b8f0 con 0x7f45c81097c0 2024-05-08T05:57:18.466 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.074+0000 7f45cea00700 1 -- 172.21.15.123:0/3468688379 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45c819f5f0 con 0x7f45c81097c0 2024-05-08T05:57:18.466 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.074+0000 7f45c6000700 1 -- 172.21.15.123:0/3468688379 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45bc023900 con 0x7f45c81097c0 2024-05-08T05:57:18.466 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.074+0000 7f45cea00700 1 -- 172.21.15.123:0/3468688379 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45c81a2280 con 0x7f45c81097c0 2024-05-08T05:57:18.466 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.075+0000 7f45c6000700 1 -- 172.21.15.123:0/3468688379 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f45bc023b70 con 0x7f45c81097c0 2024-05-08T05:57:18.466 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.075+0000 7f45cea00700 1 -- 172.21.15.123:0/3468688379 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45c806a5e0 con 0x7f45c81097c0 2024-05-08T05:57:18.466 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.075+0000 7f45c6000700 1 --2- 172.21.15.123:0/3468688379 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f45b003e4c0 0x7f45b0040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:18.466 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.076+0000 7f45c6000700 1 -- 172.21.15.123:0/3468688379 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f45bc053dd0 con 0x7f45c81097c0 2024-05-08T05:57:18.466 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.076+0000 7f45ccc00700 1 --2- 172.21.15.123:0/3468688379 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f45b003e4c0 0x7f45b0040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:18.467 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.076+0000 7f45ccc00700 1 --2- 172.21.15.123:0/3468688379 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f45b003e4c0 0x7f45b0040980 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f45b4006fd0 tx=0x7f45b4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:18.467 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.082+0000 7f45c6000700 1 -- 172.21.15.123:0/3468688379 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f45bc01f030 con 0x7f45c81097c0 2024-05-08T05:57:18.467 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.346+0000 7f45cea00700 1 -- 172.21.15.123:0/3468688379 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f45c81a29c0 con 0x7f45c81097c0 2024-05-08T05:57:18.467 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.352+0000 7f45c6000700 1 -- 172.21.15.123:0/3468688379 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f45bc00b3e0 con 0x7f45c81097c0 2024-05-08T05:57:18.467 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.355+0000 7f45cea00700 1 -- 172.21.15.123:0/3468688379 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f45b003e4c0 msgr2=0x7f45b0040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:18.467 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.355+0000 7f45cea00700 1 --2- 172.21.15.123:0/3468688379 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f45b003e4c0 0x7f45b0040980 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f45b4006fd0 tx=0x7f45b4006e40 comp rx=0 tx=0).stop 2024-05-08T05:57:18.467 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.356+0000 7f45cea00700 1 -- 172.21.15.123:0/3468688379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45c81097c0 msgr2=0x7f45c819edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:18.467 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.356+0000 7f45cea00700 1 --2- 172.21.15.123:0/3468688379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45c81097c0 0x7f45c819edc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f45bc004660 tx=0x7f45bc011aa0 comp rx=0 tx=0).stop 2024-05-08T05:57:18.467 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.356+0000 7f45cea00700 1 -- 172.21.15.123:0/3468688379 shutdown_connections 2024-05-08T05:57:18.467 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.356+0000 7f45cea00700 1 --2- 172.21.15.123:0/3468688379 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f45b003e4c0 0x7f45b0040980 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:18.467 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.356+0000 7f45cea00700 1 --2- 172.21.15.123:0/3468688379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45c81097c0 0x7f45c819edc0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:18.467 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.356+0000 7f45cea00700 1 -- 172.21.15.123:0/3468688379 >> 172.21.15.123:0/3468688379 conn(0x7f45c81031b0 msgr2=0x7f45c8103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:18.467 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.356+0000 7f45cea00700 1 -- 172.21.15.123:0/3468688379 shutdown_connections 2024-05-08T05:57:18.467 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:17.356+0000 7f45cea00700 1 -- 172.21.15.123:0/3468688379 wait complete. 2024-05-08T05:57:18.616 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:18 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/3468688379' entity='client.admin' 2024-05-08T05:57:19.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:19 smithi123 ceph-mon[29787]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:57:19.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:19 smithi123 ceph-mon[29787]: from='mgr.14120 172.21.15.123:0/1121855348' entity='mgr.x' 2024-05-08T05:57:19.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:19 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/4081837780' entity='client.admin' 2024-05-08T05:57:20.164 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.156+0000 7f773ce00700 1 Processor -- start 2024-05-08T05:57:20.164 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.156+0000 7f773ce00700 1 -- start start 2024-05-08T05:57:20.164 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.156+0000 7f773ce00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7738108ef0 0x7f7738109310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:20.165 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.156+0000 7f773ce00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7738109850 con 0x7f7738108ef0 2024-05-08T05:57:20.165 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.157+0000 7f7735e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7738108ef0 0x7f7738109310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:20.165 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.157+0000 7f7735e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7738108ef0 0x7f7738109310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55302/0 (socket says 172.21.15.123:55302) 2024-05-08T05:57:20.165 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.157+0000 7f7735e00700 1 -- 172.21.15.123:0/769199273 learned_addr learned my addr 172.21.15.123:0/769199273 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:57:20.165 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.158+0000 7f7735e00700 1 -- 172.21.15.123:0/769199273 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f773810a090 con 0x7f7738108ef0 2024-05-08T05:57:20.165 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.158+0000 7f7735e00700 1 --2- 172.21.15.123:0/769199273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7738108ef0 0x7f7738109310 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f77280098a0 tx=0x7f7728009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10de70ba33fc16b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:20.165 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.158+0000 7f7734a00700 1 -- 172.21.15.123:0/769199273 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7728004160 con 0x7f7738108ef0 2024-05-08T05:57:20.165 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.158+0000 7f7734a00700 1 -- 172.21.15.123:0/769199273 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7728004300 con 0x7f7738108ef0 2024-05-08T05:57:20.165 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.158+0000 7f7734a00700 1 -- 172.21.15.123:0/769199273 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7728011a20 con 0x7f7738108ef0 2024-05-08T05:57:20.165 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.159+0000 7f773ce00700 1 -- 172.21.15.123:0/769199273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7738108ef0 msgr2=0x7f7738109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:20.165 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.159+0000 7f773ce00700 1 --2- 172.21.15.123:0/769199273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7738108ef0 0x7f7738109310 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f77280098a0 tx=0x7f7728009bb0 comp rx=0 tx=0).stop 2024-05-08T05:57:20.165 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.160+0000 7f773ce00700 1 -- 172.21.15.123:0/769199273 shutdown_connections 2024-05-08T05:57:20.165 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.160+0000 7f773ce00700 1 --2- 172.21.15.123:0/769199273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7738108ef0 0x7f7738109310 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:20.165 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.160+0000 7f773ce00700 1 -- 172.21.15.123:0/769199273 >> 172.21.15.123:0/769199273 conn(0x7f7738104520 msgr2=0x7f7738106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:20.166 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.160+0000 7f773ce00700 1 -- 172.21.15.123:0/769199273 shutdown_connections 2024-05-08T05:57:20.166 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.160+0000 7f773ce00700 1 -- 172.21.15.123:0/769199273 wait complete. 2024-05-08T05:57:20.166 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.161+0000 7f773ce00700 1 Processor -- start 2024-05-08T05:57:20.166 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.161+0000 7f773ce00700 1 -- start start 2024-05-08T05:57:20.166 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.162+0000 7f773ce00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7738108ef0 0x7f773807f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:20.166 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.162+0000 7f773ce00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f773810d350 con 0x7f7738108ef0 2024-05-08T05:57:20.166 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.162+0000 7f7735e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7738108ef0 0x7f773807f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:20.166 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.162+0000 7f7735e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7738108ef0 0x7f773807f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55306/0 (socket says 172.21.15.123:55306) 2024-05-08T05:57:20.166 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.162+0000 7f7735e00700 1 -- 172.21.15.123:0/4081837780 learned_addr learned my addr 172.21.15.123:0/4081837780 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:57:20.166 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.162+0000 7f7735e00700 1 -- 172.21.15.123:0/4081837780 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f773807f960 con 0x7f7738108ef0 2024-05-08T05:57:20.166 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.163+0000 7f7735e00700 1 --2- 172.21.15.123:0/4081837780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7738108ef0 0x7f773807f420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7728000c00 tx=0x7f7728011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:20.166 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.163+0000 7f772ea00700 1 -- 172.21.15.123:0/4081837780 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7728003aa0 con 0x7f7738108ef0 2024-05-08T05:57:20.166 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.163+0000 7f772ea00700 1 -- 172.21.15.123:0/4081837780 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f772801b8f0 con 0x7f7738108ef0 2024-05-08T05:57:20.166 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.163+0000 7f772ea00700 1 -- 172.21.15.123:0/4081837780 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7728023890 con 0x7f7738108ef0 2024-05-08T05:57:20.167 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.163+0000 7f773ce00700 1 -- 172.21.15.123:0/4081837780 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f773807db50 con 0x7f7738108ef0 2024-05-08T05:57:20.167 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.163+0000 7f773ce00700 1 -- 172.21.15.123:0/4081837780 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f773807e090 con 0x7f7738108ef0 2024-05-08T05:57:20.167 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.165+0000 7f772ea00700 1 -- 172.21.15.123:0/4081837780 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f772801b440 con 0x7f7738108ef0 2024-05-08T05:57:20.167 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.165+0000 7f773ce00700 1 -- 172.21.15.123:0/4081837780 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7738064b00 con 0x7f7738108ef0 2024-05-08T05:57:20.167 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.165+0000 7f772ea00700 1 --2- 172.21.15.123:0/4081837780 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f771c03e4c0 0x7f771c040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:20.167 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.165+0000 7f772ea00700 1 -- 172.21.15.123:0/4081837780 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7728053e70 con 0x7f7738108ef0 2024-05-08T05:57:20.167 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.166+0000 7f7735400700 1 --2- 172.21.15.123:0/4081837780 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f771c03e4c0 0x7f771c040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:20.167 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.166+0000 7f7735400700 1 --2- 172.21.15.123:0/4081837780 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f771c03e4c0 0x7f771c040980 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7724006fd0 tx=0x7f7724006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:20.167 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.176+0000 7f772ea00700 1 -- 172.21.15.123:0/4081837780 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f772801f030 con 0x7f7738108ef0 2024-05-08T05:57:20.167 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.417+0000 7f773ce00700 1 -- 172.21.15.123:0/4081837780 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f773807e8a0 con 0x7f7738108ef0 2024-05-08T05:57:20.167 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.421+0000 7f772ea00700 1 -- 172.21.15.123:0/4081837780 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f7728016260 con 0x7f7738108ef0 2024-05-08T05:57:20.167 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.424+0000 7f773ce00700 1 -- 172.21.15.123:0/4081837780 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f771c03e4c0 msgr2=0x7f771c040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:20.167 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.424+0000 7f773ce00700 1 --2- 172.21.15.123:0/4081837780 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f771c03e4c0 0x7f771c040980 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7724006fd0 tx=0x7f7724006e40 comp rx=0 tx=0).stop 2024-05-08T05:57:20.167 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.424+0000 7f773ce00700 1 -- 172.21.15.123:0/4081837780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7738108ef0 msgr2=0x7f773807f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:20.167 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.424+0000 7f773ce00700 1 --2- 172.21.15.123:0/4081837780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7738108ef0 0x7f773807f420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7728000c00 tx=0x7f7728011aa0 comp rx=0 tx=0).stop 2024-05-08T05:57:20.168 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.424+0000 7f773ce00700 1 -- 172.21.15.123:0/4081837780 shutdown_connections 2024-05-08T05:57:20.168 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.425+0000 7f773ce00700 1 --2- 172.21.15.123:0/4081837780 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f771c03e4c0 0x7f771c040980 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:20.168 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.425+0000 7f773ce00700 1 --2- 172.21.15.123:0/4081837780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7738108ef0 0x7f773807f420 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:20.168 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.425+0000 7f773ce00700 1 -- 172.21.15.123:0/4081837780 >> 172.21.15.123:0/4081837780 conn(0x7f7738104520 msgr2=0x7f7738105080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:20.168 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.425+0000 7f773ce00700 1 -- 172.21.15.123:0/4081837780 shutdown_connections 2024-05-08T05:57:20.168 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:19.425+0000 7f773ce00700 1 -- 172.21.15.123:0/4081837780 wait complete. 2024-05-08T05:57:20.168 INFO:teuthology.orchestra.run.smithi123.stdout:Enabling the dashboard module... 2024-05-08T05:57:21.873 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:21 smithi123 ceph-mon[29787]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:57:21.873 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:21 smithi123 ceph-mon[29787]: from='mgr.14120 172.21.15.123:0/1121855348' entity='mgr.x' 2024-05-08T05:57:21.873 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:21 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/1621689815' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-08T05:57:21.873 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:21 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: ignoring --setuser ceph since I am not root 2024-05-08T05:57:21.873 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:21 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: ignoring --setgroup ceph since I am not root 2024-05-08T05:57:21.874 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:21 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:57:21.871+0000 7f78b8333200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T05:57:22.140 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:22 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:57:22.121+0000 7f78b8333200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:57:22.455 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:22 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:57:22.252+0000 7f78b8333200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T05:57:22.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:22 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/1621689815' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-08T05:57:22.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:22 smithi123 ceph-mon[29787]: mgrmap e9: x(active, since 25s) 2024-05-08T05:57:22.955 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:22 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:57:22.659+0000 7f78b8333200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T05:57:22.956 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:22 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:57:22.776+0000 7f78b8333200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T05:57:23.332 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.802+0000 7fcd83000700 1 Processor -- start 2024-05-08T05:57:23.332 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.803+0000 7fcd83000700 1 -- start start 2024-05-08T05:57:23.332 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.803+0000 7fcd83000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd7c10a540 0x7fcd7c10a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:23.333 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.803+0000 7fcd83000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7c10aea0 con 0x7fcd7c10a540 2024-05-08T05:57:23.333 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.804+0000 7fcd7be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd7c10a540 0x7fcd7c10a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:23.333 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.804+0000 7fcd7be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd7c10a540 0x7fcd7c10a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55318/0 (socket says 172.21.15.123:55318) 2024-05-08T05:57:23.333 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.804+0000 7fcd7be00700 1 -- 172.21.15.123:0/1776990014 learned_addr learned my addr 172.21.15.123:0/1776990014 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:57:23.333 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.804+0000 7fcd7be00700 1 -- 172.21.15.123:0/1776990014 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd7c108bc0 con 0x7fcd7c10a540 2024-05-08T05:57:23.333 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.804+0000 7fcd7be00700 1 --2- 172.21.15.123:0/1776990014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd7c10a540 0x7fcd7c10a960 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c0098a0 tx=0x7fcd6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445cc9a23e017b8c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:23.333 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.805+0000 7fcd7b400700 1 -- 172.21.15.123:0/1776990014 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd6c004160 con 0x7fcd7c10a540 2024-05-08T05:57:23.334 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.805+0000 7fcd7b400700 1 -- 172.21.15.123:0/1776990014 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd6c004300 con 0x7fcd7c10a540 2024-05-08T05:57:23.334 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.805+0000 7fcd7b400700 1 -- 172.21.15.123:0/1776990014 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd6c011a20 con 0x7fcd7c10a540 2024-05-08T05:57:23.334 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.805+0000 7fcd83000700 1 -- 172.21.15.123:0/1776990014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd7c10a540 msgr2=0x7fcd7c10a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:23.334 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.805+0000 7fcd83000700 1 --2- 172.21.15.123:0/1776990014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd7c10a540 0x7fcd7c10a960 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c0098a0 tx=0x7fcd6c009bb0 comp rx=0 tx=0).stop 2024-05-08T05:57:23.334 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.806+0000 7fcd83000700 1 -- 172.21.15.123:0/1776990014 shutdown_connections 2024-05-08T05:57:23.334 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.806+0000 7fcd83000700 1 --2- 172.21.15.123:0/1776990014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd7c10a540 0x7fcd7c10a960 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:23.334 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.806+0000 7fcd83000700 1 -- 172.21.15.123:0/1776990014 >> 172.21.15.123:0/1776990014 conn(0x7fcd7c104580 msgr2=0x7fcd7c1069a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:23.334 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.806+0000 7fcd83000700 1 -- 172.21.15.123:0/1776990014 shutdown_connections 2024-05-08T05:57:23.334 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.806+0000 7fcd83000700 1 -- 172.21.15.123:0/1776990014 wait complete. 2024-05-08T05:57:23.334 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.807+0000 7fcd83000700 1 Processor -- start 2024-05-08T05:57:23.334 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.807+0000 7fcd83000700 1 -- start start 2024-05-08T05:57:23.335 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.808+0000 7fcd83000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd7c10a540 0x7fcd7c19f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:23.335 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.808+0000 7fcd83000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7c10d320 con 0x7fcd7c10a540 2024-05-08T05:57:23.335 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.808+0000 7fcd7be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd7c10a540 0x7fcd7c19f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:23.335 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.808+0000 7fcd7be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd7c10a540 0x7fcd7c19f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55332/0 (socket says 172.21.15.123:55332) 2024-05-08T05:57:23.335 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.808+0000 7fcd7be00700 1 -- 172.21.15.123:0/1621689815 learned_addr learned my addr 172.21.15.123:0/1621689815 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:57:23.335 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.808+0000 7fcd7be00700 1 -- 172.21.15.123:0/1621689815 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd7c19f5c0 con 0x7fcd7c10a540 2024-05-08T05:57:23.335 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.809+0000 7fcd7be00700 1 --2- 172.21.15.123:0/1621689815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd7c10a540 0x7fcd7c19f080 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c004660 tx=0x7fcd6c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:23.335 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.809+0000 7fcd79600700 1 -- 172.21.15.123:0/1621689815 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd6c003aa0 con 0x7fcd7c10a540 2024-05-08T05:57:23.335 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.809+0000 7fcd79600700 1 -- 172.21.15.123:0/1621689815 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd6c01b8f0 con 0x7fcd7c10a540 2024-05-08T05:57:23.336 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.809+0000 7fcd83000700 1 -- 172.21.15.123:0/1621689815 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd7c19f850 con 0x7fcd7c10a540 2024-05-08T05:57:23.336 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.809+0000 7fcd79600700 1 -- 172.21.15.123:0/1621689815 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd6c023900 con 0x7fcd7c10a540 2024-05-08T05:57:23.336 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.809+0000 7fcd83000700 1 -- 172.21.15.123:0/1621689815 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd7c1a2540 con 0x7fcd7c10a540 2024-05-08T05:57:23.336 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.810+0000 7fcd83000700 1 -- 172.21.15.123:0/1621689815 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd7c064b00 con 0x7fcd7c10a540 2024-05-08T05:57:23.336 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.810+0000 7fcd79600700 1 -- 172.21.15.123:0/1621689815 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fcd6c023b70 con 0x7fcd7c10a540 2024-05-08T05:57:23.336 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.811+0000 7fcd79600700 1 --2- 172.21.15.123:0/1621689815 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7fcd5c03e4c0 0x7fcd5c040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:23.336 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.811+0000 7fcd79600700 1 -- 172.21.15.123:0/1621689815 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcd6c053d70 con 0x7fcd7c10a540 2024-05-08T05:57:23.336 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.811+0000 7fcd73400700 1 --2- 172.21.15.123:0/1621689815 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7fcd5c03e4c0 0x7fcd5c040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:23.337 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.812+0000 7fcd73400700 1 --2- 172.21.15.123:0/1621689815 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7fcd5c03e4c0 0x7fcd5c040980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcd64006fd0 tx=0x7fcd64006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:23.337 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:20.817+0000 7fcd79600700 1 -- 172.21.15.123:0/1621689815 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcd6c01f030 con 0x7fcd7c10a540 2024-05-08T05:57:23.337 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:21.115+0000 7fcd83000700 1 -- 172.21.15.123:0/1621689815 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fcd7c1a2a30 con 0x7fcd7c10a540 2024-05-08T05:57:23.337 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:21.555+0000 7fcd79600700 1 -- 172.21.15.123:0/1621689815 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fcd6c016260 con 0x7fcd7c10a540 2024-05-08T05:57:23.337 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:21.558+0000 7fcd79600700 1 -- 172.21.15.123:0/1621689815 <== mon.0 v2:172.21.15.123:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7fcd6c01acb0 con 0x7fcd7c10a540 2024-05-08T05:57:23.337 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:21.561+0000 7fcd83000700 1 -- 172.21.15.123:0/1621689815 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7fcd5c03e4c0 msgr2=0x7fcd5c040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:23.339 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:21.561+0000 7fcd83000700 1 --2- 172.21.15.123:0/1621689815 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7fcd5c03e4c0 0x7fcd5c040980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcd64006fd0 tx=0x7fcd64006e40 comp rx=0 tx=0).stop 2024-05-08T05:57:23.339 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:21.561+0000 7fcd83000700 1 -- 172.21.15.123:0/1621689815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd7c10a540 msgr2=0x7fcd7c19f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:23.339 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:21.561+0000 7fcd83000700 1 --2- 172.21.15.123:0/1621689815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd7c10a540 0x7fcd7c19f080 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c004660 tx=0x7fcd6c011aa0 comp rx=0 tx=0).stop 2024-05-08T05:57:23.339 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:21.561+0000 7fcd83000700 1 -- 172.21.15.123:0/1621689815 shutdown_connections 2024-05-08T05:57:23.339 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:21.561+0000 7fcd83000700 1 --2- 172.21.15.123:0/1621689815 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7fcd5c03e4c0 0x7fcd5c040980 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:23.339 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:21.561+0000 7fcd83000700 1 --2- 172.21.15.123:0/1621689815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd7c10a540 0x7fcd7c19f080 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:23.340 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:21.561+0000 7fcd83000700 1 -- 172.21.15.123:0/1621689815 >> 172.21.15.123:0/1621689815 conn(0x7fcd7c104580 msgr2=0x7fcd7c105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:23.340 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:21.562+0000 7fcd83000700 1 -- 172.21.15.123:0/1621689815 shutdown_connections 2024-05-08T05:57:23.340 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:21.562+0000 7fcd83000700 1 -- 172.21.15.123:0/1621689815 wait complete. 2024-05-08T05:57:24.537 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:24 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/1443772281' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-08T05:57:24.537 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:24 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:57:24.362+0000 7f78b8333200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T05:57:24.835 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:24 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:57:24.605+0000 7f78b8333200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T05:57:24.835 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:24 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:57:24.834+0000 7f78b8333200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T05:57:24.939 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout { 2024-05-08T05:57:24.940 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-08T05:57:24.940 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-08T05:57:24.940 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-08T05:57:24.940 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-08T05:57:24.940 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout } 2024-05-08T05:57:24.940 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.927+0000 7f63dfe00700 1 Processor -- start 2024-05-08T05:57:24.940 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.927+0000 7f63dfe00700 1 -- start start 2024-05-08T05:57:24.940 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.927+0000 7f63dfe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63e0107610 0x7f63e0107a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:24.940 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.927+0000 7f63dfe00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f63e007ce50 con 0x7f63e0107610 2024-05-08T05:57:24.940 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.928+0000 7f63dea00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63e0107610 0x7f63e0107a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:24.940 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.928+0000 7f63dea00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63e0107610 0x7f63e0107a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55356/0 (socket says 172.21.15.123:55356) 2024-05-08T05:57:24.940 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.928+0000 7f63dea00700 1 -- 172.21.15.123:0/3665567651 learned_addr learned my addr 172.21.15.123:0/3665567651 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:57:24.940 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.929+0000 7f63dea00700 1 -- 172.21.15.123:0/3665567651 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e0107f70 con 0x7f63e0107610 2024-05-08T05:57:24.941 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.929+0000 7f63dea00700 1 --2- 172.21.15.123:0/3665567651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63e0107610 0x7f63e0107a30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f63d00098a0 tx=0x7f63d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ce9f645f4ae7f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:24.941 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.929+0000 7f63dd600700 1 -- 172.21.15.123:0/3665567651 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63d0004160 con 0x7f63e0107610 2024-05-08T05:57:24.941 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.929+0000 7f63dd600700 1 -- 172.21.15.123:0/3665567651 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63d0004300 con 0x7f63e0107610 2024-05-08T05:57:24.941 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.929+0000 7f63dd600700 1 -- 172.21.15.123:0/3665567651 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63d0011a20 con 0x7f63e0107610 2024-05-08T05:57:24.941 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.930+0000 7f63dfe00700 1 -- 172.21.15.123:0/3665567651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63e0107610 msgr2=0x7f63e0107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:24.941 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.930+0000 7f63dfe00700 1 --2- 172.21.15.123:0/3665567651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63e0107610 0x7f63e0107a30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f63d00098a0 tx=0x7f63d0009bb0 comp rx=0 tx=0).stop 2024-05-08T05:57:24.941 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.930+0000 7f63dfe00700 1 -- 172.21.15.123:0/3665567651 shutdown_connections 2024-05-08T05:57:24.941 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.930+0000 7f63dfe00700 1 --2- 172.21.15.123:0/3665567651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63e0107610 0x7f63e0107a30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:24.941 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.930+0000 7f63dfe00700 1 -- 172.21.15.123:0/3665567651 >> 172.21.15.123:0/3665567651 conn(0x7f63e01031b0 msgr2=0x7f63e0105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:24.941 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.930+0000 7f63dfe00700 1 -- 172.21.15.123:0/3665567651 shutdown_connections 2024-05-08T05:57:24.941 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.930+0000 7f63dfe00700 1 -- 172.21.15.123:0/3665567651 wait complete. 2024-05-08T05:57:24.941 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.931+0000 7f63dfe00700 1 Processor -- start 2024-05-08T05:57:24.941 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.931+0000 7f63dfe00700 1 -- start start 2024-05-08T05:57:24.941 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.932+0000 7f63dfe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63e0107610 0x7f63e019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:24.941 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.932+0000 7f63dfe00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f63e0108fe0 con 0x7f63e0107610 2024-05-08T05:57:24.942 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.932+0000 7f63dea00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63e0107610 0x7f63e019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:24.942 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.932+0000 7f63dea00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63e0107610 0x7f63e019ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55358/0 (socket says 172.21.15.123:55358) 2024-05-08T05:57:24.942 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.932+0000 7f63dea00700 1 -- 172.21.15.123:0/1443772281 learned_addr learned my addr 172.21.15.123:0/1443772281 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:57:24.942 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.932+0000 7f63dea00700 1 -- 172.21.15.123:0/1443772281 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e019f350 con 0x7f63e0107610 2024-05-08T05:57:24.942 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.933+0000 7f63dea00700 1 --2- 172.21.15.123:0/1443772281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63e0107610 0x7f63e019ee10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f63d0004660 tx=0x7f63d0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:24.942 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.933+0000 7f63d7400700 1 -- 172.21.15.123:0/1443772281 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63d0003aa0 con 0x7f63e0107610 2024-05-08T05:57:24.942 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.933+0000 7f63d7400700 1 -- 172.21.15.123:0/1443772281 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63d001b8f0 con 0x7f63e0107610 2024-05-08T05:57:24.942 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.933+0000 7f63dfe00700 1 -- 172.21.15.123:0/1443772281 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63e019f5e0 con 0x7f63e0107610 2024-05-08T05:57:24.942 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.933+0000 7f63d7400700 1 -- 172.21.15.123:0/1443772281 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63d0023900 con 0x7f63e0107610 2024-05-08T05:57:24.942 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.933+0000 7f63dfe00700 1 -- 172.21.15.123:0/1443772281 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63e01a22d0 con 0x7f63e0107610 2024-05-08T05:57:24.942 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.934+0000 7f63dfe00700 1 -- 172.21.15.123:0/1443772281 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63e006a5e0 con 0x7f63e0107610 2024-05-08T05:57:24.942 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.934+0000 7f63d7400700 1 -- 172.21.15.123:0/1443772281 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f63d0023bc0 con 0x7f63e0107610 2024-05-08T05:57:24.942 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.935+0000 7f63d7400700 1 --2- 172.21.15.123:0/1443772281 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f63c403e4c0 0x7f63c4040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:24.942 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.935+0000 7f63d7400700 1 -- 172.21.15.123:0/1443772281 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f63d0053f80 con 0x7f63e0107610 2024-05-08T05:57:24.943 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.935+0000 7f63de000700 1 -- 172.21.15.123:0/1443772281 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f63c403e4c0 msgr2=0x7f63c4040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/3057688649 2024-05-08T05:57:24.943 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.935+0000 7f63de000700 1 --2- 172.21.15.123:0/1443772281 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f63c403e4c0 0x7f63c4040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T05:57:24.943 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:23.941+0000 7f63d7400700 1 -- 172.21.15.123:0/1443772281 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f63d001f030 con 0x7f63e0107610 2024-05-08T05:57:24.943 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:24.136+0000 7f63de000700 1 -- 172.21.15.123:0/1443772281 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f63c403e4c0 msgr2=0x7f63c4040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/3057688649 2024-05-08T05:57:24.943 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:24.136+0000 7f63de000700 1 --2- 172.21.15.123:0/1443772281 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f63c403e4c0 0x7f63c4040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T05:57:24.943 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:24.228+0000 7f63dfe00700 1 -- 172.21.15.123:0/1443772281 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f63e002e780 con 0x7f63e0107610 2024-05-08T05:57:24.943 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:24.229+0000 7f63d7400700 1 -- 172.21.15.123:0/1443772281 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f63d001baa0 con 0x7f63e0107610 2024-05-08T05:57:24.943 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:24.231+0000 7f63dfe00700 1 -- 172.21.15.123:0/1443772281 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f63c403e4c0 msgr2=0x7f63c4040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T05:57:24.943 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:24.231+0000 7f63dfe00700 1 --2- 172.21.15.123:0/1443772281 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f63c403e4c0 0x7f63c4040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:24.943 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:24.231+0000 7f63dfe00700 1 -- 172.21.15.123:0/1443772281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63e0107610 msgr2=0x7f63e019ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:24.943 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:24.231+0000 7f63dfe00700 1 --2- 172.21.15.123:0/1443772281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63e0107610 0x7f63e019ee10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f63d0004660 tx=0x7f63d0011aa0 comp rx=0 tx=0).stop 2024-05-08T05:57:24.943 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:24.232+0000 7f63dfe00700 1 -- 172.21.15.123:0/1443772281 shutdown_connections 2024-05-08T05:57:24.943 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:24.232+0000 7f63dfe00700 1 --2- 172.21.15.123:0/1443772281 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f63c403e4c0 0x7f63c4040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:24.943 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:24.232+0000 7f63dfe00700 1 --2- 172.21.15.123:0/1443772281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63e0107610 0x7f63e019ee10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:24.944 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:24.232+0000 7f63dfe00700 1 -- 172.21.15.123:0/1443772281 >> 172.21.15.123:0/1443772281 conn(0x7f63e01031b0 msgr2=0x7f63e0195a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:24.944 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:24.232+0000 7f63dfe00700 1 -- 172.21.15.123:0/1443772281 shutdown_connections 2024-05-08T05:57:24.944 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:24.232+0000 7f63dfe00700 1 -- 172.21.15.123:0/1443772281 wait complete. 2024-05-08T05:57:24.944 INFO:teuthology.orchestra.run.smithi123.stdout:Waiting for the mgr to restart... 2024-05-08T05:57:24.944 INFO:teuthology.orchestra.run.smithi123.stdout:Waiting for mgr epoch 9... 2024-05-08T05:57:25.362 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:25 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:57:25.219+0000 7f78b8333200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T05:57:25.362 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:25 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:57:25.348+0000 7f78b8333200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T05:57:25.705 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:25 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:57:25.606+0000 7f78b8333200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T05:57:26.134 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:25 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:57:25.858+0000 7f78b8333200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:57:26.921 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:26 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:57:26.637+0000 7f78b8333200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T05:57:26.921 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:26 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:57:26.920+0000 7f78b8333200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-08T05:57:27.955 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:27 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:57:27.475+0000 7f78b8333200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T05:57:27.955 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:27 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:57:27.604+0000 7f78b8333200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T05:57:28.455 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:28 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:57:28.064+0000 7f78b8333200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T05:57:28.816 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:28 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:57:28.514+0000 7f78b8333200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T05:57:28.816 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:28 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:57:28.646+0000 7f78b8333200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T05:57:29.119 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:28 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:57:28.984+0000 7f78b8333200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T05:57:29.455 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:29 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:57:29.168+0000 7f78b8333200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T05:57:29.455 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:29 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:57:29.300+0000 7f78b8333200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T05:57:30.007 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:29 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:57:29.711+0000 7f78b8333200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T05:57:30.007 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:29 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:57:29.840+0000 7f78b8333200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T05:57:30.884 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:30 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:57:30.602+0000 7f78b8333200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T05:57:30.885 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 05:57:30 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T05:57:30.883+0000 7f78b8333200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T05:57:31.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:30 smithi123 ceph-mon[29787]: Active manager daemon x restarted 2024-05-08T05:57:31.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:30 smithi123 ceph-mon[29787]: Activating manager daemon x 2024-05-08T05:57:31.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:30 smithi123 ceph-mon[29787]: osdmap e3: 0 total, 0 up, 0 in 2024-05-08T05:57:31.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:30 smithi123 ceph-mon[29787]: mgrmap e10: x(active, starting, since 0.0115124s) 2024-05-08T05:57:31.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:30 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:57:31.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:30 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T05:57:31.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:30 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T05:57:31.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:30 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T05:57:31.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:30 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T05:57:31.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:30 smithi123 ceph-mon[29787]: Manager daemon x is now available 2024-05-08T05:57:32.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:31 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:57:32.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:31 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T05:57:32.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:31 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T05:57:32.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:31 smithi123 ceph-mon[29787]: [08/May/2024:05:57:31] ENGINE Bus STARTING 2024-05-08T05:57:32.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:31 smithi123 ceph-mon[29787]: [08/May/2024:05:57:31] ENGINE Serving on https://172.21.15.123:7150 2024-05-08T05:57:32.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:31 smithi123 ceph-mon[29787]: [08/May/2024:05:57:31] ENGINE Serving on http://172.21.15.123:8765 2024-05-08T05:57:32.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:31 smithi123 ceph-mon[29787]: [08/May/2024:05:57:31] ENGINE Bus STARTED 2024-05-08T05:57:32.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:31 smithi123 ceph-mon[29787]: mgrmap e11: x(active, since 1.00935s) 2024-05-08T05:57:33.094 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:32 smithi123 ceph-mon[29787]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-08T05:57:33.106 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:32 smithi123 ceph-mon[29787]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-08T05:57:33.114 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout { 2024-05-08T05:57:33.114 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-08T05:57:33.114 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-08T05:57:33.114 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout } 2024-05-08T05:57:33.114 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.524+0000 7f5150600700 1 Processor -- start 2024-05-08T05:57:33.114 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.524+0000 7f5150600700 1 -- start start 2024-05-08T05:57:33.114 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.525+0000 7f5150600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51481099f0 0x7f514810bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:33.115 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.525+0000 7f5150600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f514807ce10 con 0x7f51481099f0 2024-05-08T05:57:33.115 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.525+0000 7f514dc00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51481099f0 0x7f514810bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:33.115 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.525+0000 7f514dc00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51481099f0 0x7f514810bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53082/0 (socket says 172.21.15.123:53082) 2024-05-08T05:57:33.115 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.525+0000 7f514dc00700 1 -- 172.21.15.123:0/3875831422 learned_addr learned my addr 172.21.15.123:0/3875831422 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:57:33.115 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.525+0000 7f514dc00700 1 -- 172.21.15.123:0/3875831422 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f514810c320 con 0x7f51481099f0 2024-05-08T05:57:33.115 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.526+0000 7f514dc00700 1 --2- 172.21.15.123:0/3875831422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51481099f0 0x7f514810bde0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f513c0098a0 tx=0x7f513c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87bb980bbb0a2c98 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:33.115 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.526+0000 7f5147e00700 1 -- 172.21.15.123:0/3875831422 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f513c004160 con 0x7f51481099f0 2024-05-08T05:57:33.115 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.526+0000 7f5147e00700 1 -- 172.21.15.123:0/3875831422 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f513c004300 con 0x7f51481099f0 2024-05-08T05:57:33.115 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.526+0000 7f5147e00700 1 -- 172.21.15.123:0/3875831422 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f513c011a20 con 0x7f51481099f0 2024-05-08T05:57:33.115 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.527+0000 7f5150600700 1 -- 172.21.15.123:0/3875831422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51481099f0 msgr2=0x7f514810bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:33.115 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.527+0000 7f5150600700 1 --2- 172.21.15.123:0/3875831422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51481099f0 0x7f514810bde0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f513c0098a0 tx=0x7f513c009bb0 comp rx=0 tx=0).stop 2024-05-08T05:57:33.116 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.527+0000 7f5150600700 1 -- 172.21.15.123:0/3875831422 shutdown_connections 2024-05-08T05:57:33.116 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.527+0000 7f5150600700 1 --2- 172.21.15.123:0/3875831422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51481099f0 0x7f514810bde0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:33.116 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.527+0000 7f5150600700 1 -- 172.21.15.123:0/3875831422 >> 172.21.15.123:0/3875831422 conn(0x7f5148103380 msgr2=0x7f51481057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:33.116 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.527+0000 7f5150600700 1 -- 172.21.15.123:0/3875831422 shutdown_connections 2024-05-08T05:57:33.116 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.527+0000 7f5150600700 1 -- 172.21.15.123:0/3875831422 wait complete. 2024-05-08T05:57:33.116 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.528+0000 7f5150600700 1 Processor -- start 2024-05-08T05:57:33.117 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.528+0000 7f5150600700 1 -- start start 2024-05-08T05:57:33.117 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.528+0000 7f5150600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51481099f0 0x7f514819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:33.117 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.528+0000 7f5150600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f514810d270 con 0x7f51481099f0 2024-05-08T05:57:33.117 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.528+0000 7f514dc00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51481099f0 0x7f514819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:33.117 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.528+0000 7f514dc00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51481099f0 0x7f514819cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53096/0 (socket says 172.21.15.123:53096) 2024-05-08T05:57:33.117 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.528+0000 7f514dc00700 1 -- 172.21.15.123:0/224855585 learned_addr learned my addr 172.21.15.123:0/224855585 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:57:33.117 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.529+0000 7f514dc00700 1 -- 172.21.15.123:0/224855585 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f514819d240 con 0x7f51481099f0 2024-05-08T05:57:33.117 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.529+0000 7f514dc00700 1 --2- 172.21.15.123:0/224855585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51481099f0 0x7f514819cd00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f513c000c00 tx=0x7f513c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:33.117 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.529+0000 7f5146000700 1 -- 172.21.15.123:0/224855585 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f513c003aa0 con 0x7f51481099f0 2024-05-08T05:57:33.117 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.529+0000 7f5146000700 1 -- 172.21.15.123:0/224855585 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f513c01b8f0 con 0x7f51481099f0 2024-05-08T05:57:33.118 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.529+0000 7f5146000700 1 -- 172.21.15.123:0/224855585 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f513c023870 con 0x7f51481099f0 2024-05-08T05:57:33.118 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.529+0000 7f5150600700 1 -- 172.21.15.123:0/224855585 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f514819d4d0 con 0x7f51481099f0 2024-05-08T05:57:33.118 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.529+0000 7f5150600700 1 -- 172.21.15.123:0/224855585 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51481a01b0 con 0x7f51481099f0 2024-05-08T05:57:33.118 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.531+0000 7f5146000700 1 -- 172.21.15.123:0/224855585 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f513c023b70 con 0x7f51481099f0 2024-05-08T05:57:33.118 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.531+0000 7f5146000700 1 --2- 172.21.15.123:0/224855585 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f513003e4c0 0x7f5130040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:33.118 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.531+0000 7f5146000700 1 -- 172.21.15.123:0/224855585 --> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5130041090 con 0x7f513003e4c0 2024-05-08T05:57:33.118 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.531+0000 7f5146000700 1 -- 172.21.15.123:0/224855585 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f513c053e30 con 0x7f51481099f0 2024-05-08T05:57:33.118 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.531+0000 7f514d200700 1 -- 172.21.15.123:0/224855585 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f513003e4c0 msgr2=0x7f5130040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/3057688649 2024-05-08T05:57:33.118 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.531+0000 7f514d200700 1 --2- 172.21.15.123:0/224855585 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f513003e4c0 0x7f5130040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T05:57:33.119 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.732+0000 7f514d200700 1 -- 172.21.15.123:0/224855585 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f513003e4c0 msgr2=0x7f5130040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/3057688649 2024-05-08T05:57:33.119 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:25.732+0000 7f514d200700 1 --2- 172.21.15.123:0/224855585 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f513003e4c0 0x7f5130040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T05:57:33.119 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:26.133+0000 7f514d200700 1 -- 172.21.15.123:0/224855585 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f513003e4c0 msgr2=0x7f5130040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/3057688649 2024-05-08T05:57:33.119 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:26.133+0000 7f514d200700 1 --2- 172.21.15.123:0/224855585 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f513003e4c0 0x7f5130040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T05:57:33.119 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:26.934+0000 7f514d200700 1 -- 172.21.15.123:0/224855585 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f513003e4c0 msgr2=0x7f5130040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/3057688649 2024-05-08T05:57:33.119 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:26.934+0000 7f514d200700 1 --2- 172.21.15.123:0/224855585 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f513003e4c0 0x7f5130040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T05:57:33.119 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:28.536+0000 7f514d200700 1 -- 172.21.15.123:0/224855585 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f513003e4c0 msgr2=0x7f5130040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/3057688649 2024-05-08T05:57:33.119 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:28.536+0000 7f514d200700 1 --2- 172.21.15.123:0/224855585 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f513003e4c0 0x7f5130040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-08T05:57:33.119 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:30.895+0000 7f5146000700 1 -- 172.21.15.123:0/224855585 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7f513c02b530 con 0x7f51481099f0 2024-05-08T05:57:33.119 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:30.895+0000 7f5146000700 1 -- 172.21.15.123:0/224855585 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f513003e4c0 msgr2=0x7f5130040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T05:57:33.119 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:30.895+0000 7f5146000700 1 --2- 172.21.15.123:0/224855585 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f513003e4c0 0x7f5130040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:33.119 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:31.894+0000 7f5146000700 1 -- 172.21.15.123:0/224855585 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f513c02dd00 con 0x7f51481099f0 2024-05-08T05:57:33.120 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:31.894+0000 7f5146000700 1 --2- 172.21.15.123:0/224855585 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5130041f40 0x7f5130044330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:33.120 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:31.894+0000 7f5146000700 1 -- 172.21.15.123:0/224855585 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f513c008e50 con 0x7f5130041f40 2024-05-08T05:57:33.120 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:31.895+0000 7f514d200700 1 --2- 172.21.15.123:0/224855585 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5130041f40 0x7f5130044330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:33.120 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:31.896+0000 7f514d200700 1 --2- 172.21.15.123:0/224855585 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5130041f40 0x7f5130044330 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5134003a10 tx=0x7f51340092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:33.120 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:31.897+0000 7f5146000700 1 -- 172.21.15.123:0/224855585 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f513c008e50 con 0x7f5130041f40 2024-05-08T05:57:33.120 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:31.906+0000 7f5150600700 1 -- 172.21.15.123:0/224855585 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f514806a5e0 con 0x7f5130041f40 2024-05-08T05:57:33.120 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:31.907+0000 7f5146000700 1 -- 172.21.15.123:0/224855585 <== mgr.14152 v2:172.21.15.123:6800/4169246006 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f514806a5e0 con 0x7f5130041f40 2024-05-08T05:57:33.120 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:31.907+0000 7f5150600700 1 -- 172.21.15.123:0/224855585 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5130041f40 msgr2=0x7f5130044330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:33.120 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:31.907+0000 7f5150600700 1 --2- 172.21.15.123:0/224855585 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5130041f40 0x7f5130044330 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5134003a10 tx=0x7f51340092b0 comp rx=0 tx=0).stop 2024-05-08T05:57:33.120 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:31.907+0000 7f5150600700 1 -- 172.21.15.123:0/224855585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51481099f0 msgr2=0x7f514819cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:33.120 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:31.907+0000 7f5150600700 1 --2- 172.21.15.123:0/224855585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51481099f0 0x7f514819cd00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f513c000c00 tx=0x7f513c011aa0 comp rx=0 tx=0).stop 2024-05-08T05:57:33.120 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:31.908+0000 7f5150600700 1 -- 172.21.15.123:0/224855585 shutdown_connections 2024-05-08T05:57:33.121 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:31.908+0000 7f5150600700 1 --2- 172.21.15.123:0/224855585 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5130041f40 0x7f5130044330 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:33.121 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:31.908+0000 7f5150600700 1 --2- 172.21.15.123:0/224855585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51481099f0 0x7f514819cd00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:33.121 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:31.908+0000 7f5150600700 1 --2- 172.21.15.123:0/224855585 >> [v2:172.21.15.123:6800/3057688649,v1:172.21.15.123:6801/3057688649] conn(0x7f513003e4c0 0x7f5130040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:33.121 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:31.908+0000 7f5150600700 1 -- 172.21.15.123:0/224855585 >> 172.21.15.123:0/224855585 conn(0x7f5148103380 msgr2=0x7f5148103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:33.121 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:31.908+0000 7f5150600700 1 -- 172.21.15.123:0/224855585 shutdown_connections 2024-05-08T05:57:33.121 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:31.908+0000 7f5150600700 1 -- 172.21.15.123:0/224855585 wait complete. 2024-05-08T05:57:33.121 INFO:teuthology.orchestra.run.smithi123.stdout:mgr epoch 9 is available 2024-05-08T05:57:33.121 INFO:teuthology.orchestra.run.smithi123.stdout:Generating a dashboard self-signed certificate... 2024-05-08T05:57:34.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:33 smithi123 ceph-mon[29787]: mgrmap e12: x(active, since 2s) 2024-05-08T05:57:34.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:33 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:57:35.173 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-08T05:57:35.173 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.047+0000 7f48cb000700 1 Processor -- start 2024-05-08T05:57:35.173 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.047+0000 7f48cb000700 1 -- start start 2024-05-08T05:57:35.174 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.048+0000 7f48cb000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48c4108e70 0x7f48c4109290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:35.174 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.048+0000 7f48cb000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f48c41097d0 con 0x7f48c4108e70 2024-05-08T05:57:35.174 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.048+0000 7f48c3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48c4108e70 0x7f48c4109290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:35.174 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.049+0000 7f48c3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48c4108e70 0x7f48c4109290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53168/0 (socket says 172.21.15.123:53168) 2024-05-08T05:57:35.174 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.049+0000 7f48c3e00700 1 -- 172.21.15.123:0/2672350327 learned_addr learned my addr 172.21.15.123:0/2672350327 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:57:35.174 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.049+0000 7f48c3e00700 1 -- 172.21.15.123:0/2672350327 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c410a060 con 0x7f48c4108e70 2024-05-08T05:57:35.174 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.049+0000 7f48c3e00700 1 --2- 172.21.15.123:0/2672350327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48c4108e70 0x7f48c4109290 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f48b40098a0 tx=0x7f48b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e7f7c174d6ff19 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:35.174 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.050+0000 7f48c2a00700 1 -- 172.21.15.123:0/2672350327 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48b4004160 con 0x7f48c4108e70 2024-05-08T05:57:35.174 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.050+0000 7f48c2a00700 1 -- 172.21.15.123:0/2672350327 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48b4004300 con 0x7f48c4108e70 2024-05-08T05:57:35.174 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.050+0000 7f48c2a00700 1 -- 172.21.15.123:0/2672350327 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48b4011a20 con 0x7f48c4108e70 2024-05-08T05:57:35.174 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.051+0000 7f48cb000700 1 -- 172.21.15.123:0/2672350327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48c4108e70 msgr2=0x7f48c4109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:35.175 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.051+0000 7f48cb000700 1 --2- 172.21.15.123:0/2672350327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48c4108e70 0x7f48c4109290 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f48b40098a0 tx=0x7f48b4009bb0 comp rx=0 tx=0).stop 2024-05-08T05:57:35.175 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.051+0000 7f48cb000700 1 -- 172.21.15.123:0/2672350327 shutdown_connections 2024-05-08T05:57:35.175 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.051+0000 7f48cb000700 1 --2- 172.21.15.123:0/2672350327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48c4108e70 0x7f48c4109290 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:35.175 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.051+0000 7f48cb000700 1 -- 172.21.15.123:0/2672350327 >> 172.21.15.123:0/2672350327 conn(0x7f48c4104440 msgr2=0x7f48c41068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:35.175 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.051+0000 7f48cb000700 1 -- 172.21.15.123:0/2672350327 shutdown_connections 2024-05-08T05:57:35.175 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.051+0000 7f48cb000700 1 -- 172.21.15.123:0/2672350327 wait complete. 2024-05-08T05:57:35.175 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.052+0000 7f48cb000700 1 Processor -- start 2024-05-08T05:57:35.175 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.052+0000 7f48cb000700 1 -- start start 2024-05-08T05:57:35.175 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.053+0000 7f48cb000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48c4108e70 0x7f48c419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:35.175 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.053+0000 7f48cb000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f48c410d320 con 0x7f48c4108e70 2024-05-08T05:57:35.175 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.053+0000 7f48c3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48c4108e70 0x7f48c419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:35.175 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.053+0000 7f48c3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48c4108e70 0x7f48c419efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53182/0 (socket says 172.21.15.123:53182) 2024-05-08T05:57:35.175 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.053+0000 7f48c3e00700 1 -- 172.21.15.123:0/1206027138 learned_addr learned my addr 172.21.15.123:0/1206027138 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:57:35.176 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.053+0000 7f48c3e00700 1 -- 172.21.15.123:0/1206027138 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c419f4e0 con 0x7f48c4108e70 2024-05-08T05:57:35.176 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.054+0000 7f48c3e00700 1 --2- 172.21.15.123:0/1206027138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48c4108e70 0x7f48c419efa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f48b4000c00 tx=0x7f48b4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:35.176 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.054+0000 7f48c0c00700 1 -- 172.21.15.123:0/1206027138 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48b4003aa0 con 0x7f48c4108e70 2024-05-08T05:57:35.176 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.054+0000 7f48c0c00700 1 -- 172.21.15.123:0/1206027138 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48b401b8f0 con 0x7f48c4108e70 2024-05-08T05:57:35.176 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.054+0000 7f48c0c00700 1 -- 172.21.15.123:0/1206027138 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48b4023890 con 0x7f48c4108e70 2024-05-08T05:57:35.176 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.054+0000 7f48cb000700 1 -- 172.21.15.123:0/1206027138 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48c419f770 con 0x7f48c4108e70 2024-05-08T05:57:35.176 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.054+0000 7f48cb000700 1 -- 172.21.15.123:0/1206027138 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48c41a2460 con 0x7f48c4108e70 2024-05-08T05:57:35.176 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.055+0000 7f48cb000700 1 -- 172.21.15.123:0/1206027138 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48c4064b00 con 0x7f48c4108e70 2024-05-08T05:57:35.176 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.056+0000 7f48c0c00700 1 -- 172.21.15.123:0/1206027138 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f48b401b440 con 0x7f48c4108e70 2024-05-08T05:57:35.176 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.056+0000 7f48c0c00700 1 --2- 172.21.15.123:0/1206027138 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f48a803e520 0x7f48a80409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:35.176 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.056+0000 7f48c0c00700 1 -- 172.21.15.123:0/1206027138 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f48b4053f00 con 0x7f48c4108e70 2024-05-08T05:57:35.176 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.056+0000 7f48c3400700 1 --2- 172.21.15.123:0/1206027138 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f48a803e520 0x7f48a80409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:35.176 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.057+0000 7f48c3400700 1 --2- 172.21.15.123:0/1206027138 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f48a803e520 0x7f48a80409e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f48b0006fd0 tx=0x7f48b0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:35.176 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.062+0000 7f48c0c00700 1 -- 172.21.15.123:0/1206027138 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f48b401f030 con 0x7f48c4108e70 2024-05-08T05:57:35.177 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.300+0000 7f48cb000700 1 -- 172.21.15.123:0/1206027138 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f48c4105730 con 0x7f48a803e520 2024-05-08T05:57:35.177 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.471+0000 7f48c0c00700 1 -- 172.21.15.123:0/1206027138 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f48c4105730 con 0x7f48a803e520 2024-05-08T05:57:35.177 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.474+0000 7f48cb000700 1 -- 172.21.15.123:0/1206027138 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f48a803e520 msgr2=0x7f48a80409e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:35.177 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.474+0000 7f48cb000700 1 --2- 172.21.15.123:0/1206027138 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f48a803e520 0x7f48a80409e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f48b0006fd0 tx=0x7f48b0006e40 comp rx=0 tx=0).stop 2024-05-08T05:57:35.177 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.474+0000 7f48cb000700 1 -- 172.21.15.123:0/1206027138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48c4108e70 msgr2=0x7f48c419efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:35.177 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.474+0000 7f48cb000700 1 --2- 172.21.15.123:0/1206027138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48c4108e70 0x7f48c419efa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f48b4000c00 tx=0x7f48b4011aa0 comp rx=0 tx=0).stop 2024-05-08T05:57:35.177 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.474+0000 7f48cb000700 1 -- 172.21.15.123:0/1206027138 shutdown_connections 2024-05-08T05:57:35.177 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.474+0000 7f48cb000700 1 --2- 172.21.15.123:0/1206027138 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f48a803e520 0x7f48a80409e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:35.177 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.474+0000 7f48cb000700 1 --2- 172.21.15.123:0/1206027138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48c4108e70 0x7f48c419efa0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:35.178 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.474+0000 7f48cb000700 1 -- 172.21.15.123:0/1206027138 >> 172.21.15.123:0/1206027138 conn(0x7f48c4104440 msgr2=0x7f48c4105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:35.178 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.475+0000 7f48cb000700 1 -- 172.21.15.123:0/1206027138 shutdown_connections 2024-05-08T05:57:35.178 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:34.475+0000 7f48cb000700 1 -- 172.21.15.123:0/1206027138 wait complete. 2024-05-08T05:57:35.178 INFO:teuthology.orchestra.run.smithi123.stdout:Creating initial admin user... 2024-05-08T05:57:35.771 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:35 smithi123 ceph-mon[29787]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:57:35.771 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:35 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:57:35.771 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:35 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:57:36.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:36 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:57:36.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:36 smithi123 ceph-mon[29787]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:57:36.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:36 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:57:37.066 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$Gog0qscQPCqLWBq51dvRDuWsdpnltyyXIGjTmjrPnISDOq6xWKIW.", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715147856, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-08T05:57:37.066 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.846+0000 7f9911a00700 1 Processor -- start 2024-05-08T05:57:37.066 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.846+0000 7f9911a00700 1 -- start start 2024-05-08T05:57:37.066 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.846+0000 7f9911a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f990c108e70 0x7f990c109290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:37.066 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.846+0000 7f9911a00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f990c1097d0 con 0x7f990c108e70 2024-05-08T05:57:37.066 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.847+0000 7f990aa00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f990c108e70 0x7f990c109290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:37.066 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.847+0000 7f990aa00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f990c108e70 0x7f990c109290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37430/0 (socket says 172.21.15.123:37430) 2024-05-08T05:57:37.066 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.847+0000 7f990aa00700 1 -- 172.21.15.123:0/1643626821 learned_addr learned my addr 172.21.15.123:0/1643626821 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:57:37.067 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.848+0000 7f990aa00700 1 -- 172.21.15.123:0/1643626821 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f990c10a060 con 0x7f990c108e70 2024-05-08T05:57:37.067 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.848+0000 7f990aa00700 1 --2- 172.21.15.123:0/1643626821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f990c108e70 0x7f990c109290 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f98fc0098a0 tx=0x7f98fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c119dd240da6aef8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:37.067 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.848+0000 7f9909600700 1 -- 172.21.15.123:0/1643626821 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98fc004160 con 0x7f990c108e70 2024-05-08T05:57:37.067 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.848+0000 7f9909600700 1 -- 172.21.15.123:0/1643626821 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98fc004300 con 0x7f990c108e70 2024-05-08T05:57:37.067 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.848+0000 7f9909600700 1 -- 172.21.15.123:0/1643626821 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98fc011a20 con 0x7f990c108e70 2024-05-08T05:57:37.067 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.849+0000 7f9911a00700 1 -- 172.21.15.123:0/1643626821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f990c108e70 msgr2=0x7f990c109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:37.067 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.849+0000 7f9911a00700 1 --2- 172.21.15.123:0/1643626821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f990c108e70 0x7f990c109290 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f98fc0098a0 tx=0x7f98fc009bb0 comp rx=0 tx=0).stop 2024-05-08T05:57:37.067 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.849+0000 7f9911a00700 1 -- 172.21.15.123:0/1643626821 shutdown_connections 2024-05-08T05:57:37.067 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.849+0000 7f9911a00700 1 --2- 172.21.15.123:0/1643626821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f990c108e70 0x7f990c109290 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:37.067 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.849+0000 7f9911a00700 1 -- 172.21.15.123:0/1643626821 >> 172.21.15.123:0/1643626821 conn(0x7f990c104440 msgr2=0x7f990c1068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:37.067 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.849+0000 7f9911a00700 1 -- 172.21.15.123:0/1643626821 shutdown_connections 2024-05-08T05:57:37.067 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.849+0000 7f9911a00700 1 -- 172.21.15.123:0/1643626821 wait complete. 2024-05-08T05:57:37.067 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.851+0000 7f9911a00700 1 Processor -- start 2024-05-08T05:57:37.067 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.851+0000 7f9911a00700 1 -- start start 2024-05-08T05:57:37.068 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.851+0000 7f9911a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f990c108e70 0x7f990c07eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:37.068 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.851+0000 7f9911a00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f990c10d320 con 0x7f990c108e70 2024-05-08T05:57:37.068 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.851+0000 7f990aa00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f990c108e70 0x7f990c07eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:37.068 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.851+0000 7f990aa00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f990c108e70 0x7f990c07eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37436/0 (socket says 172.21.15.123:37436) 2024-05-08T05:57:37.068 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.851+0000 7f990aa00700 1 -- 172.21.15.123:0/1824620881 learned_addr learned my addr 172.21.15.123:0/1824620881 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:57:37.068 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.852+0000 7f990aa00700 1 -- 172.21.15.123:0/1824620881 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f990c07f0d0 con 0x7f990c108e70 2024-05-08T05:57:37.068 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.852+0000 7f990aa00700 1 --2- 172.21.15.123:0/1824620881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f990c108e70 0x7f990c07eb90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f98fc000c00 tx=0x7f98fc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:37.068 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.852+0000 7f9903400700 1 -- 172.21.15.123:0/1824620881 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98fc003aa0 con 0x7f990c108e70 2024-05-08T05:57:37.068 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.852+0000 7f9903400700 1 -- 172.21.15.123:0/1824620881 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98fc01b8f0 con 0x7f990c108e70 2024-05-08T05:57:37.068 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.853+0000 7f9911a00700 1 -- 172.21.15.123:0/1824620881 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f990c07d2c0 con 0x7f990c108e70 2024-05-08T05:57:37.068 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.853+0000 7f9903400700 1 -- 172.21.15.123:0/1824620881 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98fc023890 con 0x7f990c108e70 2024-05-08T05:57:37.068 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.853+0000 7f9911a00700 1 -- 172.21.15.123:0/1824620881 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f990c07d800 con 0x7f990c108e70 2024-05-08T05:57:37.068 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.854+0000 7f9903400700 1 -- 172.21.15.123:0/1824620881 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f98fc01b440 con 0x7f990c108e70 2024-05-08T05:57:37.068 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.854+0000 7f9911a00700 1 -- 172.21.15.123:0/1824620881 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f990c064b00 con 0x7f990c108e70 2024-05-08T05:57:37.069 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.855+0000 7f9903400700 1 --2- 172.21.15.123:0/1824620881 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f98f003e4c0 0x7f98f0040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:37.069 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.855+0000 7f9903400700 1 -- 172.21.15.123:0/1824620881 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f98fc053fe0 con 0x7f990c108e70 2024-05-08T05:57:37.069 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.855+0000 7f990a000700 1 --2- 172.21.15.123:0/1824620881 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f98f003e4c0 0x7f98f0040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:37.069 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.856+0000 7f990a000700 1 --2- 172.21.15.123:0/1824620881 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f98f003e4c0 0x7f98f0040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f98f8006fd0 tx=0x7f98f8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:37.069 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:35.861+0000 7f9903400700 1 -- 172.21.15.123:0/1824620881 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f98fc01f030 con 0x7f990c108e70 2024-05-08T05:57:37.069 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:36.114+0000 7f9911a00700 1 -- 172.21.15.123:0/1824620881 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f990c07da80 con 0x7f98f003e4c0 2024-05-08T05:57:37.069 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:36.355+0000 7f9903400700 1 -- 172.21.15.123:0/1824620881 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f990c07da80 con 0x7f98f003e4c0 2024-05-08T05:57:37.069 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:36.359+0000 7f9911a00700 1 -- 172.21.15.123:0/1824620881 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f98f003e4c0 msgr2=0x7f98f0040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:37.069 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:36.359+0000 7f9911a00700 1 --2- 172.21.15.123:0/1824620881 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f98f003e4c0 0x7f98f0040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f98f8006fd0 tx=0x7f98f8006e40 comp rx=0 tx=0).stop 2024-05-08T05:57:37.069 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:36.359+0000 7f9911a00700 1 -- 172.21.15.123:0/1824620881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f990c108e70 msgr2=0x7f990c07eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:37.069 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:36.359+0000 7f9911a00700 1 --2- 172.21.15.123:0/1824620881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f990c108e70 0x7f990c07eb90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f98fc000c00 tx=0x7f98fc011aa0 comp rx=0 tx=0).stop 2024-05-08T05:57:37.069 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:36.359+0000 7f9911a00700 1 -- 172.21.15.123:0/1824620881 shutdown_connections 2024-05-08T05:57:37.069 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:36.359+0000 7f9911a00700 1 --2- 172.21.15.123:0/1824620881 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f98f003e4c0 0x7f98f0040980 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:37.069 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:36.359+0000 7f9911a00700 1 --2- 172.21.15.123:0/1824620881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f990c108e70 0x7f990c07eb90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:37.070 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:36.359+0000 7f9911a00700 1 -- 172.21.15.123:0/1824620881 >> 172.21.15.123:0/1824620881 conn(0x7f990c104440 msgr2=0x7f990c104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:37.070 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:36.359+0000 7f9911a00700 1 -- 172.21.15.123:0/1824620881 shutdown_connections 2024-05-08T05:57:37.070 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:36.359+0000 7f9911a00700 1 -- 172.21.15.123:0/1824620881 wait complete. 2024-05-08T05:57:37.070 INFO:teuthology.orchestra.run.smithi123.stdout:Fetching dashboard port number... 2024-05-08T05:57:38.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:38 smithi123 ceph-mon[29787]: mgrmap e13: x(active, since 6s) 2024-05-08T05:57:38.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:38 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/3233028540' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-08T05:57:39.022 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stdout 8443 2024-05-08T05:57:39.022 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.940+0000 7f9641c00700 1 Processor -- start 2024-05-08T05:57:39.022 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.940+0000 7f9641c00700 1 -- start start 2024-05-08T05:57:39.022 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.940+0000 7f9641c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f963c10b220 0x7f963c10b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:39.023 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.940+0000 7f9641c00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f963c10bc10 con 0x7f963c10b220 2024-05-08T05:57:39.023 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.941+0000 7f963aa00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f963c10b220 0x7f963c10b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:39.025 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.941+0000 7f963aa00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f963c10b220 0x7f963c10b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37440/0 (socket says 172.21.15.123:37440) 2024-05-08T05:57:39.025 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.941+0000 7f963aa00700 1 -- 172.21.15.123:0/3980361532 learned_addr learned my addr 172.21.15.123:0/3980361532 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:57:39.025 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.941+0000 7f963aa00700 1 -- 172.21.15.123:0/3980361532 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f963c10c300 con 0x7f963c10b220 2024-05-08T05:57:39.025 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.942+0000 7f963aa00700 1 --2- 172.21.15.123:0/3980361532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f963c10b220 0x7f963c10b640 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f962c0098a0 tx=0x7f962c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d11855b164e600c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:39.025 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.942+0000 7f9639600700 1 -- 172.21.15.123:0/3980361532 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f962c004160 con 0x7f963c10b220 2024-05-08T05:57:39.025 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.942+0000 7f9639600700 1 -- 172.21.15.123:0/3980361532 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f962c004300 con 0x7f963c10b220 2024-05-08T05:57:39.026 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.942+0000 7f9639600700 1 -- 172.21.15.123:0/3980361532 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f962c011a20 con 0x7f963c10b220 2024-05-08T05:57:39.026 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.942+0000 7f9641c00700 1 -- 172.21.15.123:0/3980361532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f963c10b220 msgr2=0x7f963c10b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:39.026 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.942+0000 7f9641c00700 1 --2- 172.21.15.123:0/3980361532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f963c10b220 0x7f963c10b640 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f962c0098a0 tx=0x7f962c009bb0 comp rx=0 tx=0).stop 2024-05-08T05:57:39.026 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.943+0000 7f9641c00700 1 -- 172.21.15.123:0/3980361532 shutdown_connections 2024-05-08T05:57:39.026 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.943+0000 7f9641c00700 1 --2- 172.21.15.123:0/3980361532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f963c10b220 0x7f963c10b640 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:39.026 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.943+0000 7f9641c00700 1 -- 172.21.15.123:0/3980361532 >> 172.21.15.123:0/3980361532 conn(0x7f963c07e6e0 msgr2=0x7f963c07eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:39.026 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.943+0000 7f9641c00700 1 -- 172.21.15.123:0/3980361532 shutdown_connections 2024-05-08T05:57:39.026 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.943+0000 7f9641c00700 1 -- 172.21.15.123:0/3980361532 wait complete. 2024-05-08T05:57:39.026 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.944+0000 7f9641c00700 1 Processor -- start 2024-05-08T05:57:39.026 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.944+0000 7f9641c00700 1 -- start start 2024-05-08T05:57:39.026 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.944+0000 7f9641c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f963c10b220 0x7f963c19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:39.026 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.944+0000 7f9641c00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f963c10f6b0 con 0x7f963c10b220 2024-05-08T05:57:39.026 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.944+0000 7f963aa00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f963c10b220 0x7f963c19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:39.026 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.944+0000 7f963aa00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f963c10b220 0x7f963c19efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37446/0 (socket says 172.21.15.123:37446) 2024-05-08T05:57:39.026 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.944+0000 7f963aa00700 1 -- 172.21.15.123:0/3233028540 learned_addr learned my addr 172.21.15.123:0/3233028540 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:57:39.027 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.945+0000 7f963aa00700 1 -- 172.21.15.123:0/3233028540 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f963c19f520 con 0x7f963c10b220 2024-05-08T05:57:39.029 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.945+0000 7f963aa00700 1 --2- 172.21.15.123:0/3233028540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f963c10b220 0x7f963c19efe0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f962c0116e0 tx=0x7f962c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:39.029 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.945+0000 7f9633400700 1 -- 172.21.15.123:0/3233028540 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f962c003aa0 con 0x7f963c10b220 2024-05-08T05:57:39.029 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.945+0000 7f9633400700 1 -- 172.21.15.123:0/3233028540 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f962c01b8f0 con 0x7f963c10b220 2024-05-08T05:57:39.029 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.946+0000 7f9641c00700 1 -- 172.21.15.123:0/3233028540 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f963c19f7b0 con 0x7f963c10b220 2024-05-08T05:57:39.030 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.946+0000 7f9633400700 1 -- 172.21.15.123:0/3233028540 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f962c0237c0 con 0x7f963c10b220 2024-05-08T05:57:39.030 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.946+0000 7f9641c00700 1 -- 172.21.15.123:0/3233028540 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f963c1a24a0 con 0x7f963c10b220 2024-05-08T05:57:39.030 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.947+0000 7f9633400700 1 -- 172.21.15.123:0/3233028540 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f962c01b440 con 0x7f963c10b220 2024-05-08T05:57:39.030 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.948+0000 7f9641c00700 1 -- 172.21.15.123:0/3233028540 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f963c10f830 con 0x7f963c10b220 2024-05-08T05:57:39.030 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.948+0000 7f9633400700 1 --2- 172.21.15.123:0/3233028540 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f962003e570 0x7f9620040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:39.030 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.948+0000 7f9633400700 1 -- 172.21.15.123:0/3233028540 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f962c054100 con 0x7f963c10b220 2024-05-08T05:57:39.030 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.948+0000 7f963a000700 1 --2- 172.21.15.123:0/3233028540 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f962003e570 0x7f9620040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:39.030 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.949+0000 7f963a000700 1 --2- 172.21.15.123:0/3233028540 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f962003e570 0x7f9620040a30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9628006fd0 tx=0x7f9628006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:39.030 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:37.954+0000 7f9633400700 1 -- 172.21.15.123:0/3233028540 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f962c01f030 con 0x7f963c10b220 2024-05-08T05:57:39.030 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:38.188+0000 7f9641c00700 1 -- 172.21.15.123:0/3233028540 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f963c1a2670 con 0x7f963c10b220 2024-05-08T05:57:39.030 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:38.189+0000 7f9633400700 1 -- 172.21.15.123:0/3233028540 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f962c023d90 con 0x7f963c10b220 2024-05-08T05:57:39.030 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:38.192+0000 7f9641c00700 1 -- 172.21.15.123:0/3233028540 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f962003e570 msgr2=0x7f9620040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:39.030 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:38.192+0000 7f9641c00700 1 --2- 172.21.15.123:0/3233028540 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f962003e570 0x7f9620040a30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9628006fd0 tx=0x7f9628006e40 comp rx=0 tx=0).stop 2024-05-08T05:57:39.030 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:38.192+0000 7f9641c00700 1 -- 172.21.15.123:0/3233028540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f963c10b220 msgr2=0x7f963c19efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:39.031 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:38.192+0000 7f9641c00700 1 --2- 172.21.15.123:0/3233028540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f963c10b220 0x7f963c19efe0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f962c0116e0 tx=0x7f962c011aa0 comp rx=0 tx=0).stop 2024-05-08T05:57:39.031 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:38.193+0000 7f9641c00700 1 -- 172.21.15.123:0/3233028540 shutdown_connections 2024-05-08T05:57:39.031 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:38.193+0000 7f9641c00700 1 --2- 172.21.15.123:0/3233028540 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f962003e570 0x7f9620040a30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:39.031 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:38.193+0000 7f9641c00700 1 --2- 172.21.15.123:0/3233028540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f963c10b220 0x7f963c19efe0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:39.031 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:38.193+0000 7f9641c00700 1 -- 172.21.15.123:0/3233028540 >> 172.21.15.123:0/3233028540 conn(0x7f963c07e6e0 msgr2=0x7f963c108dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:39.031 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:38.193+0000 7f9641c00700 1 -- 172.21.15.123:0/3233028540 shutdown_connections 2024-05-08T05:57:39.031 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:38.193+0000 7f9641c00700 1 -- 172.21.15.123:0/3233028540 wait complete. 2024-05-08T05:57:39.059 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T05:57:39.059 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout disabled 2024-05-08T05:57:39.084 INFO:teuthology.orchestra.run.smithi123.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T05:57:39.085 INFO:teuthology.orchestra.run.smithi123.stdout:systemctl: stdout inactive 2024-05-08T05:57:39.085 INFO:teuthology.orchestra.run.smithi123.stdout:firewalld.service is not enabled 2024-05-08T05:57:39.085 INFO:teuthology.orchestra.run.smithi123.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-08T05:57:39.092 INFO:teuthology.orchestra.run.smithi123.stdout:Ceph Dashboard is now available at: 2024-05-08T05:57:39.092 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T05:57:39.092 INFO:teuthology.orchestra.run.smithi123.stdout: URL: https://smithi123.front.sepia.ceph.com:8443/ 2024-05-08T05:57:39.092 INFO:teuthology.orchestra.run.smithi123.stdout: User: admin 2024-05-08T05:57:39.093 INFO:teuthology.orchestra.run.smithi123.stdout: Password: n67arplpqb 2024-05-08T05:57:39.093 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T05:57:39.093 INFO:teuthology.orchestra.run.smithi123.stdout:Saving cluster configuration to /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/config directory 2024-05-08T05:57:39.096 INFO:teuthology.orchestra.run.smithi123.stdout:Enabling autotune for osd_memory_target 2024-05-08T05:57:41.077 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.102+0000 7f1242000700 1 Processor -- start 2024-05-08T05:57:41.077 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.102+0000 7f1242000700 1 -- start start 2024-05-08T05:57:41.077 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.103+0000 7f1242000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f123c10b120 0x7f123c10b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:41.077 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.103+0000 7f1242000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f123c10bb30 con 0x7f123c10b120 2024-05-08T05:57:41.077 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.103+0000 7f1240c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f123c10b120 0x7f123c10b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:41.077 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.103+0000 7f1240c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f123c10b120 0x7f123c10b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37450/0 (socket says 172.21.15.123:37450) 2024-05-08T05:57:41.077 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.103+0000 7f1240c00700 1 -- 172.21.15.123:0/3095281233 learned_addr learned my addr 172.21.15.123:0/3095281233 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:57:41.078 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.104+0000 7f1240c00700 1 -- 172.21.15.123:0/3095281233 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f123c10bcb0 con 0x7f123c10b120 2024-05-08T05:57:41.078 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.104+0000 7f1240c00700 1 --2- 172.21.15.123:0/3095281233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f123c10b120 0x7f123c10b560 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f12300098a0 tx=0x7f1230009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155fdc3c3b68f691 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:41.078 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.105+0000 7f123b400700 1 -- 172.21.15.123:0/3095281233 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1230004160 con 0x7f123c10b120 2024-05-08T05:57:41.078 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.105+0000 7f123b400700 1 -- 172.21.15.123:0/3095281233 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1230004300 con 0x7f123c10b120 2024-05-08T05:57:41.078 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.105+0000 7f123b400700 1 -- 172.21.15.123:0/3095281233 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1230011a20 con 0x7f123c10b120 2024-05-08T05:57:41.078 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.105+0000 7f1242000700 1 -- 172.21.15.123:0/3095281233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f123c10b120 msgr2=0x7f123c10b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:41.078 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.105+0000 7f1242000700 1 --2- 172.21.15.123:0/3095281233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f123c10b120 0x7f123c10b560 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f12300098a0 tx=0x7f1230009bb0 comp rx=0 tx=0).stop 2024-05-08T05:57:41.078 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.105+0000 7f1242000700 1 -- 172.21.15.123:0/3095281233 shutdown_connections 2024-05-08T05:57:41.079 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.105+0000 7f1242000700 1 --2- 172.21.15.123:0/3095281233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f123c10b120 0x7f123c10b560 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:41.079 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.105+0000 7f1242000700 1 -- 172.21.15.123:0/3095281233 >> 172.21.15.123:0/3095281233 conn(0x7f123c07e350 msgr2=0x7f123c07e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:41.079 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.106+0000 7f1242000700 1 -- 172.21.15.123:0/3095281233 shutdown_connections 2024-05-08T05:57:41.079 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.106+0000 7f1242000700 1 -- 172.21.15.123:0/3095281233 wait complete. 2024-05-08T05:57:41.079 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.107+0000 7f1242000700 1 Processor -- start 2024-05-08T05:57:41.079 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.107+0000 7f1242000700 1 -- start start 2024-05-08T05:57:41.079 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.107+0000 7f1242000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f123c10b120 0x7f123c19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:41.079 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.107+0000 7f1242000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f123c10f390 con 0x7f123c10b120 2024-05-08T05:57:41.079 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.108+0000 7f1240c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f123c10b120 0x7f123c19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:41.079 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.108+0000 7f1240c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f123c10b120 0x7f123c19ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37454/0 (socket says 172.21.15.123:37454) 2024-05-08T05:57:41.079 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.108+0000 7f1240c00700 1 -- 172.21.15.123:0/72039865 learned_addr learned my addr 172.21.15.123:0/72039865 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:57:41.079 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.108+0000 7f1240c00700 1 -- 172.21.15.123:0/72039865 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f123c19f3c0 con 0x7f123c10b120 2024-05-08T05:57:41.079 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.109+0000 7f1240c00700 1 --2- 172.21.15.123:0/72039865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f123c10b120 0x7f123c19ee80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f12300119f0 tx=0x7f1230011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:41.079 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.109+0000 7f1239600700 1 -- 172.21.15.123:0/72039865 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12300039e0 con 0x7f123c10b120 2024-05-08T05:57:41.080 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.109+0000 7f1239600700 1 -- 172.21.15.123:0/72039865 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f123001b8f0 con 0x7f123c10b120 2024-05-08T05:57:41.080 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.109+0000 7f1239600700 1 -- 172.21.15.123:0/72039865 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12300237a0 con 0x7f123c10b120 2024-05-08T05:57:41.080 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.109+0000 7f1242000700 1 -- 172.21.15.123:0/72039865 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f123c19f650 con 0x7f123c10b120 2024-05-08T05:57:41.080 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.109+0000 7f1242000700 1 -- 172.21.15.123:0/72039865 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f123c1a2340 con 0x7f123c10b120 2024-05-08T05:57:41.080 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.110+0000 7f1242000700 1 -- 172.21.15.123:0/72039865 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f123c06a5e0 con 0x7f123c10b120 2024-05-08T05:57:41.080 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.110+0000 7f1239600700 1 -- 172.21.15.123:0/72039865 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f123001b440 con 0x7f123c10b120 2024-05-08T05:57:41.080 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.111+0000 7f1239600700 1 --2- 172.21.15.123:0/72039865 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f122403e570 0x7f1224040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:41.080 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.111+0000 7f1239600700 1 -- 172.21.15.123:0/72039865 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1230054060 con 0x7f123c10b120 2024-05-08T05:57:41.080 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.111+0000 7f123be00700 1 --2- 172.21.15.123:0/72039865 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f122403e570 0x7f1224040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:41.080 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.112+0000 7f123be00700 1 --2- 172.21.15.123:0/72039865 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f122403e570 0x7f1224040a30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1228006fd0 tx=0x7f1228006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:41.080 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.117+0000 7f1239600700 1 -- 172.21.15.123:0/72039865 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f123001f030 con 0x7f123c10b120 2024-05-08T05:57:41.080 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.347+0000 7f1242000700 1 -- 172.21.15.123:0/72039865 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f123c1a2510 con 0x7f123c10b120 2024-05-08T05:57:41.080 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.348+0000 7f1239600700 1 -- 172.21.15.123:0/72039865 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f123c10bcb0 con 0x7f123c10b120 2024-05-08T05:57:41.080 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.351+0000 7f1242000700 1 -- 172.21.15.123:0/72039865 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f122403e570 msgr2=0x7f1224040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:41.081 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.351+0000 7f1242000700 1 --2- 172.21.15.123:0/72039865 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f122403e570 0x7f1224040a30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1228006fd0 tx=0x7f1228006e40 comp rx=0 tx=0).stop 2024-05-08T05:57:41.081 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.351+0000 7f1242000700 1 -- 172.21.15.123:0/72039865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f123c10b120 msgr2=0x7f123c19ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:41.081 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.351+0000 7f1242000700 1 --2- 172.21.15.123:0/72039865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f123c10b120 0x7f123c19ee80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f12300119f0 tx=0x7f1230011ad0 comp rx=0 tx=0).stop 2024-05-08T05:57:41.081 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.351+0000 7f1242000700 1 -- 172.21.15.123:0/72039865 shutdown_connections 2024-05-08T05:57:41.081 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.351+0000 7f1242000700 1 --2- 172.21.15.123:0/72039865 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f122403e570 0x7f1224040a30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:41.081 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.351+0000 7f1242000700 1 --2- 172.21.15.123:0/72039865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f123c10b120 0x7f123c19ee80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:41.081 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.351+0000 7f1242000700 1 -- 172.21.15.123:0/72039865 >> 172.21.15.123:0/72039865 conn(0x7f123c07e350 msgr2=0x7f123c108db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:41.081 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.351+0000 7f1242000700 1 -- 172.21.15.123:0/72039865 shutdown_connections 2024-05-08T05:57:41.081 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:40.351+0000 7f1242000700 1 -- 172.21.15.123:0/72039865 wait complete. 2024-05-08T05:57:42.693 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.671+0000 7f9cdc600700 1 Processor -- start 2024-05-08T05:57:42.693 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.671+0000 7f9cdc600700 1 -- start start 2024-05-08T05:57:42.693 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.672+0000 7f9cdc600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cd407ee80 0x7f9cd407d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:42.694 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.672+0000 7f9cdc600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd407d8b0 con 0x7f9cd407ee80 2024-05-08T05:57:42.694 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.672+0000 7f9cd9c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cd407ee80 0x7f9cd407d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:42.694 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.672+0000 7f9cd9c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cd407ee80 0x7f9cd407d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37468/0 (socket says 172.21.15.123:37468) 2024-05-08T05:57:42.694 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.672+0000 7f9cd9c00700 1 -- 172.21.15.123:0/1503606070 learned_addr learned my addr 172.21.15.123:0/1503606070 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:57:42.694 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.673+0000 7f9cd9c00700 1 -- 172.21.15.123:0/1503606070 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd407da30 con 0x7f9cd407ee80 2024-05-08T05:57:42.694 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.673+0000 7f9cd9c00700 1 --2- 172.21.15.123:0/1503606070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cd407ee80 0x7f9cd407d2e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9cc80098a0 tx=0x7f9cc8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfb85f87c4310729 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:42.694 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.673+0000 7f9cd3e00700 1 -- 172.21.15.123:0/1503606070 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9cc8004160 con 0x7f9cd407ee80 2024-05-08T05:57:42.694 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.673+0000 7f9cd3e00700 1 -- 172.21.15.123:0/1503606070 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cc8004300 con 0x7f9cd407ee80 2024-05-08T05:57:42.694 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.673+0000 7f9cd3e00700 1 -- 172.21.15.123:0/1503606070 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9cc8011a20 con 0x7f9cd407ee80 2024-05-08T05:57:42.694 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.674+0000 7f9cdc600700 1 -- 172.21.15.123:0/1503606070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cd407ee80 msgr2=0x7f9cd407d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:42.694 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.674+0000 7f9cdc600700 1 --2- 172.21.15.123:0/1503606070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cd407ee80 0x7f9cd407d2e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9cc80098a0 tx=0x7f9cc8009bb0 comp rx=0 tx=0).stop 2024-05-08T05:57:42.694 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.674+0000 7f9cdc600700 1 -- 172.21.15.123:0/1503606070 shutdown_connections 2024-05-08T05:57:42.694 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.674+0000 7f9cdc600700 1 --2- 172.21.15.123:0/1503606070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cd407ee80 0x7f9cd407d2e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:42.695 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.674+0000 7f9cdc600700 1 -- 172.21.15.123:0/1503606070 >> 172.21.15.123:0/1503606070 conn(0x7f9cd4104440 msgr2=0x7f9cd41068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:42.695 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.674+0000 7f9cdc600700 1 -- 172.21.15.123:0/1503606070 shutdown_connections 2024-05-08T05:57:42.695 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.674+0000 7f9cdc600700 1 -- 172.21.15.123:0/1503606070 wait complete. 2024-05-08T05:57:42.695 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.675+0000 7f9cdc600700 1 Processor -- start 2024-05-08T05:57:42.695 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.675+0000 7f9cdc600700 1 -- start start 2024-05-08T05:57:42.695 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.676+0000 7f9cdc600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cd407ee80 0x7f9cd419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:42.695 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.676+0000 7f9cdc600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd410f4a0 con 0x7f9cd407ee80 2024-05-08T05:57:42.695 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.676+0000 7f9cd9c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cd407ee80 0x7f9cd419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:42.695 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.676+0000 7f9cd9c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cd407ee80 0x7f9cd419ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37480/0 (socket says 172.21.15.123:37480) 2024-05-08T05:57:42.695 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.676+0000 7f9cd9c00700 1 -- 172.21.15.123:0/1476438879 learned_addr learned my addr 172.21.15.123:0/1476438879 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:57:42.695 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.676+0000 7f9cd9c00700 1 -- 172.21.15.123:0/1476438879 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd419f4d0 con 0x7f9cd407ee80 2024-05-08T05:57:42.695 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.677+0000 7f9cd9c00700 1 --2- 172.21.15.123:0/1476438879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cd407ee80 0x7f9cd419ef90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8004660 tx=0x7f9cc8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:42.695 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.677+0000 7f9cd2000700 1 -- 172.21.15.123:0/1476438879 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9cc8003aa0 con 0x7f9cd407ee80 2024-05-08T05:57:42.695 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.677+0000 7f9cd2000700 1 -- 172.21.15.123:0/1476438879 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cc801b8f0 con 0x7f9cd407ee80 2024-05-08T05:57:42.695 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.677+0000 7f9cdc600700 1 -- 172.21.15.123:0/1476438879 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cd419f760 con 0x7f9cd407ee80 2024-05-08T05:57:42.695 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.677+0000 7f9cd2000700 1 -- 172.21.15.123:0/1476438879 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9cc8023900 con 0x7f9cd407ee80 2024-05-08T05:57:42.696 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.677+0000 7f9cdc600700 1 -- 172.21.15.123:0/1476438879 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cd41a2450 con 0x7f9cd407ee80 2024-05-08T05:57:42.696 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.678+0000 7f9cdc600700 1 -- 172.21.15.123:0/1476438879 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cd4064b00 con 0x7f9cd407ee80 2024-05-08T05:57:42.696 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.678+0000 7f9cd2000700 1 -- 172.21.15.123:0/1476438879 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f9cc801baa0 con 0x7f9cd407ee80 2024-05-08T05:57:42.696 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.679+0000 7f9cd2000700 1 --2- 172.21.15.123:0/1476438879 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9cbc03e5c0 0x7f9cbc040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:42.696 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.679+0000 7f9cd2000700 1 -- 172.21.15.123:0/1476438879 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9cc8054020 con 0x7f9cd407ee80 2024-05-08T05:57:42.696 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.679+0000 7f9cd9200700 1 --2- 172.21.15.123:0/1476438879 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9cbc03e5c0 0x7f9cbc040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:42.696 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.680+0000 7f9cd9200700 1 --2- 172.21.15.123:0/1476438879 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9cbc03e5c0 0x7f9cbc040a80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0006fd0 tx=0x7f9cc0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:42.696 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:41.684+0000 7f9cd2000700 1 -- 172.21.15.123:0/1476438879 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9cc801f030 con 0x7f9cd407ee80 2024-05-08T05:57:42.696 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:42.034+0000 7f9cdc600700 1 -- 172.21.15.123:0/1476438879 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f9cd419fc40 con 0x7f9cd407ee80 2024-05-08T05:57:42.696 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:42.040+0000 7f9cd2000700 1 -- 172.21.15.123:0/1476438879 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f9cd419fc40 con 0x7f9cd407ee80 2024-05-08T05:57:42.696 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:42.042+0000 7f9cdc600700 1 -- 172.21.15.123:0/1476438879 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9cbc03e5c0 msgr2=0x7f9cbc040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:42.696 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:42.042+0000 7f9cdc600700 1 --2- 172.21.15.123:0/1476438879 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9cbc03e5c0 0x7f9cbc040a80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0006fd0 tx=0x7f9cc0006e40 comp rx=0 tx=0).stop 2024-05-08T05:57:42.696 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:42.042+0000 7f9cdc600700 1 -- 172.21.15.123:0/1476438879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cd407ee80 msgr2=0x7f9cd419ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:42.696 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:42.042+0000 7f9cdc600700 1 --2- 172.21.15.123:0/1476438879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cd407ee80 0x7f9cd419ef90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8004660 tx=0x7f9cc8011aa0 comp rx=0 tx=0).stop 2024-05-08T05:57:42.696 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:42.043+0000 7f9cdc600700 1 -- 172.21.15.123:0/1476438879 shutdown_connections 2024-05-08T05:57:42.697 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:42.043+0000 7f9cdc600700 1 --2- 172.21.15.123:0/1476438879 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9cbc03e5c0 0x7f9cbc040a80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:42.697 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:42.043+0000 7f9cdc600700 1 --2- 172.21.15.123:0/1476438879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9cd407ee80 0x7f9cd419ef90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:42.697 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:42.043+0000 7f9cdc600700 1 -- 172.21.15.123:0/1476438879 >> 172.21.15.123:0/1476438879 conn(0x7f9cd4104440 msgr2=0x7f9cd4104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:42.697 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:42.043+0000 7f9cdc600700 1 -- 172.21.15.123:0/1476438879 shutdown_connections 2024-05-08T05:57:42.697 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr 2024-05-08T05:57:42.043+0000 7f9cdc600700 1 -- 172.21.15.123:0/1476438879 wait complete. 2024-05-08T05:57:42.697 INFO:teuthology.orchestra.run.smithi123.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-08T05:57:42.697 INFO:teuthology.orchestra.run.smithi123.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-08T05:57:42.697 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T05:57:42.697 INFO:teuthology.orchestra.run.smithi123.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-08T05:57:42.697 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T05:57:42.697 INFO:teuthology.orchestra.run.smithi123.stdout:Or, if you are only running a single cluster on this host: 2024-05-08T05:57:42.697 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T05:57:42.697 INFO:teuthology.orchestra.run.smithi123.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-08T05:57:42.697 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T05:57:42.697 INFO:teuthology.orchestra.run.smithi123.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-08T05:57:42.697 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T05:57:42.698 INFO:teuthology.orchestra.run.smithi123.stdout: ceph telemetry on 2024-05-08T05:57:42.698 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T05:57:42.698 INFO:teuthology.orchestra.run.smithi123.stdout:For more information see: 2024-05-08T05:57:42.698 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T05:57:42.698 INFO:teuthology.orchestra.run.smithi123.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-08T05:57:42.698 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T05:57:42.698 INFO:teuthology.orchestra.run.smithi123.stdout:Bootstrap complete. 2024-05-08T05:57:42.783 INFO:tasks.cephadm:Fetching config... 2024-05-08T05:57:42.783 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-05-08T05:57:42.783 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-08T05:57:42.833 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-08T05:57:42.833 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-05-08T05:57:42.833 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-08T05:57:42.913 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-08T05:57:42.913 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-05-08T05:57:42.913 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-08T05:57:43.030 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-08T05:57:43.031 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-05-08T05:57:43.031 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-08T05:57:43.078 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-08T05:57:43.079 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCykLmj8Zu9o7AUQyt7sQ18YoTeooUkGU/LifWKlxlY36ROfvKzfhzppZ/GWMnMkB54BnuD0VuUZgUXwIFb3oTtRbr5BTxH1gDa6VI+rbD+n7lGO9ujf0QNU8sil0SKEucUdFASihyw8xbO9l/wpmBatDIPzIKtGp9R6Oh6pfBVZnL0r0/ZUn0IUe0j7Q0fXIaMKXNOAKS80V/Xr3dj9Hfj/j268iD2yK8BXVqPmbhU9drY7uz3C22+3upFmx3MfClBZcm3CDVYpRit3N2yD/jBMExbm6QqaOm55uE1DM1zx6RmuHWrs6ikO6hd6VDQVdBwimBdb5TGP+420KTZJkI3t/bd23RloeNWvx5uvCZ/li4TIZ2T3ObjNBhIRrcNnixLIXmNgZT2F5JDKjICvhAOvkdY/vM8yr3TL4a5Z/FEbjsbnxIiIGQiRDe+btrzDbOsqR35e36hYW1dRTJe5Bf1kQjcP24mplck1P2j68FBIlUG1af44F2i94YjkJSolM0= ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-08T05:57:43.226 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:43 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/1476438879' entity='client.admin' 2024-05-08T05:57:43.232 INFO:teuthology.orchestra.run.smithi123.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCykLmj8Zu9o7AUQyt7sQ18YoTeooUkGU/LifWKlxlY36ROfvKzfhzppZ/GWMnMkB54BnuD0VuUZgUXwIFb3oTtRbr5BTxH1gDa6VI+rbD+n7lGO9ujf0QNU8sil0SKEucUdFASihyw8xbO9l/wpmBatDIPzIKtGp9R6Oh6pfBVZnL0r0/ZUn0IUe0j7Q0fXIaMKXNOAKS80V/Xr3dj9Hfj/j268iD2yK8BXVqPmbhU9drY7uz3C22+3upFmx3MfClBZcm3CDVYpRit3N2yD/jBMExbm6QqaOm55uE1DM1zx6RmuHWrs6ikO6hd6VDQVdBwimBdb5TGP+420KTZJkI3t/bd23RloeNWvx5uvCZ/li4TIZ2T3ObjNBhIRrcNnixLIXmNgZT2F5JDKjICvhAOvkdY/vM8yr3TL4a5Z/FEbjsbnxIiIGQiRDe+btrzDbOsqR35e36hYW1dRTJe5Bf1kQjcP24mplck1P2j68FBIlUG1af44F2i94YjkJSolM0= ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968 2024-05-08T05:57:43.265 DEBUG:teuthology.orchestra.run.smithi163:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCykLmj8Zu9o7AUQyt7sQ18YoTeooUkGU/LifWKlxlY36ROfvKzfhzppZ/GWMnMkB54BnuD0VuUZgUXwIFb3oTtRbr5BTxH1gDa6VI+rbD+n7lGO9ujf0QNU8sil0SKEucUdFASihyw8xbO9l/wpmBatDIPzIKtGp9R6Oh6pfBVZnL0r0/ZUn0IUe0j7Q0fXIaMKXNOAKS80V/Xr3dj9Hfj/j268iD2yK8BXVqPmbhU9drY7uz3C22+3upFmx3MfClBZcm3CDVYpRit3N2yD/jBMExbm6QqaOm55uE1DM1zx6RmuHWrs6ikO6hd6VDQVdBwimBdb5TGP+420KTZJkI3t/bd23RloeNWvx5uvCZ/li4TIZ2T3ObjNBhIRrcNnixLIXmNgZT2F5JDKjICvhAOvkdY/vM8yr3TL4a5Z/FEbjsbnxIiIGQiRDe+btrzDbOsqR35e36hYW1dRTJe5Bf1kQjcP24mplck1P2j68FBIlUG1af44F2i94YjkJSolM0= ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-08T05:57:43.304 INFO:teuthology.orchestra.run.smithi163.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCykLmj8Zu9o7AUQyt7sQ18YoTeooUkGU/LifWKlxlY36ROfvKzfhzppZ/GWMnMkB54BnuD0VuUZgUXwIFb3oTtRbr5BTxH1gDa6VI+rbD+n7lGO9ujf0QNU8sil0SKEucUdFASihyw8xbO9l/wpmBatDIPzIKtGp9R6Oh6pfBVZnL0r0/ZUn0IUe0j7Q0fXIaMKXNOAKS80V/Xr3dj9Hfj/j268iD2yK8BXVqPmbhU9drY7uz3C22+3upFmx3MfClBZcm3CDVYpRit3N2yD/jBMExbm6QqaOm55uE1DM1zx6RmuHWrs6ikO6hd6VDQVdBwimBdb5TGP+420KTZJkI3t/bd23RloeNWvx5uvCZ/li4TIZ2T3ObjNBhIRrcNnixLIXmNgZT2F5JDKjICvhAOvkdY/vM8yr3TL4a5Z/FEbjsbnxIiIGQiRDe+btrzDbOsqR35e36hYW1dRTJe5Bf1kQjcP24mplck1P2j68FBIlUG1af44F2i94YjkJSolM0= ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968 2024-05-08T05:57:43.316 DEBUG:teuthology.orchestra.run.smithi192:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCykLmj8Zu9o7AUQyt7sQ18YoTeooUkGU/LifWKlxlY36ROfvKzfhzppZ/GWMnMkB54BnuD0VuUZgUXwIFb3oTtRbr5BTxH1gDa6VI+rbD+n7lGO9ujf0QNU8sil0SKEucUdFASihyw8xbO9l/wpmBatDIPzIKtGp9R6Oh6pfBVZnL0r0/ZUn0IUe0j7Q0fXIaMKXNOAKS80V/Xr3dj9Hfj/j268iD2yK8BXVqPmbhU9drY7uz3C22+3upFmx3MfClBZcm3CDVYpRit3N2yD/jBMExbm6QqaOm55uE1DM1zx6RmuHWrs6ikO6hd6VDQVdBwimBdb5TGP+420KTZJkI3t/bd23RloeNWvx5uvCZ/li4TIZ2T3ObjNBhIRrcNnixLIXmNgZT2F5JDKjICvhAOvkdY/vM8yr3TL4a5Z/FEbjsbnxIiIGQiRDe+btrzDbOsqR35e36hYW1dRTJe5Bf1kQjcP24mplck1P2j68FBIlUG1af44F2i94YjkJSolM0= ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-08T05:57:43.357 INFO:teuthology.orchestra.run.smithi192.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCykLmj8Zu9o7AUQyt7sQ18YoTeooUkGU/LifWKlxlY36ROfvKzfhzppZ/GWMnMkB54BnuD0VuUZgUXwIFb3oTtRbr5BTxH1gDa6VI+rbD+n7lGO9ujf0QNU8sil0SKEucUdFASihyw8xbO9l/wpmBatDIPzIKtGp9R6Oh6pfBVZnL0r0/ZUn0IUe0j7Q0fXIaMKXNOAKS80V/Xr3dj9Hfj/j268iD2yK8BXVqPmbhU9drY7uz3C22+3upFmx3MfClBZcm3CDVYpRit3N2yD/jBMExbm6QqaOm55uE1DM1zx6RmuHWrs6ikO6hd6VDQVdBwimBdb5TGP+420KTZJkI3t/bd23RloeNWvx5uvCZ/li4TIZ2T3ObjNBhIRrcNnixLIXmNgZT2F5JDKjICvhAOvkdY/vM8yr3TL4a5Z/FEbjsbnxIiIGQiRDe+btrzDbOsqR35e36hYW1dRTJe5Bf1kQjcP24mplck1P2j68FBIlUG1af44F2i94YjkJSolM0= ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968 2024-05-08T05:57:43.369 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-08T05:57:43.804 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:57:46.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.107+0000 7f90a4e00700 1 -- 172.21.15.123:0/3442122185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90a01096b0 msgr2=0x7f90a0109ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:46.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.107+0000 7f90a4e00700 1 --2- 172.21.15.123:0/3442122185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90a01096b0 0x7f90a0109ad0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9090009910 tx=0x7f9090009c20 comp rx=0 tx=0).stop 2024-05-08T05:57:46.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.107+0000 7f90a4e00700 1 -- 172.21.15.123:0/3442122185 shutdown_connections 2024-05-08T05:57:46.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.107+0000 7f90a4e00700 1 --2- 172.21.15.123:0/3442122185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90a01096b0 0x7f90a0109ad0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:46.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.107+0000 7f90a4e00700 1 -- 172.21.15.123:0/3442122185 >> 172.21.15.123:0/3442122185 conn(0x7f90a0078790 msgr2=0x7f90a007abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:46.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.108+0000 7f90a4e00700 1 -- 172.21.15.123:0/3442122185 shutdown_connections 2024-05-08T05:57:46.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.108+0000 7f90a4e00700 1 -- 172.21.15.123:0/3442122185 wait complete. 2024-05-08T05:57:46.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.109+0000 7f90a4e00700 1 Processor -- start 2024-05-08T05:57:46.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.109+0000 7f90a4e00700 1 -- start start 2024-05-08T05:57:46.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.109+0000 7f90a4e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90a01096b0 0x7f90a019eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:46.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.109+0000 7f90a4e00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f90a010f3d0 con 0x7f90a01096b0 2024-05-08T05:57:46.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.109+0000 7f909de00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90a01096b0 0x7f90a019eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:46.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.109+0000 7f909de00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90a01096b0 0x7f90a019eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55984/0 (socket says 172.21.15.123:55984) 2024-05-08T05:57:46.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.109+0000 7f909de00700 1 -- 172.21.15.123:0/1776787383 learned_addr learned my addr 172.21.15.123:0/1776787383 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:57:46.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.110+0000 7f909de00700 1 -- 172.21.15.123:0/1776787383 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a019f070 con 0x7f90a01096b0 2024-05-08T05:57:46.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.110+0000 7f909de00700 1 --2- 172.21.15.123:0/1776787383 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90a01096b0 0x7f90a019eb30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9090005190 tx=0x7f9090009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:46.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.110+0000 7f9096a00700 1 -- 172.21.15.123:0/1776787383 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9090027070 con 0x7f90a01096b0 2024-05-08T05:57:46.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.110+0000 7f9096a00700 1 -- 172.21.15.123:0/1776787383 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9090022470 con 0x7f90a01096b0 2024-05-08T05:57:46.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.110+0000 7f90a4e00700 1 -- 172.21.15.123:0/1776787383 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90a019f300 con 0x7f90a01096b0 2024-05-08T05:57:46.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.110+0000 7f9096a00700 1 -- 172.21.15.123:0/1776787383 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f909000f5f0 con 0x7f90a01096b0 2024-05-08T05:57:46.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.110+0000 7f90a4e00700 1 -- 172.21.15.123:0/1776787383 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90a01a1ff0 con 0x7f90a01096b0 2024-05-08T05:57:46.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.111+0000 7f90a4e00700 1 -- 172.21.15.123:0/1776787383 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90a010fb50 con 0x7f90a01096b0 2024-05-08T05:57:46.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.112+0000 7f9096a00700 1 -- 172.21.15.123:0/1776787383 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f909002a070 con 0x7f90a01096b0 2024-05-08T05:57:46.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.112+0000 7f9096a00700 1 --2- 172.21.15.123:0/1776787383 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f908403e570 0x7f9084040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:46.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.112+0000 7f9096a00700 1 -- 172.21.15.123:0/1776787383 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f909001e030 con 0x7f90a01096b0 2024-05-08T05:57:46.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.112+0000 7f909d400700 1 --2- 172.21.15.123:0/1776787383 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f908403e570 0x7f9084040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:46.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.113+0000 7f909d400700 1 --2- 172.21.15.123:0/1776787383 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f908403e570 0x7f9084040a30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f908c006fd0 tx=0x7f908c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:46.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.117+0000 7f9096a00700 1 -- 172.21.15.123:0/1776787383 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9090013070 con 0x7f90a01096b0 2024-05-08T05:57:46.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.364+0000 7f90a4e00700 1 -- 172.21.15.123:0/1776787383 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f90a002e620 con 0x7f90a01096b0 2024-05-08T05:57:46.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.369+0000 7f9096a00700 1 -- 172.21.15.123:0/1776787383 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f90900310c0 con 0x7f90a01096b0 2024-05-08T05:57:46.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.373+0000 7f90a4e00700 1 -- 172.21.15.123:0/1776787383 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f908403e570 msgr2=0x7f9084040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:46.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.373+0000 7f90a4e00700 1 --2- 172.21.15.123:0/1776787383 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f908403e570 0x7f9084040a30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f908c006fd0 tx=0x7f908c006e40 comp rx=0 tx=0).stop 2024-05-08T05:57:46.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.373+0000 7f90a4e00700 1 -- 172.21.15.123:0/1776787383 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90a01096b0 msgr2=0x7f90a019eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:46.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.373+0000 7f90a4e00700 1 --2- 172.21.15.123:0/1776787383 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90a01096b0 0x7f90a019eb30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9090005190 tx=0x7f9090009ea0 comp rx=0 tx=0).stop 2024-05-08T05:57:46.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.373+0000 7f90a4e00700 1 -- 172.21.15.123:0/1776787383 shutdown_connections 2024-05-08T05:57:46.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.373+0000 7f90a4e00700 1 --2- 172.21.15.123:0/1776787383 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f908403e570 0x7f9084040a30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:46.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.373+0000 7f90a4e00700 1 --2- 172.21.15.123:0/1776787383 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90a01096b0 0x7f90a019eb30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:46.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.373+0000 7f90a4e00700 1 -- 172.21.15.123:0/1776787383 >> 172.21.15.123:0/1776787383 conn(0x7f90a0078790 msgr2=0x7f90a00792c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:46.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.373+0000 7f90a4e00700 1 -- 172.21.15.123:0/1776787383 shutdown_connections 2024-05-08T05:57:46.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:46.374+0000 7f90a4e00700 1 -- 172.21.15.123:0/1776787383 wait complete. 2024-05-08T05:57:47.399 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-08T05:57:47.400 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-08T05:57:47.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:47 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/1776787383' entity='client.admin' 2024-05-08T05:57:47.825 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:57:50.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.503+0000 7f6035c00700 1 -- 172.21.15.123:0/1598634850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6030105120 msgr2=0x7f6030105540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:50.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.503+0000 7f6035c00700 1 --2- 172.21.15.123:0/1598634850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6030105120 0x7f6030105540 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6020009910 tx=0x7f6020009c20 comp rx=0 tx=0).stop 2024-05-08T05:57:50.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.503+0000 7f6035c00700 1 -- 172.21.15.123:0/1598634850 shutdown_connections 2024-05-08T05:57:50.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.503+0000 7f6035c00700 1 --2- 172.21.15.123:0/1598634850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6030105120 0x7f6030105540 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:50.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.503+0000 7f6035c00700 1 -- 172.21.15.123:0/1598634850 >> 172.21.15.123:0/1598634850 conn(0x7f60301006c0 msgr2=0x7f6030102b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:50.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.503+0000 7f6035c00700 1 -- 172.21.15.123:0/1598634850 shutdown_connections 2024-05-08T05:57:50.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.503+0000 7f6035c00700 1 -- 172.21.15.123:0/1598634850 wait complete. 2024-05-08T05:57:50.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.505+0000 7f6035c00700 1 Processor -- start 2024-05-08T05:57:50.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.505+0000 7f6035c00700 1 -- start start 2024-05-08T05:57:50.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.505+0000 7f6035c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6030105120 0x7f603019a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:50.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.505+0000 7f6035c00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f603010b060 con 0x7f6030105120 2024-05-08T05:57:50.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.505+0000 7f602ea00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6030105120 0x7f603019a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:50.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.505+0000 7f602ea00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6030105120 0x7f603019a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56006/0 (socket says 172.21.15.123:56006) 2024-05-08T05:57:50.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.505+0000 7f602ea00700 1 -- 172.21.15.123:0/2849126631 learned_addr learned my addr 172.21.15.123:0/2849126631 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:57:50.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.506+0000 7f602ea00700 1 -- 172.21.15.123:0/2849126631 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603019ad00 con 0x7f6030105120 2024-05-08T05:57:50.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.506+0000 7f602ea00700 1 --2- 172.21.15.123:0/2849126631 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6030105120 0x7f603019a7c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6020004fb0 tx=0x7f6020009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:50.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.506+0000 7f6027400700 1 -- 172.21.15.123:0/2849126631 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6020027070 con 0x7f6030105120 2024-05-08T05:57:50.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.506+0000 7f6027400700 1 -- 172.21.15.123:0/2849126631 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f602000be10 con 0x7f6030105120 2024-05-08T05:57:50.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.506+0000 7f6027400700 1 -- 172.21.15.123:0/2849126631 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6020022de0 con 0x7f6030105120 2024-05-08T05:57:50.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.506+0000 7f6035c00700 1 -- 172.21.15.123:0/2849126631 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f603019af90 con 0x7f6030105120 2024-05-08T05:57:50.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.507+0000 7f6035c00700 1 -- 172.21.15.123:0/2849126631 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f603019dc80 con 0x7f6030105120 2024-05-08T05:57:50.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.508+0000 7f6035c00700 1 -- 172.21.15.123:0/2849126631 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f603010b7e0 con 0x7f6030105120 2024-05-08T05:57:50.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.508+0000 7f6027400700 1 -- 172.21.15.123:0/2849126631 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f602002a070 con 0x7f6030105120 2024-05-08T05:57:50.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.509+0000 7f6027400700 1 --2- 172.21.15.123:0/2849126631 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f601403e570 0x7f6014040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:50.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.509+0000 7f6027400700 1 -- 172.21.15.123:0/2849126631 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f602001e030 con 0x7f6030105120 2024-05-08T05:57:50.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.509+0000 7f602e000700 1 --2- 172.21.15.123:0/2849126631 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f601403e570 0x7f6014040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:50.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.509+0000 7f602e000700 1 --2- 172.21.15.123:0/2849126631 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f601403e570 0x7f6014040a30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f601c006fd0 tx=0x7f601c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:50.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.514+0000 7f6027400700 1 -- 172.21.15.123:0/2849126631 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6020013070 con 0x7f6030105120 2024-05-08T05:57:50.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.749+0000 7f6035c00700 1 -- 172.21.15.123:0/2849126631 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f603002e620 con 0x7f601403e570 2024-05-08T05:57:50.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.758+0000 7f6027400700 1 -- 172.21.15.123:0/2849126631 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f603002e620 con 0x7f601403e570 2024-05-08T05:57:50.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.761+0000 7f6035c00700 1 -- 172.21.15.123:0/2849126631 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f601403e570 msgr2=0x7f6014040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:50.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.761+0000 7f6035c00700 1 --2- 172.21.15.123:0/2849126631 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f601403e570 0x7f6014040a30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f601c006fd0 tx=0x7f601c006e40 comp rx=0 tx=0).stop 2024-05-08T05:57:50.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.761+0000 7f6035c00700 1 -- 172.21.15.123:0/2849126631 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6030105120 msgr2=0x7f603019a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:50.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.761+0000 7f6035c00700 1 --2- 172.21.15.123:0/2849126631 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6030105120 0x7f603019a7c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6020004fb0 tx=0x7f6020009f60 comp rx=0 tx=0).stop 2024-05-08T05:57:50.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.761+0000 7f6035c00700 1 -- 172.21.15.123:0/2849126631 shutdown_connections 2024-05-08T05:57:50.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.761+0000 7f6035c00700 1 --2- 172.21.15.123:0/2849126631 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f601403e570 0x7f6014040a30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:50.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.761+0000 7f6035c00700 1 --2- 172.21.15.123:0/2849126631 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6030105120 0x7f603019a7c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:50.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.761+0000 7f6035c00700 1 -- 172.21.15.123:0/2849126631 >> 172.21.15.123:0/2849126631 conn(0x7f60301006c0 msgr2=0x7f60301011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:50.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.762+0000 7f6035c00700 1 -- 172.21.15.123:0/2849126631 shutdown_connections 2024-05-08T05:57:50.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:50.762+0000 7f6035c00700 1 -- 172.21.15.123:0/2849126631 wait complete. 2024-05-08T05:57:51.142 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:50 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:57:51.142 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:50 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:57:51.143 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:50 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi123", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:57:51.143 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:50 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:57:51.143 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:50 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:57:51.143 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:50 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:57:51.143 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:50 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:57:51.143 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:50 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:57:51.143 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:50 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:57:51.143 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:50 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:57:51.545 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi163 2024-05-08T05:57:51.546 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T05:57:51.546 DEBUG:teuthology.orchestra.run.smithi163:> dd of=/etc/ceph/ceph.conf 2024-05-08T05:57:51.565 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T05:57:51.565 DEBUG:teuthology.orchestra.run.smithi163:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:57:51.623 INFO:tasks.cephadm:Adding host smithi163 to orchestrator... 2024-05-08T05:57:51.623 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph orch host add smithi163 2024-05-08T05:57:52.021 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:57:52.167 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:51 smithi123 ceph-mon[29787]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:57:52.168 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:51 smithi123 ceph-mon[29787]: Updating smithi123:/etc/ceph/ceph.conf 2024-05-08T05:57:52.168 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:51 smithi123 ceph-mon[29787]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:57:52.168 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:51 smithi123 ceph-mon[29787]: Updating smithi123:/var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:57:53.068 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:52 smithi123 ceph-mon[29787]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:57:53.068 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:52 smithi123 ceph-mon[29787]: Updating smithi123:/var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:57:54.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:54.236+0000 7f02ce000700 1 -- 172.21.15.123:0/2993821360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02c810e490 msgr2=0x7f02c810e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:54.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:54.236+0000 7f02ce000700 1 --2- 172.21.15.123:0/2993821360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02c810e490 0x7f02c810e870 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f02b8009910 tx=0x7f02b8009c20 comp rx=0 tx=0).stop 2024-05-08T05:57:54.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:54.236+0000 7f02ce000700 1 -- 172.21.15.123:0/2993821360 shutdown_connections 2024-05-08T05:57:54.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:54.236+0000 7f02ce000700 1 --2- 172.21.15.123:0/2993821360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02c810e490 0x7f02c810e870 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:54.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:54.236+0000 7f02ce000700 1 -- 172.21.15.123:0/2993821360 >> 172.21.15.123:0/2993821360 conn(0x7f02c8078d00 msgr2=0x7f02c807b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:54.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:54.237+0000 7f02ce000700 1 -- 172.21.15.123:0/2993821360 shutdown_connections 2024-05-08T05:57:54.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:54.237+0000 7f02ce000700 1 -- 172.21.15.123:0/2993821360 wait complete. 2024-05-08T05:57:54.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:54.238+0000 7f02ce000700 1 Processor -- start 2024-05-08T05:57:54.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:54.238+0000 7f02ce000700 1 -- start start 2024-05-08T05:57:54.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:54.239+0000 7f02ce000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02c810e490 0x7f02c819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:54.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:54.239+0000 7f02ce000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f02c81139c0 con 0x7f02c810e490 2024-05-08T05:57:54.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:54.239+0000 7f02c7400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02c810e490 0x7f02c819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:54.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:54.239+0000 7f02c7400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02c810e490 0x7f02c819ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56024/0 (socket says 172.21.15.123:56024) 2024-05-08T05:57:54.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:54.239+0000 7f02c7400700 1 -- 172.21.15.123:0/3781974993 learned_addr learned my addr 172.21.15.123:0/3781974993 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:57:54.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:54.239+0000 7f02c7400700 1 -- 172.21.15.123:0/3781974993 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c819f450 con 0x7f02c810e490 2024-05-08T05:57:54.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:54.240+0000 7f02c7400700 1 --2- 172.21.15.123:0/3781974993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02c810e490 0x7f02c819ef10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f02b80056d0 tx=0x7f02b80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:54.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:54.240+0000 7f02bfe00700 1 -- 172.21.15.123:0/3781974993 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02b8027070 con 0x7f02c810e490 2024-05-08T05:57:54.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:54.240+0000 7f02bfe00700 1 -- 172.21.15.123:0/3781974993 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02b800be10 con 0x7f02c810e490 2024-05-08T05:57:54.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:54.240+0000 7f02bfe00700 1 -- 172.21.15.123:0/3781974993 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02b8022d00 con 0x7f02c810e490 2024-05-08T05:57:54.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:54.240+0000 7f02ce000700 1 -- 172.21.15.123:0/3781974993 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02c819f740 con 0x7f02c810e490 2024-05-08T05:57:54.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:54.240+0000 7f02ce000700 1 -- 172.21.15.123:0/3781974993 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02c81a37e0 con 0x7f02c810e490 2024-05-08T05:57:54.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:54.241+0000 7f02ce000700 1 -- 172.21.15.123:0/3781974993 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02c810abd0 con 0x7f02c810e490 2024-05-08T05:57:54.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:54.241+0000 7f02bfe00700 1 -- 172.21.15.123:0/3781974993 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f02b802a070 con 0x7f02c810e490 2024-05-08T05:57:54.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:54.242+0000 7f02bfe00700 1 --2- 172.21.15.123:0/3781974993 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f02ac03e570 0x7f02ac040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:57:54.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:54.242+0000 7f02bfe00700 1 -- 172.21.15.123:0/3781974993 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f02b801e030 con 0x7f02c810e490 2024-05-08T05:57:54.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:54.242+0000 7f02c6a00700 1 --2- 172.21.15.123:0/3781974993 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f02ac03e570 0x7f02ac040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:57:54.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:54.243+0000 7f02c6a00700 1 --2- 172.21.15.123:0/3781974993 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f02ac03e570 0x7f02ac040a30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f02b4006fd0 tx=0x7f02b4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:57:54.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:54.248+0000 7f02bfe00700 1 -- 172.21.15.123:0/3781974993 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f02b8013070 con 0x7f02c810e490 2024-05-08T05:57:54.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:54 smithi123 ceph-mon[29787]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:57:54.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:54 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:57:54.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:54 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:57:54.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:54 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:57:54.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:54.483+0000 7f02ce000700 1 -- 172.21.15.123:0/3781974993 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi163", "target": ["mon-mgr", ""]}) -- 0x7f02c81a0270 con 0x7f02ac03e570 2024-05-08T05:57:55.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:55 smithi123 ceph-mon[29787]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi163", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:57:56.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:56 smithi123 ceph-mon[29787]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:57:56.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:56 smithi123 ceph-mon[29787]: Deploying cephadm binary to smithi163 2024-05-08T05:57:58.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:57:58 smithi123 ceph-mon[29787]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:57:59.175 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:59.175+0000 7f02bfe00700 1 -- 172.21.15.123:0/3781974993 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f02c81a0270 con 0x7f02ac03e570 2024-05-08T05:57:59.175 INFO:teuthology.orchestra.run.smithi123.stdout:Added host 'smithi163' with addr '172.21.15.163' 2024-05-08T05:57:59.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:59.178+0000 7f02ce000700 1 -- 172.21.15.123:0/3781974993 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f02ac03e570 msgr2=0x7f02ac040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:59.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:59.178+0000 7f02ce000700 1 --2- 172.21.15.123:0/3781974993 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f02ac03e570 0x7f02ac040a30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f02b4006fd0 tx=0x7f02b4006e40 comp rx=0 tx=0).stop 2024-05-08T05:57:59.179 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:59.178+0000 7f02ce000700 1 -- 172.21.15.123:0/3781974993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02c810e490 msgr2=0x7f02c819ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:57:59.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:59.178+0000 7f02ce000700 1 --2- 172.21.15.123:0/3781974993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02c810e490 0x7f02c819ef10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f02b80056d0 tx=0x7f02b80057b0 comp rx=0 tx=0).stop 2024-05-08T05:57:59.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:59.178+0000 7f02ce000700 1 -- 172.21.15.123:0/3781974993 shutdown_connections 2024-05-08T05:57:59.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:59.179+0000 7f02ce000700 1 --2- 172.21.15.123:0/3781974993 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f02ac03e570 0x7f02ac040a30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:59.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:59.179+0000 7f02ce000700 1 --2- 172.21.15.123:0/3781974993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02c810e490 0x7f02c819ef10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:57:59.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:59.179+0000 7f02ce000700 1 -- 172.21.15.123:0/3781974993 >> 172.21.15.123:0/3781974993 conn(0x7f02c8078d00 msgr2=0x7f02c81110a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:57:59.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:59.179+0000 7f02ce000700 1 -- 172.21.15.123:0/3781974993 shutdown_connections 2024-05-08T05:57:59.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:57:59.179+0000 7f02ce000700 1 -- 172.21.15.123:0/3781974993 wait complete. 2024-05-08T05:57:59.931 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-08T05:58:00.342 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:58:00.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:00 smithi123 ceph-mon[29787]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:00.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:00 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:00.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:00 smithi123 ceph-mon[29787]: Added host smithi163 2024-05-08T05:58:00.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:00 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:58:00.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:00 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:01.505 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:01 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:02.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.384+0000 7f0353000700 1 -- 172.21.15.123:0/1225922904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f034c102f20 msgr2=0x7f034c103300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:02.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.384+0000 7f0353000700 1 --2- 172.21.15.123:0/1225922904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f034c102f20 0x7f034c103300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0340009910 tx=0x7f0340009c20 comp rx=0 tx=0).stop 2024-05-08T05:58:02.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.384+0000 7f0353000700 1 -- 172.21.15.123:0/1225922904 shutdown_connections 2024-05-08T05:58:02.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.384+0000 7f0353000700 1 --2- 172.21.15.123:0/1225922904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f034c102f20 0x7f034c103300 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:02.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.384+0000 7f0353000700 1 -- 172.21.15.123:0/1225922904 >> 172.21.15.123:0/1225922904 conn(0x7f034c0fe7f0 msgr2=0x7f034c100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:02.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.384+0000 7f0353000700 1 -- 172.21.15.123:0/1225922904 shutdown_connections 2024-05-08T05:58:02.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.384+0000 7f0353000700 1 -- 172.21.15.123:0/1225922904 wait complete. 2024-05-08T05:58:02.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.385+0000 7f0353000700 1 Processor -- start 2024-05-08T05:58:02.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.385+0000 7f0353000700 1 -- start start 2024-05-08T05:58:02.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.386+0000 7f0353000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f034c102f20 0x7f034c1a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:02.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.386+0000 7f0353000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f034c10d1a0 con 0x7f034c102f20 2024-05-08T05:58:02.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.386+0000 7f0351c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f034c102f20 0x7f034c1a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:02.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.386+0000 7f0351c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f034c102f20 0x7f034c1a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45810/0 (socket says 172.21.15.123:45810) 2024-05-08T05:58:02.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.386+0000 7f0351c00700 1 -- 172.21.15.123:0/2758907200 learned_addr learned my addr 172.21.15.123:0/2758907200 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:58:02.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.386+0000 7f0351c00700 1 -- 172.21.15.123:0/2758907200 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f034c1a0be0 con 0x7f034c102f20 2024-05-08T05:58:02.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.387+0000 7f0351c00700 1 --2- 172.21.15.123:0/2758907200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f034c102f20 0x7f034c1a06a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0340006010 tx=0x7f0340005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:02.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.387+0000 7f034a000700 1 -- 172.21.15.123:0/2758907200 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0340026070 con 0x7f034c102f20 2024-05-08T05:58:02.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.387+0000 7f034a000700 1 -- 172.21.15.123:0/2758907200 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f034000bba0 con 0x7f034c102f20 2024-05-08T05:58:02.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.387+0000 7f034a000700 1 -- 172.21.15.123:0/2758907200 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0340003910 con 0x7f034c102f20 2024-05-08T05:58:02.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.387+0000 7f0353000700 1 -- 172.21.15.123:0/2758907200 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f034c1a0ed0 con 0x7f034c102f20 2024-05-08T05:58:02.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.387+0000 7f0353000700 1 -- 172.21.15.123:0/2758907200 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f034c19a940 con 0x7f034c102f20 2024-05-08T05:58:02.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.389+0000 7f034a000700 1 -- 172.21.15.123:0/2758907200 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f0340029070 con 0x7f034c102f20 2024-05-08T05:58:02.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.389+0000 7f0353000700 1 -- 172.21.15.123:0/2758907200 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f034c104400 con 0x7f034c102f20 2024-05-08T05:58:02.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.389+0000 7f034a000700 1 --2- 172.21.15.123:0/2758907200 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f033403e570 0x7f0334040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:02.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.389+0000 7f034a000700 1 -- 172.21.15.123:0/2758907200 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f034001e070 con 0x7f034c102f20 2024-05-08T05:58:02.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.389+0000 7f0351200700 1 --2- 172.21.15.123:0/2758907200 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f033403e570 0x7f0334040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:02.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.390+0000 7f0351200700 1 --2- 172.21.15.123:0/2758907200 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f033403e570 0x7f0334040a30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0338006fd0 tx=0x7f0338006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:02.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.395+0000 7f034a000700 1 -- 172.21.15.123:0/2758907200 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0340013070 con 0x7f034c102f20 2024-05-08T05:58:02.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.634+0000 7f0353000700 1 -- 172.21.15.123:0/2758907200 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f034c19bd10 con 0x7f033403e570 2024-05-08T05:58:02.636 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:02 smithi123 ceph-mon[29787]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:02.636 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:02 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:02.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.637+0000 7f034a000700 1 -- 172.21.15.123:0/2758907200 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7f034c19bd10 con 0x7f033403e570 2024-05-08T05:58:02.638 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T05:58:02.639 INFO:teuthology.orchestra.run.smithi123.stdout:[{"addr": "172.21.15.123", "hostname": "smithi123", "labels": [], "status": ""}, {"addr": "172.21.15.163", "hostname": "smithi163", "labels": [], "status": ""}] 2024-05-08T05:58:02.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.640+0000 7f0353000700 1 -- 172.21.15.123:0/2758907200 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f033403e570 msgr2=0x7f0334040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:02.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.640+0000 7f0353000700 1 --2- 172.21.15.123:0/2758907200 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f033403e570 0x7f0334040a30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0338006fd0 tx=0x7f0338006e40 comp rx=0 tx=0).stop 2024-05-08T05:58:02.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.640+0000 7f0353000700 1 -- 172.21.15.123:0/2758907200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f034c102f20 msgr2=0x7f034c1a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:02.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.640+0000 7f0353000700 1 --2- 172.21.15.123:0/2758907200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f034c102f20 0x7f034c1a06a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0340006010 tx=0x7f0340005780 comp rx=0 tx=0).stop 2024-05-08T05:58:02.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.640+0000 7f0353000700 1 -- 172.21.15.123:0/2758907200 shutdown_connections 2024-05-08T05:58:02.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.640+0000 7f0353000700 1 --2- 172.21.15.123:0/2758907200 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f033403e570 0x7f0334040a30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:02.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.640+0000 7f0353000700 1 --2- 172.21.15.123:0/2758907200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f034c102f20 0x7f034c1a06a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:02.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.640+0000 7f0353000700 1 -- 172.21.15.123:0/2758907200 >> 172.21.15.123:0/2758907200 conn(0x7f034c0fe7f0 msgr2=0x7f034c0756d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:02.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.641+0000 7f0353000700 1 -- 172.21.15.123:0/2758907200 shutdown_connections 2024-05-08T05:58:02.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:02.641+0000 7f0353000700 1 -- 172.21.15.123:0/2758907200 wait complete. 2024-05-08T05:58:03.366 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi192 2024-05-08T05:58:03.366 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T05:58:03.366 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/etc/ceph/ceph.conf 2024-05-08T05:58:03.384 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T05:58:03.384 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:58:03.442 INFO:tasks.cephadm:Adding host smithi192 to orchestrator... 2024-05-08T05:58:03.442 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph orch host add smithi192 2024-05-08T05:58:03.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:03 smithi123 ceph-mon[29787]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:58:03.873 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:58:04.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:04 smithi123 ceph-mon[29787]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:06.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:06.050+0000 7f01a6800700 1 -- 172.21.15.123:0/2524212446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01a00779d0 msgr2=0x7f01a0077db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:06.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:06.050+0000 7f01a6800700 1 --2- 172.21.15.123:0/2524212446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01a00779d0 0x7f01a0077db0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0190009910 tx=0x7f0190009c20 comp rx=0 tx=0).stop 2024-05-08T05:58:06.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:06.050+0000 7f01a6800700 1 -- 172.21.15.123:0/2524212446 shutdown_connections 2024-05-08T05:58:06.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:06.050+0000 7f01a6800700 1 --2- 172.21.15.123:0/2524212446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01a00779d0 0x7f01a0077db0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:06.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:06.050+0000 7f01a6800700 1 -- 172.21.15.123:0/2524212446 >> 172.21.15.123:0/2524212446 conn(0x7f01a0100b50 msgr2=0x7f01a0102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:06.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:06.050+0000 7f01a6800700 1 -- 172.21.15.123:0/2524212446 shutdown_connections 2024-05-08T05:58:06.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:06.050+0000 7f01a6800700 1 -- 172.21.15.123:0/2524212446 wait complete. 2024-05-08T05:58:06.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:06.051+0000 7f01a6800700 1 Processor -- start 2024-05-08T05:58:06.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:06.051+0000 7f01a6800700 1 -- start start 2024-05-08T05:58:06.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:06.051+0000 7f01a6800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01a00779d0 0x7f01a019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:06.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:06.051+0000 7f01a6800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f01a01139c0 con 0x7f01a00779d0 2024-05-08T05:58:06.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:06.052+0000 7f019f400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01a00779d0 0x7f01a019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:06.119 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:06.052+0000 7f019f400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01a00779d0 0x7f01a019eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50712/0 (socket says 172.21.15.123:50712) 2024-05-08T05:58:06.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:06.052+0000 7f019f400700 1 -- 172.21.15.123:0/556854662 learned_addr learned my addr 172.21.15.123:0/556854662 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:58:06.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:06.052+0000 7f019f400700 1 -- 172.21.15.123:0/556854662 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a019f430 con 0x7f01a00779d0 2024-05-08T05:58:06.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:06.052+0000 7f019f400700 1 --2- 172.21.15.123:0/556854662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01a00779d0 0x7f01a019eef0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0190006010 tx=0x7f0190005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:06.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:06.053+0000 7f0197e00700 1 -- 172.21.15.123:0/556854662 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0190026070 con 0x7f01a00779d0 2024-05-08T05:58:06.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:06.053+0000 7f0197e00700 1 -- 172.21.15.123:0/556854662 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f019000bc00 con 0x7f01a00779d0 2024-05-08T05:58:06.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:06.053+0000 7f01a6800700 1 -- 172.21.15.123:0/556854662 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01a019f6c0 con 0x7f01a00779d0 2024-05-08T05:58:06.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:06.053+0000 7f0197e00700 1 -- 172.21.15.123:0/556854662 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0190003910 con 0x7f01a00779d0 2024-05-08T05:58:06.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:06.053+0000 7f01a6800700 1 -- 172.21.15.123:0/556854662 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01a019fb20 con 0x7f01a00779d0 2024-05-08T05:58:06.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:06.054+0000 7f01a6800700 1 -- 172.21.15.123:0/556854662 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01a0079110 con 0x7f01a00779d0 2024-05-08T05:58:06.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:06.055+0000 7f0197e00700 1 -- 172.21.15.123:0/556854662 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f0190029070 con 0x7f01a00779d0 2024-05-08T05:58:06.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:06.055+0000 7f0197e00700 1 --2- 172.21.15.123:0/556854662 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f018403e510 0x7f01840409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:06.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:06.055+0000 7f0197e00700 1 -- 172.21.15.123:0/556854662 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f019001e070 con 0x7f01a00779d0 2024-05-08T05:58:06.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:06.056+0000 7f019ea00700 1 --2- 172.21.15.123:0/556854662 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f018403e510 0x7f01840409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:06.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:06.056+0000 7f019ea00700 1 --2- 172.21.15.123:0/556854662 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f018403e510 0x7f01840409d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f018c006fd0 tx=0x7f018c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:06.121 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:06.061+0000 7f0197e00700 1 -- 172.21.15.123:0/556854662 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0190013070 con 0x7f01a00779d0 2024-05-08T05:58:06.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:06 smithi123 ceph-mon[29787]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:06.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:06 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:06.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:06 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:06.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:06 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:06.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:06 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:06.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:06 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi163", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:58:06.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:06 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:06.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:06 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:58:06.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:06 smithi123 ceph-mon[29787]: Updating smithi163:/etc/ceph/ceph.conf 2024-05-08T05:58:06.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:06 smithi123 ceph-mon[29787]: Updating smithi163:/var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:58:06.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:06 smithi123 ceph-mon[29787]: Updating smithi163:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:58:06.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:06.306+0000 7f01a6800700 1 -- 172.21.15.123:0/556854662 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi192", "target": ["mon-mgr", ""]}) -- 0x7f01a01a02c0 con 0x7f018403e510 2024-05-08T05:58:07.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:07 smithi123 ceph-mon[29787]: Updating smithi163:/var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:58:07.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:07 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:07.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:07 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:07.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:07 smithi123 ceph-mon[29787]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi192", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:58:07.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:07 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:08.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:08 smithi123 ceph-mon[29787]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:08.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:08 smithi123 ceph-mon[29787]: Deploying cephadm binary to smithi192 2024-05-08T05:58:10.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:10 smithi123 ceph-mon[29787]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:11.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:11.091+0000 7f0197e00700 1 -- 172.21.15.123:0/556854662 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f01a01a02c0 con 0x7f018403e510 2024-05-08T05:58:11.091 INFO:teuthology.orchestra.run.smithi123.stdout:Added host 'smithi192' with addr '172.21.15.192' 2024-05-08T05:58:11.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:11.094+0000 7f01a6800700 1 -- 172.21.15.123:0/556854662 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f018403e510 msgr2=0x7f01840409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:11.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:11.095+0000 7f01a6800700 1 --2- 172.21.15.123:0/556854662 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f018403e510 0x7f01840409d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f018c006fd0 tx=0x7f018c006e40 comp rx=0 tx=0).stop 2024-05-08T05:58:11.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:11.095+0000 7f01a6800700 1 -- 172.21.15.123:0/556854662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01a00779d0 msgr2=0x7f01a019eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:11.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:11.095+0000 7f01a6800700 1 --2- 172.21.15.123:0/556854662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01a00779d0 0x7f01a019eef0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0190006010 tx=0x7f0190005780 comp rx=0 tx=0).stop 2024-05-08T05:58:11.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:11.095+0000 7f01a6800700 1 -- 172.21.15.123:0/556854662 shutdown_connections 2024-05-08T05:58:11.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:11.095+0000 7f01a6800700 1 --2- 172.21.15.123:0/556854662 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f018403e510 0x7f01840409d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:11.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:11.095+0000 7f01a6800700 1 --2- 172.21.15.123:0/556854662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01a00779d0 0x7f01a019eef0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:11.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:11.095+0000 7f01a6800700 1 -- 172.21.15.123:0/556854662 >> 172.21.15.123:0/556854662 conn(0x7f01a0100b50 msgr2=0x7f01a0100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:11.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:11.095+0000 7f01a6800700 1 -- 172.21.15.123:0/556854662 shutdown_connections 2024-05-08T05:58:11.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:11.095+0000 7f01a6800700 1 -- 172.21.15.123:0/556854662 wait complete. 2024-05-08T05:58:11.889 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-08T05:58:12.310 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:58:12.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:12 smithi123 ceph-mon[29787]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:12.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:12 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:12.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:12 smithi123 ceph-mon[29787]: Added host smithi192 2024-05-08T05:58:12.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:12 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:58:12.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:12 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:13.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:13 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:14.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.359+0000 7fa362600700 1 -- 172.21.15.123:0/776452312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa35c0757c0 msgr2=0x7fa35c075ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:14.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.359+0000 7fa362600700 1 --2- 172.21.15.123:0/776452312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa35c0757c0 0x7fa35c075ba0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa34c009910 tx=0x7fa34c009c20 comp rx=0 tx=0).stop 2024-05-08T05:58:14.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.359+0000 7fa362600700 1 -- 172.21.15.123:0/776452312 shutdown_connections 2024-05-08T05:58:14.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.359+0000 7fa362600700 1 --2- 172.21.15.123:0/776452312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa35c0757c0 0x7fa35c075ba0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:14.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.359+0000 7fa362600700 1 -- 172.21.15.123:0/776452312 >> 172.21.15.123:0/776452312 conn(0x7fa35c0fe990 msgr2=0x7fa35c100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:14.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.359+0000 7fa362600700 1 -- 172.21.15.123:0/776452312 shutdown_connections 2024-05-08T05:58:14.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.359+0000 7fa362600700 1 -- 172.21.15.123:0/776452312 wait complete. 2024-05-08T05:58:14.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.360+0000 7fa362600700 1 Processor -- start 2024-05-08T05:58:14.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.360+0000 7fa362600700 1 -- start start 2024-05-08T05:58:14.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.361+0000 7fa362600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa35c0757c0 0x7fa35c1a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:14.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.361+0000 7fa362600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c10f5c0 con 0x7fa35c0757c0 2024-05-08T05:58:14.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.361+0000 7fa35b400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa35c0757c0 0x7fa35c1a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:14.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.361+0000 7fa35b400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa35c0757c0 0x7fa35c1a0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50732/0 (socket says 172.21.15.123:50732) 2024-05-08T05:58:14.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.361+0000 7fa35b400700 1 -- 172.21.15.123:0/1861069077 learned_addr learned my addr 172.21.15.123:0/1861069077 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:58:14.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.361+0000 7fa35b400700 1 -- 172.21.15.123:0/1861069077 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35c1a0cd0 con 0x7fa35c0757c0 2024-05-08T05:58:14.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.362+0000 7fa35b400700 1 --2- 172.21.15.123:0/1861069077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa35c0757c0 0x7fa35c1a0790 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa34c0056d0 tx=0x7fa34c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:14.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.362+0000 7fa353e00700 1 -- 172.21.15.123:0/1861069077 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa34c026070 con 0x7fa35c0757c0 2024-05-08T05:58:14.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.362+0000 7fa353e00700 1 -- 172.21.15.123:0/1861069077 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa34c004370 con 0x7fa35c0757c0 2024-05-08T05:58:14.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.362+0000 7fa353e00700 1 -- 172.21.15.123:0/1861069077 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa34c0039b0 con 0x7fa35c0757c0 2024-05-08T05:58:14.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.362+0000 7fa362600700 1 -- 172.21.15.123:0/1861069077 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa35c1a0f60 con 0x7fa35c0757c0 2024-05-08T05:58:14.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.362+0000 7fa362600700 1 -- 172.21.15.123:0/1861069077 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa35c19a950 con 0x7fa35c0757c0 2024-05-08T05:58:14.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.363+0000 7fa362600700 1 -- 172.21.15.123:0/1861069077 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa35c076f00 con 0x7fa35c0757c0 2024-05-08T05:58:14.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.364+0000 7fa353e00700 1 -- 172.21.15.123:0/1861069077 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7fa34c029070 con 0x7fa35c0757c0 2024-05-08T05:58:14.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.364+0000 7fa353e00700 1 --2- 172.21.15.123:0/1861069077 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa34003e510 0x7fa3400409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:14.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.364+0000 7fa353e00700 1 -- 172.21.15.123:0/1861069077 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa34c01e070 con 0x7fa35c0757c0 2024-05-08T05:58:14.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.364+0000 7fa35aa00700 1 --2- 172.21.15.123:0/1861069077 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa34003e510 0x7fa3400409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:14.365 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.365+0000 7fa35aa00700 1 --2- 172.21.15.123:0/1861069077 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa34003e510 0x7fa3400409d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa348006fd0 tx=0x7fa348006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:14.370 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.370+0000 7fa353e00700 1 -- 172.21.15.123:0/1861069077 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa34c013070 con 0x7fa35c0757c0 2024-05-08T05:58:14.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:14 smithi123 ceph-mon[29787]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:14.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:14 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:14.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.608+0000 7fa362600700 1 -- 172.21.15.123:0/1861069077 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa35c19be80 con 0x7fa34003e510 2024-05-08T05:58:14.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.610+0000 7fa353e00700 1 -- 172.21.15.123:0/1861069077 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7fa35c19be80 con 0x7fa34003e510 2024-05-08T05:58:14.610 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T05:58:14.611 INFO:teuthology.orchestra.run.smithi123.stdout:[{"addr": "172.21.15.123", "hostname": "smithi123", "labels": [], "status": ""}, {"addr": "172.21.15.163", "hostname": "smithi163", "labels": [], "status": ""}, {"addr": "172.21.15.192", "hostname": "smithi192", "labels": [], "status": ""}] 2024-05-08T05:58:14.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.613+0000 7fa362600700 1 -- 172.21.15.123:0/1861069077 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa34003e510 msgr2=0x7fa3400409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:14.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.613+0000 7fa362600700 1 --2- 172.21.15.123:0/1861069077 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa34003e510 0x7fa3400409d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa348006fd0 tx=0x7fa348006e40 comp rx=0 tx=0).stop 2024-05-08T05:58:14.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.613+0000 7fa362600700 1 -- 172.21.15.123:0/1861069077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa35c0757c0 msgr2=0x7fa35c1a0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:14.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.613+0000 7fa362600700 1 --2- 172.21.15.123:0/1861069077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa35c0757c0 0x7fa35c1a0790 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa34c0056d0 tx=0x7fa34c0057b0 comp rx=0 tx=0).stop 2024-05-08T05:58:14.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.614+0000 7fa362600700 1 -- 172.21.15.123:0/1861069077 shutdown_connections 2024-05-08T05:58:14.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.614+0000 7fa362600700 1 --2- 172.21.15.123:0/1861069077 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa34003e510 0x7fa3400409d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:14.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.614+0000 7fa362600700 1 --2- 172.21.15.123:0/1861069077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa35c0757c0 0x7fa35c1a0790 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:14.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.614+0000 7fa362600700 1 -- 172.21.15.123:0/1861069077 >> 172.21.15.123:0/1861069077 conn(0x7fa35c0fe990 msgr2=0x7fa35c1071a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:14.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.614+0000 7fa362600700 1 -- 172.21.15.123:0/1861069077 shutdown_connections 2024-05-08T05:58:14.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:14.614+0000 7fa362600700 1 -- 172.21.15.123:0/1861069077 wait complete. 2024-05-08T05:58:15.352 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:15 smithi123 ceph-mon[29787]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:58:15.407 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-08T05:58:15.407 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph osd crush tunables default 2024-05-08T05:58:15.840 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:58:16.271 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:16 smithi123 ceph-mon[29787]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:18.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:18.059+0000 7fea69400700 1 -- 172.21.15.123:0/3318565808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea641030e0 msgr2=0x7fea641034c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:18.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:18.059+0000 7fea69400700 1 --2- 172.21.15.123:0/3318565808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea641030e0 0x7fea641034c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fea54009910 tx=0x7fea54009c20 comp rx=0 tx=0).stop 2024-05-08T05:58:18.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:18.059+0000 7fea69400700 1 -- 172.21.15.123:0/3318565808 shutdown_connections 2024-05-08T05:58:18.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:18.059+0000 7fea69400700 1 --2- 172.21.15.123:0/3318565808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea641030e0 0x7fea641034c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:18.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:18.059+0000 7fea69400700 1 -- 172.21.15.123:0/3318565808 >> 172.21.15.123:0/3318565808 conn(0x7fea640fe990 msgr2=0x7fea64100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:18.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:18.060+0000 7fea69400700 1 -- 172.21.15.123:0/3318565808 shutdown_connections 2024-05-08T05:58:18.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:18.060+0000 7fea69400700 1 -- 172.21.15.123:0/3318565808 wait complete. 2024-05-08T05:58:18.061 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:18.061+0000 7fea69400700 1 Processor -- start 2024-05-08T05:58:18.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:18.061+0000 7fea69400700 1 -- start start 2024-05-08T05:58:18.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:18.061+0000 7fea69400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea641030e0 0x7fea6419e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:18.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:18.061+0000 7fea69400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fea6410d360 con 0x7fea641030e0 2024-05-08T05:58:18.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:18.062+0000 7fea62800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea641030e0 0x7fea6419e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:18.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:18.062+0000 7fea62800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea641030e0 0x7fea6419e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41624/0 (socket says 172.21.15.123:41624) 2024-05-08T05:58:18.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:18.062+0000 7fea62800700 1 -- 172.21.15.123:0/2662668964 learned_addr learned my addr 172.21.15.123:0/2662668964 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:58:18.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:18.062+0000 7fea62800700 1 -- 172.21.15.123:0/2662668964 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea6419eb90 con 0x7fea641030e0 2024-05-08T05:58:18.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:18.062+0000 7fea62800700 1 --2- 172.21.15.123:0/2662668964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea641030e0 0x7fea6419e650 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fea540056d0 tx=0x7fea540057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:18.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:18.063+0000 7fea5b400700 1 -- 172.21.15.123:0/2662668964 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea54026070 con 0x7fea641030e0 2024-05-08T05:58:18.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:18.063+0000 7fea5b400700 1 -- 172.21.15.123:0/2662668964 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea5400bba0 con 0x7fea641030e0 2024-05-08T05:58:18.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:18.063+0000 7fea69400700 1 -- 172.21.15.123:0/2662668964 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea6419ee80 con 0x7fea641030e0 2024-05-08T05:58:18.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:18.063+0000 7fea5b400700 1 -- 172.21.15.123:0/2662668964 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea54003920 con 0x7fea641030e0 2024-05-08T05:58:18.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:18.063+0000 7fea69400700 1 -- 172.21.15.123:0/2662668964 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea641988f0 con 0x7fea641030e0 2024-05-08T05:58:18.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:18.064+0000 7fea5b400700 1 -- 172.21.15.123:0/2662668964 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7fea54029070 con 0x7fea641030e0 2024-05-08T05:58:18.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:18.064+0000 7fea69400700 1 -- 172.21.15.123:0/2662668964 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea641045c0 con 0x7fea641030e0 2024-05-08T05:58:18.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:18.064+0000 7fea5b400700 1 --2- 172.21.15.123:0/2662668964 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fea4403e570 0x7fea44040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:18.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:18.065+0000 7fea5b400700 1 -- 172.21.15.123:0/2662668964 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fea5401e070 con 0x7fea641030e0 2024-05-08T05:58:18.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:18.065+0000 7fea61e00700 1 --2- 172.21.15.123:0/2662668964 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fea4403e570 0x7fea44040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:18.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:18.066+0000 7fea61e00700 1 --2- 172.21.15.123:0/2662668964 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fea4403e570 0x7fea44040a30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fea4c00ad30 tx=0x7fea4c0093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:18.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:18.070+0000 7fea5b400700 1 -- 172.21.15.123:0/2662668964 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fea54013070 con 0x7fea641030e0 2024-05-08T05:58:18.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:17 smithi123 ceph-mon[29787]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:18.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:17 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:18.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:17 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:18.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:17 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:18.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:17 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:18.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:17 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi192", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:58:18.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:17 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:18.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:17 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:58:18.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:17 smithi123 ceph-mon[29787]: Updating smithi192:/etc/ceph/ceph.conf 2024-05-08T05:58:18.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:17 smithi123 ceph-mon[29787]: Updating smithi192:/var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:58:18.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:17 smithi123 ceph-mon[29787]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:58:18.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:18.309+0000 7fea69400700 1 -- 172.21.15.123:0/2662668964 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fea640008d0 con 0x7fea641030e0 2024-05-08T05:58:19.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:19.184+0000 7fea5b400700 1 -- 172.21.15.123:0/2662668964 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fea5400d340 con 0x7fea641030e0 2024-05-08T05:58:19.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:19.188+0000 7fea69400700 1 -- 172.21.15.123:0/2662668964 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fea4403e570 msgr2=0x7fea44040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:19.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:19.188+0000 7fea69400700 1 --2- 172.21.15.123:0/2662668964 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fea4403e570 0x7fea44040a30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fea4c00ad30 tx=0x7fea4c0093f0 comp rx=0 tx=0).stop 2024-05-08T05:58:19.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:19.188+0000 7fea69400700 1 -- 172.21.15.123:0/2662668964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea641030e0 msgr2=0x7fea6419e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:19.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:19.188+0000 7fea69400700 1 --2- 172.21.15.123:0/2662668964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea641030e0 0x7fea6419e650 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fea540056d0 tx=0x7fea540057b0 comp rx=0 tx=0).stop 2024-05-08T05:58:19.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:19.188+0000 7fea69400700 1 -- 172.21.15.123:0/2662668964 shutdown_connections 2024-05-08T05:58:19.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:19.188+0000 7fea69400700 1 --2- 172.21.15.123:0/2662668964 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fea4403e570 0x7fea44040a30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:19.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:19.188+0000 7fea69400700 1 --2- 172.21.15.123:0/2662668964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea641030e0 0x7fea6419e650 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:19.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:19.188+0000 7fea69400700 1 -- 172.21.15.123:0/2662668964 >> 172.21.15.123:0/2662668964 conn(0x7fea640fe990 msgr2=0x7fea64104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:19.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:19.189+0000 7fea69400700 1 -- 172.21.15.123:0/2662668964 shutdown_connections 2024-05-08T05:58:19.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:19.189+0000 7fea69400700 1 -- 172.21.15.123:0/2662668964 wait complete. 2024-05-08T05:58:19.190 INFO:teuthology.orchestra.run.smithi123.stderr:adjusted tunables profile to default 2024-05-08T05:58:19.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:19 smithi123 ceph-mon[29787]: Updating smithi192:/var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:58:19.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:19 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:19.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:19 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:19.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:19 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:19.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:19 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/2662668964' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-08T05:58:19.957 INFO:tasks.cephadm:Adding mon.a on smithi123 2024-05-08T05:58:19.958 INFO:tasks.cephadm:Adding mon.b on smithi163 2024-05-08T05:58:19.958 INFO:tasks.cephadm:Adding mon.c on smithi192 2024-05-08T05:58:19.958 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph orch apply mon '3;smithi123:172.21.15.123=a;smithi163:172.21.15.163=b;smithi192:172.21.15.192=c' 2024-05-08T05:58:20.202 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:58:20.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:20 smithi123 ceph-mon[29787]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:20.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:20 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/2662668964' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-08T05:58:20.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:20 smithi123 ceph-mon[29787]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T05:58:21.605 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.604+0000 7f8e8fede700 1 -- 172.21.15.192:0/1835505413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e881017f0 msgr2=0x7f8e88101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:21.605 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.604+0000 7f8e8fede700 1 --2- 172.21.15.192:0/1835505413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e881017f0 0x7f8e88101bd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8e8400b0a0 tx=0x7f8e8400b3b0 comp rx=0 tx=0).stop 2024-05-08T05:58:21.605 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.604+0000 7f8e8fede700 1 -- 172.21.15.192:0/1835505413 shutdown_connections 2024-05-08T05:58:21.605 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.604+0000 7f8e8fede700 1 --2- 172.21.15.192:0/1835505413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e881017f0 0x7f8e88101bd0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:21.605 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.604+0000 7f8e8fede700 1 -- 172.21.15.192:0/1835505413 >> 172.21.15.192:0/1835505413 conn(0x7f8e88079400 msgr2=0x7f8e88079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:21.606 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.604+0000 7f8e8fede700 1 -- 172.21.15.192:0/1835505413 shutdown_connections 2024-05-08T05:58:21.606 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.604+0000 7f8e8fede700 1 -- 172.21.15.192:0/1835505413 wait complete. 2024-05-08T05:58:21.606 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.605+0000 7f8e8fede700 1 Processor -- start 2024-05-08T05:58:21.606 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.605+0000 7f8e8fede700 1 -- start start 2024-05-08T05:58:21.606 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.605+0000 7f8e8fede700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e881017f0 0x7f8e8819e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:21.606 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.605+0000 7f8e8fede700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8810d3a0 con 0x7f8e881017f0 2024-05-08T05:58:21.607 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.606+0000 7f8e8dc7a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e881017f0 0x7f8e8819e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:21.607 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.606+0000 7f8e8dc7a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e881017f0 0x7f8e8819e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.192:57444/0 (socket says 172.21.15.192:57444) 2024-05-08T05:58:21.607 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.606+0000 7f8e8dc7a700 1 -- 172.21.15.192:0/338708299 learned_addr learned my addr 172.21.15.192:0/338708299 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-08T05:58:21.607 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.606+0000 7f8e8dc7a700 1 -- 172.21.15.192:0/338708299 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e8819ead0 con 0x7f8e881017f0 2024-05-08T05:58:21.607 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.606+0000 7f8e8dc7a700 1 --2- 172.21.15.192:0/338708299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e881017f0 0x7f8e8819e590 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8e84006130 tx=0x7f8e840097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:21.607 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.607+0000 7f8e7effd700 1 -- 172.21.15.192:0/338708299 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e84021020 con 0x7f8e881017f0 2024-05-08T05:58:21.607 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.607+0000 7f8e7effd700 1 -- 172.21.15.192:0/338708299 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e840079b0 con 0x7f8e881017f0 2024-05-08T05:58:21.608 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.607+0000 7f8e8fede700 1 -- 172.21.15.192:0/338708299 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e8819ed60 con 0x7f8e881017f0 2024-05-08T05:58:21.608 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.607+0000 7f8e7effd700 1 -- 172.21.15.192:0/338708299 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e84017040 con 0x7f8e881017f0 2024-05-08T05:58:21.608 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.607+0000 7f8e8fede700 1 -- 172.21.15.192:0/338708299 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e881986b0 con 0x7f8e881017f0 2024-05-08T05:58:21.608 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.608+0000 7f8e8fede700 1 -- 172.21.15.192:0/338708299 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e8806afb0 con 0x7f8e881017f0 2024-05-08T05:58:21.609 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.609+0000 7f8e7effd700 1 -- 172.21.15.192:0/338708299 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f8e84024020 con 0x7f8e881017f0 2024-05-08T05:58:21.609 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.609+0000 7f8e7effd700 1 --2- 172.21.15.192:0/338708299 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f8e7403e510 0x7f8e740409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:21.609 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.609+0000 7f8e7effd700 1 -- 172.21.15.192:0/338708299 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8e8400f050 con 0x7f8e881017f0 2024-05-08T05:58:21.609 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.609+0000 7f8e8d479700 1 --2- 172.21.15.192:0/338708299 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f8e7403e510 0x7f8e740409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:21.610 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.610+0000 7f8e8d479700 1 --2- 172.21.15.192:0/338708299 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f8e7403e510 0x7f8e740409d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8e78006fd0 tx=0x7f8e78006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:21.625 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.625+0000 7f8e7effd700 1 -- 172.21.15.192:0/338708299 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8e8402b020 con 0x7f8e881017f0 2024-05-08T05:58:21.842 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.842+0000 7f8e8fede700 1 -- 172.21.15.192:0/338708299 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi123:172.21.15.123=a;smithi163:172.21.15.163=b;smithi192:172.21.15.192=c", "target": ["mon-mgr", ""]}) -- 0x7f8e88199b90 con 0x7f8e7403e510 2024-05-08T05:58:21.857 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.857+0000 7f8e7effd700 1 -- 172.21.15.192:0/338708299 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8e88199b90 con 0x7f8e7403e510 2024-05-08T05:58:21.857 INFO:teuthology.orchestra.run.smithi192.stdout:Scheduled mon update... 2024-05-08T05:58:21.860 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.859+0000 7f8e8fede700 1 -- 172.21.15.192:0/338708299 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f8e7403e510 msgr2=0x7f8e740409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:21.860 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.859+0000 7f8e8fede700 1 --2- 172.21.15.192:0/338708299 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f8e7403e510 0x7f8e740409d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8e78006fd0 tx=0x7f8e78006e40 comp rx=0 tx=0).stop 2024-05-08T05:58:21.860 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.859+0000 7f8e8fede700 1 -- 172.21.15.192:0/338708299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e881017f0 msgr2=0x7f8e8819e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:21.860 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.859+0000 7f8e8fede700 1 --2- 172.21.15.192:0/338708299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e881017f0 0x7f8e8819e590 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8e84006130 tx=0x7f8e840097d0 comp rx=0 tx=0).stop 2024-05-08T05:58:21.860 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.860+0000 7f8e8fede700 1 -- 172.21.15.192:0/338708299 shutdown_connections 2024-05-08T05:58:21.860 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.860+0000 7f8e8fede700 1 --2- 172.21.15.192:0/338708299 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f8e7403e510 0x7f8e740409d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:21.861 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.860+0000 7f8e8fede700 1 --2- 172.21.15.192:0/338708299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e881017f0 0x7f8e8819e590 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:21.861 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.860+0000 7f8e8fede700 1 -- 172.21.15.192:0/338708299 >> 172.21.15.192:0/338708299 conn(0x7f8e88079400 msgr2=0x7f8e88104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:21.861 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.860+0000 7f8e8fede700 1 -- 172.21.15.192:0/338708299 shutdown_connections 2024-05-08T05:58:21.861 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:21.860+0000 7f8e8fede700 1 -- 172.21.15.192:0/338708299 wait complete. 2024-05-08T05:58:22.435 DEBUG:teuthology.orchestra.run.smithi163:mon.b> sudo journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mon.b.service 2024-05-08T05:58:22.438 DEBUG:teuthology.orchestra.run.smithi192:mon.c> sudo journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mon.c.service 2024-05-08T05:58:22.440 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-08T05:58:22.440 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-08T05:58:22.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:22 smithi123 ceph-mon[29787]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:22.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:22 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:22.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:22 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:58:22.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:22 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:22.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:22 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:58:22.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:22 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:22.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:22 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:58:22.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:22 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:23.190 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:58:23.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:23 smithi123 ceph-mon[29787]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi123:172.21.15.123=a;smithi163:172.21.15.163=b;smithi192:172.21.15.192=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:58:23.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:23 smithi123 ceph-mon[29787]: Saving service mon spec with placement smithi123:172.21.15.123=a;smithi163:172.21.15.163=b;smithi192:172.21.15.192=c;count:3 2024-05-08T05:58:23.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:23 smithi123 ceph-mon[29787]: Deploying daemon mon.c on smithi192 2024-05-08T05:58:24.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:24 smithi123 ceph-mon[29787]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:26.133 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.132+0000 7f2940c58700 1 -- 172.21.15.192:0/1449470173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f293c104070 msgr2=0x7f293c104450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:26.133 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.132+0000 7f2940c58700 1 --2- 172.21.15.192:0/1449470173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f293c104070 0x7f293c104450 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f292c009910 tx=0x7f292c009c20 comp rx=0 tx=0).stop 2024-05-08T05:58:26.133 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.132+0000 7f2940c58700 1 -- 172.21.15.192:0/1449470173 shutdown_connections 2024-05-08T05:58:26.133 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.132+0000 7f2940c58700 1 --2- 172.21.15.192:0/1449470173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f293c104070 0x7f293c104450 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:26.133 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.132+0000 7f2940c58700 1 -- 172.21.15.192:0/1449470173 >> 172.21.15.192:0/1449470173 conn(0x7f293c0fd8e0 msgr2=0x7f293c0ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:26.133 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.132+0000 7f2940c58700 1 -- 172.21.15.192:0/1449470173 shutdown_connections 2024-05-08T05:58:26.133 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.132+0000 7f2940c58700 1 -- 172.21.15.192:0/1449470173 wait complete. 2024-05-08T05:58:26.134 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.133+0000 7f2940c58700 1 Processor -- start 2024-05-08T05:58:26.134 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.133+0000 7f2940c58700 1 -- start start 2024-05-08T05:58:26.134 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.134+0000 7f2940c58700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f293c104070 0x7f293c19e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:26.134 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.134+0000 7f2940c58700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f293c10d410 con 0x7f293c104070 2024-05-08T05:58:26.134 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.134+0000 7f293a59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f293c104070 0x7f293c19e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:26.134 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.134+0000 7f293a59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f293c104070 0x7f293c19e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.192:48762/0 (socket says 172.21.15.192:48762) 2024-05-08T05:58:26.135 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.134+0000 7f293a59c700 1 -- 172.21.15.192:0/3982669295 learned_addr learned my addr 172.21.15.192:0/3982669295 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-08T05:58:26.135 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.135+0000 7f293a59c700 1 -- 172.21.15.192:0/3982669295 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f293c19eb90 con 0x7f293c104070 2024-05-08T05:58:26.135 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.135+0000 7f293a59c700 1 --2- 172.21.15.192:0/3982669295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f293c104070 0x7f293c19e650 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f292c0056d0 tx=0x7f292c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:26.136 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.135+0000 7f292b7fe700 1 -- 172.21.15.192:0/3982669295 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f292c026070 con 0x7f293c104070 2024-05-08T05:58:26.136 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.135+0000 7f2940c58700 1 -- 172.21.15.192:0/3982669295 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f293c19ee20 con 0x7f293c104070 2024-05-08T05:58:26.136 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.135+0000 7f2940c58700 1 -- 172.21.15.192:0/3982669295 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f293c1987d0 con 0x7f293c104070 2024-05-08T05:58:26.136 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.135+0000 7f292b7fe700 1 -- 172.21.15.192:0/3982669295 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f292c004370 con 0x7f293c104070 2024-05-08T05:58:26.136 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.135+0000 7f292b7fe700 1 -- 172.21.15.192:0/3982669295 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f292c003990 con 0x7f293c104070 2024-05-08T05:58:26.136 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.137+0000 7f2940c58700 1 -- 172.21.15.192:0/3982669295 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f293c06ba40 con 0x7f293c104070 2024-05-08T05:58:26.137 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.137+0000 7f292b7fe700 1 -- 172.21.15.192:0/3982669295 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f292c029070 con 0x7f293c104070 2024-05-08T05:58:26.137 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.137+0000 7f292b7fe700 1 --2- 172.21.15.192:0/3982669295 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f292403e510 0x7f29240409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:26.137 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.137+0000 7f292b7fe700 1 -- 172.21.15.192:0/3982669295 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f292c01e070 con 0x7f293c104070 2024-05-08T05:58:26.137 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.138+0000 7f2939d9b700 1 --2- 172.21.15.192:0/3982669295 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f292403e510 0x7f29240409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:26.138 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.138+0000 7f2939d9b700 1 --2- 172.21.15.192:0/3982669295 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f292403e510 0x7f29240409d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2930006fd0 tx=0x7f2930006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:26.153 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.153+0000 7f292b7fe700 1 -- 172.21.15.192:0/3982669295 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f292c013070 con 0x7f293c104070 2024-05-08T05:58:26.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:26 smithi123 ceph-mon[29787]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:26.470 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.469+0000 7f2940c58700 1 -- 172.21.15.192:0/3982669295 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f293c064b00 con 0x7f293c104070 2024-05-08T05:58:26.470 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.470+0000 7f292b7fe700 1 -- 172.21.15.192:0/3982669295 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7f292c00bc70 con 0x7f293c104070 2024-05-08T05:58:26.471 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T05:58:26.471 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"85cfb2e6-0cff-11ef-bc97-c7b262605968","modified":"2024-05-08T05:56:08.254675Z","created":"2024-05-08T05:56:08.254675Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:3300","nonce":0},{"type":"v1","addr":"172.21.15.123:6789","nonce":0}]},"addr":"172.21.15.123:6789/0","public_addr":"172.21.15.123:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-08T05:58:26.473 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.473+0000 7f2940c58700 1 -- 172.21.15.192:0/3982669295 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f292403e510 msgr2=0x7f29240409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:26.473 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.473+0000 7f2940c58700 1 --2- 172.21.15.192:0/3982669295 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f292403e510 0x7f29240409d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2930006fd0 tx=0x7f2930006e40 comp rx=0 tx=0).stop 2024-05-08T05:58:26.473 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.473+0000 7f2940c58700 1 -- 172.21.15.192:0/3982669295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f293c104070 msgr2=0x7f293c19e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:26.473 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.473+0000 7f2940c58700 1 --2- 172.21.15.192:0/3982669295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f293c104070 0x7f293c19e650 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f292c0056d0 tx=0x7f292c0057b0 comp rx=0 tx=0).stop 2024-05-08T05:58:26.473 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.473+0000 7f2940c58700 1 -- 172.21.15.192:0/3982669295 shutdown_connections 2024-05-08T05:58:26.473 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.473+0000 7f2940c58700 1 --2- 172.21.15.192:0/3982669295 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f292403e510 0x7f29240409d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:26.473 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.473+0000 7f2940c58700 1 --2- 172.21.15.192:0/3982669295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f293c104070 0x7f293c19e650 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:26.473 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.473+0000 7f2940c58700 1 -- 172.21.15.192:0/3982669295 >> 172.21.15.192:0/3982669295 conn(0x7f293c0fd8e0 msgr2=0x7f293c106e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:26.474 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.473+0000 7f2940c58700 1 -- 172.21.15.192:0/3982669295 shutdown_connections 2024-05-08T05:58:26.474 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:26.473+0000 7f2940c58700 1 -- 172.21.15.192:0/3982669295 wait complete. 2024-05-08T05:58:26.474 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-05-08T05:58:26.756 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:26 smithi192 systemd[1]: Starting Ceph mon.c for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T05:58:27.435 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 podman[34645]: 2024-05-08 05:58:27.182813071 +0000 UTC m=+0.103452466 container create adfc8c6ace0c13c8f69f9eb6f7f84c47df58aee294cc7da4d39cb19c17483b6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-08T05:58:27.435 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 podman[34645]: 2024-05-08 05:58:27.096934226 +0000 UTC m=+0.017573628 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:58:27.435 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 podman[34645]: 2024-05-08 05:58:27.36004563 +0000 UTC m=+0.280685022 container init adfc8c6ace0c13c8f69f9eb6f7f84c47df58aee294cc7da4d39cb19c17483b6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mon-c, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0) 2024-05-08T05:58:27.435 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 podman[34645]: 2024-05-08 05:58:27.362799771 +0000 UTC m=+0.283439164 container start adfc8c6ace0c13c8f69f9eb6f7f84c47df58aee294cc7da4d39cb19c17483b6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mon-c, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507) 2024-05-08T05:58:27.435 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: set uid:gid to 167:167 (ceph:ceph) 2024-05-08T05:58:27.435 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: ceph version 19.0.0-3538-gbee299f4 (bee299f4ac344e4c45e550d7a80abcadd7bc3d5a) squid (dev), process ceph-mon, pid 2 2024-05-08T05:58:27.435 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: pidfile_write: ignore empty --pid-file 2024-05-08T05:58:27.435 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: load: jerasure load: lrc 2024-05-08T05:58:27.435 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: RocksDB version: 7.9.2 2024-05-08T05:58:27.435 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Git sha 0 2024-05-08T05:58:27.436 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Compile date 2024-05-07 11:18:56 2024-05-08T05:58:27.436 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: DB SUMMARY 2024-05-08T05:58:27.436 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: DB Session ID: 69IQFNV158CU5F3XETRZ 2024-05-08T05:58:27.436 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: CURRENT file: CURRENT 2024-05-08T05:58:27.436 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: IDENTITY file: IDENTITY 2024-05-08T05:58:27.436 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-08T05:58:27.436 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-05-08T05:58:27.436 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-05-08T05:58:27.438 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.error_if_exists: 0 2024-05-08T05:58:27.438 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.create_if_missing: 0 2024-05-08T05:58:27.438 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.paranoid_checks: 1 2024-05-08T05:58:27.438 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-08T05:58:27.438 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-08T05:58:27.438 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-08T05:58:27.438 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.env: 0x56105f8bccc0 2024-05-08T05:58:27.438 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.fs: PosixFileSystem 2024-05-08T05:58:27.438 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.info_log: 0x561061009da0 2024-05-08T05:58:27.439 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.max_file_opening_threads: 16 2024-05-08T05:58:27.439 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.statistics: (nil) 2024-05-08T05:58:27.439 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.use_fsync: 0 2024-05-08T05:58:27.439 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.max_log_file_size: 0 2024-05-08T05:58:27.439 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-08T05:58:27.439 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-08T05:58:27.439 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.keep_log_file_num: 1000 2024-05-08T05:58:27.439 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.recycle_log_file_num: 0 2024-05-08T05:58:27.439 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.allow_fallocate: 1 2024-05-08T05:58:27.439 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.allow_mmap_reads: 0 2024-05-08T05:58:27.439 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.allow_mmap_writes: 0 2024-05-08T05:58:27.439 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.use_direct_reads: 0 2024-05-08T05:58:27.439 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-08T05:58:27.439 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.create_missing_column_families: 0 2024-05-08T05:58:27.440 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.db_log_dir: 2024-05-08T05:58:27.440 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.wal_dir: 2024-05-08T05:58:27.440 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-08T05:58:27.440 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-08T05:58:27.440 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-08T05:58:27.440 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-08T05:58:27.440 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-08T05:58:27.440 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-08T05:58:27.440 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.advise_random_on_open: 1 2024-05-08T05:58:27.440 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.db_write_buffer_size: 0 2024-05-08T05:58:27.440 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.write_buffer_manager: 0x561060fdb0e0 2024-05-08T05:58:27.440 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-08T05:58:27.440 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-08T05:58:27.440 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-08T05:58:27.441 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.rate_limiter: (nil) 2024-05-08T05:58:27.441 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-08T05:58:27.441 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.wal_recovery_mode: 2 2024-05-08T05:58:27.441 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.enable_thread_tracking: 0 2024-05-08T05:58:27.441 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.enable_pipelined_write: 0 2024-05-08T05:58:27.441 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.unordered_write: 0 2024-05-08T05:58:27.441 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-08T05:58:27.441 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-08T05:58:27.441 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-08T05:58:27.441 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-08T05:58:27.441 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.row_cache: None 2024-05-08T05:58:27.441 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.wal_filter: None 2024-05-08T05:58:27.441 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-08T05:58:27.441 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.allow_ingest_behind: 0 2024-05-08T05:58:27.442 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.two_write_queues: 0 2024-05-08T05:58:27.442 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.manual_wal_flush: 0 2024-05-08T05:58:27.442 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.wal_compression: 0 2024-05-08T05:58:27.442 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.atomic_flush: 0 2024-05-08T05:58:27.442 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-08T05:58:27.442 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-08T05:58:27.442 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-08T05:58:27.442 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.log_readahead_size: 0 2024-05-08T05:58:27.442 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-08T05:58:27.442 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.best_efforts_recovery: 0 2024-05-08T05:58:27.442 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-08T05:58:27.442 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-08T05:58:27.442 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.allow_data_in_errors: 0 2024-05-08T05:58:27.442 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.db_host_id: __hostname__ 2024-05-08T05:58:27.442 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-08T05:58:27.443 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.max_background_jobs: 2 2024-05-08T05:58:27.443 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.max_background_compactions: -1 2024-05-08T05:58:27.443 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.max_subcompactions: 1 2024-05-08T05:58:27.443 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-08T05:58:27.443 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-08T05:58:27.443 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-08T05:58:27.443 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.max_total_wal_size: 0 2024-05-08T05:58:27.443 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-08T05:58:27.443 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-08T05:58:27.443 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-08T05:58:27.443 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-08T05:58:27.443 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.max_open_files: -1 2024-05-08T05:58:27.443 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.bytes_per_sync: 0 2024-05-08T05:58:27.443 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-08T05:58:27.444 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-08T05:58:27.444 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.compaction_readahead_size: 0 2024-05-08T05:58:27.444 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.max_background_flushes: -1 2024-05-08T05:58:27.444 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Compression algorithms supported: 2024-05-08T05:58:27.444 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-08T05:58:27.444 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: kZSTD supported: 0 2024-05-08T05:58:27.444 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: kXpressCompression supported: 0 2024-05-08T05:58:27.444 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: kLZ4HCCompression supported: 1 2024-05-08T05:58:27.444 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: kZlibCompression supported: 1 2024-05-08T05:58:27.444 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: kSnappyCompression supported: 1 2024-05-08T05:58:27.444 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: kLZ4Compression supported: 1 2024-05-08T05:58:27.444 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: kBZip2Compression supported: 0 2024-05-08T05:58:27.444 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-08T05:58:27.444 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-08T05:58:27.445 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-05-08T05:58:27.445 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-08T05:58:27.445 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-08T05:58:27.445 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.merge_operator: 2024-05-08T05:58:27.445 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.compaction_filter: None 2024-05-08T05:58:27.445 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.compaction_filter_factory: None 2024-05-08T05:58:27.445 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.sst_partitioner_factory: None 2024-05-08T05:58:27.445 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-08T05:58:27.445 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-08T05:58:27.445 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561061009f60) 2024-05-08T05:58:27.445 INFO:journalctl@ceph.mon.c.smithi192.stdout: cache_index_and_filter_blocks: 1 2024-05-08T05:58:27.445 INFO:journalctl@ceph.mon.c.smithi192.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-08T05:58:27.445 INFO:journalctl@ceph.mon.c.smithi192.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-08T05:58:27.445 INFO:journalctl@ceph.mon.c.smithi192.stdout: pin_top_level_index_and_filter: 1 2024-05-08T05:58:27.446 INFO:journalctl@ceph.mon.c.smithi192.stdout: index_type: 0 2024-05-08T05:58:27.446 INFO:journalctl@ceph.mon.c.smithi192.stdout: data_block_index_type: 0 2024-05-08T05:58:27.446 INFO:journalctl@ceph.mon.c.smithi192.stdout: index_shortening: 1 2024-05-08T05:58:27.446 INFO:journalctl@ceph.mon.c.smithi192.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-08T05:58:27.446 INFO:journalctl@ceph.mon.c.smithi192.stdout: checksum: 4 2024-05-08T05:58:27.446 INFO:journalctl@ceph.mon.c.smithi192.stdout: no_block_cache: 0 2024-05-08T05:58:27.446 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_cache: 0x56106100d1f0 2024-05-08T05:58:27.446 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_cache_name: BinnedLRUCache 2024-05-08T05:58:27.446 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_cache_options: 2024-05-08T05:58:27.446 INFO:journalctl@ceph.mon.c.smithi192.stdout: capacity : 536870912 2024-05-08T05:58:27.446 INFO:journalctl@ceph.mon.c.smithi192.stdout: num_shard_bits : 4 2024-05-08T05:58:27.446 INFO:journalctl@ceph.mon.c.smithi192.stdout: strict_capacity_limit : 0 2024-05-08T05:58:27.446 INFO:journalctl@ceph.mon.c.smithi192.stdout: high_pri_pool_ratio: 0.000 2024-05-08T05:58:27.446 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_cache_compressed: (nil) 2024-05-08T05:58:27.447 INFO:journalctl@ceph.mon.c.smithi192.stdout: persistent_cache: (nil) 2024-05-08T05:58:27.447 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_size: 4096 2024-05-08T05:58:27.447 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_size_deviation: 10 2024-05-08T05:58:27.447 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_restart_interval: 16 2024-05-08T05:58:27.447 INFO:journalctl@ceph.mon.c.smithi192.stdout: index_block_restart_interval: 1 2024-05-08T05:58:27.447 INFO:journalctl@ceph.mon.c.smithi192.stdout: metadata_block_size: 4096 2024-05-08T05:58:27.447 INFO:journalctl@ceph.mon.c.smithi192.stdout: partition_filters: 0 2024-05-08T05:58:27.447 INFO:journalctl@ceph.mon.c.smithi192.stdout: use_delta_encoding: 1 2024-05-08T05:58:27.447 INFO:journalctl@ceph.mon.c.smithi192.stdout: filter_policy: bloomfilter 2024-05-08T05:58:27.447 INFO:journalctl@ceph.mon.c.smithi192.stdout: whole_key_filtering: 1 2024-05-08T05:58:27.448 INFO:journalctl@ceph.mon.c.smithi192.stdout: verify_compression: 0 2024-05-08T05:58:27.448 INFO:journalctl@ceph.mon.c.smithi192.stdout: read_amp_bytes_per_bit: 0 2024-05-08T05:58:27.448 INFO:journalctl@ceph.mon.c.smithi192.stdout: format_version: 5 2024-05-08T05:58:27.448 INFO:journalctl@ceph.mon.c.smithi192.stdout: enable_index_compression: 1 2024-05-08T05:58:27.448 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_align: 0 2024-05-08T05:58:27.448 INFO:journalctl@ceph.mon.c.smithi192.stdout: max_auto_readahead_size: 262144 2024-05-08T05:58:27.448 INFO:journalctl@ceph.mon.c.smithi192.stdout: prepopulate_block_cache: 0 2024-05-08T05:58:27.448 INFO:journalctl@ceph.mon.c.smithi192.stdout: initial_auto_readahead_size: 8192 2024-05-08T05:58:27.448 INFO:journalctl@ceph.mon.c.smithi192.stdout: num_file_reads_for_auto_readahead: 2 2024-05-08T05:58:27.449 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.write_buffer_size: 33554432 2024-05-08T05:58:27.449 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.max_write_buffer_number: 2 2024-05-08T05:58:27.449 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.compression: NoCompression 2024-05-08T05:58:27.449 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.bottommost_compression: Disabled 2024-05-08T05:58:27.449 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.prefix_extractor: nullptr 2024-05-08T05:58:27.449 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-08T05:58:27.449 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.num_levels: 7 2024-05-08T05:58:27.449 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-08T05:58:27.449 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-08T05:58:27.449 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-08T05:58:27.449 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-08T05:58:27.449 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-08T05:58:27.449 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-08T05:58:27.449 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-08T05:58:27.450 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-08T05:58:27.450 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-08T05:58:27.450 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-08T05:58:27.450 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-08T05:58:27.450 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-08T05:58:27.450 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-08T05:58:27.450 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.compression_opts.level: 32767 2024-05-08T05:58:27.450 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.compression_opts.strategy: 0 2024-05-08T05:58:27.450 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-08T05:58:27.450 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-08T05:58:27.450 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-08T05:58:27.451 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-08T05:58:27.451 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.compression_opts.enabled: false 2024-05-08T05:58:27.451 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-08T05:58:27.451 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-08T05:58:27.451 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-08T05:58:27.451 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-08T05:58:27.451 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.target_file_size_base: 67108864 2024-05-08T05:58:27.451 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-08T05:58:27.451 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-08T05:58:27.451 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-08T05:58:27.451 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-08T05:58:27.451 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-08T05:58:27.451 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-08T05:58:27.451 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-08T05:58:27.452 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-08T05:58:27.452 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-08T05:58:27.452 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-08T05:58:27.452 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-08T05:58:27.452 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-08T05:58:27.452 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-08T05:58:27.452 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-08T05:58:27.452 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.arena_block_size: 1048576 2024-05-08T05:58:27.452 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-08T05:58:27.452 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-08T05:58:27.452 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.disable_auto_compactions: 0 2024-05-08T05:58:27.452 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-08T05:58:27.452 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-08T05:58:27.452 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-08T05:58:27.452 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-08T05:58:27.452 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-08T05:58:27.453 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-08T05:58:27.453 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-08T05:58:27.453 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-08T05:58:27.453 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-08T05:58:27.453 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-08T05:58:27.453 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-08T05:58:27.453 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.inplace_update_support: 0 2024-05-08T05:58:27.453 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-08T05:58:27.453 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-08T05:58:27.453 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-08T05:58:27.453 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-08T05:58:27.453 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.bloom_locality: 0 2024-05-08T05:58:27.453 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.max_successive_merges: 0 2024-05-08T05:58:27.453 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-08T05:58:27.454 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.paranoid_file_checks: 0 2024-05-08T05:58:27.454 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.force_consistency_checks: 1 2024-05-08T05:58:27.454 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.report_bg_io_stats: 0 2024-05-08T05:58:27.454 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.ttl: 2592000 2024-05-08T05:58:27.454 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-08T05:58:27.454 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-08T05:58:27.454 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-08T05:58:27.454 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.enable_blob_files: false 2024-05-08T05:58:27.454 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.min_blob_size: 0 2024-05-08T05:58:27.454 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.blob_file_size: 268435456 2024-05-08T05:58:27.454 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-08T05:58:27.454 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-08T05:58:27.454 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-08T05:58:27.454 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-08T05:58:27.454 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-08T05:58:27.454 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.blob_file_starting_level: 0 2024-05-08T05:58:27.455 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-08T05:58:27.455 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-08T05:58:27.455 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-08T05:58:27.455 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 95e2b297-27fb-4326-8e06-118b106cec90 2024-05-08T05:58:27.455 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715147907392694, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-08T05:58:27.455 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-08T05:58:27.455 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715147907393116, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715147907, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "95e2b297-27fb-4326-8e06-118b106cec90", "db_session_id": "69IQFNV158CU5F3XETRZ", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-08T05:58:27.455 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715147907393193, "job": 1, "event": "recovery_finished"} 2024-05-08T05:58:27.455 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-08T05:58:27.455 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-08T05:58:27.455 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56106112a000 2024-05-08T05:58:27.455 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: DB pointer 0x561061116000 2024-05-08T05:58:27.455 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-08T05:58:27.455 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-08T05:58:27.456 INFO:journalctl@ceph.mon.c.smithi192.stdout: ** DB Stats ** 2024-05-08T05:58:27.456 INFO:journalctl@ceph.mon.c.smithi192.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T05:58:27.456 INFO:journalctl@ceph.mon.c.smithi192.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-08T05:58:27.456 INFO:journalctl@ceph.mon.c.smithi192.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T05:58:27.456 INFO:journalctl@ceph.mon.c.smithi192.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T05:58:27.456 INFO:journalctl@ceph.mon.c.smithi192.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-08T05:58:27.456 INFO:journalctl@ceph.mon.c.smithi192.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T05:58:27.456 INFO:journalctl@ceph.mon.c.smithi192.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T05:58:27.456 INFO:journalctl@ceph.mon.c.smithi192.stdout: 2024-05-08T05:58:27.456 INFO:journalctl@ceph.mon.c.smithi192.stdout: ** Compaction Stats [default] ** 2024-05-08T05:58:27.456 INFO:journalctl@ceph.mon.c.smithi192.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T05:58:27.456 INFO:journalctl@ceph.mon.c.smithi192.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-08T05:58:27.456 INFO:journalctl@ceph.mon.c.smithi192.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T05:58:27.457 INFO:journalctl@ceph.mon.c.smithi192.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T05:58:27.457 INFO:journalctl@ceph.mon.c.smithi192.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T05:58:27.457 INFO:journalctl@ceph.mon.c.smithi192.stdout: 2024-05-08T05:58:27.457 INFO:journalctl@ceph.mon.c.smithi192.stdout: ** Compaction Stats [default] ** 2024-05-08T05:58:27.457 INFO:journalctl@ceph.mon.c.smithi192.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T05:58:27.457 INFO:journalctl@ceph.mon.c.smithi192.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-08T05:58:27.457 INFO:journalctl@ceph.mon.c.smithi192.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T05:58:27.457 INFO:journalctl@ceph.mon.c.smithi192.stdout: 2024-05-08T05:58:27.457 INFO:journalctl@ceph.mon.c.smithi192.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-08T05:58:27.457 INFO:journalctl@ceph.mon.c.smithi192.stdout: 2024-05-08T05:58:27.457 INFO:journalctl@ceph.mon.c.smithi192.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T05:58:27.457 INFO:journalctl@ceph.mon.c.smithi192.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-08T05:58:27.457 INFO:journalctl@ceph.mon.c.smithi192.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-08T05:58:27.457 INFO:journalctl@ceph.mon.c.smithi192.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-08T05:58:27.457 INFO:journalctl@ceph.mon.c.smithi192.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-08T05:58:27.457 INFO:journalctl@ceph.mon.c.smithi192.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-08T05:58:27.458 INFO:journalctl@ceph.mon.c.smithi192.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T05:58:27.458 INFO:journalctl@ceph.mon.c.smithi192.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T05:58:27.458 INFO:journalctl@ceph.mon.c.smithi192.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-08T05:58:27.458 INFO:journalctl@ceph.mon.c.smithi192.stdout: Block cache BinnedLRUCache@0x56106100d1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-08T05:58:27.458 INFO:journalctl@ceph.mon.c.smithi192.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-08T05:58:27.458 INFO:journalctl@ceph.mon.c.smithi192.stdout: 2024-05-08T05:58:27.458 INFO:journalctl@ceph.mon.c.smithi192.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-08T05:58:27.458 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-05-08T05:58:27.458 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: using public_addr v2:172.21.15.192:0/0 -> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] 2024-05-08T05:58:27.458 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: starting mon.c rank -1 at public addrs [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] at bind addrs [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 2024-05-08T05:58:27.458 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: mon.c@-1(???) e0 preinit fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 2024-05-08T05:58:27.458 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: mon.c@-1(synchronizing).mds e1 new map 2024-05-08T05:58:27.458 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-08T05:58:27.458 INFO:journalctl@ceph.mon.c.smithi192.stdout: e1 2024-05-08T05:58:27.458 INFO:journalctl@ceph.mon.c.smithi192.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T05:58:27.458 INFO:journalctl@ceph.mon.c.smithi192.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-08T05:58:27.459 INFO:journalctl@ceph.mon.c.smithi192.stdout: legacy client fscid: -1 2024-05-08T05:58:27.459 INFO:journalctl@ceph.mon.c.smithi192.stdout: 2024-05-08T05:58:27.459 INFO:journalctl@ceph.mon.c.smithi192.stdout: No filesystems configured 2024-05-08T05:58:27.459 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-08T05:58:27.459 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-08T05:58:27.459 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-08T05:58:27.459 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-08T05:58:27.459 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-08T05:58:27.460 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-08T05:58:27.460 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-08T05:58:27.460 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T05:58:27.460 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T05:58:27.460 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T05:58:27.460 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:27.460 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:27.460 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:27.460 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: Added host smithi163 2024-05-08T05:58:27.460 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:58:27.460 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:27.460 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:27.460 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:27.460 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:27.460 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:58:27.460 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:27.460 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:27.461 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:27.461 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:27.461 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:27.461 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:27.461 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi163", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:58:27.461 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:27.461 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:58:27.461 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: Updating smithi163:/etc/ceph/ceph.conf 2024-05-08T05:58:27.461 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: Updating smithi163:/var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:58:27.461 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: Updating smithi163:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:58:27.461 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: Updating smithi163:/var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:58:27.461 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:27.461 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:27.461 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi192", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:58:27.461 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:27.462 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:27.462 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: Deploying cephadm binary to smithi192 2024-05-08T05:58:27.462 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:27.462 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:27.462 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:27.462 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: Added host smithi192 2024-05-08T05:58:27.462 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:58:27.462 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:27.462 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:27.462 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:27.463 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:27.463 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:58:27.463 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:27.463 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:27.463 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:27.463 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:27.463 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:27.463 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:27.463 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi192", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:58:27.463 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:27.463 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:58:27.463 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: Updating smithi192:/etc/ceph/ceph.conf 2024-05-08T05:58:27.463 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: Updating smithi192:/var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:58:27.463 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:58:27.463 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: Updating smithi192:/var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:58:27.464 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:27.464 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:27.464 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:27.464 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/2662668964' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-08T05:58:27.464 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:27.464 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/2662668964' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-08T05:58:27.464 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T05:58:27.464 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:27.464 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:27.464 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:58:27.464 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:27.464 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:58:27.464 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:27.465 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:58:27.465 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:27.465 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi123:172.21.15.123=a;smithi163:172.21.15.163=b;smithi192:172.21.15.192=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:58:27.465 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: Saving service mon spec with placement smithi123:172.21.15.123=a;smithi163:172.21.15.163=b;smithi192:172.21.15.192=c;count:3 2024-05-08T05:58:27.465 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: Deploying daemon mon.c on smithi192 2024-05-08T05:58:27.465 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:27.465 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:27.465 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: from='client.? 172.21.15.192:0/3982669295' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T05:58:27.465 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 ceph-mon[34664]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-08T05:58:27.465 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 bash[34645]: adfc8c6ace0c13c8f69f9eb6f7f84c47df58aee294cc7da4d39cb19c17483b6c 2024-05-08T05:58:27.465 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:27 smithi192 systemd[1]: Started Ceph mon.c for 85cfb2e6-0cff-11ef-bc97-c7b262605968. 2024-05-08T05:58:27.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:27 smithi123 ceph-mon[29787]: from='client.? 172.21.15.192:0/3982669295' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T05:58:28.001 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-08T05:58:28.001 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-08T05:58:28.237 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:58:32.670 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.670+0000 7fad3e6cb700 1 -- 172.21.15.192:0/2976520097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad200057d0 msgr2=0x7fad20007c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:32.670 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.670+0000 7fad3e6cb700 1 --2- 172.21.15.192:0/2976520097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad200057d0 0x7fad20007c60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fad3806af00 tx=0x7fad2c00fe60 comp rx=0 tx=0).stop 2024-05-08T05:58:32.670 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.670+0000 7fad3e6cb700 1 -- 172.21.15.192:0/2976520097 shutdown_connections 2024-05-08T05:58:32.670 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.670+0000 7fad3e6cb700 1 --2- 172.21.15.192:0/2976520097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad200057d0 0x7fad20007c60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:32.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.670+0000 7fad3e6cb700 1 -- 172.21.15.192:0/2976520097 >> 172.21.15.192:0/2976520097 conn(0x7fad380782c0 msgr2=0x7fad380786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:32.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.670+0000 7fad3e6cb700 1 -- 172.21.15.192:0/2976520097 shutdown_connections 2024-05-08T05:58:32.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.670+0000 7fad3e6cb700 1 -- 172.21.15.192:0/2976520097 wait complete. 2024-05-08T05:58:32.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.671+0000 7fad3e6cb700 1 Processor -- start 2024-05-08T05:58:32.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.671+0000 7fad3e6cb700 1 -- start start 2024-05-08T05:58:32.672 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.672+0000 7fad3e6cb700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad200057d0 0x7fad3819c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:32.672 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.672+0000 7fad3e6cb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad3819cad0 0x7fad38196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:32.672 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.672+0000 7fad3e6cb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fad38108dd0 con 0x7fad3819cad0 2024-05-08T05:58:32.672 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.672+0000 7fad3e6cb700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fad381025c0 con 0x7fad200057d0 2024-05-08T05:58:32.673 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.672+0000 7fad37fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad200057d0 0x7fad3819c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:32.673 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.672+0000 7fad37fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad200057d0 0x7fad3819c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:36452/0 (socket says 172.21.15.192:36452) 2024-05-08T05:58:32.673 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.672+0000 7fad37fff700 1 -- 172.21.15.192:0/3392158130 learned_addr learned my addr 172.21.15.192:0/3392158130 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-08T05:58:32.673 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.672+0000 7fad37fff700 1 -- 172.21.15.192:0/3392158130 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad200057d0 msgr2=0x7fad3819c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-08T05:58:32.673 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.672+0000 7fad37fff700 1 -- 172.21.15.192:0/3392158130 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad200057d0 msgr2=0x7fad3819c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T05:58:32.673 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.672+0000 7fad37fff700 1 --2- 172.21.15.192:0/3392158130 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad200057d0 0x7fad3819c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T05:58:32.673 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.672+0000 7fad37fff700 1 --2- 172.21.15.192:0/3392158130 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad200057d0 0x7fad3819c590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T05:58:32.673 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.672+0000 7fad377fe700 1 --2- 172.21.15.192:0/3392158130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad3819cad0 0x7fad38196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:32.674 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.673+0000 7fad377fe700 1 -- 172.21.15.192:0/3392158130 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad200057d0 msgr2=0x7fad3819c590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T05:58:32.674 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.673+0000 7fad377fe700 1 --2- 172.21.15.192:0/3392158130 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad200057d0 0x7fad3819c590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:32.674 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.673+0000 7fad377fe700 1 -- 172.21.15.192:0/3392158130 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad3819d050 con 0x7fad3819cad0 2024-05-08T05:58:32.674 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.673+0000 7fad377fe700 1 --2- 172.21.15.192:0/3392158130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad3819cad0 0x7fad38196660 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fad2800b4d0 tx=0x7fad2800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:32.674 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.673+0000 7fad357fa700 1 -- 172.21.15.192:0/3392158130 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fad28011630 con 0x7fad3819cad0 2024-05-08T05:58:32.674 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.673+0000 7fad3e6cb700 1 -- 172.21.15.192:0/3392158130 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad38196c00 con 0x7fad3819cad0 2024-05-08T05:58:32.674 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.674+0000 7fad3e6cb700 1 -- 172.21.15.192:0/3392158130 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad38197190 con 0x7fad3819cad0 2024-05-08T05:58:32.674 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.674+0000 7fad357fa700 1 -- 172.21.15.192:0/3392158130 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad28011c80 con 0x7fad3819cad0 2024-05-08T05:58:32.675 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.674+0000 7fad357fa700 1 -- 172.21.15.192:0/3392158130 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fad2801adc0 con 0x7fad3819cad0 2024-05-08T05:58:32.675 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.675+0000 7fad3e6cb700 1 -- 172.21.15.192:0/3392158130 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad381037d0 con 0x7fad3819cad0 2024-05-08T05:58:32.675 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.675+0000 7fad357fa700 1 -- 172.21.15.192:0/3392158130 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7fad28018020 con 0x7fad3819cad0 2024-05-08T05:58:32.675 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.675+0000 7fad357fa700 1 --2- 172.21.15.192:0/3392158130 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fad2403e2b0 0x7fad24040770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:32.675 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.675+0000 7fad357fa700 1 -- 172.21.15.192:0/3392158130 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fad280521f0 con 0x7fad3819cad0 2024-05-08T05:58:32.676 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.676+0000 7fad37fff700 1 --2- 172.21.15.192:0/3392158130 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fad2403e2b0 0x7fad24040770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:32.676 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.677+0000 7fad37fff700 1 --2- 172.21.15.192:0/3392158130 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fad2403e2b0 0x7fad24040770 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fad2c005ed0 tx=0x7fad2c005ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:32.690 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.691+0000 7fad357fa700 1 -- 172.21.15.192:0/3392158130 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fad2801f020 con 0x7fad3819cad0 2024-05-08T05:58:32.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:32 smithi192 ceph-mon[34664]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:32.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:32 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:58:32.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:32 smithi192 ceph-mon[34664]: mon.a calling monitor election 2024-05-08T05:58:32.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:32 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:58:32.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:32 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:58:32.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:32 smithi192 ceph-mon[34664]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:32.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:32 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:58:32.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:32 smithi192 ceph-mon[34664]: mon.c calling monitor election 2024-05-08T05:58:32.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:32 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:58:32.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:32 smithi192 ceph-mon[34664]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:32.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:32 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:58:32.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:32 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:58:32.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:32 smithi192 ceph-mon[34664]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T05:58:32.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:32 smithi192 ceph-mon[34664]: monmap epoch 2 2024-05-08T05:58:32.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:32 smithi192 ceph-mon[34664]: fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 2024-05-08T05:58:32.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:32 smithi192 ceph-mon[34664]: last_changed 2024-05-08T05:58:27.426103+0000 2024-05-08T05:58:32.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:32 smithi192 ceph-mon[34664]: created 2024-05-08T05:56:08.254675+0000 2024-05-08T05:58:32.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:32 smithi192 ceph-mon[34664]: min_mon_release 19 (squid) 2024-05-08T05:58:32.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:32 smithi192 ceph-mon[34664]: election_strategy: 1 2024-05-08T05:58:32.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:32 smithi192 ceph-mon[34664]: 0: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.a 2024-05-08T05:58:32.803 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:32 smithi192 ceph-mon[34664]: 1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.c 2024-05-08T05:58:32.803 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:32 smithi192 ceph-mon[34664]: fsmap 2024-05-08T05:58:32.803 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:32 smithi192 ceph-mon[34664]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T05:58:32.803 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:32 smithi192 ceph-mon[34664]: mgrmap e13: x(active, since 61s) 2024-05-08T05:58:32.803 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:32 smithi192 ceph-mon[34664]: overall HEALTH_OK 2024-05-08T05:58:32.803 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:32 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:32.803 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:32 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:32.803 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:32 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:32.803 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:32 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:58:32.803 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:32 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:32.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:32 smithi123 ceph-mon[29787]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:32.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:32 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:58:32.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:32 smithi123 ceph-mon[29787]: mon.a calling monitor election 2024-05-08T05:58:32.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:32 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:58:32.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:32 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:58:32.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:32 smithi123 ceph-mon[29787]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:32.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:32 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:58:32.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:32 smithi123 ceph-mon[29787]: mon.c calling monitor election 2024-05-08T05:58:32.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:32 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:58:32.958 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:32 smithi123 ceph-mon[29787]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:32.958 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:32 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:58:32.958 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:32 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:58:32.958 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:32 smithi123 ceph-mon[29787]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T05:58:32.958 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:32 smithi123 ceph-mon[29787]: monmap epoch 2 2024-05-08T05:58:32.958 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:32 smithi123 ceph-mon[29787]: fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 2024-05-08T05:58:32.958 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:32 smithi123 ceph-mon[29787]: last_changed 2024-05-08T05:58:27.426103+0000 2024-05-08T05:58:32.958 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:32 smithi123 ceph-mon[29787]: created 2024-05-08T05:56:08.254675+0000 2024-05-08T05:58:32.958 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:32 smithi123 ceph-mon[29787]: min_mon_release 19 (squid) 2024-05-08T05:58:32.958 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:32 smithi123 ceph-mon[29787]: election_strategy: 1 2024-05-08T05:58:32.958 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:32 smithi123 ceph-mon[29787]: 0: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.a 2024-05-08T05:58:32.958 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:32 smithi123 ceph-mon[29787]: 1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.c 2024-05-08T05:58:32.958 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:32 smithi123 ceph-mon[29787]: fsmap 2024-05-08T05:58:32.958 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:32 smithi123 ceph-mon[29787]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T05:58:32.959 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:32 smithi123 ceph-mon[29787]: mgrmap e13: x(active, since 61s) 2024-05-08T05:58:32.959 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:32 smithi123 ceph-mon[29787]: overall HEALTH_OK 2024-05-08T05:58:32.959 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:32 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:32.959 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:32 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:32.959 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:32 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:32.959 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:32 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:58:32.959 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:32 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:32.979 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.979+0000 7fad3e6cb700 1 -- 172.21.15.192:0/3392158130 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fad38197500 con 0x7fad3819cad0 2024-05-08T05:58:32.981 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.980+0000 7fad357fa700 1 -- 172.21.15.192:0/3392158130 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7fad28018b80 con 0x7fad3819cad0 2024-05-08T05:58:32.981 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T05:58:32.981 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":2,"fsid":"85cfb2e6-0cff-11ef-bc97-c7b262605968","modified":"2024-05-08T05:58:27.426103Z","created":"2024-05-08T05:56:08.254675Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:3300","nonce":0},{"type":"v1","addr":"172.21.15.123:6789","nonce":0}]},"addr":"172.21.15.123:6789/0","public_addr":"172.21.15.123:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-08T05:58:32.983 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.983+0000 7fad3e6cb700 1 -- 172.21.15.192:0/3392158130 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fad2403e2b0 msgr2=0x7fad24040770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:32.983 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.983+0000 7fad3e6cb700 1 --2- 172.21.15.192:0/3392158130 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fad2403e2b0 0x7fad24040770 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fad2c005ed0 tx=0x7fad2c005ad0 comp rx=0 tx=0).stop 2024-05-08T05:58:32.983 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.983+0000 7fad3e6cb700 1 -- 172.21.15.192:0/3392158130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad3819cad0 msgr2=0x7fad38196660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:32.983 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.983+0000 7fad3e6cb700 1 --2- 172.21.15.192:0/3392158130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad3819cad0 0x7fad38196660 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fad2800b4d0 tx=0x7fad2800b890 comp rx=0 tx=0).stop 2024-05-08T05:58:32.984 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.983+0000 7fad3e6cb700 1 -- 172.21.15.192:0/3392158130 shutdown_connections 2024-05-08T05:58:32.984 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.983+0000 7fad3e6cb700 1 --2- 172.21.15.192:0/3392158130 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fad2403e2b0 0x7fad24040770 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:32.984 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.983+0000 7fad3e6cb700 1 --2- 172.21.15.192:0/3392158130 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad200057d0 0x7fad3819c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:32.984 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.983+0000 7fad3e6cb700 1 --2- 172.21.15.192:0/3392158130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad3819cad0 0x7fad38196660 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:32.984 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.983+0000 7fad3e6cb700 1 -- 172.21.15.192:0/3392158130 >> 172.21.15.192:0/3392158130 conn(0x7fad380782c0 msgr2=0x7fad380ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:32.984 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.983+0000 7fad3e6cb700 1 -- 172.21.15.192:0/3392158130 shutdown_connections 2024-05-08T05:58:32.984 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:32.983+0000 7fad3e6cb700 1 -- 172.21.15.192:0/3392158130 wait complete. 2024-05-08T05:58:32.984 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 2 2024-05-08T05:58:33.621 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:33 smithi192 ceph-mon[34664]: Deploying daemon mon.b on smithi163 2024-05-08T05:58:33.621 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:33 smithi192 ceph-mon[34664]: from='client.? 172.21.15.192:0/3392158130' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T05:58:33.621 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:33 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:58:33.651 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:33 smithi123 ceph-mon[29787]: Deploying daemon mon.b on smithi163 2024-05-08T05:58:33.651 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:33 smithi123 ceph-mon[29787]: from='client.? 172.21.15.192:0/3392158130' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T05:58:33.651 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:33 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:58:34.649 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-08T05:58:34.649 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-08T05:58:34.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:34 smithi192 ceph-mon[34664]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:34.892 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:58:34.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:34 smithi123 ceph-mon[29787]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:36.165 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.163+0000 7fed4026a700 1 -- 172.21.15.192:0/2551177178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed38102110 msgr2=0x7fed381024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:36.165 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.163+0000 7fed4026a700 1 --2- 172.21.15.192:0/2551177178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed38102110 0x7fed381024f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fed34009910 tx=0x7fed34009c20 comp rx=0 tx=0).stop 2024-05-08T05:58:36.165 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.163+0000 7fed4026a700 1 -- 172.21.15.192:0/2551177178 shutdown_connections 2024-05-08T05:58:36.165 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.163+0000 7fed4026a700 1 --2- 172.21.15.192:0/2551177178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed38102110 0x7fed381024f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:36.165 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.163+0000 7fed4026a700 1 -- 172.21.15.192:0/2551177178 >> 172.21.15.192:0/2551177178 conn(0x7fed380fd9e0 msgr2=0x7fed380ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:36.165 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.163+0000 7fed4026a700 1 -- 172.21.15.192:0/2551177178 shutdown_connections 2024-05-08T05:58:36.165 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.163+0000 7fed4026a700 1 -- 172.21.15.192:0/2551177178 wait complete. 2024-05-08T05:58:36.165 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.163+0000 7fed4026a700 1 Processor -- start 2024-05-08T05:58:36.165 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.163+0000 7fed4026a700 1 -- start start 2024-05-08T05:58:36.165 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.164+0000 7fed4026a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed38102110 0x7fed3819e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:36.166 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.164+0000 7fed4026a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed3819ecd0 0x7fed381989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:36.166 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.164+0000 7fed4026a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fed3810c390 con 0x7fed38102110 2024-05-08T05:58:36.166 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.164+0000 7fed4026a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fed3810c210 con 0x7fed3819ecd0 2024-05-08T05:58:36.166 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.164+0000 7fed3d805700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed3819ecd0 0x7fed381989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:36.166 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.164+0000 7fed3d805700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed3819ecd0 0x7fed381989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:60318/0 (socket says 172.21.15.192:60318) 2024-05-08T05:58:36.166 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.164+0000 7fed3d805700 1 -- 172.21.15.192:0/3252284663 learned_addr learned my addr 172.21.15.192:0/3252284663 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-08T05:58:36.166 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.164+0000 7fed3d805700 1 -- 172.21.15.192:0/3252284663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed3819ecd0 msgr2=0x7fed381989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-08T05:58:36.166 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.164+0000 7fed3d805700 1 -- 172.21.15.192:0/3252284663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed3819ecd0 msgr2=0x7fed381989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T05:58:36.166 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.164+0000 7fed3d805700 1 --2- 172.21.15.192:0/3252284663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed3819ecd0 0x7fed381989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T05:58:36.166 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.164+0000 7fed3e006700 1 --2- 172.21.15.192:0/3252284663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed38102110 0x7fed3819e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:36.166 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.164+0000 7fed3d805700 1 --2- 172.21.15.192:0/3252284663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed3819ecd0 0x7fed381989a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T05:58:36.166 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.165+0000 7fed3e006700 1 -- 172.21.15.192:0/3252284663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed3819ecd0 msgr2=0x7fed381989a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T05:58:36.167 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.165+0000 7fed3e006700 1 --2- 172.21.15.192:0/3252284663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed3819ecd0 0x7fed381989a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:36.167 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.165+0000 7fed3e006700 1 -- 172.21.15.192:0/3252284663 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed3819f310 con 0x7fed38102110 2024-05-08T05:58:36.167 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.165+0000 7fed3e006700 1 --2- 172.21.15.192:0/3252284663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed38102110 0x7fed3819e790 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fed34014000 tx=0x7fed34005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:36.168 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.165+0000 7fed2f7fe700 1 -- 172.21.15.192:0/3252284663 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fed34005dc0 con 0x7fed38102110 2024-05-08T05:58:36.168 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.165+0000 7fed2f7fe700 1 -- 172.21.15.192:0/3252284663 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed3400b750 con 0x7fed38102110 2024-05-08T05:58:36.168 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.165+0000 7fed4026a700 1 -- 172.21.15.192:0/3252284663 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed38198ee0 con 0x7fed38102110 2024-05-08T05:58:36.168 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.165+0000 7fed2f7fe700 1 -- 172.21.15.192:0/3252284663 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fed340039a0 con 0x7fed38102110 2024-05-08T05:58:36.168 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.166+0000 7fed4026a700 1 -- 172.21.15.192:0/3252284663 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed38199470 con 0x7fed38102110 2024-05-08T05:58:36.168 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.166+0000 7fed4026a700 1 -- 172.21.15.192:0/3252284663 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed381997e0 con 0x7fed38102110 2024-05-08T05:58:36.168 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.167+0000 7fed2f7fe700 1 -- 172.21.15.192:0/3252284663 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7fed34029070 con 0x7fed38102110 2024-05-08T05:58:36.168 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.167+0000 7fed2f7fe700 1 --2- 172.21.15.192:0/3252284663 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fed2403e2b0 0x7fed24040770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:36.169 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.168+0000 7fed2f7fe700 1 -- 172.21.15.192:0/3252284663 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fed3401e070 con 0x7fed38102110 2024-05-08T05:58:36.169 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.168+0000 7fed3d805700 1 --2- 172.21.15.192:0/3252284663 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fed2403e2b0 0x7fed24040770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:36.169 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.169+0000 7fed3d805700 1 --2- 172.21.15.192:0/3252284663 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fed2403e2b0 0x7fed24040770 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fed28009180 tx=0x7fed28008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:36.184 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.184+0000 7fed2f7fe700 1 -- 172.21.15.192:0/3252284663 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fed34030020 con 0x7fed38102110 2024-05-08T05:58:36.215 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 podman[34499]: 2024-05-08 05:58:36.056313045 +0000 UTC m=+0.018416575 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:58:36.392 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.392+0000 7fed2f7fe700 1 -- 172.21.15.192:0/3252284663 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed34003b40 con 0x7fed38102110 2024-05-08T05:58:36.476 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:36.475+0000 7fed4026a700 1 -- 172.21.15.192:0/3252284663 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fed38199e80 con 0x7fed38102110 2024-05-08T05:58:36.481 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 podman[34499]: 2024-05-08 05:58:36.333788564 +0000 UTC m=+0.295892076 container init 72a80a2675fe76851a0b33442a0c56c468c74517efd98d98c6f35e97024d9f3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mon-b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f) 2024-05-08T05:58:36.481 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 podman[34499]: 2024-05-08 05:58:36.336817215 +0000 UTC m=+0.298920729 container start 72a80a2675fe76851a0b33442a0c56c468c74517efd98d98c6f35e97024d9f3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mon-b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-08T05:58:36.481 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: set uid:gid to 167:167 (ceph:ceph) 2024-05-08T05:58:36.481 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: ceph version 19.0.0-3538-gbee299f4 (bee299f4ac344e4c45e550d7a80abcadd7bc3d5a) squid (dev), process ceph-mon, pid 2 2024-05-08T05:58:36.481 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: pidfile_write: ignore empty --pid-file 2024-05-08T05:58:36.481 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: load: jerasure load: lrc 2024-05-08T05:58:36.481 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: RocksDB version: 7.9.2 2024-05-08T05:58:36.482 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Git sha 0 2024-05-08T05:58:36.482 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Compile date 2024-05-07 11:18:56 2024-05-08T05:58:36.482 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: DB SUMMARY 2024-05-08T05:58:36.482 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: DB Session ID: E7UELEVKLWMGS7FWETM4 2024-05-08T05:58:36.482 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: CURRENT file: CURRENT 2024-05-08T05:58:36.482 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: IDENTITY file: IDENTITY 2024-05-08T05:58:36.482 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-08T05:58:36.482 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-05-08T05:58:36.482 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-05-08T05:58:36.482 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.error_if_exists: 0 2024-05-08T05:58:36.482 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.create_if_missing: 0 2024-05-08T05:58:36.482 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.paranoid_checks: 1 2024-05-08T05:58:36.482 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-08T05:58:36.482 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-08T05:58:36.483 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-08T05:58:36.483 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.env: 0x5574dbba3cc0 2024-05-08T05:58:36.483 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.fs: PosixFileSystem 2024-05-08T05:58:36.483 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.info_log: 0x5574de219da0 2024-05-08T05:58:36.483 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.max_file_opening_threads: 16 2024-05-08T05:58:36.484 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.statistics: (nil) 2024-05-08T05:58:36.484 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.use_fsync: 0 2024-05-08T05:58:36.484 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.max_log_file_size: 0 2024-05-08T05:58:36.484 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-08T05:58:36.484 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-08T05:58:36.484 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.keep_log_file_num: 1000 2024-05-08T05:58:36.484 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.recycle_log_file_num: 0 2024-05-08T05:58:36.484 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.allow_fallocate: 1 2024-05-08T05:58:36.484 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.allow_mmap_reads: 0 2024-05-08T05:58:36.484 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.allow_mmap_writes: 0 2024-05-08T05:58:36.484 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.use_direct_reads: 0 2024-05-08T05:58:36.484 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-08T05:58:36.484 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.create_missing_column_families: 0 2024-05-08T05:58:36.485 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.db_log_dir: 2024-05-08T05:58:36.485 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.wal_dir: 2024-05-08T05:58:36.485 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-08T05:58:36.485 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-08T05:58:36.485 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-08T05:58:36.485 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-08T05:58:36.485 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-08T05:58:36.485 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-08T05:58:36.485 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.advise_random_on_open: 1 2024-05-08T05:58:36.485 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.db_write_buffer_size: 0 2024-05-08T05:58:36.485 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.write_buffer_manager: 0x5574de1eb0e0 2024-05-08T05:58:36.485 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-08T05:58:36.485 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-08T05:58:36.486 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-08T05:58:36.486 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.rate_limiter: (nil) 2024-05-08T05:58:36.486 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-08T05:58:36.486 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.wal_recovery_mode: 2 2024-05-08T05:58:36.486 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.enable_thread_tracking: 0 2024-05-08T05:58:36.486 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.enable_pipelined_write: 0 2024-05-08T05:58:36.486 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.unordered_write: 0 2024-05-08T05:58:36.486 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-08T05:58:36.486 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-08T05:58:36.486 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-08T05:58:36.486 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-08T05:58:36.486 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.row_cache: None 2024-05-08T05:58:36.486 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.wal_filter: None 2024-05-08T05:58:36.486 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-08T05:58:36.486 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.allow_ingest_behind: 0 2024-05-08T05:58:36.487 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.two_write_queues: 0 2024-05-08T05:58:36.487 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.manual_wal_flush: 0 2024-05-08T05:58:36.487 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.wal_compression: 0 2024-05-08T05:58:36.487 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.atomic_flush: 0 2024-05-08T05:58:36.487 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-08T05:58:36.487 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-08T05:58:36.487 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-08T05:58:36.487 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.log_readahead_size: 0 2024-05-08T05:58:36.487 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-08T05:58:36.487 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.best_efforts_recovery: 0 2024-05-08T05:58:36.487 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-08T05:58:36.487 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-08T05:58:36.487 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.allow_data_in_errors: 0 2024-05-08T05:58:36.487 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.db_host_id: __hostname__ 2024-05-08T05:58:36.487 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-08T05:58:36.488 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.max_background_jobs: 2 2024-05-08T05:58:36.488 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.max_background_compactions: -1 2024-05-08T05:58:36.488 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.max_subcompactions: 1 2024-05-08T05:58:36.488 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-08T05:58:36.488 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-08T05:58:36.488 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-08T05:58:36.488 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.max_total_wal_size: 0 2024-05-08T05:58:36.488 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-08T05:58:36.488 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-08T05:58:36.488 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-08T05:58:36.488 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-08T05:58:36.488 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.max_open_files: -1 2024-05-08T05:58:36.488 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.bytes_per_sync: 0 2024-05-08T05:58:36.488 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-08T05:58:36.489 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-08T05:58:36.489 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.compaction_readahead_size: 0 2024-05-08T05:58:36.489 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.max_background_flushes: -1 2024-05-08T05:58:36.489 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Compression algorithms supported: 2024-05-08T05:58:36.489 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-08T05:58:36.489 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: kZSTD supported: 0 2024-05-08T05:58:36.489 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: kXpressCompression supported: 0 2024-05-08T05:58:36.489 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: kLZ4HCCompression supported: 1 2024-05-08T05:58:36.489 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: kZlibCompression supported: 1 2024-05-08T05:58:36.489 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: kSnappyCompression supported: 1 2024-05-08T05:58:36.489 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: kLZ4Compression supported: 1 2024-05-08T05:58:36.489 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: kBZip2Compression supported: 0 2024-05-08T05:58:36.489 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-08T05:58:36.489 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-08T05:58:36.489 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-05-08T05:58:36.490 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-08T05:58:36.490 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-08T05:58:36.490 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.merge_operator: 2024-05-08T05:58:36.490 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.compaction_filter: None 2024-05-08T05:58:36.490 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.compaction_filter_factory: None 2024-05-08T05:58:36.490 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.sst_partitioner_factory: None 2024-05-08T05:58:36.490 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-08T05:58:36.490 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-08T05:58:36.490 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5574de219f60) 2024-05-08T05:58:36.490 INFO:journalctl@ceph.mon.b.smithi163.stdout: cache_index_and_filter_blocks: 1 2024-05-08T05:58:36.490 INFO:journalctl@ceph.mon.b.smithi163.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-08T05:58:36.490 INFO:journalctl@ceph.mon.b.smithi163.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-08T05:58:36.490 INFO:journalctl@ceph.mon.b.smithi163.stdout: pin_top_level_index_and_filter: 1 2024-05-08T05:58:36.490 INFO:journalctl@ceph.mon.b.smithi163.stdout: index_type: 0 2024-05-08T05:58:36.491 INFO:journalctl@ceph.mon.b.smithi163.stdout: data_block_index_type: 0 2024-05-08T05:58:36.491 INFO:journalctl@ceph.mon.b.smithi163.stdout: index_shortening: 1 2024-05-08T05:58:36.491 INFO:journalctl@ceph.mon.b.smithi163.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-08T05:58:36.491 INFO:journalctl@ceph.mon.b.smithi163.stdout: checksum: 4 2024-05-08T05:58:36.491 INFO:journalctl@ceph.mon.b.smithi163.stdout: no_block_cache: 0 2024-05-08T05:58:36.491 INFO:journalctl@ceph.mon.b.smithi163.stdout: block_cache: 0x5574de21d1f0 2024-05-08T05:58:36.491 INFO:journalctl@ceph.mon.b.smithi163.stdout: block_cache_name: BinnedLRUCache 2024-05-08T05:58:36.491 INFO:journalctl@ceph.mon.b.smithi163.stdout: block_cache_options: 2024-05-08T05:58:36.491 INFO:journalctl@ceph.mon.b.smithi163.stdout: capacity : 536870912 2024-05-08T05:58:36.491 INFO:journalctl@ceph.mon.b.smithi163.stdout: num_shard_bits : 4 2024-05-08T05:58:36.491 INFO:journalctl@ceph.mon.b.smithi163.stdout: strict_capacity_limit : 0 2024-05-08T05:58:36.491 INFO:journalctl@ceph.mon.b.smithi163.stdout: high_pri_pool_ratio: 0.000 2024-05-08T05:58:36.491 INFO:journalctl@ceph.mon.b.smithi163.stdout: block_cache_compressed: (nil) 2024-05-08T05:58:36.491 INFO:journalctl@ceph.mon.b.smithi163.stdout: persistent_cache: (nil) 2024-05-08T05:58:36.491 INFO:journalctl@ceph.mon.b.smithi163.stdout: block_size: 4096 2024-05-08T05:58:36.492 INFO:journalctl@ceph.mon.b.smithi163.stdout: block_size_deviation: 10 2024-05-08T05:58:36.492 INFO:journalctl@ceph.mon.b.smithi163.stdout: block_restart_interval: 16 2024-05-08T05:58:36.492 INFO:journalctl@ceph.mon.b.smithi163.stdout: index_block_restart_interval: 1 2024-05-08T05:58:36.492 INFO:journalctl@ceph.mon.b.smithi163.stdout: metadata_block_size: 4096 2024-05-08T05:58:36.492 INFO:journalctl@ceph.mon.b.smithi163.stdout: partition_filters: 0 2024-05-08T05:58:36.492 INFO:journalctl@ceph.mon.b.smithi163.stdout: use_delta_encoding: 1 2024-05-08T05:58:36.492 INFO:journalctl@ceph.mon.b.smithi163.stdout: filter_policy: bloomfilter 2024-05-08T05:58:36.492 INFO:journalctl@ceph.mon.b.smithi163.stdout: whole_key_filtering: 1 2024-05-08T05:58:36.492 INFO:journalctl@ceph.mon.b.smithi163.stdout: verify_compression: 0 2024-05-08T05:58:36.492 INFO:journalctl@ceph.mon.b.smithi163.stdout: read_amp_bytes_per_bit: 0 2024-05-08T05:58:36.492 INFO:journalctl@ceph.mon.b.smithi163.stdout: format_version: 5 2024-05-08T05:58:36.492 INFO:journalctl@ceph.mon.b.smithi163.stdout: enable_index_compression: 1 2024-05-08T05:58:36.492 INFO:journalctl@ceph.mon.b.smithi163.stdout: block_align: 0 2024-05-08T05:58:36.492 INFO:journalctl@ceph.mon.b.smithi163.stdout: max_auto_readahead_size: 262144 2024-05-08T05:58:36.493 INFO:journalctl@ceph.mon.b.smithi163.stdout: prepopulate_block_cache: 0 2024-05-08T05:58:36.493 INFO:journalctl@ceph.mon.b.smithi163.stdout: initial_auto_readahead_size: 8192 2024-05-08T05:58:36.493 INFO:journalctl@ceph.mon.b.smithi163.stdout: num_file_reads_for_auto_readahead: 2 2024-05-08T05:58:36.493 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.write_buffer_size: 33554432 2024-05-08T05:58:36.493 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.max_write_buffer_number: 2 2024-05-08T05:58:36.493 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.compression: NoCompression 2024-05-08T05:58:36.493 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.bottommost_compression: Disabled 2024-05-08T05:58:36.493 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.prefix_extractor: nullptr 2024-05-08T05:58:36.493 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-08T05:58:36.493 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.num_levels: 7 2024-05-08T05:58:36.493 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-08T05:58:36.493 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-08T05:58:36.493 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-08T05:58:36.493 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-08T05:58:36.493 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-08T05:58:36.493 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-08T05:58:36.493 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-08T05:58:36.494 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-08T05:58:36.494 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-08T05:58:36.494 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-08T05:58:36.494 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-08T05:58:36.494 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-08T05:58:36.494 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-08T05:58:36.494 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.compression_opts.level: 32767 2024-05-08T05:58:36.494 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.compression_opts.strategy: 0 2024-05-08T05:58:36.494 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-08T05:58:36.494 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-08T05:58:36.494 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-08T05:58:36.494 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-08T05:58:36.494 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.compression_opts.enabled: false 2024-05-08T05:58:36.494 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-08T05:58:36.495 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-08T05:58:36.495 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-08T05:58:36.495 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-08T05:58:36.495 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.target_file_size_base: 67108864 2024-05-08T05:58:36.495 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-08T05:58:36.495 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-08T05:58:36.495 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-08T05:58:36.495 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-08T05:58:36.495 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-08T05:58:36.495 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-08T05:58:36.496 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-08T05:58:36.497 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-08T05:58:36.497 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-08T05:58:36.497 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-08T05:58:36.497 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-08T05:58:36.497 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-08T05:58:36.497 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-08T05:58:36.497 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-08T05:58:36.497 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.arena_block_size: 1048576 2024-05-08T05:58:36.497 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-08T05:58:36.497 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-08T05:58:36.497 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.disable_auto_compactions: 0 2024-05-08T05:58:36.497 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-08T05:58:36.497 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-08T05:58:36.497 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-08T05:58:36.497 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-08T05:58:36.498 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-08T05:58:36.498 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-08T05:58:36.498 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-08T05:58:36.498 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-08T05:58:36.498 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-08T05:58:36.498 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-08T05:58:36.498 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-08T05:58:36.498 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.inplace_update_support: 0 2024-05-08T05:58:36.498 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-08T05:58:36.498 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-08T05:58:36.498 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-08T05:58:36.498 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-08T05:58:36.498 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.bloom_locality: 0 2024-05-08T05:58:36.498 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.max_successive_merges: 0 2024-05-08T05:58:36.499 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-08T05:58:36.499 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.paranoid_file_checks: 0 2024-05-08T05:58:36.499 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.force_consistency_checks: 1 2024-05-08T05:58:36.499 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.report_bg_io_stats: 0 2024-05-08T05:58:36.499 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.ttl: 2592000 2024-05-08T05:58:36.499 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-08T05:58:36.499 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-08T05:58:36.499 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-08T05:58:36.499 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.enable_blob_files: false 2024-05-08T05:58:36.499 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.min_blob_size: 0 2024-05-08T05:58:36.499 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.blob_file_size: 268435456 2024-05-08T05:58:36.499 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-08T05:58:36.499 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-08T05:58:36.499 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-08T05:58:36.499 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-08T05:58:36.499 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-08T05:58:36.500 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.blob_file_starting_level: 0 2024-05-08T05:58:36.500 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-08T05:58:36.500 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-08T05:58:36.500 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-08T05:58:36.500 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d392ff15-c004-4baf-89ee-36a796534c18 2024-05-08T05:58:36.500 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715147916366967, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-08T05:58:36.500 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-08T05:58:36.500 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715147916367389, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715147916, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d392ff15-c004-4baf-89ee-36a796534c18", "db_session_id": "E7UELEVKLWMGS7FWETM4", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-08T05:58:36.500 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715147916367469, "job": 1, "event": "recovery_finished"} 2024-05-08T05:58:36.500 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-08T05:58:36.500 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-08T05:58:36.500 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5574de33a000 2024-05-08T05:58:36.500 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: DB pointer 0x5574de326000 2024-05-08T05:58:36.500 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-08T05:58:36.500 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-08T05:58:36.500 INFO:journalctl@ceph.mon.b.smithi163.stdout: ** DB Stats ** 2024-05-08T05:58:36.501 INFO:journalctl@ceph.mon.b.smithi163.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T05:58:36.501 INFO:journalctl@ceph.mon.b.smithi163.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-08T05:58:36.501 INFO:journalctl@ceph.mon.b.smithi163.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T05:58:36.501 INFO:journalctl@ceph.mon.b.smithi163.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T05:58:36.501 INFO:journalctl@ceph.mon.b.smithi163.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-08T05:58:36.501 INFO:journalctl@ceph.mon.b.smithi163.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T05:58:36.501 INFO:journalctl@ceph.mon.b.smithi163.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T05:58:36.501 INFO:journalctl@ceph.mon.b.smithi163.stdout: 2024-05-08T05:58:36.501 INFO:journalctl@ceph.mon.b.smithi163.stdout: ** Compaction Stats [default] ** 2024-05-08T05:58:36.501 INFO:journalctl@ceph.mon.b.smithi163.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T05:58:36.501 INFO:journalctl@ceph.mon.b.smithi163.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-08T05:58:36.501 INFO:journalctl@ceph.mon.b.smithi163.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T05:58:36.501 INFO:journalctl@ceph.mon.b.smithi163.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T05:58:36.501 INFO:journalctl@ceph.mon.b.smithi163.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T05:58:36.502 INFO:journalctl@ceph.mon.b.smithi163.stdout: 2024-05-08T05:58:36.502 INFO:journalctl@ceph.mon.b.smithi163.stdout: ** Compaction Stats [default] ** 2024-05-08T05:58:36.502 INFO:journalctl@ceph.mon.b.smithi163.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T05:58:36.502 INFO:journalctl@ceph.mon.b.smithi163.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-08T05:58:36.502 INFO:journalctl@ceph.mon.b.smithi163.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T05:58:36.502 INFO:journalctl@ceph.mon.b.smithi163.stdout: 2024-05-08T05:58:36.502 INFO:journalctl@ceph.mon.b.smithi163.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-08T05:58:36.502 INFO:journalctl@ceph.mon.b.smithi163.stdout: 2024-05-08T05:58:36.502 INFO:journalctl@ceph.mon.b.smithi163.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T05:58:36.502 INFO:journalctl@ceph.mon.b.smithi163.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-08T05:58:36.502 INFO:journalctl@ceph.mon.b.smithi163.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-08T05:58:36.502 INFO:journalctl@ceph.mon.b.smithi163.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-08T05:58:36.502 INFO:journalctl@ceph.mon.b.smithi163.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-08T05:58:36.502 INFO:journalctl@ceph.mon.b.smithi163.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-08T05:58:36.502 INFO:journalctl@ceph.mon.b.smithi163.stdout: Cumulative compaction: 0.00 GB write, 0.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T05:58:36.503 INFO:journalctl@ceph.mon.b.smithi163.stdout: Interval compaction: 0.00 GB write, 0.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T05:58:36.503 INFO:journalctl@ceph.mon.b.smithi163.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-08T05:58:36.503 INFO:journalctl@ceph.mon.b.smithi163.stdout: Block cache BinnedLRUCache@0x5574de21d1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-05-08T05:58:36.503 INFO:journalctl@ceph.mon.b.smithi163.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-08T05:58:36.503 INFO:journalctl@ceph.mon.b.smithi163.stdout: 2024-05-08T05:58:36.503 INFO:journalctl@ceph.mon.b.smithi163.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-08T05:58:36.503 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-05-08T05:58:36.503 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: using public_addr v2:172.21.15.163:0/0 -> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] 2024-05-08T05:58:36.503 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: starting mon.b rank -1 at public addrs [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] at bind addrs [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 2024-05-08T05:58:36.503 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: mon.b@-1(???) e0 preinit fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 2024-05-08T05:58:36.503 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: mon.b@-1(synchronizing).mds e1 new map 2024-05-08T05:58:36.503 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: mon.b@-1(synchronizing).mds e1 print_map 2024-05-08T05:58:36.503 INFO:journalctl@ceph.mon.b.smithi163.stdout: e1 2024-05-08T05:58:36.504 INFO:journalctl@ceph.mon.b.smithi163.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T05:58:36.504 INFO:journalctl@ceph.mon.b.smithi163.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-08T05:58:36.504 INFO:journalctl@ceph.mon.b.smithi163.stdout: legacy client fscid: -1 2024-05-08T05:58:36.504 INFO:journalctl@ceph.mon.b.smithi163.stdout: 2024-05-08T05:58:36.504 INFO:journalctl@ceph.mon.b.smithi163.stdout: No filesystems configured 2024-05-08T05:58:36.504 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-08T05:58:36.504 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-08T05:58:36.504 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-08T05:58:36.504 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-08T05:58:36.504 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-08T05:58:36.504 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-08T05:58:36.504 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-08T05:58:36.504 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T05:58:36.504 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T05:58:36.504 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T05:58:36.504 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:36.505 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:36.505 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:36.505 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: Added host smithi163 2024-05-08T05:58:36.505 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:58:36.505 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:36.505 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:36.505 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:36.505 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:36.505 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:58:36.505 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:36.505 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:36.505 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:36.505 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:36.505 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:36.505 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:36.506 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi163", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:58:36.506 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:36.506 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:58:36.506 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: Updating smithi163:/etc/ceph/ceph.conf 2024-05-08T05:58:36.506 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: Updating smithi163:/var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:58:36.506 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: Updating smithi163:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:58:36.506 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: Updating smithi163:/var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:58:36.506 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:36.506 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:36.506 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi192", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:58:36.506 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:36.506 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:36.506 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: Deploying cephadm binary to smithi192 2024-05-08T05:58:36.506 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:36.507 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:36.507 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:36.507 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: Added host smithi192 2024-05-08T05:58:36.507 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:58:36.507 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:36.507 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:36.507 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:36.507 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:36.507 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:58:36.507 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:36.507 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:36.507 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:36.507 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:36.507 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:36.507 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:36.507 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi192", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:58:36.508 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:36.508 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:58:36.508 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: Updating smithi192:/etc/ceph/ceph.conf 2024-05-08T05:58:36.508 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: Updating smithi192:/var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:58:36.508 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:58:36.508 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: Updating smithi192:/var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:58:36.508 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:36.508 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:36.508 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:36.508 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/2662668964' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-08T05:58:36.508 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:36.508 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/2662668964' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-08T05:58:36.508 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T05:58:36.508 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:36.509 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:36.509 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:58:36.509 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:36.509 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:58:36.509 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:36.509 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:58:36.509 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:36.509 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi123:172.21.15.123=a;smithi163:172.21.15.163=b;smithi192:172.21.15.192=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:58:36.509 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: Saving service mon spec with placement smithi123:172.21.15.123=a;smithi163:172.21.15.163=b;smithi192:172.21.15.192=c;count:3 2024-05-08T05:58:36.509 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: Deploying daemon mon.c on smithi192 2024-05-08T05:58:36.509 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:36.509 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:36.509 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='client.? 172.21.15.192:0/3982669295' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T05:58:36.509 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:36.509 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:58:36.509 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: mon.a calling monitor election 2024-05-08T05:58:36.510 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:58:36.510 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:58:36.510 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:36.510 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:58:36.510 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: mon.c calling monitor election 2024-05-08T05:58:36.510 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:58:36.510 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:36.510 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:58:36.510 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:58:36.510 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T05:58:36.510 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: monmap epoch 2 2024-05-08T05:58:36.510 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 2024-05-08T05:58:36.510 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: last_changed 2024-05-08T05:58:27.426103+0000 2024-05-08T05:58:36.510 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: created 2024-05-08T05:56:08.254675+0000 2024-05-08T05:58:36.510 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: min_mon_release 19 (squid) 2024-05-08T05:58:36.511 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: election_strategy: 1 2024-05-08T05:58:36.511 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: 0: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.a 2024-05-08T05:58:36.511 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: 1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.c 2024-05-08T05:58:36.511 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: fsmap 2024-05-08T05:58:36.511 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T05:58:36.511 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: mgrmap e13: x(active, since 61s) 2024-05-08T05:58:36.511 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: overall HEALTH_OK 2024-05-08T05:58:36.511 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:36.511 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:36.511 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:36.511 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:58:36.511 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:36.511 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: Deploying daemon mon.b on smithi163 2024-05-08T05:58:36.511 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='client.? 172.21.15.192:0/3392158130' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T05:58:36.511 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:58:36.511 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:36.511 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 ceph-mon[34520]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-08T05:58:36.512 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 bash[34499]: 72a80a2675fe76851a0b33442a0c56c468c74517efd98d98c6f35e97024d9f3c 2024-05-08T05:58:36.512 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:36 smithi163 systemd[1]: Started Ceph mon.b for 85cfb2e6-0cff-11ef-bc97-c7b262605968. 2024-05-08T05:58:41.418 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-08T05:58:41.419 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":3,"fsid":"85cfb2e6-0cff-11ef-bc97-c7b262605968","modified":"2024-05-08T05:58:36.390559Z","created":"2024-05-08T05:56:08.254675Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:3300","nonce":0},{"type":"v1","addr":"172.21.15.123:6789","nonce":0}]},"addr":"172.21.15.123:6789/0","public_addr":"172.21.15.123:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:3300","nonce":0},{"type":"v1","addr":"172.21.15.163:6789","nonce":0}]},"addr":"172.21.15.163:6789/0","public_addr":"172.21.15.163:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-05-08T05:58:41.419 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:41.417+0000 7fed2f7fe700 1 -- 172.21.15.192:0/3252284663 <== mon.0 v2:172.21.15.123:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1289 (secure 0 0 0) 0x7fed34013070 con 0x7fed38102110 2024-05-08T05:58:41.421 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:41.421+0000 7fed4026a700 1 -- 172.21.15.192:0/3252284663 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fed2403e2b0 msgr2=0x7fed24040770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:41.421 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:41.421+0000 7fed4026a700 1 --2- 172.21.15.192:0/3252284663 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fed2403e2b0 0x7fed24040770 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fed28009180 tx=0x7fed28008040 comp rx=0 tx=0).stop 2024-05-08T05:58:41.422 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:41.421+0000 7fed4026a700 1 -- 172.21.15.192:0/3252284663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed38102110 msgr2=0x7fed3819e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:41.422 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:41.421+0000 7fed4026a700 1 --2- 172.21.15.192:0/3252284663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed38102110 0x7fed3819e790 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fed34014000 tx=0x7fed34005030 comp rx=0 tx=0).stop 2024-05-08T05:58:41.422 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:41.421+0000 7fed4026a700 1 -- 172.21.15.192:0/3252284663 shutdown_connections 2024-05-08T05:58:41.422 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:41.421+0000 7fed4026a700 1 --2- 172.21.15.192:0/3252284663 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fed2403e2b0 0x7fed24040770 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:41.422 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:41.421+0000 7fed4026a700 1 --2- 172.21.15.192:0/3252284663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed38102110 0x7fed3819e790 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:41.422 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:41.421+0000 7fed4026a700 1 --2- 172.21.15.192:0/3252284663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed3819ecd0 0x7fed381989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:41.422 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:41.421+0000 7fed4026a700 1 -- 172.21.15.192:0/3252284663 >> 172.21.15.192:0/3252284663 conn(0x7fed380fd9e0 msgr2=0x7fed3806b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:41.422 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:41.421+0000 7fed4026a700 1 -- 172.21.15.192:0/3252284663 shutdown_connections 2024-05-08T05:58:41.422 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:41.421+0000 7fed4026a700 1 -- 172.21.15.192:0/3252284663 wait complete. 2024-05-08T05:58:41.423 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 3 2024-05-08T05:58:41.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:41 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:58:41.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:41 smithi163 ceph-mon[34520]: mon.a calling monitor election 2024-05-08T05:58:41.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:41 smithi163 ceph-mon[34520]: mon.c calling monitor election 2024-05-08T05:58:41.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:41 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:58:41.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:41 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:58:41.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:41 smithi163 ceph-mon[34520]: from='client.? 172.21.15.192:0/3252284663' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T05:58:41.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:41 smithi163 ceph-mon[34520]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:41.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:41 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:58:41.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:41 smithi163 ceph-mon[34520]: mon.b calling monitor election 2024-05-08T05:58:41.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:41 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:58:41.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:41 smithi163 ceph-mon[34520]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:41.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:41 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:58:41.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:41 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:58:41.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:41 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:58:41.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:41 smithi163 ceph-mon[34520]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-08T05:58:41.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:41 smithi163 ceph-mon[34520]: monmap epoch 3 2024-05-08T05:58:41.790 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:41 smithi163 ceph-mon[34520]: fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 2024-05-08T05:58:41.790 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:41 smithi163 ceph-mon[34520]: last_changed 2024-05-08T05:58:36.390559+0000 2024-05-08T05:58:41.790 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:41 smithi163 ceph-mon[34520]: created 2024-05-08T05:56:08.254675+0000 2024-05-08T05:58:41.790 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:41 smithi163 ceph-mon[34520]: min_mon_release 19 (squid) 2024-05-08T05:58:41.790 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:41 smithi163 ceph-mon[34520]: election_strategy: 1 2024-05-08T05:58:41.790 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:41 smithi163 ceph-mon[34520]: 0: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.a 2024-05-08T05:58:41.790 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:41 smithi163 ceph-mon[34520]: 1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.c 2024-05-08T05:58:41.790 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:41 smithi163 ceph-mon[34520]: 2: [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] mon.b 2024-05-08T05:58:41.790 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:41 smithi163 ceph-mon[34520]: fsmap 2024-05-08T05:58:41.790 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:41 smithi163 ceph-mon[34520]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T05:58:41.790 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:41 smithi163 ceph-mon[34520]: mgrmap e13: x(active, since 70s) 2024-05-08T05:58:41.790 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:41 smithi163 ceph-mon[34520]: overall HEALTH_OK 2024-05-08T05:58:41.790 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:41 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:41.790 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:41 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:41.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:41 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:58:41.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:41 smithi192 ceph-mon[34664]: mon.a calling monitor election 2024-05-08T05:58:41.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:41 smithi192 ceph-mon[34664]: mon.c calling monitor election 2024-05-08T05:58:41.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:41 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:58:41.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:41 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:58:41.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:41 smithi192 ceph-mon[34664]: from='client.? 172.21.15.192:0/3252284663' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T05:58:41.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:41 smithi192 ceph-mon[34664]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:41.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:41 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:58:41.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:41 smithi192 ceph-mon[34664]: mon.b calling monitor election 2024-05-08T05:58:41.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:41 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:58:41.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:41 smithi192 ceph-mon[34664]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:41.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:41 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:58:41.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:41 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:58:41.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:41 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:58:41.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:41 smithi192 ceph-mon[34664]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-08T05:58:41.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:41 smithi192 ceph-mon[34664]: monmap epoch 3 2024-05-08T05:58:41.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:41 smithi192 ceph-mon[34664]: fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 2024-05-08T05:58:41.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:41 smithi192 ceph-mon[34664]: last_changed 2024-05-08T05:58:36.390559+0000 2024-05-08T05:58:41.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:41 smithi192 ceph-mon[34664]: created 2024-05-08T05:56:08.254675+0000 2024-05-08T05:58:41.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:41 smithi192 ceph-mon[34664]: min_mon_release 19 (squid) 2024-05-08T05:58:41.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:41 smithi192 ceph-mon[34664]: election_strategy: 1 2024-05-08T05:58:41.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:41 smithi192 ceph-mon[34664]: 0: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.a 2024-05-08T05:58:41.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:41 smithi192 ceph-mon[34664]: 1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.c 2024-05-08T05:58:41.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:41 smithi192 ceph-mon[34664]: 2: [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] mon.b 2024-05-08T05:58:41.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:41 smithi192 ceph-mon[34664]: fsmap 2024-05-08T05:58:41.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:41 smithi192 ceph-mon[34664]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T05:58:41.803 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:41 smithi192 ceph-mon[34664]: mgrmap e13: x(active, since 70s) 2024-05-08T05:58:41.803 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:41 smithi192 ceph-mon[34664]: overall HEALTH_OK 2024-05-08T05:58:41.803 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:41 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:41.803 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:41 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:41.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:41 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:58:41.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:41 smithi123 ceph-mon[29787]: mon.a calling monitor election 2024-05-08T05:58:41.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:41 smithi123 ceph-mon[29787]: mon.c calling monitor election 2024-05-08T05:58:41.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:41 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:58:41.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:41 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:58:41.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:41 smithi123 ceph-mon[29787]: from='client.? 172.21.15.192:0/3252284663' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T05:58:41.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:41 smithi123 ceph-mon[29787]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:41.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:41 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:58:41.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:41 smithi123 ceph-mon[29787]: mon.b calling monitor election 2024-05-08T05:58:41.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:41 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:58:41.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:41 smithi123 ceph-mon[29787]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:41.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:41 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:58:41.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:41 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:58:41.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:41 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:58:41.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:41 smithi123 ceph-mon[29787]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-08T05:58:41.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:41 smithi123 ceph-mon[29787]: monmap epoch 3 2024-05-08T05:58:41.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:41 smithi123 ceph-mon[29787]: fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 2024-05-08T05:58:41.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:41 smithi123 ceph-mon[29787]: last_changed 2024-05-08T05:58:36.390559+0000 2024-05-08T05:58:41.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:41 smithi123 ceph-mon[29787]: created 2024-05-08T05:56:08.254675+0000 2024-05-08T05:58:41.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:41 smithi123 ceph-mon[29787]: min_mon_release 19 (squid) 2024-05-08T05:58:41.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:41 smithi123 ceph-mon[29787]: election_strategy: 1 2024-05-08T05:58:41.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:41 smithi123 ceph-mon[29787]: 0: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.a 2024-05-08T05:58:41.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:41 smithi123 ceph-mon[29787]: 1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.c 2024-05-08T05:58:41.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:41 smithi123 ceph-mon[29787]: 2: [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] mon.b 2024-05-08T05:58:41.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:41 smithi123 ceph-mon[29787]: fsmap 2024-05-08T05:58:41.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:41 smithi123 ceph-mon[29787]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T05:58:41.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:41 smithi123 ceph-mon[29787]: mgrmap e13: x(active, since 70s) 2024-05-08T05:58:41.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:41 smithi123 ceph-mon[29787]: overall HEALTH_OK 2024-05-08T05:58:41.958 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:41 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:41.958 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:41 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:42.144 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-08T05:58:42.144 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph config generate-minimal-conf 2024-05-08T05:58:42.575 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:58:42.749 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:42 smithi123 ceph-mon[29787]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:42.749 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:42 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:42.749 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:42 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:42.749 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:42 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:58:42.749 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:42 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:58:42.779 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:42 smithi163 ceph-mon[34520]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:42.779 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:42 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:42.779 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:42 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:42.779 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:42 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:58:42.779 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:42 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:58:42.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:42 smithi192 ceph-mon[34664]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:42.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:42 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:42.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:42 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:42.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:42 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:58:42.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:42 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:58:44.263 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:44 smithi123 ceph-mon[29787]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:44.263 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:44 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:44.264 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:44 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:44.264 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:44 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:44.264 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:44 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:44.264 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:44 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:44.264 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:44 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:58:44.264 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:44 smithi123 ceph-mon[29787]: Updating smithi123:/etc/ceph/ceph.conf 2024-05-08T05:58:44.264 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:44 smithi123 ceph-mon[29787]: Updating smithi163:/etc/ceph/ceph.conf 2024-05-08T05:58:44.264 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:44 smithi123 ceph-mon[29787]: Updating smithi192:/etc/ceph/ceph.conf 2024-05-08T05:58:44.264 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:44 smithi123 ceph-mon[29787]: Updating smithi163:/var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:58:44.264 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:44 smithi123 ceph-mon[29787]: Updating smithi192:/var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:58:44.264 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:44 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:44.264 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:44 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:44.264 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:44 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:44.264 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:44 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:44.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:44 smithi163 ceph-mon[34520]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:44.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:44 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:44.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:44 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:44.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:44 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:44.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:44 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:44.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:44 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:44.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:44 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:58:44.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:44 smithi163 ceph-mon[34520]: Updating smithi123:/etc/ceph/ceph.conf 2024-05-08T05:58:44.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:44 smithi163 ceph-mon[34520]: Updating smithi163:/etc/ceph/ceph.conf 2024-05-08T05:58:44.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:44 smithi163 ceph-mon[34520]: Updating smithi192:/etc/ceph/ceph.conf 2024-05-08T05:58:44.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:44 smithi163 ceph-mon[34520]: Updating smithi163:/var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:58:44.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:44 smithi163 ceph-mon[34520]: Updating smithi192:/var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:58:44.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:44 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:44.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:44 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:44.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:44 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:44.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:44 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:44.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:44 smithi192 ceph-mon[34664]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:44.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:44 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:44.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:44 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:44.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:44 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:44.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:44 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:44.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:44 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:44.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:44 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:58:44.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:44 smithi192 ceph-mon[34664]: Updating smithi123:/etc/ceph/ceph.conf 2024-05-08T05:58:44.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:44 smithi192 ceph-mon[34664]: Updating smithi163:/etc/ceph/ceph.conf 2024-05-08T05:58:44.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:44 smithi192 ceph-mon[34664]: Updating smithi192:/etc/ceph/ceph.conf 2024-05-08T05:58:44.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:44 smithi192 ceph-mon[34664]: Updating smithi163:/var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:58:44.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:44 smithi192 ceph-mon[34664]: Updating smithi192:/var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:58:44.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:44 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:44.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:44 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:44.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:44 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:44.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:44 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:44.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.714+0000 7f7972200700 1 -- 172.21.15.123:0/159519428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f796c1052f0 msgr2=0x7f796c1056d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:44.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.714+0000 7f7972200700 1 --2- 172.21.15.123:0/159519428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f796c1052f0 0x7f796c1056d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f795c009910 tx=0x7f795c009c20 comp rx=0 tx=0).stop 2024-05-08T05:58:44.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.714+0000 7f7972200700 1 -- 172.21.15.123:0/159519428 shutdown_connections 2024-05-08T05:58:44.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.714+0000 7f7972200700 1 --2- 172.21.15.123:0/159519428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f796c1052f0 0x7f796c1056d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:44.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.714+0000 7f7972200700 1 -- 172.21.15.123:0/159519428 >> 172.21.15.123:0/159519428 conn(0x7f796c100bc0 msgr2=0x7f796c102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:44.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.715+0000 7f7972200700 1 -- 172.21.15.123:0/159519428 shutdown_connections 2024-05-08T05:58:44.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.715+0000 7f7972200700 1 -- 172.21.15.123:0/159519428 wait complete. 2024-05-08T05:58:44.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.716+0000 7f7972200700 1 Processor -- start 2024-05-08T05:58:44.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.716+0000 7f7972200700 1 -- start start 2024-05-08T05:58:44.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.716+0000 7f7972200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f796c1052f0 0x7f796c1a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:44.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.716+0000 7f7972200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f796c1a0fa0 0x7f796c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:44.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.716+0000 7f7972200700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f796c19b070 0x7f796c19b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:44.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.716+0000 7f7972200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f796c10f570 con 0x7f796c1a0fa0 2024-05-08T05:58:44.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.716+0000 7f7972200700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f796c10f3f0 con 0x7f796c1052f0 2024-05-08T05:58:44.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.716+0000 7f7972200700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f796c10f6f0 con 0x7f796c19b070 2024-05-08T05:58:44.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.717+0000 7f796b400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f796c1052f0 0x7f796c1a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:44.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.717+0000 7f796b400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f796c1052f0 0x7f796c1a0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:55498/0 (socket says 172.21.15.123:55498) 2024-05-08T05:58:44.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.717+0000 7f796b400700 1 -- 172.21.15.123:0/2650540677 learned_addr learned my addr 172.21.15.123:0/2650540677 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:58:44.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.717+0000 7f796aa00700 1 --2- 172.21.15.123:0/2650540677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f796c1a0fa0 0x7f796c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:44.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.717+0000 7f796be00700 1 --2- 172.21.15.123:0/2650540677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f796c19b070 0x7f796c19b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:44.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.717+0000 7f796b400700 1 -- 172.21.15.123:0/2650540677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f796c19b070 msgr2=0x7f796c19b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:44.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.717+0000 7f796b400700 1 --2- 172.21.15.123:0/2650540677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f796c19b070 0x7f796c19b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:44.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.717+0000 7f796b400700 1 -- 172.21.15.123:0/2650540677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f796c1a0fa0 msgr2=0x7f796c19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:44.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.717+0000 7f796b400700 1 --2- 172.21.15.123:0/2650540677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f796c1a0fa0 0x7f796c19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:44.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.717+0000 7f796b400700 1 -- 172.21.15.123:0/2650540677 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f796c1a7810 con 0x7f796c1052f0 2024-05-08T05:58:44.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.717+0000 7f796be00700 1 --2- 172.21.15.123:0/2650540677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f796c19b070 0x7f796c19b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:58:44.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.717+0000 7f796b400700 1 --2- 172.21.15.123:0/2650540677 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f796c1052f0 0x7f796c1a0a60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f795c004f80 tx=0x7f795c004fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:44.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.718+0000 7f7963e00700 1 -- 172.21.15.123:0/2650540677 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795c026070 con 0x7f796c1052f0 2024-05-08T05:58:44.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.718+0000 7f7963e00700 1 -- 172.21.15.123:0/2650540677 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f795c00b6f0 con 0x7f796c1052f0 2024-05-08T05:58:44.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.718+0000 7f7972200700 1 -- 172.21.15.123:0/2650540677 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f796c1a7aa0 con 0x7f796c1052f0 2024-05-08T05:58:44.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.718+0000 7f7963e00700 1 -- 172.21.15.123:0/2650540677 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795c0038e0 con 0x7f796c1052f0 2024-05-08T05:58:44.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.718+0000 7f7972200700 1 -- 172.21.15.123:0/2650540677 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f796c1a8030 con 0x7f796c1052f0 2024-05-08T05:58:44.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.719+0000 7f7963e00700 1 -- 172.21.15.123:0/2650540677 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f795c029070 con 0x7f796c1052f0 2024-05-08T05:58:44.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.719+0000 7f7972200700 1 -- 172.21.15.123:0/2650540677 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f796c1067d0 con 0x7f796c1052f0 2024-05-08T05:58:44.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.719+0000 7f7963e00700 1 --2- 172.21.15.123:0/2650540677 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f795003e9a0 0x7f7950040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:44.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.720+0000 7f7963e00700 1 -- 172.21.15.123:0/2650540677 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f795c053280 con 0x7f796c1052f0 2024-05-08T05:58:44.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.720+0000 7f796aa00700 1 --2- 172.21.15.123:0/2650540677 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f795003e9a0 0x7f7950040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:44.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.720+0000 7f796aa00700 1 --2- 172.21.15.123:0/2650540677 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f795003e9a0 0x7f7950040e60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7958009ad0 tx=0x7f7958009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:44.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.726+0000 7f7963e00700 1 -- 172.21.15.123:0/2650540677 <== mon.2 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f795c013070 con 0x7f796c1052f0 2024-05-08T05:58:44.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.970+0000 7f7972200700 1 -- 172.21.15.123:0/2650540677 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f796c1a8200 con 0x7f796c1052f0 2024-05-08T05:58:44.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.971+0000 7f7963e00700 1 -- 172.21.15.123:0/2650540677 <== mon.2 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7f795c030130 con 0x7f796c1052f0 2024-05-08T05:58:44.972 INFO:teuthology.orchestra.run.smithi123.stdout:# minimal ceph.conf for 85cfb2e6-0cff-11ef-bc97-c7b262605968 2024-05-08T05:58:44.972 INFO:teuthology.orchestra.run.smithi123.stdout:[global] 2024-05-08T05:58:44.972 INFO:teuthology.orchestra.run.smithi123.stdout: fsid = 85cfb2e6-0cff-11ef-bc97-c7b262605968 2024-05-08T05:58:44.972 INFO:teuthology.orchestra.run.smithi123.stdout: mon_host = [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] 2024-05-08T05:58:44.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.975+0000 7f7972200700 1 -- 172.21.15.123:0/2650540677 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f795003e9a0 msgr2=0x7f7950040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:44.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.975+0000 7f7972200700 1 --2- 172.21.15.123:0/2650540677 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f795003e9a0 0x7f7950040e60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7958009ad0 tx=0x7f7958009210 comp rx=0 tx=0).stop 2024-05-08T05:58:44.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.975+0000 7f7972200700 1 -- 172.21.15.123:0/2650540677 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f796c1052f0 msgr2=0x7f796c1a0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:44.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.975+0000 7f7972200700 1 --2- 172.21.15.123:0/2650540677 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f796c1052f0 0x7f796c1a0a60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f795c004f80 tx=0x7f795c004fb0 comp rx=0 tx=0).stop 2024-05-08T05:58:44.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.975+0000 7f7972200700 1 -- 172.21.15.123:0/2650540677 shutdown_connections 2024-05-08T05:58:44.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.975+0000 7f7972200700 1 --2- 172.21.15.123:0/2650540677 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f795003e9a0 0x7f7950040e60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:44.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.975+0000 7f7972200700 1 --2- 172.21.15.123:0/2650540677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f796c19b070 0x7f796c19b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:44.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.975+0000 7f7972200700 1 --2- 172.21.15.123:0/2650540677 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f796c1052f0 0x7f796c1a0a60 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:44.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.975+0000 7f7972200700 1 --2- 172.21.15.123:0/2650540677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f796c1a0fa0 0x7f796c19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:44.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.975+0000 7f7972200700 1 -- 172.21.15.123:0/2650540677 >> 172.21.15.123:0/2650540677 conn(0x7f796c100bc0 msgr2=0x7f796c106cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:44.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.975+0000 7f7972200700 1 -- 172.21.15.123:0/2650540677 shutdown_connections 2024-05-08T05:58:44.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:44.975+0000 7f7972200700 1 -- 172.21.15.123:0/2650540677 wait complete. 2024-05-08T05:58:45.631 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:45 smithi123 ceph-mon[29787]: Updating smithi123:/var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:58:45.631 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:45 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:45.631 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:45 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:45.631 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:45 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:45.632 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:45 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:45.632 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:45 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:45.632 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:45 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:45.632 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:45 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:45.632 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:45 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:58:45.632 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:45 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T05:58:45.632 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:45 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:45.632 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:45 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/2650540677' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:45.682 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-08T05:58:45.682 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-05-08T05:58:45.682 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.conf 2024-05-08T05:58:45.746 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-05-08T05:58:45.746 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:58:45.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:45 smithi163 ceph-mon[34520]: Updating smithi123:/var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:58:45.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:45 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:45.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:45 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:45.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:45 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:45.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:45 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:45.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:45 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:45.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:45 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:45.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:45 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:45.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:45 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:58:45.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:45 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T05:58:45.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:45 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:45.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:45 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/2650540677' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:45.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:45 smithi192 ceph-mon[34664]: Updating smithi123:/var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:58:45.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:45 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:45.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:45 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:45.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:45 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:45.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:45 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:45.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:45 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:45.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:45 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:45.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:45 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:45.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:45 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:58:45.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:45 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T05:58:45.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:45 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:45.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:45 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/2650540677' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:45.850 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T05:58:45.851 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/ceph/ceph.conf 2024-05-08T05:58:45.884 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T05:58:45.884 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:58:45.951 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T05:58:45.951 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/ceph/ceph.conf 2024-05-08T05:58:45.983 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T05:58:45.983 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:58:46.050 INFO:tasks.cephadm:Adding mgr.x on smithi123 2024-05-08T05:58:46.051 INFO:tasks.cephadm:Adding mgr.y on smithi163 2024-05-08T05:58:46.051 INFO:tasks.cephadm:Adding mgr.z on smithi192 2024-05-08T05:58:46.051 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph orch apply mgr '3;smithi123=x;smithi163=y;smithi192=z' 2024-05-08T05:58:46.325 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:58:46.738 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:46 smithi192 ceph-mon[34664]: Reconfiguring mon.a (unknown last config time)... 2024-05-08T05:58:46.739 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:46 smithi192 ceph-mon[34664]: Reconfiguring daemon mon.a on smithi123 2024-05-08T05:58:46.739 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:46 smithi192 ceph-mon[34664]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:46.739 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:46 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:46.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:46 smithi163 ceph-mon[34520]: Reconfiguring mon.a (unknown last config time)... 2024-05-08T05:58:46.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:46 smithi163 ceph-mon[34520]: Reconfiguring daemon mon.a on smithi123 2024-05-08T05:58:46.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:46 smithi163 ceph-mon[34520]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:46.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:46 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:46.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:46 smithi123 ceph-mon[29787]: Reconfiguring mon.a (unknown last config time)... 2024-05-08T05:58:46.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:46 smithi123 ceph-mon[29787]: Reconfiguring daemon mon.a on smithi123 2024-05-08T05:58:46.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:46 smithi123 ceph-mon[29787]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:46.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:46 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:47.816 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:47.814+0000 7f085b982700 1 -- 172.21.15.192:0/892273000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0854101330 msgr2=0x7f0854101750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:47.817 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:47.814+0000 7f085b982700 1 --2- 172.21.15.192:0/892273000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0854101330 0x7f0854101750 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0844009910 tx=0x7f0844009c20 comp rx=0 tx=0).stop 2024-05-08T05:58:47.817 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:47.814+0000 7f085b982700 1 -- 172.21.15.192:0/892273000 shutdown_connections 2024-05-08T05:58:47.817 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:47.814+0000 7f085b982700 1 --2- 172.21.15.192:0/892273000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0854101330 0x7f0854101750 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:47.817 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:47.814+0000 7f085b982700 1 -- 172.21.15.192:0/892273000 >> 172.21.15.192:0/892273000 conn(0x7f08540fcc50 msgr2=0x7f08540ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:47.817 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:47.814+0000 7f085b982700 1 -- 172.21.15.192:0/892273000 shutdown_connections 2024-05-08T05:58:47.817 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:47.814+0000 7f085b982700 1 -- 172.21.15.192:0/892273000 wait complete. 2024-05-08T05:58:47.817 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:47.815+0000 7f085b982700 1 Processor -- start 2024-05-08T05:58:47.817 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:47.815+0000 7f085b982700 1 -- start start 2024-05-08T05:58:47.817 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:47.815+0000 7f085b982700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0854101330 0x7f08541987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:47.817 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:47.815+0000 7f085b982700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0854198d10 0x7f085419dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:47.817 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:47.815+0000 7f085b982700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0854199190 0x7f08541a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:47.818 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:47.815+0000 7f085b982700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0854108f30 con 0x7f0854101330 2024-05-08T05:58:47.818 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:47.815+0000 7f085b982700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0854108db0 con 0x7f0854198d10 2024-05-08T05:58:47.818 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:47.815+0000 7f085b982700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f08541090b0 con 0x7f0854199190 2024-05-08T05:58:47.818 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:47.815+0000 7f0859f1f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0854199190 0x7f08541a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:47.818 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:47.815+0000 7f0859f1f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0854199190 0x7f08541a0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:48684/0 (socket says 172.21.15.192:48684) 2024-05-08T05:58:47.818 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:47.815+0000 7f0859f1f700 1 -- 172.21.15.192:0/3973877657 learned_addr learned my addr 172.21.15.192:0/3973877657 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-08T05:58:47.818 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:47.815+0000 7f0859f1f700 1 -- 172.21.15.192:0/3973877657 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0854198d10 msgr2=0x7f085419dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:47.818 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:47.815+0000 7f0858f1d700 1 --2- 172.21.15.192:0/3973877657 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0854198d10 0x7f085419dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:47.818 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:47.815+0000 7f0859f1f700 1 --2- 172.21.15.192:0/3973877657 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0854198d10 0x7f085419dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:47.818 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:47.815+0000 7f0859f1f700 1 -- 172.21.15.192:0/3973877657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0854101330 msgr2=0x7f08541987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:47.818 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:47.815+0000 7f0859f1f700 1 --2- 172.21.15.192:0/3973877657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0854101330 0x7f08541987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:47.818 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:47.815+0000 7f0859f1f700 1 -- 172.21.15.192:0/3973877657 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08541a0be0 con 0x7f0854199190 2024-05-08T05:58:47.818 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:47.815+0000 7f0858f1d700 1 --2- 172.21.15.192:0/3973877657 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0854198d10 0x7f085419dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:58:47.818 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:47.815+0000 7f0859f1f700 1 --2- 172.21.15.192:0/3973877657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0854199190 0x7f08541a0310 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f084c00ba50 tx=0x7f084c00be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:47.819 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:47.816+0000 7f084a7fc700 1 -- 172.21.15.192:0/3973877657 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c019070 con 0x7f0854199190 2024-05-08T05:58:47.819 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:47.816+0000 7f084a7fc700 1 -- 172.21.15.192:0/3973877657 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f084c014440 con 0x7f0854199190 2024-05-08T05:58:47.819 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:47.816+0000 7f085b982700 1 -- 172.21.15.192:0/3973877657 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08541a0ed0 con 0x7f0854199190 2024-05-08T05:58:47.819 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:47.816+0000 7f084a7fc700 1 -- 172.21.15.192:0/3973877657 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c007590 con 0x7f0854199190 2024-05-08T05:58:47.819 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:47.816+0000 7f085b982700 1 -- 172.21.15.192:0/3973877657 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08541a13b0 con 0x7f0854199190 2024-05-08T05:58:47.819 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:47.816+0000 7f084a7fc700 1 -- 172.21.15.192:0/3973877657 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f084c01c070 con 0x7f0854199190 2024-05-08T05:58:47.819 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:47.817+0000 7f085b982700 1 -- 172.21.15.192:0/3973877657 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0854064b00 con 0x7f0854199190 2024-05-08T05:58:47.819 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:47.817+0000 7f084a7fc700 1 --2- 172.21.15.192:0/3973877657 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f084003e9a0 0x7f0840040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:47.819 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:47.817+0000 7f084a7fc700 1 -- 172.21.15.192:0/3973877657 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f084c051f90 con 0x7f0854199190 2024-05-08T05:58:47.820 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:47.817+0000 7f085971e700 1 --2- 172.21.15.192:0/3973877657 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f084003e9a0 0x7f0840040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:47.820 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:47.818+0000 7f085971e700 1 --2- 172.21.15.192:0/3973877657 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f084003e9a0 0x7f0840040e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0844005e00 tx=0x7f0844005260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:47.822 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:47.822+0000 7f084a7fc700 1 -- 172.21.15.192:0/3973877657 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f084c022170 con 0x7f0854199190 2024-05-08T05:58:48.039 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:48.039+0000 7f085b982700 1 -- 172.21.15.192:0/3973877657 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi123=x;smithi163=y;smithi192=z", "target": ["mon-mgr", ""]}) -- 0x7f08540febe0 con 0x7f084003e9a0 2024-05-08T05:58:48.054 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:48.052+0000 7f084a7fc700 1 -- 172.21.15.192:0/3973877657 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f08540febe0 con 0x7f084003e9a0 2024-05-08T05:58:48.054 INFO:teuthology.orchestra.run.smithi192.stdout:Scheduled mgr update... 2024-05-08T05:58:48.055 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:48.054+0000 7f085b982700 1 -- 172.21.15.192:0/3973877657 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f084003e9a0 msgr2=0x7f0840040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:48.055 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:48.054+0000 7f085b982700 1 --2- 172.21.15.192:0/3973877657 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f084003e9a0 0x7f0840040e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0844005e00 tx=0x7f0844005260 comp rx=0 tx=0).stop 2024-05-08T05:58:48.055 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:48.054+0000 7f085b982700 1 -- 172.21.15.192:0/3973877657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0854199190 msgr2=0x7f08541a0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:48.055 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:48.054+0000 7f085b982700 1 --2- 172.21.15.192:0/3973877657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0854199190 0x7f08541a0310 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f084c00ba50 tx=0x7f084c00be10 comp rx=0 tx=0).stop 2024-05-08T05:58:48.055 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:48.055+0000 7f085b982700 1 -- 172.21.15.192:0/3973877657 shutdown_connections 2024-05-08T05:58:48.055 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:48.055+0000 7f085b982700 1 --2- 172.21.15.192:0/3973877657 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f084003e9a0 0x7f0840040e60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:48.055 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:48.055+0000 7f085b982700 1 --2- 172.21.15.192:0/3973877657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0854101330 0x7f08541987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:48.055 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:48.055+0000 7f085b982700 1 --2- 172.21.15.192:0/3973877657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0854199190 0x7f08541a0310 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:48.056 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:48.055+0000 7f085b982700 1 --2- 172.21.15.192:0/3973877657 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0854198d10 0x7f085419dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:48.056 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:48.055+0000 7f085b982700 1 -- 172.21.15.192:0/3973877657 >> 172.21.15.192:0/3973877657 conn(0x7f08540fcc50 msgr2=0x7f08540fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:48.056 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:48.055+0000 7f085b982700 1 -- 172.21.15.192:0/3973877657 shutdown_connections 2024-05-08T05:58:48.056 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:58:48.055+0000 7f085b982700 1 -- 172.21.15.192:0/3973877657 wait complete. 2024-05-08T05:58:48.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:48 smithi192 ceph-mon[34664]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:48.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:48 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:48.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:48 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:48.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:48 smithi192 ceph-mon[34664]: Reconfiguring mon.b (monmap changed)... 2024-05-08T05:58:48.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:48 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:58:48.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:48 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T05:58:48.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:48 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:48.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:48 smithi192 ceph-mon[34664]: Reconfiguring daemon mon.b on smithi163 2024-05-08T05:58:48.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:48 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:48.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:48 smithi123 ceph-mon[29787]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:48.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:48 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:48.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:48 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:48.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:48 smithi123 ceph-mon[29787]: Reconfiguring mon.b (monmap changed)... 2024-05-08T05:58:48.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:48 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:58:48.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:48 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T05:58:48.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:48 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:48.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:48 smithi123 ceph-mon[29787]: Reconfiguring daemon mon.b on smithi163 2024-05-08T05:58:48.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:48 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:48.518 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:48 smithi163 ceph-mon[34520]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:48.518 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:48 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:48.519 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:48 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:48.519 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:48 smithi163 ceph-mon[34520]: Reconfiguring mon.b (monmap changed)... 2024-05-08T05:58:48.519 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:48 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:58:48.519 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:48 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T05:58:48.519 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:48 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:48.519 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:48 smithi163 ceph-mon[34520]: Reconfiguring daemon mon.b on smithi163 2024-05-08T05:58:48.519 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:48 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:48.574 DEBUG:teuthology.orchestra.run.smithi163:mgr.y> sudo journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mgr.y.service 2024-05-08T05:58:48.577 DEBUG:teuthology.orchestra.run.smithi192:mgr.z> sudo journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mgr.z.service 2024-05-08T05:58:48.579 INFO:tasks.cephadm:Deploying OSDs... 2024-05-08T05:58:48.579 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-05-08T05:58:48.579 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T05:58:48.638 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T05:58:48.639 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_1 2024-05-08T05:58:48.691 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T05:58:48.692 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:58:48.692 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-05-08T05:58:48.692 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:58:48.692 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:58:48.692 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-05-08 05:57:48.462255639 +0000 2024-05-08T05:58:48.692 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-05-08 05:52:28.338137139 +0000 2024-05-08T05:58:48.692 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-05-08 05:52:28.338137139 +0000 2024-05-08T05:58:48.692 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-05-08 05:52:28.338137139 +0000 2024-05-08T05:58:48.693 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T05:58:48.770 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-05-08T05:58:48.770 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-05-08T05:58:48.770 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000358839 s, 1.4 MB/s 2024-05-08T05:58:48.773 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T05:58:48.828 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_2 2024-05-08T05:58:48.880 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T05:58:48.880 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:58:48.881 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-05-08T05:58:48.881 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:58:48.881 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:58:48.881 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-05-08 05:57:48.520254308 +0000 2024-05-08T05:58:48.881 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-05-08 05:52:28.940123184 +0000 2024-05-08T05:58:48.881 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-05-08 05:52:28.940123184 +0000 2024-05-08T05:58:48.881 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-05-08 05:52:28.940123184 +0000 2024-05-08T05:58:48.882 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T05:58:48.959 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-05-08T05:58:48.960 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-05-08T05:58:48.960 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000331498 s, 1.5 MB/s 2024-05-08T05:58:48.963 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T05:58:49.018 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_3 2024-05-08T05:58:49.067 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T05:58:49.068 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:58:49.068 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-05-08T05:58:49.068 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:58:49.068 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:58:49.068 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-05-08 05:57:48.584252839 +0000 2024-05-08T05:58:49.068 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-05-08 05:52:29.529109530 +0000 2024-05-08T05:58:49.068 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-05-08 05:52:29.529109530 +0000 2024-05-08T05:58:49.068 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-05-08 05:52:29.529109530 +0000 2024-05-08T05:58:49.068 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T05:58:49.145 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-05-08T05:58:49.145 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-05-08T05:58:49.145 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000365588 s, 1.4 MB/s 2024-05-08T05:58:49.149 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T05:58:49.204 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_4 2024-05-08T05:58:49.254 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T05:58:49.254 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:58:49.254 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-05-08T05:58:49.254 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:58:49.255 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:58:49.255 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-05-08 05:57:48.649251347 +0000 2024-05-08T05:58:49.255 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-05-08 05:52:30.154095042 +0000 2024-05-08T05:58:49.255 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-05-08 05:52:30.154095042 +0000 2024-05-08T05:58:49.255 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-05-08 05:52:30.154095042 +0000 2024-05-08T05:58:49.255 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T05:58:49.331 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-05-08T05:58:49.331 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-05-08T05:58:49.331 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000342557 s, 1.5 MB/s 2024-05-08T05:58:49.335 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T05:58:49.391 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-08T05:58:49.391 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T05:58:49.408 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T05:58:49.409 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_1 2024-05-08T05:58:49.464 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T05:58:49.465 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:58:49.465 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-05-08T05:58:49.465 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:58:49.465 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:58:49.465 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-05-08 05:58:35.855886165 +0000 2024-05-08T05:58:49.465 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-05-08 05:51:32.814654488 +0000 2024-05-08T05:58:49.465 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-05-08 05:51:32.814654488 +0000 2024-05-08T05:58:49.465 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: 2024-05-08 05:51:32.814654488 +0000 2024-05-08T05:58:49.465 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T05:58:49.528 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-05-08T05:58:49.528 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-05-08T05:58:49.529 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000252993 s, 2.0 MB/s 2024-05-08T05:58:49.530 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T05:58:49.586 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_2 2024-05-08T05:58:49.642 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T05:58:49.642 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:58:49.642 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-05-08T05:58:49.643 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:58:49.643 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:58:49.643 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-05-08 05:58:35.856886143 +0000 2024-05-08T05:58:49.643 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-05-08 05:51:33.152646889 +0000 2024-05-08T05:58:49.643 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-05-08 05:51:33.152646889 +0000 2024-05-08T05:58:49.643 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: 2024-05-08 05:51:33.152646889 +0000 2024-05-08T05:58:49.643 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T05:58:49.705 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-05-08T05:58:49.705 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-05-08T05:58:49.705 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000182618 s, 2.8 MB/s 2024-05-08T05:58:49.707 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T05:58:49.762 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_3 2024-05-08T05:58:49.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:49 smithi163 ceph-mon[34520]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi123=x;smithi163=y;smithi192=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:58:49.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:49 smithi163 ceph-mon[34520]: Saving service mgr spec with placement smithi123=x;smithi163=y;smithi192=z;count:3 2024-05-08T05:58:49.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:49 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:49.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:49 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:49.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:49 smithi163 ceph-mon[34520]: Reconfiguring mon.c (monmap changed)... 2024-05-08T05:58:49.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:49 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:58:49.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:49 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T05:58:49.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:49 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:49.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:49 smithi163 ceph-mon[34520]: Reconfiguring daemon mon.c on smithi192 2024-05-08T05:58:49.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:49 smithi192 ceph-mon[34664]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi123=x;smithi163=y;smithi192=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:58:49.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:49 smithi192 ceph-mon[34664]: Saving service mgr spec with placement smithi123=x;smithi163=y;smithi192=z;count:3 2024-05-08T05:58:49.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:49 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:49.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:49 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:49.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:49 smithi192 ceph-mon[34664]: Reconfiguring mon.c (monmap changed)... 2024-05-08T05:58:49.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:49 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:58:49.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:49 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T05:58:49.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:49 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:49.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:49 smithi192 ceph-mon[34664]: Reconfiguring daemon mon.c on smithi192 2024-05-08T05:58:49.803 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T05:58:49.803 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:58:49.803 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-05-08T05:58:49.803 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:58:49.803 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:58:49.803 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-05-08 05:58:35.856886143 +0000 2024-05-08T05:58:49.803 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-05-08 05:51:33.482639470 +0000 2024-05-08T05:58:49.803 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-05-08 05:51:33.482639470 +0000 2024-05-08T05:58:49.803 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: 2024-05-08 05:51:33.482639470 +0000 2024-05-08T05:58:49.804 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T05:58:49.865 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-05-08T05:58:49.866 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-05-08T05:58:49.866 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000198203 s, 2.6 MB/s 2024-05-08T05:58:49.867 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T05:58:49.923 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_4 2024-05-08T05:58:49.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:49 smithi123 ceph-mon[29787]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi123=x;smithi163=y;smithi192=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:58:49.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:49 smithi123 ceph-mon[29787]: Saving service mgr spec with placement smithi123=x;smithi163=y;smithi192=z;count:3 2024-05-08T05:58:49.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:49 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:49.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:49 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:49.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:49 smithi123 ceph-mon[29787]: Reconfiguring mon.c (monmap changed)... 2024-05-08T05:58:49.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:49 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:58:49.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:49 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T05:58:49.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:49 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:49.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:49 smithi123 ceph-mon[29787]: Reconfiguring daemon mon.c on smithi192 2024-05-08T05:58:49.978 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T05:58:49.978 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:58:49.978 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-05-08T05:58:49.978 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:58:49.978 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:58:49.978 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-05-08 05:58:35.857886120 +0000 2024-05-08T05:58:49.978 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-05-08 05:51:33.806632185 +0000 2024-05-08T05:58:49.978 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-05-08 05:51:33.806632185 +0000 2024-05-08T05:58:49.979 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: 2024-05-08 05:51:33.806632185 +0000 2024-05-08T05:58:49.979 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T05:58:50.041 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-05-08T05:58:50.041 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-05-08T05:58:50.041 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000190462 s, 2.7 MB/s 2024-05-08T05:58:50.042 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T05:58:50.098 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-08T05:58:50.098 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T05:58:50.118 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T05:58:50.118 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_1 2024-05-08T05:58:50.173 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T05:58:50.174 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:58:50.174 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-08T05:58:50.174 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:58:50.174 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:58:50.174 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-05-08 05:58:26.458658538 +0000 2024-05-08T05:58:50.174 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-05-08 05:51:19.465524017 +0000 2024-05-08T05:58:50.174 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-05-08 05:51:19.465524017 +0000 2024-05-08T05:58:50.174 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-05-08 05:51:19.465524017 +0000 2024-05-08T05:58:50.174 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T05:58:50.237 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-05-08T05:58:50.237 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-05-08T05:58:50.237 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000204823 s, 2.5 MB/s 2024-05-08T05:58:50.239 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T05:58:50.294 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_2 2024-05-08T05:58:50.350 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T05:58:50.351 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:58:50.351 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-05-08T05:58:50.351 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:58:50.351 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:58:50.351 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-05-08 05:58:26.459658518 +0000 2024-05-08T05:58:50.351 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-05-08 05:51:19.805517272 +0000 2024-05-08T05:58:50.351 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-05-08 05:51:19.805517272 +0000 2024-05-08T05:58:50.351 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-05-08 05:51:19.805517272 +0000 2024-05-08T05:58:50.351 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T05:58:50.414 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-05-08T05:58:50.414 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-05-08T05:58:50.414 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000193458 s, 2.6 MB/s 2024-05-08T05:58:50.415 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T05:58:50.472 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_3 2024-05-08T05:58:50.527 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T05:58:50.527 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:58:50.528 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-05-08T05:58:50.528 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:58:50.528 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:58:50.528 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-05-08 05:58:26.459658518 +0000 2024-05-08T05:58:50.528 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-05-08 05:51:20.147510487 +0000 2024-05-08T05:58:50.528 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-05-08 05:51:20.147510487 +0000 2024-05-08T05:58:50.528 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-05-08 05:51:20.147510487 +0000 2024-05-08T05:58:50.528 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T05:58:50.591 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-05-08T05:58:50.591 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-05-08T05:58:50.591 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000203858 s, 2.5 MB/s 2024-05-08T05:58:50.592 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T05:58:50.648 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_4 2024-05-08T05:58:50.704 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T05:58:50.704 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:58:50.704 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-05-08T05:58:50.704 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:58:50.704 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:58:50.704 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-05-08 05:58:26.459658518 +0000 2024-05-08T05:58:50.704 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-05-08 05:51:20.475503979 +0000 2024-05-08T05:58:50.704 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-05-08 05:51:20.475503979 +0000 2024-05-08T05:58:50.704 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-05-08 05:51:20.475503979 +0000 2024-05-08T05:58:50.705 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T05:58:50.767 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-05-08T05:58:50.768 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-05-08T05:58:50.768 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000185404 s, 2.8 MB/s 2024-05-08T05:58:50.770 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T05:58:50.826 INFO:tasks.cephadm:Deploying osd.0 on smithi123 with /dev/vg_nvme/lv_4... 2024-05-08T05:58:50.826 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-08T05:58:50.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:50 smithi192 ceph-mon[34664]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:50.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:50 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:50.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:50 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:50.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:50 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:58:50.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:50 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:50.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:50 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:58:50.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:50 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:50.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:50 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T05:58:50.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:50 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T05:58:50.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:50 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T05:58:50.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:50 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:50.901 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:50 smithi123 ceph-mon[29787]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:50.901 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:50 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:50.901 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:50 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:50.901 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:50 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:58:50.901 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:50 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:50.901 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:50 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:58:50.901 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:50 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:50.901 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:50 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T05:58:50.901 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:50 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T05:58:50.901 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:50 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T05:58:50.901 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:50 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:51.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:50 smithi163 ceph-mon[34520]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:51.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:50 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:51.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:50 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:51.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:50 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:58:51.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:50 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:51.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:50 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:58:51.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:50 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:51.040 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:50 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T05:58:51.040 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:50 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T05:58:51.040 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:50 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T05:58:51.040 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:50 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:51.254 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:58:51.692 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:51 smithi123 ceph-mon[29787]: Deploying daemon mgr.z on smithi192 2024-05-08T05:58:51.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:51 smithi192 ceph-mon[34664]: Deploying daemon mgr.z on smithi192 2024-05-08T05:58:52.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:51 smithi163 ceph-mon[34520]: Deploying daemon mgr.z on smithi192 2024-05-08T05:58:52.217 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 08 05:58:52 smithi192 systemd[1]: Starting Ceph mgr.z for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T05:58:52.551 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 08 05:58:52 smithi192 podman[36514]: 2024-05-08 05:58:52.337240436 +0000 UTC m=+0.017947065 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:58:52.551 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 08 05:58:52 smithi192 podman[36514]: 2024-05-08 05:58:52.440625891 +0000 UTC m=+0.121332542 container create 85c10a5c884bb25b4a5e75d28456fa7ad2b7042ae3645654bee97bb1e77783e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f) 2024-05-08T05:58:52.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:52 smithi192 ceph-mon[34664]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:52.821 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 08 05:58:52 smithi192 podman[36514]: 2024-05-08 05:58:52.659811891 +0000 UTC m=+0.340518507 container init 85c10a5c884bb25b4a5e75d28456fa7ad2b7042ae3645654bee97bb1e77783e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-z, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T05:58:52.821 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 08 05:58:52 smithi192 podman[36514]: 2024-05-08 05:58:52.662990404 +0000 UTC m=+0.343697025 container start 85c10a5c884bb25b4a5e75d28456fa7ad2b7042ae3645654bee97bb1e77783e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-z, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T05:58:52.821 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 08 05:58:52 smithi192 ceph-mgr[36533]: -- 172.21.15.192:0/2451066982 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e166dde5a0 con 0x55e166db3000 2024-05-08T05:58:52.821 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 08 05:58:52 smithi192 bash[36514]: 85c10a5c884bb25b4a5e75d28456fa7ad2b7042ae3645654bee97bb1e77783e9 2024-05-08T05:58:52.821 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 08 05:58:52 smithi192 systemd[1]: Started Ceph mgr.z for 85cfb2e6-0cff-11ef-bc97-c7b262605968. 2024-05-08T05:58:52.891 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:52 smithi123 ceph-mon[29787]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:52.939 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:52 smithi163 ceph-mon[34520]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:53.105 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 08 05:58:52 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-z[36529]: 2024-05-08T05:58:52.820+0000 7fc81d927200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T05:58:53.105 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 08 05:58:53 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-z[36529]: 2024-05-08T05:58:53.003+0000 7fc81d927200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:58:53.105 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 08 05:58:53 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-z[36529]: 2024-05-08T05:58:53.105+0000 7fc81d927200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T05:58:53.783 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 08 05:58:53 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-z[36529]: 2024-05-08T05:58:53.412+0000 7fc81d927200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T05:58:53.783 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 08 05:58:53 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-z[36529]: 2024-05-08T05:58:53.497+0000 7fc81d927200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T05:58:54.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:53 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:54.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:53 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:54.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:53 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:54.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:53 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T05:58:54.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:53 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T05:58:54.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:53 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T05:58:54.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:53 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:54.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:53 smithi163 ceph-mon[34520]: Deploying daemon mgr.y on smithi163 2024-05-08T05:58:54.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:53 smithi163 ceph-mon[34520]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:54.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:53 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:54.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:53 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:54.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:53 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:54.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:53 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T05:58:54.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:53 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T05:58:54.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:53 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T05:58:54.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:53 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:54.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:53 smithi192 ceph-mon[34664]: Deploying daemon mgr.y on smithi163 2024-05-08T05:58:54.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:53 smithi192 ceph-mon[34664]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:54.117 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:53 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:54.117 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:53 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:54.117 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:53 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:54.118 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:53 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T05:58:54.118 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:53 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T05:58:54.118 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:53 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T05:58:54.118 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:53 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:54.118 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:53 smithi123 ceph-mon[29787]: Deploying daemon mgr.y on smithi163 2024-05-08T05:58:54.118 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:53 smithi123 ceph-mon[29787]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:54.118 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-08T05:58:54.118 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-08T05:58:54.118 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2024-05-08T05:58:54.119 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2024-05-08T05:58:54.119 INFO:teuthology.orchestra.run.smithi123.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0349474 s, 300 MB/s 2024-05-08T05:58:54.119 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2024-05-08T05:58:54.905 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi123:vg_nvme/lv_4 2024-05-08T05:58:55.023 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 08 05:58:54 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-z[36529]: 2024-05-08T05:58:54.677+0000 7fc81d927200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T05:58:55.023 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 08 05:58:54 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-z[36529]: 2024-05-08T05:58:54.844+0000 7fc81d927200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T05:58:55.023 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 08 05:58:55 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-z[36529]: 2024-05-08T05:58:55.022+0000 7fc81d927200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T05:58:55.329 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:58:55.538 INFO:journalctl@ceph.mgr.y.smithi163.stdout:May 08 05:58:55 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-y[35918]: 2024-05-08T05:58:55.403+0000 7f6ced3a1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T05:58:55.640 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 08 05:58:55 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-z[36529]: 2024-05-08T05:58:55.328+0000 7fc81d927200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T05:58:55.640 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 08 05:58:55 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-z[36529]: 2024-05-08T05:58:55.425+0000 7fc81d927200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T05:58:55.640 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 08 05:58:55 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-z[36529]: 2024-05-08T05:58:55.640+0000 7fc81d927200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T05:58:56.025 INFO:journalctl@ceph.mgr.y.smithi163.stdout:May 08 05:58:55 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-y[35918]: 2024-05-08T05:58:55.593+0000 7f6ced3a1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:58:56.026 INFO:journalctl@ceph.mgr.y.smithi163.stdout:May 08 05:58:55 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-y[35918]: 2024-05-08T05:58:55.697+0000 7f6ced3a1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T05:58:56.026 INFO:journalctl@ceph.mgr.y.smithi163.stdout:May 08 05:58:56 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-y[35918]: 2024-05-08T05:58:56.025+0000 7f6ced3a1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T05:58:56.050 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 08 05:58:55 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-z[36529]: 2024-05-08T05:58:55.839+0000 7fc81d927200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:58:56.288 INFO:journalctl@ceph.mgr.y.smithi163.stdout:May 08 05:58:56 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-y[35918]: 2024-05-08T05:58:56.115+0000 7f6ced3a1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T05:58:56.676 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:56 smithi123 ceph-mon[29787]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:56.676 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:56 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:56.676 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:56 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:56.676 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:56 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:56.676 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:56 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:56.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:56 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:58:56.677 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:56 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:56.706 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 08 05:58:56 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-z[36529]: 2024-05-08T05:58:56.483+0000 7fc81d927200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T05:58:56.706 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 08 05:58:56 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-z[36529]: 2024-05-08T05:58:56.706+0000 7fc81d927200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-08T05:58:56.707 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:56 smithi192 ceph-mon[34664]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:56.707 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:56 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:56.707 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:56 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:56.707 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:56 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:56.707 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:56 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:56.707 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:56 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:58:56.708 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:56 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:56.720 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:56 smithi163 ceph-mon[34520]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:56.720 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:56 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:56.720 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:56 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:56.720 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:56 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:56.720 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:56 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:56.720 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:56 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:58:56.720 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:56 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:57.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.537+0000 7f2c50400700 1 -- 172.21.15.123:0/1418420533 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2c48104750 msgr2=0x7f2c48104bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:57.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.537+0000 7f2c50400700 1 --2- 172.21.15.123:0/1418420533 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2c48104750 0x7f2c48104bb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2c340099c0 tx=0x7f2c34009cd0 comp rx=0 tx=0).stop 2024-05-08T05:58:57.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.537+0000 7f2c50400700 1 -- 172.21.15.123:0/1418420533 shutdown_connections 2024-05-08T05:58:57.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.537+0000 7f2c50400700 1 --2- 172.21.15.123:0/1418420533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c48103550 0x7f2c48103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:57.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.537+0000 7f2c50400700 1 --2- 172.21.15.123:0/1418420533 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c481050f0 0x7f2c4810b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:57.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.537+0000 7f2c50400700 1 --2- 172.21.15.123:0/1418420533 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2c48104750 0x7f2c48104bb0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:57.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.537+0000 7f2c50400700 1 -- 172.21.15.123:0/1418420533 >> 172.21.15.123:0/1418420533 conn(0x7f2c480feaf0 msgr2=0x7f2c48100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:58:57.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.537+0000 7f2c50400700 1 -- 172.21.15.123:0/1418420533 shutdown_connections 2024-05-08T05:58:57.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.537+0000 7f2c50400700 1 -- 172.21.15.123:0/1418420533 wait complete. 2024-05-08T05:58:57.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.538+0000 7f2c50400700 1 Processor -- start 2024-05-08T05:58:57.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.539+0000 7f2c50400700 1 -- start start 2024-05-08T05:58:57.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.539+0000 7f2c50400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2c48103550 0x7f2c48077240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:57.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.539+0000 7f2c50400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c48104750 0x7f2c48075890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:57.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.539+0000 7f2c50400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c481050f0 0x7f2c48075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:57.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.539+0000 7f2c50400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4810f040 con 0x7f2c48104750 2024-05-08T05:58:57.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.539+0000 7f2c50400700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4810eec0 con 0x7f2c48103550 2024-05-08T05:58:57.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.539+0000 7f2c50400700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4810f1c0 con 0x7f2c481050f0 2024-05-08T05:58:57.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.539+0000 7f2c4da00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2c48103550 0x7f2c48077240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:57.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.539+0000 7f2c4da00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2c48103550 0x7f2c48077240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:35632/0 (socket says 172.21.15.123:35632) 2024-05-08T05:58:57.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.539+0000 7f2c4da00700 1 -- 172.21.15.123:0/2276765334 learned_addr learned my addr 172.21.15.123:0/2276765334 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T05:58:57.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.539+0000 7f2c4e400700 1 --2- 172.21.15.123:0/2276765334 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c481050f0 0x7f2c48075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:57.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.539+0000 7f2c4d000700 1 --2- 172.21.15.123:0/2276765334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c48104750 0x7f2c48075890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:57.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.540+0000 7f2c4da00700 1 -- 172.21.15.123:0/2276765334 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c481050f0 msgr2=0x7f2c48075dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:57.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.540+0000 7f2c4da00700 1 --2- 172.21.15.123:0/2276765334 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c481050f0 0x7f2c48075dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:57.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.540+0000 7f2c4da00700 1 -- 172.21.15.123:0/2276765334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c48104750 msgr2=0x7f2c48075890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:58:57.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.540+0000 7f2c4da00700 1 --2- 172.21.15.123:0/2276765334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c48104750 0x7f2c48075890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:58:57.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.540+0000 7f2c4da00700 1 -- 172.21.15.123:0/2276765334 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c48076680 con 0x7f2c48103550 2024-05-08T05:58:57.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.540+0000 7f2c4da00700 1 --2- 172.21.15.123:0/2276765334 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2c48103550 0x7f2c48077240 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c00e970 tx=0x7f2c3c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:57.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.540+0000 7f2c46000700 1 -- 172.21.15.123:0/2276765334 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c004220 con 0x7f2c48103550 2024-05-08T05:58:57.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.540+0000 7f2c46000700 1 -- 172.21.15.123:0/2276765334 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c3c0043c0 con 0x7f2c48103550 2024-05-08T05:58:57.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.540+0000 7f2c50400700 1 -- 172.21.15.123:0/2276765334 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c481a37c0 con 0x7f2c48103550 2024-05-08T05:58:57.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.540+0000 7f2c46000700 1 -- 172.21.15.123:0/2276765334 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c003ce0 con 0x7f2c48103550 2024-05-08T05:58:57.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.540+0000 7f2c50400700 1 -- 172.21.15.123:0/2276765334 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c481a3d00 con 0x7f2c48103550 2024-05-08T05:58:57.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.542+0000 7f2c50400700 1 -- 172.21.15.123:0/2276765334 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c4806a190 con 0x7f2c48103550 2024-05-08T05:58:57.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.542+0000 7f2c46000700 1 -- 172.21.15.123:0/2276765334 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f2c3c018430 con 0x7f2c48103550 2024-05-08T05:58:57.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.542+0000 7f2c46000700 1 --2- 172.21.15.123:0/2276765334 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f2c3003e750 0x7f2c30040c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:58:57.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.542+0000 7f2c46000700 1 -- 172.21.15.123:0/2276765334 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2c3c052b80 con 0x7f2c48103550 2024-05-08T05:58:57.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.543+0000 7f2c4d000700 1 --2- 172.21.15.123:0/2276765334 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f2c3003e750 0x7f2c30040c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:58:57.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.543+0000 7f2c4d000700 1 --2- 172.21.15.123:0/2276765334 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f2c3003e750 0x7f2c30040c10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2c34000c00 tx=0x7f2c34005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:58:57.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.547+0000 7f2c46000700 1 -- 172.21.15.123:0/2276765334 <== mon.2 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2c3c0173d0 con 0x7f2c48103550 2024-05-08T05:58:57.550 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 08 05:58:57 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-z[36529]: 2024-05-08T05:58:57.158+0000 7fc81d927200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T05:58:57.550 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 08 05:58:57 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-z[36529]: 2024-05-08T05:58:57.254+0000 7fc81d927200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T05:58:57.720 INFO:journalctl@ceph.mgr.y.smithi163.stdout:May 08 05:58:57 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-y[35918]: 2024-05-08T05:58:57.376+0000 7f6ced3a1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T05:58:57.720 INFO:journalctl@ceph.mgr.y.smithi163.stdout:May 08 05:58:57 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-y[35918]: 2024-05-08T05:58:57.548+0000 7f6ced3a1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T05:58:57.720 INFO:journalctl@ceph.mgr.y.smithi163.stdout:May 08 05:58:57 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-y[35918]: 2024-05-08T05:58:57.719+0000 7f6ced3a1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T05:58:57.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:58:57.789+0000 7f2c50400700 1 -- 172.21.15.123:0/2276765334 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f2c480638e0 con 0x7f2c3003e750 2024-05-08T05:58:57.980 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 08 05:58:57 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-z[36529]: 2024-05-08T05:58:57.628+0000 7fc81d927200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T05:58:58.288 INFO:journalctl@ceph.mgr.y.smithi163.stdout:May 08 05:58:58 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-y[35918]: 2024-05-08T05:58:58.006+0000 7f6ced3a1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T05:58:58.288 INFO:journalctl@ceph.mgr.y.smithi163.stdout:May 08 05:58:58 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-y[35918]: 2024-05-08T05:58:58.099+0000 7f6ced3a1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T05:58:58.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:58 smithi163 ceph-mon[34520]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:58.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:58 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:58.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:58 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:58.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:58 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:58.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:58 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:58.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:58 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:58.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:58 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:58:58.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:58 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:58.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:58 smithi163 ceph-mon[34520]: Reconfiguring mgr.x (unknown last config time)... 2024-05-08T05:58:58.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:58 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T05:58:58.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:58 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T05:58:58.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:58 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:58.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:58 smithi163 ceph-mon[34520]: Reconfiguring daemon mgr.x on smithi123 2024-05-08T05:58:58.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:58 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:58:58.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:58 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:58:58.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:58 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:58.300 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 08 05:58:57 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-z[36529]: 2024-05-08T05:58:57.979+0000 7fc81d927200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T05:58:58.300 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 08 05:58:58 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-z[36529]: 2024-05-08T05:58:58.071+0000 7fc81d927200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T05:58:58.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:58 smithi192 ceph-mon[34664]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:58.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:58 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:58.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:58 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:58.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:58 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:58.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:58 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:58.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:58 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:58.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:58 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:58:58.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:58 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:58.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:58 smithi192 ceph-mon[34664]: Reconfiguring mgr.x (unknown last config time)... 2024-05-08T05:58:58.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:58 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T05:58:58.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:58 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T05:58:58.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:58 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:58.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:58 smithi192 ceph-mon[34664]: Reconfiguring daemon mgr.x on smithi123 2024-05-08T05:58:58.302 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:58 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:58:58.302 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:58 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:58:58.302 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:58 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:58.365 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:58 smithi123 ceph-mon[29787]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:58:58.365 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:58 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:58.365 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:58 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:58.365 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:58 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:58.365 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:58 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:58.365 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:58 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:58.366 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:58 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:58:58.366 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:58 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:58:58.366 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:58 smithi123 ceph-mon[29787]: Reconfiguring mgr.x (unknown last config time)... 2024-05-08T05:58:58.366 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:58 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T05:58:58.366 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:58 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T05:58:58.366 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:58 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:58.366 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:58 smithi123 ceph-mon[29787]: Reconfiguring daemon mgr.x on smithi123 2024-05-08T05:58:58.366 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:58 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:58:58.366 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:58 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:58:58.366 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:58 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:58.788 INFO:journalctl@ceph.mgr.y.smithi163.stdout:May 08 05:58:58 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-y[35918]: 2024-05-08T05:58:58.303+0000 7f6ced3a1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T05:58:58.788 INFO:journalctl@ceph.mgr.y.smithi163.stdout:May 08 05:58:58 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-y[35918]: 2024-05-08T05:58:58.493+0000 7f6ced3a1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:58:58.800 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 08 05:58:58 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-z[36529]: 2024-05-08T05:58:58.313+0000 7fc81d927200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T05:58:58.800 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 08 05:58:58 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-z[36529]: 2024-05-08T05:58:58.445+0000 7fc81d927200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T05:58:58.800 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 08 05:58:58 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-z[36529]: 2024-05-08T05:58:58.539+0000 7fc81d927200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T05:58:59.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:58:59 smithi192 ceph-mon[34664]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:58:59.301 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 08 05:58:58 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-z[36529]: 2024-05-08T05:58:58.867+0000 7fc81d927200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T05:58:59.301 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 08 05:58:58 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-z[36529]: 2024-05-08T05:58:58.958+0000 7fc81d927200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T05:58:59.387 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:58:59 smithi123 ceph-mon[29787]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:58:59.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:58:59 smithi163 ceph-mon[34520]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:58:59.538 INFO:journalctl@ceph.mgr.y.smithi163.stdout:May 08 05:58:59 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-y[35918]: 2024-05-08T05:58:59.094+0000 7f6ced3a1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T05:58:59.538 INFO:journalctl@ceph.mgr.y.smithi163.stdout:May 08 05:58:59 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-y[35918]: 2024-05-08T05:58:59.313+0000 7f6ced3a1200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-08T05:59:00.038 INFO:journalctl@ceph.mgr.y.smithi163.stdout:May 08 05:58:59 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-y[35918]: 2024-05-08T05:58:59.738+0000 7f6ced3a1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T05:59:00.038 INFO:journalctl@ceph.mgr.y.smithi163.stdout:May 08 05:58:59 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-y[35918]: 2024-05-08T05:58:59.833+0000 7f6ced3a1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T05:59:00.050 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 08 05:58:59 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-z[36529]: 2024-05-08T05:58:59.555+0000 7fc81d927200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T05:59:00.050 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 08 05:58:59 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-z[36529]: 2024-05-08T05:58:59.772+0000 7fc81d927200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T05:59:00.522 INFO:journalctl@ceph.mgr.y.smithi163.stdout:May 08 05:59:00 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-y[35918]: 2024-05-08T05:59:00.204+0000 7f6ced3a1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T05:59:00.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:59:00.551+0000 7f2c46000700 1 -- 172.21.15.123:0/2276765334 <== mon.2 v2:172.21.15.163:3300/0 7 ==== mgrmap(e 14) ==== 99938+0+0 (secure 0 0 0) 0x7f2c3c01dd20 con 0x7f2c48103550 2024-05-08T05:59:00.659 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:00 smithi123 ceph-mon[29787]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:00.659 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:00 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:00.660 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:00 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:00.660 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:00 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:59:00.660 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:00 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:59:00.660 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:00 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:59:00.660 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:00 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:00.660 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:00 smithi123 ceph-mon[29787]: from='mgr.? 172.21.15.192:0/4258357953' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T05:59:00.660 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:00 smithi123 ceph-mon[29787]: Standby manager daemon z started 2024-05-08T05:59:00.660 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:00 smithi123 ceph-mon[29787]: from='mgr.? 172.21.15.192:0/4258357953' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T05:59:00.660 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:00 smithi123 ceph-mon[29787]: from='mgr.? 172.21.15.192:0/4258357953' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T05:59:00.660 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:00 smithi123 ceph-mon[29787]: from='mgr.? 172.21.15.192:0/4258357953' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T05:59:00.788 INFO:journalctl@ceph.mgr.y.smithi163.stdout:May 08 05:59:00 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-y[35918]: 2024-05-08T05:59:00.557+0000 7f6ced3a1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T05:59:00.788 INFO:journalctl@ceph.mgr.y.smithi163.stdout:May 08 05:59:00 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-y[35918]: 2024-05-08T05:59:00.649+0000 7f6ced3a1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T05:59:00.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:00 smithi163 ceph-mon[34520]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:00.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:00 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:00.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:00 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:00.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:00 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:59:00.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:00 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:59:00.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:00 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:59:00.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:00 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:00.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:00 smithi163 ceph-mon[34520]: from='mgr.? 172.21.15.192:0/4258357953' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T05:59:00.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:00 smithi163 ceph-mon[34520]: Standby manager daemon z started 2024-05-08T05:59:00.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:00 smithi163 ceph-mon[34520]: from='mgr.? 172.21.15.192:0/4258357953' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T05:59:00.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:00 smithi163 ceph-mon[34520]: from='mgr.? 172.21.15.192:0/4258357953' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T05:59:00.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:00 smithi163 ceph-mon[34520]: from='mgr.? 172.21.15.192:0/4258357953' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T05:59:00.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:00 smithi192 ceph-mon[34664]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:00.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:00 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:00.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:00 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:00.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:00 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:59:00.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:00 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:59:00.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:00 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:59:00.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:00 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:00.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:00 smithi192 ceph-mon[34664]: from='mgr.? 172.21.15.192:0/4258357953' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T05:59:00.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:00 smithi192 ceph-mon[34664]: Standby manager daemon z started 2024-05-08T05:59:00.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:00 smithi192 ceph-mon[34664]: from='mgr.? 172.21.15.192:0/4258357953' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T05:59:00.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:00 smithi192 ceph-mon[34664]: from='mgr.? 172.21.15.192:0/4258357953' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T05:59:00.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:00 smithi192 ceph-mon[34664]: from='mgr.? 172.21.15.192:0/4258357953' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T05:59:01.288 INFO:journalctl@ceph.mgr.y.smithi163.stdout:May 08 05:59:00 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-y[35918]: 2024-05-08T05:59:00.891+0000 7f6ced3a1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T05:59:01.288 INFO:journalctl@ceph.mgr.y.smithi163.stdout:May 08 05:59:01 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-y[35918]: 2024-05-08T05:59:01.022+0000 7f6ced3a1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T05:59:01.288 INFO:journalctl@ceph.mgr.y.smithi163.stdout:May 08 05:59:01 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-y[35918]: 2024-05-08T05:59:01.116+0000 7f6ced3a1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T05:59:01.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:01 smithi163 ceph-mon[34520]: mgrmap e14: x(active, since 89s), standbys: z 2024-05-08T05:59:01.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:01 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T05:59:01.788 INFO:journalctl@ceph.mgr.y.smithi163.stdout:May 08 05:59:01 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-y[35918]: 2024-05-08T05:59:01.443+0000 7f6ced3a1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T05:59:01.788 INFO:journalctl@ceph.mgr.y.smithi163.stdout:May 08 05:59:01 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-y[35918]: 2024-05-08T05:59:01.534+0000 7f6ced3a1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T05:59:01.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:01 smithi123 ceph-mon[29787]: mgrmap e14: x(active, since 89s), standbys: z 2024-05-08T05:59:01.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:01 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T05:59:02.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:01 smithi192 ceph-mon[34664]: mgrmap e14: x(active, since 89s), standbys: z 2024-05-08T05:59:02.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:01 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T05:59:02.538 INFO:journalctl@ceph.mgr.y.smithi163.stdout:May 08 05:59:02 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-y[35918]: 2024-05-08T05:59:02.129+0000 7f6ced3a1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T05:59:02.538 INFO:journalctl@ceph.mgr.y.smithi163.stdout:May 08 05:59:02 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-y[35918]: 2024-05-08T05:59:02.344+0000 7f6ced3a1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T05:59:02.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:59:02.567+0000 7f2c46000700 1 -- 172.21.15.123:0/2276765334 <== mon.2 v2:172.21.15.163:3300/0 8 ==== mgrmap(e 15) ==== 149597+0+0 (secure 0 0 0) 0x7f2c3c0653a0 con 0x7f2c48103550 2024-05-08T05:59:02.858 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:02 smithi123 ceph-mon[29787]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:02.858 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:02 smithi123 ceph-mon[29787]: from='mgr.? 172.21.15.163:0/1335834141' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T05:59:02.858 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:02 smithi123 ceph-mon[29787]: Standby manager daemon y started 2024-05-08T05:59:02.858 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:02 smithi123 ceph-mon[29787]: from='mgr.? 172.21.15.163:0/1335834141' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T05:59:02.859 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:02 smithi123 ceph-mon[29787]: from='mgr.? 172.21.15.163:0/1335834141' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T05:59:02.859 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:02 smithi123 ceph-mon[29787]: from='mgr.? 172.21.15.163:0/1335834141' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T05:59:02.859 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:02 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/3510458831' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a14fde9b-bfb1-40a7-a105-305e8981a778"}]: dispatch 2024-05-08T05:59:02.859 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:02 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/3510458831' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a14fde9b-bfb1-40a7-a105-305e8981a778"}]': finished 2024-05-08T05:59:02.859 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:02 smithi123 ceph-mon[29787]: osdmap e5: 1 total, 0 up, 1 in 2024-05-08T05:59:02.859 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:02 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:59:03.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:02 smithi163 ceph-mon[34520]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:03.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:02 smithi163 ceph-mon[34520]: from='mgr.? 172.21.15.163:0/1335834141' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T05:59:03.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:02 smithi163 ceph-mon[34520]: Standby manager daemon y started 2024-05-08T05:59:03.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:02 smithi163 ceph-mon[34520]: from='mgr.? 172.21.15.163:0/1335834141' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T05:59:03.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:02 smithi163 ceph-mon[34520]: from='mgr.? 172.21.15.163:0/1335834141' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T05:59:03.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:02 smithi163 ceph-mon[34520]: from='mgr.? 172.21.15.163:0/1335834141' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T05:59:03.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:02 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/3510458831' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a14fde9b-bfb1-40a7-a105-305e8981a778"}]: dispatch 2024-05-08T05:59:03.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:02 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/3510458831' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a14fde9b-bfb1-40a7-a105-305e8981a778"}]': finished 2024-05-08T05:59:03.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:02 smithi163 ceph-mon[34520]: osdmap e5: 1 total, 0 up, 1 in 2024-05-08T05:59:03.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:02 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:59:03.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:02 smithi192 ceph-mon[34664]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:03.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:02 smithi192 ceph-mon[34664]: from='mgr.? 172.21.15.163:0/1335834141' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T05:59:03.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:02 smithi192 ceph-mon[34664]: Standby manager daemon y started 2024-05-08T05:59:03.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:02 smithi192 ceph-mon[34664]: from='mgr.? 172.21.15.163:0/1335834141' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T05:59:03.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:02 smithi192 ceph-mon[34664]: from='mgr.? 172.21.15.163:0/1335834141' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T05:59:03.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:02 smithi192 ceph-mon[34664]: from='mgr.? 172.21.15.163:0/1335834141' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T05:59:03.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:02 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/3510458831' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a14fde9b-bfb1-40a7-a105-305e8981a778"}]: dispatch 2024-05-08T05:59:03.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:02 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/3510458831' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a14fde9b-bfb1-40a7-a105-305e8981a778"}]': finished 2024-05-08T05:59:03.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:02 smithi192 ceph-mon[34664]: osdmap e5: 1 total, 0 up, 1 in 2024-05-08T05:59:03.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:02 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:59:03.884 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:03 smithi123 ceph-mon[29787]: mgrmap e15: x(active, since 91s), standbys: z, y 2024-05-08T05:59:03.884 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:03 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T05:59:03.884 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:03 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/1441961430' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:59:04.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:03 smithi163 ceph-mon[34520]: mgrmap e15: x(active, since 91s), standbys: z, y 2024-05-08T05:59:04.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:03 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T05:59:04.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:03 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/1441961430' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:59:04.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:03 smithi192 ceph-mon[34664]: mgrmap e15: x(active, since 91s), standbys: z, y 2024-05-08T05:59:04.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:03 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T05:59:04.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:03 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/1441961430' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:59:04.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:04 smithi123 ceph-mon[29787]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:05.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:04 smithi163 ceph-mon[34520]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:05.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:04 smithi192 ceph-mon[34664]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:06.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:06 smithi123 ceph-mon[29787]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:07.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:06 smithi163 ceph-mon[34520]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:07.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:06 smithi192 ceph-mon[34664]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:08.862 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:08 smithi123 ceph-mon[29787]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:09.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:08 smithi163 ceph-mon[34520]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:09.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:08 smithi192 ceph-mon[34664]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:10.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:10 smithi123 ceph-mon[29787]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:11.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:10 smithi163 ceph-mon[34520]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:11.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:10 smithi192 ceph-mon[34664]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:12.909 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:12 smithi123 ceph-mon[29787]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:12.909 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:12 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-08T05:59:12.909 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:12 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:59:12.909 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:12 smithi123 ceph-mon[29787]: Deploying daemon osd.0 on smithi123 2024-05-08T05:59:13.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:12 smithi163 ceph-mon[34520]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:13.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:12 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-08T05:59:13.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:12 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:59:13.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:12 smithi163 ceph-mon[34520]: Deploying daemon osd.0 on smithi123 2024-05-08T05:59:13.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:12 smithi192 ceph-mon[34664]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:13.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:12 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-08T05:59:13.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:12 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:59:13.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:12 smithi192 ceph-mon[34664]: Deploying daemon osd.0 on smithi123 2024-05-08T05:59:14.877 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:14 smithi123 ceph-mon[29787]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:15.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:14 smithi163 ceph-mon[34520]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:15.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:14 smithi192 ceph-mon[34664]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:16.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:16 smithi123 ceph-mon[29787]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:17.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:16 smithi163 ceph-mon[34520]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:17.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:16 smithi192 ceph-mon[34664]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:18.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:18 smithi123 ceph-mon[29787]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:19.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:18 smithi163 ceph-mon[34520]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:19.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:18 smithi192 ceph-mon[34664]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:20.880 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:20 smithi123 ceph-mon[29787]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:20.880 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:20 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:20.880 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:20 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:20.880 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:20 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:59:21.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:20 smithi163 ceph-mon[34520]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:21.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:20 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:21.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:20 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:21.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:20 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:59:21.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:20 smithi192 ceph-mon[34664]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:21.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:20 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:21.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:20 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:21.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:20 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:59:22.918 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:22 smithi163 ceph-mon[34520]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:22.920 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:22 smithi123 ceph-mon[29787]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:23.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:22 smithi192 ceph-mon[34664]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:24.020 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:23 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:24.021 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:23 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:24.021 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:23 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:59:24.021 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:23 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:59:24.021 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:23 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:24.021 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:23 smithi123 ceph-mon[29787]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:24.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:23 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:24.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:23 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:24.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:23 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:59:24.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:23 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:59:24.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:23 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:24.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:23 smithi163 ceph-mon[34520]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:24.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:23 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:24.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:23 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:24.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:23 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:59:24.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:23 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:59:24.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:23 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:24.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:23 smithi192 ceph-mon[34664]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:24.990 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 0 on host 'smithi123' 2024-05-08T05:59:24.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:59:24.989+0000 7f2c46000700 1 -- 172.21.15.123:0/2276765334 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2c480638e0 con 0x7f2c3003e750 2024-05-08T05:59:24.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:59:24.993+0000 7f2c50400700 1 -- 172.21.15.123:0/2276765334 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f2c3003e750 msgr2=0x7f2c30040c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:24.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:59:24.993+0000 7f2c50400700 1 --2- 172.21.15.123:0/2276765334 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f2c3003e750 0x7f2c30040c10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2c34000c00 tx=0x7f2c34005fb0 comp rx=0 tx=0).stop 2024-05-08T05:59:24.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:59:24.993+0000 7f2c50400700 1 -- 172.21.15.123:0/2276765334 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2c48103550 msgr2=0x7f2c48077240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:24.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:59:24.993+0000 7f2c50400700 1 --2- 172.21.15.123:0/2276765334 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2c48103550 0x7f2c48077240 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c00e970 tx=0x7f2c3c00ec80 comp rx=0 tx=0).stop 2024-05-08T05:59:24.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:59:24.993+0000 7f2c50400700 1 -- 172.21.15.123:0/2276765334 shutdown_connections 2024-05-08T05:59:24.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:59:24.993+0000 7f2c50400700 1 --2- 172.21.15.123:0/2276765334 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f2c3003e750 0x7f2c30040c10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:24.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:59:24.993+0000 7f2c50400700 1 --2- 172.21.15.123:0/2276765334 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c481050f0 0x7f2c48075dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:24.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:59:24.993+0000 7f2c50400700 1 --2- 172.21.15.123:0/2276765334 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2c48103550 0x7f2c48077240 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:24.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:59:24.993+0000 7f2c50400700 1 --2- 172.21.15.123:0/2276765334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c48104750 0x7f2c48075890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:24.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:59:24.993+0000 7f2c50400700 1 -- 172.21.15.123:0/2276765334 >> 172.21.15.123:0/2276765334 conn(0x7f2c480feaf0 msgr2=0x7f2c48107a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:24.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:59:24.993+0000 7f2c50400700 1 -- 172.21.15.123:0/2276765334 shutdown_connections 2024-05-08T05:59:24.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T05:59:24.994+0000 7f2c50400700 1 -- 172.21.15.123:0/2276765334 wait complete. 2024-05-08T05:59:25.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:24 smithi163 ceph-mon[34520]: from='osd.0 [v2:172.21.15.123:6802/3608618058,v1:172.21.15.123:6803/3608618058]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T05:59:25.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:24 smithi163 ceph-mon[34520]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T05:59:25.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:24 smithi192 ceph-mon[34664]: from='osd.0 [v2:172.21.15.123:6802/3608618058,v1:172.21.15.123:6803/3608618058]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T05:59:25.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:24 smithi192 ceph-mon[34664]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T05:59:25.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:24 smithi123 ceph-mon[29787]: from='osd.0 [v2:172.21.15.123:6802/3608618058,v1:172.21.15.123:6803/3608618058]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T05:59:25.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:24 smithi123 ceph-mon[29787]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T05:59:25.743 DEBUG:teuthology.orchestra.run.smithi123:osd.0> sudo journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@osd.0.service 2024-05-08T05:59:25.747 INFO:tasks.cephadm:Deploying osd.1 on smithi163 with /dev/vg_nvme/lv_4... 2024-05-08T05:59:25.747 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-08T05:59:25.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:25 smithi123 ceph-mon[29787]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-08T05:59:25.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:25 smithi123 ceph-mon[29787]: from='osd.0 [v2:172.21.15.123:6802/3608618058,v1:172.21.15.123:6803/3608618058]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-08T05:59:25.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:25 smithi123 ceph-mon[29787]: osdmap e6: 1 total, 0 up, 1 in 2024-05-08T05:59:25.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:25 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:59:25.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:25 smithi123 ceph-mon[29787]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-08T05:59:25.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:25 smithi123 ceph-mon[29787]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:25.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:25 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:25.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:25 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:59:25.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:25 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:25.956 INFO:journalctl@ceph.osd.0.smithi123.stdout:May 08 05:59:25 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-0[44030]: 2024-05-08T05:59:25.819+0000 7fea72600700 -1 osd.0 0 waiting for initial osdmap 2024-05-08T05:59:25.957 INFO:journalctl@ceph.osd.0.smithi123.stdout:May 08 05:59:25 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-0[44030]: 2024-05-08T05:59:25.823+0000 7fea6d400700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:59:25.996 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T05:59:26.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:25 smithi163 ceph-mon[34520]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-08T05:59:26.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:25 smithi163 ceph-mon[34520]: from='osd.0 [v2:172.21.15.123:6802/3608618058,v1:172.21.15.123:6803/3608618058]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-08T05:59:26.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:25 smithi163 ceph-mon[34520]: osdmap e6: 1 total, 0 up, 1 in 2024-05-08T05:59:26.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:25 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:59:26.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:25 smithi163 ceph-mon[34520]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-08T05:59:26.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:25 smithi163 ceph-mon[34520]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:26.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:25 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:26.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:25 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:59:26.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:25 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:26.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:25 smithi192 ceph-mon[34664]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-08T05:59:26.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:25 smithi192 ceph-mon[34664]: from='osd.0 [v2:172.21.15.123:6802/3608618058,v1:172.21.15.123:6803/3608618058]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-08T05:59:26.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:25 smithi192 ceph-mon[34664]: osdmap e6: 1 total, 0 up, 1 in 2024-05-08T05:59:26.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:25 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:59:26.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:25 smithi192 ceph-mon[34664]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-08T05:59:26.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:25 smithi192 ceph-mon[34664]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:26.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:25 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:26.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:25 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:59:26.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:25 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:27.125 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:26 smithi163 ceph-mon[34520]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-05-08T05:59:27.126 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:26 smithi163 ceph-mon[34520]: osdmap e7: 1 total, 0 up, 1 in 2024-05-08T05:59:27.126 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:26 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:59:27.126 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:26 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:59:27.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:26 smithi123 ceph-mon[29787]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-05-08T05:59:27.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:26 smithi123 ceph-mon[29787]: osdmap e7: 1 total, 0 up, 1 in 2024-05-08T05:59:27.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:26 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:59:27.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:26 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:59:27.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:26 smithi192 ceph-mon[34664]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-05-08T05:59:27.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:26 smithi192 ceph-mon[34664]: osdmap e7: 1 total, 0 up, 1 in 2024-05-08T05:59:27.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:26 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:59:27.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:26 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:59:27.744 INFO:teuthology.orchestra.run.smithi163.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-08T05:59:27.744 INFO:teuthology.orchestra.run.smithi163.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-08T05:59:27.744 INFO:teuthology.orchestra.run.smithi163.stderr: stderr: 10+0 records in 2024-05-08T05:59:27.744 INFO:teuthology.orchestra.run.smithi163.stderr:10+0 records out 2024-05-08T05:59:27.744 INFO:teuthology.orchestra.run.smithi163.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114954 s, 912 MB/s 2024-05-08T05:59:27.744 INFO:teuthology.orchestra.run.smithi163.stderr:--> Zapping successful for: 2024-05-08T05:59:27.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:27 smithi123 ceph-mon[29787]: purged_snaps scrub starts 2024-05-08T05:59:27.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:27 smithi123 ceph-mon[29787]: purged_snaps scrub ok 2024-05-08T05:59:27.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:27 smithi123 ceph-mon[29787]: osd.0 [v2:172.21.15.123:6802/3608618058,v1:172.21.15.123:6803/3608618058] boot 2024-05-08T05:59:27.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:27 smithi123 ceph-mon[29787]: osdmap e8: 1 total, 1 up, 1 in 2024-05-08T05:59:27.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:27 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:59:27.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:27 smithi123 ceph-mon[29787]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:28.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:27 smithi163 ceph-mon[34520]: purged_snaps scrub starts 2024-05-08T05:59:28.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:27 smithi163 ceph-mon[34520]: purged_snaps scrub ok 2024-05-08T05:59:28.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:27 smithi163 ceph-mon[34520]: osd.0 [v2:172.21.15.123:6802/3608618058,v1:172.21.15.123:6803/3608618058] boot 2024-05-08T05:59:28.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:27 smithi163 ceph-mon[34520]: osdmap e8: 1 total, 1 up, 1 in 2024-05-08T05:59:28.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:27 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:59:28.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:27 smithi163 ceph-mon[34520]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:28.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:27 smithi192 ceph-mon[34664]: purged_snaps scrub starts 2024-05-08T05:59:28.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:27 smithi192 ceph-mon[34664]: purged_snaps scrub ok 2024-05-08T05:59:28.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:27 smithi192 ceph-mon[34664]: osd.0 [v2:172.21.15.123:6802/3608618058,v1:172.21.15.123:6803/3608618058] boot 2024-05-08T05:59:28.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:27 smithi192 ceph-mon[34664]: osdmap e8: 1 total, 1 up, 1 in 2024-05-08T05:59:28.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:59:28.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:27 smithi192 ceph-mon[34664]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:59:28.304 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi163:vg_nvme/lv_4 2024-05-08T05:59:28.535 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T05:59:28.930 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:28 smithi163 ceph-mon[34520]: osdmap e9: 1 total, 1 up, 1 in 2024-05-08T05:59:29.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:28 smithi123 ceph-mon[29787]: osdmap e9: 1 total, 1 up, 1 in 2024-05-08T05:59:29.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:28 smithi192 ceph-mon[34664]: osdmap e9: 1 total, 1 up, 1 in 2024-05-08T05:59:29.922 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:29 smithi163 ceph-mon[34520]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:59:29.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:29 smithi123 ceph-mon[29787]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:59:30.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.028+0000 7f7761ddc700 1 -- 172.21.15.163:0/1992152413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f775c102410 msgr2=0x7f775c102830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:30.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.028+0000 7f7761ddc700 1 --2- 172.21.15.163:0/1992152413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f775c102410 0x7f775c102830 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f774c0099c0 tx=0x7f774c009cd0 comp rx=0 tx=0).stop 2024-05-08T05:59:30.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.028+0000 7f7761ddc700 1 -- 172.21.15.163:0/1992152413 shutdown_connections 2024-05-08T05:59:30.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.028+0000 7f7761ddc700 1 --2- 172.21.15.163:0/1992152413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f775c103fb0 0x7f775c10a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:30.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.028+0000 7f7761ddc700 1 --2- 172.21.15.163:0/1992152413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f775c102410 0x7f775c102830 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:30.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.028+0000 7f7761ddc700 1 --2- 172.21.15.163:0/1992152413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f775c103610 0x7f775c103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:30.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.028+0000 7f7761ddc700 1 -- 172.21.15.163:0/1992152413 >> 172.21.15.163:0/1992152413 conn(0x7f775c0fd9d0 msgr2=0x7f775c0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:30.031 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.028+0000 7f7761ddc700 1 -- 172.21.15.163:0/1992152413 shutdown_connections 2024-05-08T05:59:30.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.028+0000 7f7761ddc700 1 -- 172.21.15.163:0/1992152413 wait complete. 2024-05-08T05:59:30.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.029+0000 7f7761ddc700 1 Processor -- start 2024-05-08T05:59:30.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.029+0000 7f7761ddc700 1 -- start start 2024-05-08T05:59:30.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.029+0000 7f7761ddc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f775c102410 0x7f775c199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:30.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.029+0000 7f7761ddc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f775c103610 0x7f775c199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:30.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.029+0000 7f7761ddc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f775c103fb0 0x7f775c1a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:30.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.029+0000 7f7761ddc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f775c10df80 con 0x7f775c103fb0 2024-05-08T05:59:30.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.029+0000 7f7761ddc700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f775c10de00 con 0x7f775c102410 2024-05-08T05:59:30.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.029+0000 7f7761ddc700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f775c10e100 con 0x7f775c103610 2024-05-08T05:59:30.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.030+0000 7f775b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f775c102410 0x7f775c199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:30.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.030+0000 7f775b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f775c102410 0x7f775c199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46954/0 (socket says 172.21.15.163:46954) 2024-05-08T05:59:30.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.030+0000 7f775b7fe700 1 -- 172.21.15.163:0/1540942305 learned_addr learned my addr 172.21.15.163:0/1540942305 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T05:59:30.032 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.030+0000 7f7752ffd700 1 --2- 172.21.15.163:0/1540942305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f775c103610 0x7f775c199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:30.033 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.030+0000 7f775b7fe700 1 -- 172.21.15.163:0/1540942305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f775c103610 msgr2=0x7f775c199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:30.033 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.030+0000 7f775b7fe700 1 --2- 172.21.15.163:0/1540942305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f775c103610 0x7f775c199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:30.033 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.030+0000 7f775b7fe700 1 -- 172.21.15.163:0/1540942305 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f775c103fb0 msgr2=0x7f775c1a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:30.033 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.030+0000 7f775b7fe700 1 --2- 172.21.15.163:0/1540942305 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f775c103fb0 0x7f775c1a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:30.033 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.030+0000 7f775b7fe700 1 -- 172.21.15.163:0/1540942305 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f775c1a13d0 con 0x7f775c102410 2024-05-08T05:59:30.033 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.030+0000 7f7752ffd700 1 --2- 172.21.15.163:0/1540942305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f775c103610 0x7f775c199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:59:30.033 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.030+0000 7f775b7fe700 1 --2- 172.21.15.163:0/1540942305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f775c102410 0x7f775c199770 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f774c009f50 tx=0x7f774c003750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:30.033 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.030+0000 7f77597fa700 1 -- 172.21.15.163:0/1540942305 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c027070 con 0x7f775c102410 2024-05-08T05:59:30.033 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.030+0000 7f7761ddc700 1 -- 172.21.15.163:0/1540942305 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f775c1a1660 con 0x7f775c102410 2024-05-08T05:59:30.033 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.030+0000 7f77597fa700 1 -- 172.21.15.163:0/1540942305 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f774c022470 con 0x7f775c102410 2024-05-08T05:59:30.033 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.030+0000 7f77597fa700 1 -- 172.21.15.163:0/1540942305 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c00f6c0 con 0x7f775c102410 2024-05-08T05:59:30.033 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.030+0000 7f7761ddc700 1 -- 172.21.15.163:0/1540942305 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f775c1a1c10 con 0x7f775c102410 2024-05-08T05:59:30.034 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.031+0000 7f7761ddc700 1 -- 172.21.15.163:0/1540942305 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f775c06a190 con 0x7f775c102410 2024-05-08T05:59:30.034 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.032+0000 7f77597fa700 1 -- 172.21.15.163:0/1540942305 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 15) ==== 149597+0+0 (secure 0 0 0) 0x7f774c02a070 con 0x7f775c102410 2024-05-08T05:59:30.034 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.033+0000 7f77597fa700 1 --2- 172.21.15.163:0/1540942305 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f77480b27f0 0x7f77480b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:30.034 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.033+0000 7f77597fa700 1 -- 172.21.15.163:0/1540942305 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f774c0e1810 con 0x7f775c102410 2024-05-08T05:59:30.034 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.034+0000 7f7752ffd700 1 --2- 172.21.15.163:0/1540942305 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f77480b27f0 0x7f77480b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:30.035 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.035+0000 7f7752ffd700 1 --2- 172.21.15.163:0/1540942305 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f77480b27f0 0x7f77480b4cb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f77440099d0 tx=0x7f7744008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:30.038 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.037+0000 7f77597fa700 1 -- 172.21.15.163:0/1540942305 <== mon.2 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f774c013070 con 0x7f775c102410 2024-05-08T05:59:30.248 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:30.247+0000 7f7761ddc700 1 -- 172.21.15.163:0/1540942305 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f775c06c300 con 0x7f77480b27f0 2024-05-08T05:59:30.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:29 smithi192 ceph-mon[34664]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:59:31.174 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:30 smithi163 ceph-mon[34520]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:59:31.174 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:30 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:59:31.174 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:30 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:59:31.174 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:30 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:59:31.174 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:30 smithi163 ceph-mon[34520]: Detected new or changed devices on smithi123 2024-05-08T05:59:31.174 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:30 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:31.174 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:30 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:31.174 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:30 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:59:31.174 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:30 smithi163 ceph-mon[34520]: Adjusting osd_memory_target on smithi123 to 17089M 2024-05-08T05:59:31.174 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:30 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:31.175 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:30 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:59:31.175 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:30 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:59:31.175 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:30 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:31.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:30 smithi123 ceph-mon[29787]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:59:31.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:30 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:59:31.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:30 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:59:31.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:30 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:59:31.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:30 smithi123 ceph-mon[29787]: Detected new or changed devices on smithi123 2024-05-08T05:59:31.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:30 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:31.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:30 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:31.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:30 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:59:31.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:30 smithi123 ceph-mon[29787]: Adjusting osd_memory_target on smithi123 to 17089M 2024-05-08T05:59:31.207 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:30 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:31.207 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:30 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:59:31.207 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:30 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:59:31.207 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:30 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:31.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:30 smithi192 ceph-mon[34664]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:59:31.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:30 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:59:31.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:30 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:59:31.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:30 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:59:31.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:30 smithi192 ceph-mon[34664]: Detected new or changed devices on smithi123 2024-05-08T05:59:31.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:30 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:31.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:30 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:31.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:30 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:59:31.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:30 smithi192 ceph-mon[34664]: Adjusting osd_memory_target on smithi123 to 17089M 2024-05-08T05:59:31.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:30 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:31.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:30 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:59:31.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:30 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:59:31.302 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:30 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:32.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:31 smithi163 ceph-mon[34520]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:59:32.109 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:31 smithi123 ceph-mon[29787]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:59:32.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:31 smithi192 ceph-mon[34664]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:59:33.200 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:32 smithi163 ceph-mon[34520]: from='client.? 172.21.15.163:0/3630241134' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e023ae3d-a82e-4bb6-8fef-889dd07fe183"}]: dispatch 2024-05-08T05:59:33.201 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:32 smithi163 ceph-mon[34520]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e023ae3d-a82e-4bb6-8fef-889dd07fe183"}]: dispatch 2024-05-08T05:59:33.201 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:32 smithi163 ceph-mon[34520]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e023ae3d-a82e-4bb6-8fef-889dd07fe183"}]': finished 2024-05-08T05:59:33.201 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:32 smithi163 ceph-mon[34520]: osdmap e10: 2 total, 1 up, 2 in 2024-05-08T05:59:33.201 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:32 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:59:33.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:32 smithi123 ceph-mon[29787]: from='client.? 172.21.15.163:0/3630241134' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e023ae3d-a82e-4bb6-8fef-889dd07fe183"}]: dispatch 2024-05-08T05:59:33.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:32 smithi123 ceph-mon[29787]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e023ae3d-a82e-4bb6-8fef-889dd07fe183"}]: dispatch 2024-05-08T05:59:33.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:32 smithi123 ceph-mon[29787]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e023ae3d-a82e-4bb6-8fef-889dd07fe183"}]': finished 2024-05-08T05:59:33.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:32 smithi123 ceph-mon[29787]: osdmap e10: 2 total, 1 up, 2 in 2024-05-08T05:59:33.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:32 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:59:33.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:32 smithi192 ceph-mon[34664]: from='client.? 172.21.15.163:0/3630241134' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e023ae3d-a82e-4bb6-8fef-889dd07fe183"}]: dispatch 2024-05-08T05:59:33.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:32 smithi192 ceph-mon[34664]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e023ae3d-a82e-4bb6-8fef-889dd07fe183"}]: dispatch 2024-05-08T05:59:33.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:32 smithi192 ceph-mon[34664]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e023ae3d-a82e-4bb6-8fef-889dd07fe183"}]': finished 2024-05-08T05:59:33.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:32 smithi192 ceph-mon[34664]: osdmap e10: 2 total, 1 up, 2 in 2024-05-08T05:59:33.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:32 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:59:33.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:33 smithi123 ceph-mon[29787]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:59:33.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:33 smithi123 ceph-mon[29787]: from='client.? 172.21.15.163:0/2282494432' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:59:34.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:33 smithi163 ceph-mon[34520]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:59:34.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:33 smithi163 ceph-mon[34520]: from='client.? 172.21.15.163:0/2282494432' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:59:34.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:33 smithi192 ceph-mon[34664]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:59:34.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:33 smithi192 ceph-mon[34664]: from='client.? 172.21.15.163:0/2282494432' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:59:36.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:35 smithi123 ceph-mon[29787]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:59:36.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:35 smithi163 ceph-mon[34520]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:59:36.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:35 smithi192 ceph-mon[34664]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:59:38.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:37 smithi123 ceph-mon[29787]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:59:38.275 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:37 smithi163 ceph-mon[34520]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:59:38.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:37 smithi192 ceph-mon[34664]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:59:40.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:39 smithi163 ceph-mon[34520]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:59:40.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:39 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-08T05:59:40.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:39 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:59:40.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:39 smithi123 ceph-mon[29787]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:59:40.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:39 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-08T05:59:40.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:39 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:59:40.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:39 smithi192 ceph-mon[34664]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:59:40.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:39 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-08T05:59:40.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:39 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:59:41.231 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:40 smithi163 ceph-mon[34520]: Deploying daemon osd.1 on smithi163 2024-05-08T05:59:41.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:40 smithi192 ceph-mon[34664]: Deploying daemon osd.1 on smithi163 2024-05-08T05:59:41.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:40 smithi123 ceph-mon[29787]: Deploying daemon osd.1 on smithi163 2024-05-08T05:59:42.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:41 smithi163 ceph-mon[34520]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:59:42.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:41 smithi192 ceph-mon[34664]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:59:42.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:41 smithi123 ceph-mon[29787]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:59:44.121 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:43 smithi163 ceph-mon[34520]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:59:44.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:43 smithi123 ceph-mon[29787]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:59:44.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:43 smithi192 ceph-mon[34664]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:59:46.166 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:45 smithi163 ceph-mon[34520]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:59:46.166 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:45 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:46.167 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:45 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:46.167 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:45 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:59:46.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:45 smithi192 ceph-mon[34664]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:59:46.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:45 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:46.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:45 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:46.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:45 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:59:46.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:45 smithi123 ceph-mon[29787]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:59:46.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:45 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:46.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:45 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:46.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:45 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:59:48.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:47 smithi163 ceph-mon[34520]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:59:48.048 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:48.048+0000 7f77597fa700 1 -- 172.21.15.163:0/1540942305 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f775c06c300 con 0x7f77480b27f0 2024-05-08T05:59:48.049 INFO:teuthology.orchestra.run.smithi163.stdout:Created osd(s) 1 on host 'smithi163' 2024-05-08T05:59:48.051 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:48.050+0000 7f7761ddc700 1 -- 172.21.15.163:0/1540942305 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f77480b27f0 msgr2=0x7f77480b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:48.051 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:48.050+0000 7f7761ddc700 1 --2- 172.21.15.163:0/1540942305 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f77480b27f0 0x7f77480b4cb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f77440099d0 tx=0x7f7744008040 comp rx=0 tx=0).stop 2024-05-08T05:59:48.051 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:48.050+0000 7f7761ddc700 1 -- 172.21.15.163:0/1540942305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f775c102410 msgr2=0x7f775c199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:48.051 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:48.050+0000 7f7761ddc700 1 --2- 172.21.15.163:0/1540942305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f775c102410 0x7f775c199770 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f774c009f50 tx=0x7f774c003750 comp rx=0 tx=0).stop 2024-05-08T05:59:48.051 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:48.051+0000 7f7761ddc700 1 -- 172.21.15.163:0/1540942305 shutdown_connections 2024-05-08T05:59:48.051 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:48.051+0000 7f7761ddc700 1 --2- 172.21.15.163:0/1540942305 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f77480b27f0 0x7f77480b4cb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:48.051 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:48.051+0000 7f7761ddc700 1 --2- 172.21.15.163:0/1540942305 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f775c103fb0 0x7f775c1a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:48.051 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:48.051+0000 7f7761ddc700 1 --2- 172.21.15.163:0/1540942305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f775c102410 0x7f775c199770 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:48.051 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:48.051+0000 7f7761ddc700 1 --2- 172.21.15.163:0/1540942305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f775c103610 0x7f775c199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:48.051 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:48.051+0000 7f7761ddc700 1 -- 172.21.15.163:0/1540942305 >> 172.21.15.163:0/1540942305 conn(0x7f775c0fd9d0 msgr2=0x7f775c06bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:48.051 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:48.051+0000 7f7761ddc700 1 -- 172.21.15.163:0/1540942305 shutdown_connections 2024-05-08T05:59:48.051 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T05:59:48.051+0000 7f7761ddc700 1 -- 172.21.15.163:0/1540942305 wait complete. 2024-05-08T05:59:48.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:47 smithi192 ceph-mon[34664]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:59:48.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:47 smithi123 ceph-mon[29787]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:59:48.702 DEBUG:teuthology.orchestra.run.smithi163:osd.1> sudo journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@osd.1.service 2024-05-08T05:59:48.705 INFO:tasks.cephadm:Deploying osd.2 on smithi192 with /dev/vg_nvme/lv_4... 2024-05-08T05:59:48.705 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-08T05:59:48.950 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:59:49.014 INFO:journalctl@ceph.osd.1.smithi163.stdout:May 08 05:59:48 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-1[39131]: 2024-05-08T05:59:48.837+0000 7f861b0d47c0 -1 osd.1 0 log_to_monitors true 2024-05-08T05:59:49.279 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:49 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:49.279 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:49 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:49.279 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:49 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:49.279 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:49 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:49.279 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:49 smithi192 ceph-mon[34664]: from='osd.1 [v2:172.21.15.163:6800/1884753449,v1:172.21.15.163:6801/1884753449]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T05:59:49.279 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:49 smithi192 ceph-mon[34664]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T05:59:49.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:49 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:49.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:49 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:49.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:49 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:49.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:49 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:49.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:49 smithi163 ceph-mon[34520]: from='osd.1 [v2:172.21.15.163:6800/1884753449,v1:172.21.15.163:6801/1884753449]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T05:59:49.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:49 smithi163 ceph-mon[34520]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T05:59:49.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:49 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:49.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:49 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:49.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:49 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:49.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:49 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:49.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:49 smithi123 ceph-mon[29787]: from='osd.1 [v2:172.21.15.163:6800/1884753449,v1:172.21.15.163:6801/1884753449]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T05:59:49.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:49 smithi123 ceph-mon[29787]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T05:59:50.224 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:50 smithi192 ceph-mon[34664]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:59:50.224 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:50 smithi192 ceph-mon[34664]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-08T05:59:50.225 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:50 smithi192 ceph-mon[34664]: from='osd.1 [v2:172.21.15.163:6800/1884753449,v1:172.21.15.163:6801/1884753449]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-05-08T05:59:50.225 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:50 smithi192 ceph-mon[34664]: osdmap e11: 2 total, 1 up, 2 in 2024-05-08T05:59:50.225 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:50 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:59:50.225 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:50 smithi192 ceph-mon[34664]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-05-08T05:59:50.449 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:50 smithi163 ceph-mon[34520]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:59:50.449 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:50 smithi163 ceph-mon[34520]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-08T05:59:50.449 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:50 smithi163 ceph-mon[34520]: from='osd.1 [v2:172.21.15.163:6800/1884753449,v1:172.21.15.163:6801/1884753449]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-05-08T05:59:50.449 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:50 smithi163 ceph-mon[34520]: osdmap e11: 2 total, 1 up, 2 in 2024-05-08T05:59:50.449 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:50 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:59:50.449 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:50 smithi163 ceph-mon[34520]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-05-08T05:59:50.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:50 smithi123 ceph-mon[29787]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:59:50.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:50 smithi123 ceph-mon[29787]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-08T05:59:50.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:50 smithi123 ceph-mon[29787]: from='osd.1 [v2:172.21.15.163:6800/1884753449,v1:172.21.15.163:6801/1884753449]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-05-08T05:59:50.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:50 smithi123 ceph-mon[29787]: osdmap e11: 2 total, 1 up, 2 in 2024-05-08T05:59:50.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:50 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:59:50.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:50 smithi123 ceph-mon[29787]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-05-08T05:59:50.788 INFO:journalctl@ceph.osd.1.smithi163.stdout:May 08 05:59:50 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-1[39131]: 2024-05-08T05:59:50.613+0000 7f8610738700 -1 osd.1 0 waiting for initial osdmap 2024-05-08T05:59:50.788 INFO:journalctl@ceph.osd.1.smithi163.stdout:May 08 05:59:50 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-1[39131]: 2024-05-08T05:59:50.616+0000 7f8608523700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:59:50.790 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-08T05:59:50.790 INFO:teuthology.orchestra.run.smithi192.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-08T05:59:50.790 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10+0 records in 2024-05-08T05:59:50.790 INFO:teuthology.orchestra.run.smithi192.stderr:10+0 records out 2024-05-08T05:59:50.790 INFO:teuthology.orchestra.run.smithi192.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112639 s, 931 MB/s 2024-05-08T05:59:50.790 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping successful for: 2024-05-08T05:59:51.431 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:51 smithi192 ceph-mon[34664]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-05-08T05:59:51.431 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:51 smithi192 ceph-mon[34664]: osdmap e12: 2 total, 1 up, 2 in 2024-05-08T05:59:51.431 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:51 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:59:51.431 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:51 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:59:51.432 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:51 smithi192 ceph-mon[34664]: from='osd.1 ' entity='osd.1' 2024-05-08T05:59:51.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:51 smithi123 ceph-mon[29787]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-05-08T05:59:51.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:51 smithi123 ceph-mon[29787]: osdmap e12: 2 total, 1 up, 2 in 2024-05-08T05:59:51.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:51 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:59:51.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:51 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:59:51.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:51 smithi123 ceph-mon[29787]: from='osd.1 ' entity='osd.1' 2024-05-08T05:59:51.458 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:51 smithi163 ceph-mon[34520]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-05-08T05:59:51.459 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:51 smithi163 ceph-mon[34520]: osdmap e12: 2 total, 1 up, 2 in 2024-05-08T05:59:51.459 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:51 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:59:51.459 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:51 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:59:51.459 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:51 smithi163 ceph-mon[34520]: from='osd.1 ' entity='osd.1' 2024-05-08T05:59:51.461 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi192:vg_nvme/lv_4 2024-05-08T05:59:51.703 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:59:52.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:52 smithi123 ceph-mon[29787]: purged_snaps scrub starts 2024-05-08T05:59:52.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:52 smithi123 ceph-mon[29787]: purged_snaps scrub ok 2024-05-08T05:59:52.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:52 smithi123 ceph-mon[29787]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:59:52.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:52 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:59:52.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:52 smithi123 ceph-mon[29787]: osd.1 [v2:172.21.15.163:6800/1884753449,v1:172.21.15.163:6801/1884753449] boot 2024-05-08T05:59:52.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:52 smithi123 ceph-mon[29787]: osdmap e13: 2 total, 2 up, 2 in 2024-05-08T05:59:52.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:52 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:59:52.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:52 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:52.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:52 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:52.457 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:52 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:59:52.457 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:52 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:52.457 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:52 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:59:52.457 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:52 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:59:52.457 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:52 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:52.457 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:52 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:59:52.457 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:52 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:59:52.457 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:52 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:59:52.457 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:52 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:52.496 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:52 smithi192 ceph-mon[34664]: purged_snaps scrub starts 2024-05-08T05:59:52.496 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:52 smithi192 ceph-mon[34664]: purged_snaps scrub ok 2024-05-08T05:59:52.497 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:52 smithi192 ceph-mon[34664]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:59:52.497 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:52 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:59:52.497 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:52 smithi192 ceph-mon[34664]: osd.1 [v2:172.21.15.163:6800/1884753449,v1:172.21.15.163:6801/1884753449] boot 2024-05-08T05:59:52.497 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:52 smithi192 ceph-mon[34664]: osdmap e13: 2 total, 2 up, 2 in 2024-05-08T05:59:52.497 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:52 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:59:52.497 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:52 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:52.497 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:52 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:52.497 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:52 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:59:52.497 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:52 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:52.497 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:52 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:59:52.497 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:52 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:59:52.497 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:52 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:52.497 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:52 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:59:52.497 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:52 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:59:52.497 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:52 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:59:52.498 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:52 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:52.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:52 smithi163 ceph-mon[34520]: purged_snaps scrub starts 2024-05-08T05:59:52.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:52 smithi163 ceph-mon[34520]: purged_snaps scrub ok 2024-05-08T05:59:52.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:52 smithi163 ceph-mon[34520]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:59:52.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:52 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:59:52.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:52 smithi163 ceph-mon[34520]: osd.1 [v2:172.21.15.163:6800/1884753449,v1:172.21.15.163:6801/1884753449] boot 2024-05-08T05:59:52.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:52 smithi163 ceph-mon[34520]: osdmap e13: 2 total, 2 up, 2 in 2024-05-08T05:59:52.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:52 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:59:52.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:52 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:52.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:52 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:52.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:52 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:59:52.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:52 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:52.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:52 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:59:52.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:52 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:59:52.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:52 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:52.540 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:52 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:59:52.540 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:52 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:59:52.540 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:52 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:59:52.540 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:52 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T05:59:53.237 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.234+0000 7fa44bdb5700 1 -- 172.21.15.192:0/3902780368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa444103500 msgr2=0x7fa444103920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:53.237 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.234+0000 7fa44bdb5700 1 --2- 172.21.15.192:0/3902780368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa444103500 0x7fa444103920 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa440009990 tx=0x7fa440009ca0 comp rx=0 tx=0).stop 2024-05-08T05:59:53.238 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.234+0000 7fa44bdb5700 1 -- 172.21.15.192:0/3902780368 shutdown_connections 2024-05-08T05:59:53.238 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.234+0000 7fa44bdb5700 1 --2- 172.21.15.192:0/3902780368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4441050a0 0x7fa44410b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:53.238 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.234+0000 7fa44bdb5700 1 --2- 172.21.15.192:0/3902780368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa444103500 0x7fa444103920 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:53.238 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.234+0000 7fa44bdb5700 1 --2- 172.21.15.192:0/3902780368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa444104700 0x7fa444104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:53.238 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.234+0000 7fa44bdb5700 1 -- 172.21.15.192:0/3902780368 >> 172.21.15.192:0/3902780368 conn(0x7fa4440feaa0 msgr2=0x7fa444100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:59:53.238 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.234+0000 7fa44bdb5700 1 -- 172.21.15.192:0/3902780368 shutdown_connections 2024-05-08T05:59:53.238 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.234+0000 7fa44bdb5700 1 -- 172.21.15.192:0/3902780368 wait complete. 2024-05-08T05:59:53.238 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.234+0000 7fa44bdb5700 1 Processor -- start 2024-05-08T05:59:53.238 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.234+0000 7fa44bdb5700 1 -- start start 2024-05-08T05:59:53.238 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.234+0000 7fa44bdb5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa444103500 0x7fa444199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:53.238 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.234+0000 7fa44bdb5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa444104700 0x7fa444199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:53.239 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.235+0000 7fa44bdb5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4441050a0 0x7fa4441a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:53.239 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.235+0000 7fa44bdb5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa44410f070 con 0x7fa444103500 2024-05-08T05:59:53.239 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.235+0000 7fa44bdb5700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa44410eef0 con 0x7fa444104700 2024-05-08T05:59:53.239 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.235+0000 7fa44bdb5700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa44410f1f0 con 0x7fa4441050a0 2024-05-08T05:59:53.239 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.235+0000 7fa44a352700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4441050a0 0x7fa4441a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:53.239 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.235+0000 7fa44a352700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4441050a0 0x7fa4441a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:38462/0 (socket says 172.21.15.192:38462) 2024-05-08T05:59:53.239 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.235+0000 7fa44a352700 1 -- 172.21.15.192:0/755541615 learned_addr learned my addr 172.21.15.192:0/755541615 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-08T05:59:53.239 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.235+0000 7fa449350700 1 --2- 172.21.15.192:0/755541615 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa444104700 0x7fa444199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:53.239 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.235+0000 7fa44a352700 1 -- 172.21.15.192:0/755541615 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa444104700 msgr2=0x7fa444199cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:53.239 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.235+0000 7fa44a352700 1 --2- 172.21.15.192:0/755541615 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa444104700 0x7fa444199cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:53.239 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.235+0000 7fa44a352700 1 -- 172.21.15.192:0/755541615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa444103500 msgr2=0x7fa444199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:59:53.239 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.235+0000 7fa44a352700 1 --2- 172.21.15.192:0/755541615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa444103500 0x7fa444199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:59:53.239 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.235+0000 7fa44a352700 1 -- 172.21.15.192:0/755541615 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4441a13f0 con 0x7fa4441050a0 2024-05-08T05:59:53.240 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.235+0000 7fa44a352700 1 --2- 172.21.15.192:0/755541615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4441050a0 0x7fa4441a0d50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa43c007f10 tx=0x7fa43c00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:53.240 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.235+0000 7fa43affd700 1 -- 172.21.15.192:0/755541615 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43c004bb0 con 0x7fa4441050a0 2024-05-08T05:59:53.240 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.235+0000 7fa43affd700 1 -- 172.21.15.192:0/755541615 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa43c004d50 con 0x7fa4441050a0 2024-05-08T05:59:53.240 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.235+0000 7fa44bdb5700 1 -- 172.21.15.192:0/755541615 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4441a16e0 con 0x7fa4441050a0 2024-05-08T05:59:53.240 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.235+0000 7fa43affd700 1 -- 172.21.15.192:0/755541615 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43c005a30 con 0x7fa4441050a0 2024-05-08T05:59:53.240 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.235+0000 7fa44bdb5700 1 -- 172.21.15.192:0/755541615 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4441a1c20 con 0x7fa4441050a0 2024-05-08T05:59:53.240 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.236+0000 7fa44bdb5700 1 -- 172.21.15.192:0/755541615 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa44406a190 con 0x7fa4441050a0 2024-05-08T05:59:53.241 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.237+0000 7fa43affd700 1 -- 172.21.15.192:0/755541615 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 15) ==== 149597+0+0 (secure 0 0 0) 0x7fa43c01a020 con 0x7fa4441050a0 2024-05-08T05:59:53.241 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.238+0000 7fa43affd700 1 --2- 172.21.15.192:0/755541615 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa4300b27f0 0x7fa4300b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:59:53.241 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.238+0000 7fa43affd700 1 -- 172.21.15.192:0/755541615 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fa43c0e06f0 con 0x7fa4441050a0 2024-05-08T05:59:53.241 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.239+0000 7fa449b51700 1 --2- 172.21.15.192:0/755541615 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa4300b27f0 0x7fa4300b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:59:53.241 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.240+0000 7fa449b51700 1 --2- 172.21.15.192:0/755541615 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa4300b27f0 0x7fa4300b4cb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa440003790 tx=0x7fa440003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:59:53.243 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.242+0000 7fa43affd700 1 -- 172.21.15.192:0/755541615 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa43c010070 con 0x7fa4441050a0 2024-05-08T05:59:53.450 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T05:59:53.449+0000 7fa44bdb5700 1 -- 172.21.15.192:0/755541615 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa4440638e0 con 0x7fa4300b27f0 2024-05-08T05:59:53.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:53 smithi192 ceph-mon[34664]: Detected new or changed devices on smithi163 2024-05-08T05:59:53.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:53 smithi192 ceph-mon[34664]: Adjusting osd_memory_target on smithi163 to 17088M 2024-05-08T05:59:53.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:53 smithi192 ceph-mon[34664]: osdmap e14: 2 total, 2 up, 2 in 2024-05-08T05:59:53.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:53 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:59:53.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:53 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:59:53.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:53 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:59:53.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:53 smithi123 ceph-mon[29787]: Detected new or changed devices on smithi163 2024-05-08T05:59:53.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:53 smithi123 ceph-mon[29787]: Adjusting osd_memory_target on smithi163 to 17088M 2024-05-08T05:59:53.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:53 smithi123 ceph-mon[29787]: osdmap e14: 2 total, 2 up, 2 in 2024-05-08T05:59:53.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:53 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:59:53.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:53 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:59:53.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:53 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:59:54.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:53 smithi163 ceph-mon[34520]: Detected new or changed devices on smithi163 2024-05-08T05:59:54.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:53 smithi163 ceph-mon[34520]: Adjusting osd_memory_target on smithi163 to 17088M 2024-05-08T05:59:54.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:53 smithi163 ceph-mon[34520]: osdmap e14: 2 total, 2 up, 2 in 2024-05-08T05:59:54.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:53 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:59:54.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:53 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:59:54.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:53 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:59:54.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:54 smithi163 ceph-mon[34520]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:59:54.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:54 smithi163 ceph-mon[34520]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:59:54.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:54 smithi123 ceph-mon[29787]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:59:54.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:54 smithi123 ceph-mon[29787]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:59:54.989 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:54 smithi192 ceph-mon[34664]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:59:54.989 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:54 smithi192 ceph-mon[34664]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:59:56.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:56 smithi192 ceph-mon[34664]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:59:56.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:56 smithi192 ceph-mon[34664]: from='client.? 172.21.15.192:0/2329836750' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69f4cfca-63cb-4862-9480-d79891b0f338"}]: dispatch 2024-05-08T05:59:56.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:56 smithi192 ceph-mon[34664]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69f4cfca-63cb-4862-9480-d79891b0f338"}]: dispatch 2024-05-08T05:59:56.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:56 smithi192 ceph-mon[34664]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "69f4cfca-63cb-4862-9480-d79891b0f338"}]': finished 2024-05-08T05:59:56.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:56 smithi192 ceph-mon[34664]: osdmap e15: 3 total, 2 up, 3 in 2024-05-08T05:59:56.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:56 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:59:56.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:56 smithi123 ceph-mon[29787]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:59:56.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:56 smithi123 ceph-mon[29787]: from='client.? 172.21.15.192:0/2329836750' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69f4cfca-63cb-4862-9480-d79891b0f338"}]: dispatch 2024-05-08T05:59:56.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:56 smithi123 ceph-mon[29787]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69f4cfca-63cb-4862-9480-d79891b0f338"}]: dispatch 2024-05-08T05:59:56.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:56 smithi123 ceph-mon[29787]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "69f4cfca-63cb-4862-9480-d79891b0f338"}]': finished 2024-05-08T05:59:56.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:56 smithi123 ceph-mon[29787]: osdmap e15: 3 total, 2 up, 3 in 2024-05-08T05:59:56.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:56 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:59:57.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:56 smithi163 ceph-mon[34520]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:59:57.042 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:56 smithi163 ceph-mon[34520]: from='client.? 172.21.15.192:0/2329836750' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69f4cfca-63cb-4862-9480-d79891b0f338"}]: dispatch 2024-05-08T05:59:57.042 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:56 smithi163 ceph-mon[34520]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69f4cfca-63cb-4862-9480-d79891b0f338"}]: dispatch 2024-05-08T05:59:57.042 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:56 smithi163 ceph-mon[34520]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "69f4cfca-63cb-4862-9480-d79891b0f338"}]': finished 2024-05-08T05:59:57.042 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:56 smithi163 ceph-mon[34520]: osdmap e15: 3 total, 2 up, 3 in 2024-05-08T05:59:57.042 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:56 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:59:57.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:57 smithi123 ceph-mon[29787]: from='client.? 172.21.15.192:0/2615027141' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:59:58.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:57 smithi163 ceph-mon[34520]: from='client.? 172.21.15.192:0/2615027141' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:59:58.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:57 smithi192 ceph-mon[34664]: from='client.? 172.21.15.192:0/2615027141' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:59:58.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 05:59:58 smithi123 ceph-mon[29787]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:59:59.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 05:59:58 smithi163 ceph-mon[34520]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:59:59.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 05:59:58 smithi192 ceph-mon[34664]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T06:00:00.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:00 smithi123 ceph-mon[29787]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T06:00:00.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:00 smithi123 ceph-mon[29787]: overall HEALTH_OK 2024-05-08T06:00:00.966 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:00 smithi192 ceph-mon[34664]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T06:00:00.966 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:00 smithi192 ceph-mon[34664]: overall HEALTH_OK 2024-05-08T06:00:01.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:00 smithi163 ceph-mon[34520]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T06:00:01.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:00 smithi163 ceph-mon[34520]: overall HEALTH_OK 2024-05-08T06:00:02.761 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:02 smithi192 ceph-mon[34664]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T06:00:02.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:02 smithi123 ceph-mon[29787]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T06:00:03.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:02 smithi163 ceph-mon[34520]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T06:00:03.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:03 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-08T06:00:03.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:03 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:00:03.973 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:03 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-08T06:00:03.973 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:03 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:00:04.001 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:03 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-08T06:00:04.001 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:03 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:00:04.930 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:04 smithi192 ceph-mon[34664]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T06:00:04.931 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:04 smithi192 ceph-mon[34664]: Deploying daemon osd.2 on smithi192 2024-05-08T06:00:04.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:04 smithi123 ceph-mon[29787]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T06:00:04.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:04 smithi123 ceph-mon[29787]: Deploying daemon osd.2 on smithi192 2024-05-08T06:00:05.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:04 smithi163 ceph-mon[34520]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T06:00:05.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:04 smithi163 ceph-mon[34520]: Deploying daemon osd.2 on smithi192 2024-05-08T06:00:06.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:06 smithi123 ceph-mon[29787]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T06:00:07.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:06 smithi163 ceph-mon[34520]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T06:00:07.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:06 smithi192 ceph-mon[34664]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T06:00:08.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:08 smithi123 ceph-mon[29787]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T06:00:08.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:08 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:08.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:08 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:08.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:08 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:00:08.967 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:08 smithi192 ceph-mon[34664]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T06:00:08.968 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:08 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:08.968 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:08 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:08.968 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:08 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:00:09.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:08 smithi163 ceph-mon[34520]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T06:00:09.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:08 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:09.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:08 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:09.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:08 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:00:10.904 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:10 smithi192 ceph-mon[34664]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T06:00:10.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:10 smithi123 ceph-mon[29787]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T06:00:11.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:10 smithi163 ceph-mon[34520]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T06:00:11.935 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:00:11.935+0000 7fa43affd700 1 -- 172.21.15.192:0/755541615 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa4440638e0 con 0x7fa4300b27f0 2024-05-08T06:00:11.935 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 2 on host 'smithi192' 2024-05-08T06:00:11.937 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:00:11.937+0000 7fa44bdb5700 1 -- 172.21.15.192:0/755541615 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa4300b27f0 msgr2=0x7fa4300b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:11.937 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:00:11.937+0000 7fa44bdb5700 1 --2- 172.21.15.192:0/755541615 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa4300b27f0 0x7fa4300b4cb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa440003790 tx=0x7fa440003860 comp rx=0 tx=0).stop 2024-05-08T06:00:11.938 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:00:11.937+0000 7fa44bdb5700 1 -- 172.21.15.192:0/755541615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4441050a0 msgr2=0x7fa4441a0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:11.938 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:00:11.937+0000 7fa44bdb5700 1 --2- 172.21.15.192:0/755541615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4441050a0 0x7fa4441a0d50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa43c007f10 tx=0x7fa43c00d3b0 comp rx=0 tx=0).stop 2024-05-08T06:00:11.938 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:00:11.937+0000 7fa44bdb5700 1 -- 172.21.15.192:0/755541615 shutdown_connections 2024-05-08T06:00:11.938 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:00:11.937+0000 7fa44bdb5700 1 --2- 172.21.15.192:0/755541615 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa4300b27f0 0x7fa4300b4cb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:11.938 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:00:11.937+0000 7fa44bdb5700 1 --2- 172.21.15.192:0/755541615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa444103500 0x7fa444199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:11.938 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:00:11.937+0000 7fa44bdb5700 1 --2- 172.21.15.192:0/755541615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4441050a0 0x7fa4441a0d50 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:11.938 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:00:11.937+0000 7fa44bdb5700 1 --2- 172.21.15.192:0/755541615 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa444104700 0x7fa444199cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:11.938 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:00:11.937+0000 7fa44bdb5700 1 -- 172.21.15.192:0/755541615 >> 172.21.15.192:0/755541615 conn(0x7fa4440feaa0 msgr2=0x7fa444107b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:00:11.938 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:00:11.937+0000 7fa44bdb5700 1 -- 172.21.15.192:0/755541615 shutdown_connections 2024-05-08T06:00:11.938 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:00:11.937+0000 7fa44bdb5700 1 -- 172.21.15.192:0/755541615 wait complete. 2024-05-08T06:00:12.548 DEBUG:teuthology.orchestra.run.smithi192:osd.2> sudo journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@osd.2.service 2024-05-08T06:00:12.550 INFO:tasks.cephadm:Deploying osd.3 on smithi123 with /dev/vg_nvme/lv_3... 2024-05-08T06:00:12.550 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-08T06:00:12.796 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:12 smithi192 ceph-mon[34664]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T06:00:12.796 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:12 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:12.796 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:12 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:12.796 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:12 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:12.797 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:12 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:12.797 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:12 smithi192 ceph-mon[34664]: from='osd.2 [v2:172.21.15.192:6800/4161570404,v1:172.21.15.192:6801/4161570404]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T06:00:12.797 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:12 smithi192 ceph-mon[34664]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T06:00:12.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:12 smithi123 ceph-mon[29787]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T06:00:12.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:12 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:12.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:12 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:12.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:12 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:12.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:12 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:12.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:12 smithi123 ceph-mon[29787]: from='osd.2 [v2:172.21.15.192:6800/4161570404,v1:172.21.15.192:6801/4161570404]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T06:00:12.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:12 smithi123 ceph-mon[29787]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T06:00:12.987 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:00:13.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:12 smithi163 ceph-mon[34520]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T06:00:13.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:12 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:13.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:12 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:13.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:12 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:13.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:12 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:13.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:12 smithi163 ceph-mon[34520]: from='osd.2 [v2:172.21.15.192:6800/4161570404,v1:172.21.15.192:6801/4161570404]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T06:00:13.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:12 smithi163 ceph-mon[34520]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T06:00:14.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:13 smithi163 ceph-mon[34520]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T06:00:14.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:13 smithi163 ceph-mon[34520]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-08T06:00:14.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:13 smithi163 ceph-mon[34520]: from='osd.2 [v2:172.21.15.192:6800/4161570404,v1:172.21.15.192:6801/4161570404]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-08T06:00:14.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:13 smithi163 ceph-mon[34520]: osdmap e16: 3 total, 2 up, 3 in 2024-05-08T06:00:14.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:13 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T06:00:14.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:13 smithi163 ceph-mon[34520]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-08T06:00:14.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:13 smithi192 ceph-mon[34664]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T06:00:14.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:13 smithi192 ceph-mon[34664]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-08T06:00:14.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:13 smithi192 ceph-mon[34664]: from='osd.2 [v2:172.21.15.192:6800/4161570404,v1:172.21.15.192:6801/4161570404]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-08T06:00:14.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:13 smithi192 ceph-mon[34664]: osdmap e16: 3 total, 2 up, 3 in 2024-05-08T06:00:14.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:13 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T06:00:14.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:13 smithi192 ceph-mon[34664]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-08T06:00:14.451 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:13 smithi123 ceph-mon[29787]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T06:00:14.451 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:13 smithi123 ceph-mon[29787]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-08T06:00:14.451 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:13 smithi123 ceph-mon[29787]: from='osd.2 [v2:172.21.15.192:6800/4161570404,v1:172.21.15.192:6801/4161570404]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-08T06:00:14.451 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:13 smithi123 ceph-mon[29787]: osdmap e16: 3 total, 2 up, 3 in 2024-05-08T06:00:14.452 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:13 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T06:00:14.452 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:13 smithi123 ceph-mon[29787]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-08T06:00:14.801 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 08 06:00:14 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-2[39865]: 2024-05-08T06:00:14.395+0000 7fed84526700 -1 osd.2 0 waiting for initial osdmap 2024-05-08T06:00:14.801 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 08 06:00:14 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-2[39865]: 2024-05-08T06:00:14.398+0000 7fed7c311700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T06:00:15.090 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:14 smithi123 ceph-mon[29787]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-05-08T06:00:15.090 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:14 smithi123 ceph-mon[29787]: osdmap e17: 3 total, 2 up, 3 in 2024-05-08T06:00:15.090 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:14 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T06:00:15.090 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:14 smithi123 ceph-mon[29787]: from='osd.2 ' entity='osd.2' 2024-05-08T06:00:15.090 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:14 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T06:00:15.215 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:14 smithi192 ceph-mon[34664]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-05-08T06:00:15.215 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:14 smithi192 ceph-mon[34664]: osdmap e17: 3 total, 2 up, 3 in 2024-05-08T06:00:15.215 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:14 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T06:00:15.215 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:14 smithi192 ceph-mon[34664]: from='osd.2 ' entity='osd.2' 2024-05-08T06:00:15.215 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:14 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T06:00:15.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:14 smithi163 ceph-mon[34520]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-05-08T06:00:15.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:14 smithi163 ceph-mon[34520]: osdmap e17: 3 total, 2 up, 3 in 2024-05-08T06:00:15.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:14 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T06:00:15.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:14 smithi163 ceph-mon[34520]: from='osd.2 ' entity='osd.2' 2024-05-08T06:00:15.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:14 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T06:00:15.973 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-08T06:00:15.973 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-08T06:00:15.973 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2024-05-08T06:00:15.974 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2024-05-08T06:00:15.974 INFO:teuthology.orchestra.run.smithi123.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0564889 s, 186 MB/s 2024-05-08T06:00:15.974 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 2024-05-08T06:00:15.974 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2024-05-08T06:00:16.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:16 smithi123 ceph-mon[29787]: purged_snaps scrub starts 2024-05-08T06:00:16.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:16 smithi123 ceph-mon[29787]: purged_snaps scrub ok 2024-05-08T06:00:16.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:16 smithi123 ceph-mon[29787]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T06:00:16.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:16 smithi123 ceph-mon[29787]: osd.2 [v2:172.21.15.192:6800/4161570404,v1:172.21.15.192:6801/4161570404] boot 2024-05-08T06:00:16.707 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:16 smithi123 ceph-mon[29787]: osdmap e18: 3 total, 3 up, 3 in 2024-05-08T06:00:16.707 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:16 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T06:00:16.707 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:16 smithi123 ceph-mon[29787]: Detected new or changed devices on smithi192 2024-05-08T06:00:16.707 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:16 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:16.707 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:16 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:16.707 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:16 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:00:16.707 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:16 smithi123 ceph-mon[29787]: Adjusting osd_memory_target on smithi192 to 17088M 2024-05-08T06:00:16.707 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:16 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:16.707 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:16 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:00:16.707 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:16 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:00:16.707 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:16 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:16.707 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:16 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:00:16.707 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:16 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:00:16.707 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:16 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:00:16.708 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:16 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:16.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:16 smithi163 ceph-mon[34520]: purged_snaps scrub starts 2024-05-08T06:00:16.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:16 smithi163 ceph-mon[34520]: purged_snaps scrub ok 2024-05-08T06:00:16.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:16 smithi163 ceph-mon[34520]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T06:00:16.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:16 smithi163 ceph-mon[34520]: osd.2 [v2:172.21.15.192:6800/4161570404,v1:172.21.15.192:6801/4161570404] boot 2024-05-08T06:00:16.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:16 smithi163 ceph-mon[34520]: osdmap e18: 3 total, 3 up, 3 in 2024-05-08T06:00:16.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:16 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T06:00:16.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:16 smithi163 ceph-mon[34520]: Detected new or changed devices on smithi192 2024-05-08T06:00:16.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:16 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:16.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:16 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:16.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:16 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:00:16.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:16 smithi163 ceph-mon[34520]: Adjusting osd_memory_target on smithi192 to 17088M 2024-05-08T06:00:16.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:16 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:16.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:16 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:00:16.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:16 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:00:16.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:16 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:16.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:16 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:00:16.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:16 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:00:16.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:16 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:00:16.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:16 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:16.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:16 smithi192 ceph-mon[34664]: purged_snaps scrub starts 2024-05-08T06:00:16.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:16 smithi192 ceph-mon[34664]: purged_snaps scrub ok 2024-05-08T06:00:16.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:16 smithi192 ceph-mon[34664]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T06:00:16.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:16 smithi192 ceph-mon[34664]: osd.2 [v2:172.21.15.192:6800/4161570404,v1:172.21.15.192:6801/4161570404] boot 2024-05-08T06:00:16.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:16 smithi192 ceph-mon[34664]: osdmap e18: 3 total, 3 up, 3 in 2024-05-08T06:00:16.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:16 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T06:00:16.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:16 smithi192 ceph-mon[34664]: Detected new or changed devices on smithi192 2024-05-08T06:00:16.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:16 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:16.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:16 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:16.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:16 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:00:16.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:16 smithi192 ceph-mon[34664]: Adjusting osd_memory_target on smithi192 to 17088M 2024-05-08T06:00:16.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:16 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:16.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:16 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:00:16.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:16 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:00:16.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:16 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:16.803 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:16 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:00:16.803 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:16 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:00:16.803 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:16 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:00:16.803 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:16 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:16.831 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi123:vg_nvme/lv_3 2024-05-08T06:00:17.237 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:00:17.572 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:17 smithi123 ceph-mon[29787]: osdmap e19: 3 total, 3 up, 3 in 2024-05-08T06:00:17.573 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:17 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-08T06:00:17.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:17 smithi163 ceph-mon[34520]: osdmap e19: 3 total, 3 up, 3 in 2024-05-08T06:00:17.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:17 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-08T06:00:17.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:17 smithi192 ceph-mon[34664]: osdmap e19: 3 total, 3 up, 3 in 2024-05-08T06:00:17.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:17 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-08T06:00:18.704 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:18 smithi123 ceph-mon[29787]: pgmap v92: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:18.704 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:18 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-08T06:00:18.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:18 smithi123 ceph-mon[29787]: osdmap e20: 3 total, 3 up, 3 in 2024-05-08T06:00:18.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:18 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T06:00:18.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:18 smithi163 ceph-mon[34520]: pgmap v92: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:18.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:18 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-08T06:00:18.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:18 smithi163 ceph-mon[34520]: osdmap e20: 3 total, 3 up, 3 in 2024-05-08T06:00:18.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:18 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T06:00:18.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:18 smithi192 ceph-mon[34664]: pgmap v92: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:18.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:18 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-08T06:00:18.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:18 smithi192 ceph-mon[34664]: osdmap e20: 3 total, 3 up, 3 in 2024-05-08T06:00:18.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:18 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T06:00:18.955 INFO:journalctl@ceph.osd.0.smithi123.stdout:May 08 06:00:18 smithi123 sudo[48388]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T06:00:18.955 INFO:journalctl@ceph.osd.0.smithi123.stdout:May 08 06:00:18 smithi123 sudo[48388]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T06:00:18.955 INFO:journalctl@ceph.osd.0.smithi123.stdout:May 08 06:00:18 smithi123 sudo[48388]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T06:00:18.955 INFO:journalctl@ceph.osd.0.smithi123.stdout:May 08 06:00:18 smithi123 sudo[48388]: pam_unix(sudo:session): session closed for user root 2024-05-08T06:00:18.955 INFO:journalctl@ceph.osd.0.smithi123.stdout:May 08 06:00:18 smithi123 sudo[48403]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T06:00:18.955 INFO:journalctl@ceph.osd.0.smithi123.stdout:May 08 06:00:18 smithi123 sudo[48403]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T06:00:18.955 INFO:journalctl@ceph.osd.0.smithi123.stdout:May 08 06:00:18 smithi123 sudo[48403]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T06:00:18.955 INFO:journalctl@ceph.osd.0.smithi123.stdout:May 08 06:00:18 smithi123 sudo[48403]: pam_unix(sudo:session): session closed for user root 2024-05-08T06:00:19.288 INFO:journalctl@ceph.osd.1.smithi163.stdout:May 08 06:00:19 smithi163 sudo[43246]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T06:00:19.288 INFO:journalctl@ceph.osd.1.smithi163.stdout:May 08 06:00:19 smithi163 sudo[43246]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T06:00:19.288 INFO:journalctl@ceph.osd.1.smithi163.stdout:May 08 06:00:19 smithi163 sudo[43246]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T06:00:19.288 INFO:journalctl@ceph.osd.1.smithi163.stdout:May 08 06:00:19 smithi163 sudo[43246]: pam_unix(sudo:session): session closed for user root 2024-05-08T06:00:19.288 INFO:journalctl@ceph.osd.1.smithi163.stdout:May 08 06:00:19 smithi163 sudo[43249]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T06:00:19.288 INFO:journalctl@ceph.osd.1.smithi163.stdout:May 08 06:00:19 smithi163 sudo[43249]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T06:00:19.288 INFO:journalctl@ceph.osd.1.smithi163.stdout:May 08 06:00:19 smithi163 sudo[43249]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T06:00:19.288 INFO:journalctl@ceph.osd.1.smithi163.stdout:May 08 06:00:19 smithi163 sudo[43249]: pam_unix(sudo:session): session closed for user root 2024-05-08T06:00:19.352 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:19 smithi123 sudo[48408]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T06:00:19.352 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:19 smithi123 sudo[48408]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T06:00:19.352 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:19 smithi123 sudo[48408]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T06:00:19.352 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:19 smithi123 sudo[48408]: pam_unix(sudo:session): session closed for user root 2024-05-08T06:00:19.352 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:19 smithi123 sudo[48414]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T06:00:19.352 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:19 smithi123 sudo[48414]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T06:00:19.352 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:19 smithi123 sudo[48414]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T06:00:19.352 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:19 smithi123 sudo[48414]: pam_unix(sudo:session): session closed for user root 2024-05-08T06:00:19.459 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:19 smithi192 sudo[43933]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T06:00:19.459 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:19 smithi192 sudo[43933]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T06:00:19.459 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:19 smithi192 sudo[43933]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T06:00:19.459 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:19 smithi192 sudo[43933]: pam_unix(sudo:session): session closed for user root 2024-05-08T06:00:19.459 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:19 smithi192 sudo[43936]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T06:00:19.459 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:19 smithi192 sudo[43936]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T06:00:19.459 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:19 smithi192 sudo[43936]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T06:00:19.459 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:19 smithi192 sudo[43936]: pam_unix(sudo:session): session closed for user root 2024-05-08T06:00:19.460 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 08 06:00:19 smithi192 sudo[43927]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T06:00:19.460 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 08 06:00:19 smithi192 sudo[43927]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T06:00:19.460 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 08 06:00:19 smithi192 sudo[43927]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T06:00:19.460 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 08 06:00:19 smithi192 sudo[43927]: pam_unix(sudo:session): session closed for user root 2024-05-08T06:00:19.460 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 08 06:00:19 smithi192 sudo[43930]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T06:00:19.460 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 08 06:00:19 smithi192 sudo[43930]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T06:00:19.460 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 08 06:00:19 smithi192 sudo[43930]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T06:00:19.460 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 08 06:00:19 smithi192 sudo[43930]: pam_unix(sudo:session): session closed for user root 2024-05-08T06:00:19.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.526+0000 7effac600700 1 -- 172.21.15.123:0/2120932260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa4075f00 msgr2=0x7effa4076380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:19.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.526+0000 7effac600700 1 --2- 172.21.15.123:0/2120932260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa4075f00 0x7effa4076380 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7eff90009990 tx=0x7eff90009ca0 comp rx=0 tx=0).stop 2024-05-08T06:00:19.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.526+0000 7effac600700 1 -- 172.21.15.123:0/2120932260 shutdown_connections 2024-05-08T06:00:19.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.526+0000 7effac600700 1 --2- 172.21.15.123:0/2120932260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effa40774d0 0x7effa4075930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:19.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.526+0000 7effac600700 1 --2- 172.21.15.123:0/2120932260 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7effa410b3e0 0x7effa410d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:19.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.526+0000 7effac600700 1 --2- 172.21.15.123:0/2120932260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa4075f00 0x7effa4076380 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:19.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.526+0000 7effac600700 1 -- 172.21.15.123:0/2120932260 >> 172.21.15.123:0/2120932260 conn(0x7effa40fea70 msgr2=0x7effa4100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:00:19.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.526+0000 7effac600700 1 -- 172.21.15.123:0/2120932260 shutdown_connections 2024-05-08T06:00:19.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.526+0000 7effac600700 1 -- 172.21.15.123:0/2120932260 wait complete. 2024-05-08T06:00:19.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.528+0000 7effac600700 1 Processor -- start 2024-05-08T06:00:19.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.528+0000 7effac600700 1 -- start start 2024-05-08T06:00:19.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.528+0000 7effac600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effa4075f00 0x7effa419b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:19.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.528+0000 7effac600700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa40774d0 0x7effa419be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:19.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.528+0000 7effac600700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7effa410b3e0 0x7effa41a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:19.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.529+0000 7effac600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7effa4111210 con 0x7effa4075f00 2024-05-08T06:00:19.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.529+0000 7effac600700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7effa4111090 con 0x7effa40774d0 2024-05-08T06:00:19.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.529+0000 7effac600700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7effa4111390 con 0x7effa410b3e0 2024-05-08T06:00:19.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.529+0000 7effa9200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa40774d0 0x7effa419be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:19.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.529+0000 7effa9200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa40774d0 0x7effa419be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:48312/0 (socket says 172.21.15.123:48312) 2024-05-08T06:00:19.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.529+0000 7effa9200700 1 -- 172.21.15.123:0/3654899476 learned_addr learned my addr 172.21.15.123:0/3654899476 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:00:19.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.529+0000 7effaa600700 1 --2- 172.21.15.123:0/3654899476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7effa410b3e0 0x7effa41a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:19.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.529+0000 7effa9c00700 1 --2- 172.21.15.123:0/3654899476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effa4075f00 0x7effa419b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:19.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.529+0000 7effaa600700 1 -- 172.21.15.123:0/3654899476 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa40774d0 msgr2=0x7effa419be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:19.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.529+0000 7effaa600700 1 --2- 172.21.15.123:0/3654899476 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa40774d0 0x7effa419be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:19.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.529+0000 7effaa600700 1 -- 172.21.15.123:0/3654899476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effa4075f00 msgr2=0x7effa419b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:19.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.529+0000 7effaa600700 1 --2- 172.21.15.123:0/3654899476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effa4075f00 0x7effa419b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:19.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.529+0000 7effaa600700 1 -- 172.21.15.123:0/3654899476 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa41a3520 con 0x7effa410b3e0 2024-05-08T06:00:19.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.529+0000 7effa9c00700 1 --2- 172.21.15.123:0/3654899476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effa4075f00 0x7effa419b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:00:19.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.530+0000 7effaa600700 1 --2- 172.21.15.123:0/3654899476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7effa410b3e0 0x7effa41a2e80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7eff9c009f60 tx=0x7eff9c009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:00:19.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.530+0000 7effa2000700 1 -- 172.21.15.123:0/3654899476 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9c0092e0 con 0x7effa410b3e0 2024-05-08T06:00:19.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.530+0000 7effa2000700 1 -- 172.21.15.123:0/3654899476 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff9c004bb0 con 0x7effa410b3e0 2024-05-08T06:00:19.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.530+0000 7effa2000700 1 -- 172.21.15.123:0/3654899476 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9c004280 con 0x7effa410b3e0 2024-05-08T06:00:19.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.530+0000 7effac600700 1 -- 172.21.15.123:0/3654899476 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa41a37b0 con 0x7effa410b3e0 2024-05-08T06:00:19.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.530+0000 7effac600700 1 -- 172.21.15.123:0/3654899476 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effa41a3d40 con 0x7effa410b3e0 2024-05-08T06:00:19.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.531+0000 7effac600700 1 -- 172.21.15.123:0/3654899476 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effa406a190 con 0x7effa410b3e0 2024-05-08T06:00:19.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.533+0000 7effa2000700 1 -- 172.21.15.123:0/3654899476 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 15) ==== 149597+0+0 (secure 0 0 0) 0x7eff9c01b030 con 0x7effa410b3e0 2024-05-08T06:00:19.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.534+0000 7effa2000700 1 --2- 172.21.15.123:0/3654899476 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7eff880b2870 0x7eff880b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:19.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.535+0000 7effa2000700 1 -- 172.21.15.123:0/3654899476 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 3237+0+0 (secure 0 0 0) 0x7eff9c0e0840 con 0x7effa410b3e0 2024-05-08T06:00:19.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.535+0000 7effa9c00700 1 --2- 172.21.15.123:0/3654899476 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7eff880b2870 0x7eff880b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:19.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.535+0000 7effa9c00700 1 --2- 172.21.15.123:0/3654899476 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7eff880b2870 0x7eff880b4d30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7effa40778b0 tx=0x7eff94009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:00:19.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.539+0000 7effa2000700 1 -- 172.21.15.123:0/3654899476 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7eff9c022020 con 0x7effa410b3e0 2024-05-08T06:00:19.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:19 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-08T06:00:19.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:19 smithi123 ceph-mon[29787]: osdmap e21: 3 total, 3 up, 3 in 2024-05-08T06:00:19.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:19 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:19.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:19 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:19.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:19 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:19.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:19 smithi123 ceph-mon[29787]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T06:00:19.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:19 smithi123 ceph-mon[29787]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T06:00:19.707 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:19 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T06:00:19.707 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:19 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T06:00:19.707 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:19 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T06:00:19.707 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:19 smithi123 ceph-mon[29787]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T06:00:19.707 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:19 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T06:00:19.707 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:19 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T06:00:19.707 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:19 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T06:00:19.707 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:19 smithi123 ceph-mon[29787]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T06:00:19.707 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:19 smithi123 ceph-mon[29787]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T06:00:19.707 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:19 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T06:00:19.707 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:19 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T06:00:19.707 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:19 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T06:00:19.707 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:19 smithi123 ceph-mon[29787]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T06:00:19.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:19 smithi163 sudo[43252]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T06:00:19.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:19 smithi163 sudo[43252]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T06:00:19.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:19 smithi163 sudo[43252]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T06:00:19.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:19 smithi163 sudo[43252]: pam_unix(sudo:session): session closed for user root 2024-05-08T06:00:19.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:19 smithi163 sudo[43255]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T06:00:19.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:19 smithi163 sudo[43255]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T06:00:19.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:19 smithi163 sudo[43255]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T06:00:19.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:19 smithi163 sudo[43255]: pam_unix(sudo:session): session closed for user root 2024-05-08T06:00:19.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:19 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-08T06:00:19.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:19 smithi163 ceph-mon[34520]: osdmap e21: 3 total, 3 up, 3 in 2024-05-08T06:00:19.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:19 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:19.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:19 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:19.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:19 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:19.790 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:19 smithi163 ceph-mon[34520]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T06:00:19.790 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:19 smithi163 ceph-mon[34520]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T06:00:19.790 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:19 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T06:00:19.790 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:19 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T06:00:19.790 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:19 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T06:00:19.790 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:19 smithi163 ceph-mon[34520]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T06:00:19.790 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:19 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T06:00:19.790 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:19 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T06:00:19.790 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:19 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T06:00:19.790 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:19 smithi163 ceph-mon[34520]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T06:00:19.790 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:19 smithi163 ceph-mon[34520]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T06:00:19.790 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:19 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T06:00:19.790 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:19 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T06:00:19.791 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:19 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T06:00:19.791 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:19 smithi163 ceph-mon[34520]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T06:00:19.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.795+0000 7effac600700 1 -- 172.21.15.123:0/3654899476 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7effa40639a0 con 0x7eff880b2870 2024-05-08T06:00:19.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:19 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-08T06:00:19.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:19 smithi192 ceph-mon[34664]: osdmap e21: 3 total, 3 up, 3 in 2024-05-08T06:00:19.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:19 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:19.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:19 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:19.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:19 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:19.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:19 smithi192 ceph-mon[34664]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T06:00:19.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:19 smithi192 ceph-mon[34664]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T06:00:19.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:19 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T06:00:19.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:19 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T06:00:19.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:19 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T06:00:19.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:19 smithi192 ceph-mon[34664]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T06:00:19.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:19 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T06:00:19.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:19 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T06:00:19.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:19 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T06:00:19.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:19 smithi192 ceph-mon[34664]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T06:00:19.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:19 smithi192 ceph-mon[34664]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T06:00:19.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:19 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T06:00:19.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:19 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T06:00:19.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:19 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T06:00:19.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:19 smithi192 ceph-mon[34664]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T06:00:19.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:19.993+0000 7effa2000700 1 -- 172.21.15.123:0/3654899476 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7eff9c012050 con 0x7effa410b3e0 2024-05-08T06:00:20.725 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:20 smithi123 ceph-mon[29787]: pgmap v95: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:20.725 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:20 smithi123 ceph-mon[29787]: osdmap e22: 3 total, 3 up, 3 in 2024-05-08T06:00:20.725 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:20 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T06:00:20.725 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:20 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T06:00:20.725 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:20 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:00:20.725 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:20 smithi123 ceph-mon[29787]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-08T06:00:20.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:20 smithi163 ceph-mon[34520]: pgmap v95: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:20.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:20 smithi163 ceph-mon[34520]: osdmap e22: 3 total, 3 up, 3 in 2024-05-08T06:00:20.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:20 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T06:00:20.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:20 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T06:00:20.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:20 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:00:20.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:20 smithi163 ceph-mon[34520]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-08T06:00:20.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:20 smithi192 ceph-mon[34664]: pgmap v95: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:20.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:20 smithi192 ceph-mon[34664]: osdmap e22: 3 total, 3 up, 3 in 2024-05-08T06:00:20.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:20 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T06:00:20.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:20 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T06:00:20.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:20 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:00:20.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:20 smithi192 ceph-mon[34664]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-08T06:00:21.778 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:21 smithi123 ceph-mon[29787]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:00:21.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:21 smithi163 ceph-mon[34520]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:00:21.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:21 smithi192 ceph-mon[34664]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:00:22.573 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:22 smithi123 ceph-mon[29787]: pgmap v97: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:22.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:22 smithi163 ceph-mon[34520]: pgmap v97: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:22.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:22 smithi192 ceph-mon[34664]: pgmap v97: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:24.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:24 smithi163 ceph-mon[34520]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:24.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:24 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/1850485269' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71bb5ecd-c88c-45fb-8b51-0eefb53941c2"}]: dispatch 2024-05-08T06:00:24.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:24 smithi163 ceph-mon[34520]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71bb5ecd-c88c-45fb-8b51-0eefb53941c2"}]: dispatch 2024-05-08T06:00:24.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:24 smithi163 ceph-mon[34520]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71bb5ecd-c88c-45fb-8b51-0eefb53941c2"}]': finished 2024-05-08T06:00:24.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:24 smithi163 ceph-mon[34520]: osdmap e23: 4 total, 3 up, 4 in 2024-05-08T06:00:24.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:24 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T06:00:24.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:24 smithi192 ceph-mon[34664]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:24.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:24 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/1850485269' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71bb5ecd-c88c-45fb-8b51-0eefb53941c2"}]: dispatch 2024-05-08T06:00:24.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:24 smithi192 ceph-mon[34664]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71bb5ecd-c88c-45fb-8b51-0eefb53941c2"}]: dispatch 2024-05-08T06:00:24.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:24 smithi192 ceph-mon[34664]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71bb5ecd-c88c-45fb-8b51-0eefb53941c2"}]': finished 2024-05-08T06:00:24.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:24 smithi192 ceph-mon[34664]: osdmap e23: 4 total, 3 up, 4 in 2024-05-08T06:00:24.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:24 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T06:00:24.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:24 smithi123 ceph-mon[29787]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:24.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:24 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/1850485269' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71bb5ecd-c88c-45fb-8b51-0eefb53941c2"}]: dispatch 2024-05-08T06:00:24.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:24 smithi123 ceph-mon[29787]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71bb5ecd-c88c-45fb-8b51-0eefb53941c2"}]: dispatch 2024-05-08T06:00:24.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:24 smithi123 ceph-mon[29787]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71bb5ecd-c88c-45fb-8b51-0eefb53941c2"}]': finished 2024-05-08T06:00:24.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:24 smithi123 ceph-mon[29787]: osdmap e23: 4 total, 3 up, 4 in 2024-05-08T06:00:24.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:24 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T06:00:25.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:25 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/1733746865' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T06:00:25.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:25 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/1733746865' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T06:00:25.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:25 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/1733746865' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T06:00:26.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:26 smithi163 ceph-mon[34520]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:26.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:26 smithi192 ceph-mon[34664]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:26.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:26 smithi123 ceph-mon[29787]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:28.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:28 smithi163 ceph-mon[34520]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:28.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:28 smithi192 ceph-mon[34664]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:28.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:28 smithi123 ceph-mon[29787]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:30.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:30 smithi123 ceph-mon[29787]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:30.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:30 smithi163 ceph-mon[34520]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:30.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:30 smithi192 ceph-mon[34664]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:32.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:32 smithi163 ceph-mon[34520]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:32.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:32 smithi192 ceph-mon[34664]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:32.816 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:32 smithi123 ceph-mon[29787]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:33.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:33 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-08T06:00:33.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:33 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:00:33.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:33 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-08T06:00:33.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:33 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:00:33.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:33 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-08T06:00:33.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:33 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:00:34.639 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:34 smithi123 ceph-mon[29787]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:34.639 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:34 smithi123 ceph-mon[29787]: Deploying daemon osd.3 on smithi123 2024-05-08T06:00:34.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:34 smithi163 ceph-mon[34520]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:34.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:34 smithi163 ceph-mon[34520]: Deploying daemon osd.3 on smithi123 2024-05-08T06:00:34.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:34 smithi192 ceph-mon[34664]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:34.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:34 smithi192 ceph-mon[34664]: Deploying daemon osd.3 on smithi123 2024-05-08T06:00:36.653 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:36 smithi123 ceph-mon[29787]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:36.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:36 smithi163 ceph-mon[34520]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:36.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:36 smithi192 ceph-mon[34664]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:38.564 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:38 smithi123 ceph-mon[29787]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:38.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:38 smithi163 ceph-mon[34520]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:38.796 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:38 smithi192 ceph-mon[34664]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:40.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:40 smithi123 ceph-mon[29787]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:40.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:40 smithi163 ceph-mon[34520]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:40.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:40 smithi192 ceph-mon[34664]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:42.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:42 smithi123 ceph-mon[29787]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:42.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:42 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:42.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:42 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:42.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:42 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:00:42.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:42 smithi163 ceph-mon[34520]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:42.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:42 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:42.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:42 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:42.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:42 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:00:42.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:42 smithi192 ceph-mon[34664]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:42.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:42 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:42.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:42 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:42.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:42 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:00:44.669 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:44 smithi123 ceph-mon[29787]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:44.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:44 smithi163 ceph-mon[34520]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:44.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:44 smithi192 ceph-mon[34664]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:46.792 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:46 smithi123 ceph-mon[29787]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:46.793 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:46 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:46.793 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:46 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:46.793 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:46 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:00:46.793 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:46 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:00:46.793 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:46 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:46.793 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:46 smithi123 ceph-mon[29787]: from='osd.3 [v2:172.21.15.123:6810/1243003258,v1:172.21.15.123:6811/1243003258]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-08T06:00:46.793 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:46 smithi123 ceph-mon[29787]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-08T06:00:46.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:46 smithi192 ceph-mon[34664]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:46.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:46 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:46.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:46 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:46.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:46 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:00:46.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:46 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:00:46.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:46 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:46.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:46 smithi192 ceph-mon[34664]: from='osd.3 [v2:172.21.15.123:6810/1243003258,v1:172.21.15.123:6811/1243003258]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-08T06:00:46.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:46 smithi192 ceph-mon[34664]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-08T06:00:46.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:46.893+0000 7effa2000700 1 -- 172.21.15.123:0/3654899476 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7effa40639a0 con 0x7eff880b2870 2024-05-08T06:00:46.895 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 3 on host 'smithi123' 2024-05-08T06:00:46.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:46.897+0000 7effac600700 1 -- 172.21.15.123:0/3654899476 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7eff880b2870 msgr2=0x7eff880b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:46.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:46.897+0000 7effac600700 1 --2- 172.21.15.123:0/3654899476 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7eff880b2870 0x7eff880b4d30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7effa40778b0 tx=0x7eff94009260 comp rx=0 tx=0).stop 2024-05-08T06:00:46.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:46.897+0000 7effac600700 1 -- 172.21.15.123:0/3654899476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7effa410b3e0 msgr2=0x7effa41a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:46.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:46.897+0000 7effac600700 1 --2- 172.21.15.123:0/3654899476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7effa410b3e0 0x7effa41a2e80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7eff9c009f60 tx=0x7eff9c009f90 comp rx=0 tx=0).stop 2024-05-08T06:00:46.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:46.897+0000 7effac600700 1 -- 172.21.15.123:0/3654899476 shutdown_connections 2024-05-08T06:00:46.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:46.897+0000 7effac600700 1 --2- 172.21.15.123:0/3654899476 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7eff880b2870 0x7eff880b4d30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:46.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:46.897+0000 7effac600700 1 --2- 172.21.15.123:0/3654899476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effa4075f00 0x7effa419b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:46.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:46.897+0000 7effac600700 1 --2- 172.21.15.123:0/3654899476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7effa410b3e0 0x7effa41a2e80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:46.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:46.897+0000 7effac600700 1 --2- 172.21.15.123:0/3654899476 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effa40774d0 0x7effa419be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:46.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:46.897+0000 7effac600700 1 -- 172.21.15.123:0/3654899476 >> 172.21.15.123:0/3654899476 conn(0x7effa40fea70 msgr2=0x7effa41072e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:00:46.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:46.897+0000 7effac600700 1 -- 172.21.15.123:0/3654899476 shutdown_connections 2024-05-08T06:00:46.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:00:46.897+0000 7effac600700 1 -- 172.21.15.123:0/3654899476 wait complete. 2024-05-08T06:00:47.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:46 smithi163 ceph-mon[34520]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:47.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:46 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:47.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:46 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:47.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:46 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:00:47.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:46 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:00:47.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:46 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:47.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:46 smithi163 ceph-mon[34520]: from='osd.3 [v2:172.21.15.123:6810/1243003258,v1:172.21.15.123:6811/1243003258]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-08T06:00:47.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:46 smithi163 ceph-mon[34520]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-08T06:00:47.672 DEBUG:teuthology.orchestra.run.smithi123:osd.3> sudo journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@osd.3.service 2024-05-08T06:00:47.676 INFO:tasks.cephadm:Deploying osd.4 on smithi163 with /dev/vg_nvme/lv_3... 2024-05-08T06:00:47.676 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-08T06:00:47.921 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T06:00:48.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:47 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:48.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:47 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:48.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:47 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:00:48.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:47 smithi163 ceph-mon[34520]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-08T06:00:48.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:47 smithi163 ceph-mon[34520]: from='osd.3 [v2:172.21.15.123:6810/1243003258,v1:172.21.15.123:6811/1243003258]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-08T06:00:48.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:47 smithi163 ceph-mon[34520]: osdmap e24: 4 total, 3 up, 4 in 2024-05-08T06:00:48.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:47 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T06:00:48.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:47 smithi163 ceph-mon[34520]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-08T06:00:48.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:47 smithi163 ceph-mon[34520]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:48.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:47 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:48.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:47 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:48.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:47 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:00:48.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:47 smithi123 ceph-mon[29787]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-08T06:00:48.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:47 smithi123 ceph-mon[29787]: from='osd.3 [v2:172.21.15.123:6810/1243003258,v1:172.21.15.123:6811/1243003258]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-08T06:00:48.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:47 smithi123 ceph-mon[29787]: osdmap e24: 4 total, 3 up, 4 in 2024-05-08T06:00:48.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:47 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T06:00:48.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:47 smithi123 ceph-mon[29787]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-08T06:00:48.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:47 smithi123 ceph-mon[29787]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:48.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:47 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:48.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:47 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:48.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:47 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:00:48.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:47 smithi192 ceph-mon[34664]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-08T06:00:48.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:47 smithi192 ceph-mon[34664]: from='osd.3 [v2:172.21.15.123:6810/1243003258,v1:172.21.15.123:6811/1243003258]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-08T06:00:48.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:47 smithi192 ceph-mon[34664]: osdmap e24: 4 total, 3 up, 4 in 2024-05-08T06:00:48.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:47 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T06:00:48.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:47 smithi192 ceph-mon[34664]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-08T06:00:48.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:47 smithi192 ceph-mon[34664]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:48.934 INFO:journalctl@ceph.osd.3.smithi123.stdout:May 08 06:00:48 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-3[50712]: 2024-05-08T06:00:48.615+0000 7fac29200700 -1 osd.3 0 waiting for initial osdmap 2024-05-08T06:00:48.935 INFO:journalctl@ceph.osd.3.smithi123.stdout:May 08 06:00:48 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-3[50712]: 2024-05-08T06:00:48.641+0000 7fac1f000700 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T06:00:48.994 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:48 smithi163 ceph-mon[34520]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-05-08T06:00:48.994 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:48 smithi163 ceph-mon[34520]: osdmap e25: 4 total, 3 up, 4 in 2024-05-08T06:00:48.994 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:48 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T06:00:48.995 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:48 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T06:00:48.995 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:48 smithi163 ceph-mon[34520]: osdmap e26: 4 total, 3 up, 4 in 2024-05-08T06:00:48.995 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:48 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T06:00:48.995 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:48 smithi163 ceph-mon[34520]: from='osd.3 ' entity='osd.3' 2024-05-08T06:00:49.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:48 smithi123 ceph-mon[29787]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-05-08T06:00:49.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:48 smithi123 ceph-mon[29787]: osdmap e25: 4 total, 3 up, 4 in 2024-05-08T06:00:49.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:48 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T06:00:49.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:48 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T06:00:49.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:48 smithi123 ceph-mon[29787]: osdmap e26: 4 total, 3 up, 4 in 2024-05-08T06:00:49.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:48 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T06:00:49.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:48 smithi123 ceph-mon[29787]: from='osd.3 ' entity='osd.3' 2024-05-08T06:00:49.302 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:48 smithi192 ceph-mon[34664]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-05-08T06:00:49.304 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:48 smithi192 ceph-mon[34664]: osdmap e25: 4 total, 3 up, 4 in 2024-05-08T06:00:49.306 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:48 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T06:00:49.306 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:48 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T06:00:49.306 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:48 smithi192 ceph-mon[34664]: osdmap e26: 4 total, 3 up, 4 in 2024-05-08T06:00:49.306 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:48 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T06:00:49.306 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:48 smithi192 ceph-mon[34664]: from='osd.3 ' entity='osd.3' 2024-05-08T06:00:49.809 INFO:teuthology.orchestra.run.smithi163.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-08T06:00:49.809 INFO:teuthology.orchestra.run.smithi163.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-08T06:00:49.809 INFO:teuthology.orchestra.run.smithi163.stderr: stderr: 10+0 records in 2024-05-08T06:00:49.809 INFO:teuthology.orchestra.run.smithi163.stderr:10+0 records out 2024-05-08T06:00:49.809 INFO:teuthology.orchestra.run.smithi163.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113166 s, 927 MB/s 2024-05-08T06:00:49.810 INFO:teuthology.orchestra.run.smithi163.stderr:--> Zapping successful for: 2024-05-08T06:00:50.133 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:49 smithi163 ceph-mon[34520]: purged_snaps scrub starts 2024-05-08T06:00:50.133 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:49 smithi163 ceph-mon[34520]: purged_snaps scrub ok 2024-05-08T06:00:50.133 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:49 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T06:00:50.134 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:49 smithi163 ceph-mon[34520]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:50.134 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:49 smithi163 ceph-mon[34520]: osd.3 [v2:172.21.15.123:6810/1243003258,v1:172.21.15.123:6811/1243003258] boot 2024-05-08T06:00:50.134 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:49 smithi163 ceph-mon[34520]: osdmap e27: 4 total, 4 up, 4 in 2024-05-08T06:00:50.134 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:49 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T06:00:50.193 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:49 smithi123 ceph-mon[29787]: purged_snaps scrub starts 2024-05-08T06:00:50.193 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:49 smithi123 ceph-mon[29787]: purged_snaps scrub ok 2024-05-08T06:00:50.193 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:49 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T06:00:50.193 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:49 smithi123 ceph-mon[29787]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:50.194 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:49 smithi123 ceph-mon[29787]: osd.3 [v2:172.21.15.123:6810/1243003258,v1:172.21.15.123:6811/1243003258] boot 2024-05-08T06:00:50.194 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:49 smithi123 ceph-mon[29787]: osdmap e27: 4 total, 4 up, 4 in 2024-05-08T06:00:50.194 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:49 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T06:00:50.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:49 smithi192 ceph-mon[34664]: purged_snaps scrub starts 2024-05-08T06:00:50.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:49 smithi192 ceph-mon[34664]: purged_snaps scrub ok 2024-05-08T06:00:50.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:49 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T06:00:50.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:49 smithi192 ceph-mon[34664]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T06:00:50.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:49 smithi192 ceph-mon[34664]: osd.3 [v2:172.21.15.123:6810/1243003258,v1:172.21.15.123:6811/1243003258] boot 2024-05-08T06:00:50.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:49 smithi192 ceph-mon[34664]: osdmap e27: 4 total, 4 up, 4 in 2024-05-08T06:00:50.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:49 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T06:00:50.522 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi163:vg_nvme/lv_3 2024-05-08T06:00:50.774 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T06:00:51.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:51 smithi163 ceph-mon[34520]: osdmap e28: 4 total, 4 up, 4 in 2024-05-08T06:00:51.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:51 smithi192 ceph-mon[34664]: osdmap e28: 4 total, 4 up, 4 in 2024-05-08T06:00:51.607 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:51 smithi123 ceph-mon[29787]: osdmap e28: 4 total, 4 up, 4 in 2024-05-08T06:00:52.352 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:52 smithi163 ceph-mon[34520]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T06:00:52.352 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:52 smithi163 ceph-mon[34520]: osdmap e29: 4 total, 4 up, 4 in 2024-05-08T06:00:52.383 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.381+0000 7fec0ad96700 1 -- 172.21.15.163:0/672292936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec04103610 msgr2=0x7fec04103a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:52.383 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.381+0000 7fec0ad96700 1 --2- 172.21.15.163:0/672292936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec04103610 0x7fec04103a70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7febf00099c0 tx=0x7febf0009cd0 comp rx=0 tx=0).stop 2024-05-08T06:00:52.383 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.381+0000 7fec0ad96700 1 -- 172.21.15.163:0/672292936 shutdown_connections 2024-05-08T06:00:52.383 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.381+0000 7fec0ad96700 1 --2- 172.21.15.163:0/672292936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec04103fb0 0x7fec0410a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:52.383 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.381+0000 7fec0ad96700 1 --2- 172.21.15.163:0/672292936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec04102410 0x7fec04102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:52.383 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.381+0000 7fec0ad96700 1 --2- 172.21.15.163:0/672292936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec04103610 0x7fec04103a70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:52.383 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.381+0000 7fec0ad96700 1 -- 172.21.15.163:0/672292936 >> 172.21.15.163:0/672292936 conn(0x7fec040fd9d0 msgr2=0x7fec040ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:00:52.383 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.381+0000 7fec0ad96700 1 -- 172.21.15.163:0/672292936 shutdown_connections 2024-05-08T06:00:52.384 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.381+0000 7fec0ad96700 1 -- 172.21.15.163:0/672292936 wait complete. 2024-05-08T06:00:52.384 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.381+0000 7fec0ad96700 1 Processor -- start 2024-05-08T06:00:52.384 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.381+0000 7fec0ad96700 1 -- start start 2024-05-08T06:00:52.384 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.381+0000 7fec0ad96700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec04102410 0x7fec04199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:52.384 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.381+0000 7fec0ad96700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec04103610 0x7fec04199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:52.384 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.382+0000 7fec0ad96700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec04103fb0 0x7fec041a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:52.384 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.382+0000 7fec0ad96700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fec0410df20 con 0x7fec04103fb0 2024-05-08T06:00:52.384 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.382+0000 7fec0ad96700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fec0410dda0 con 0x7fec04103610 2024-05-08T06:00:52.384 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.382+0000 7fec0ad96700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fec0410e0a0 con 0x7fec04102410 2024-05-08T06:00:52.384 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.382+0000 7fec03fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec04103610 0x7fec04199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:52.384 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.382+0000 7fec03fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec04103610 0x7fec04199cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40282/0 (socket says 172.21.15.163:40282) 2024-05-08T06:00:52.384 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.382+0000 7fec03fff700 1 -- 172.21.15.163:0/3675781297 learned_addr learned my addr 172.21.15.163:0/3675781297 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:00:52.384 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.382+0000 7fec08b32700 1 --2- 172.21.15.163:0/3675781297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec04102410 0x7fec04199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:52.385 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.382+0000 7fec08b32700 1 -- 172.21.15.163:0/3675781297 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec04103610 msgr2=0x7fec04199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:52.385 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.382+0000 7fec08b32700 1 --2- 172.21.15.163:0/3675781297 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec04103610 0x7fec04199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:52.385 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.382+0000 7fec08b32700 1 -- 172.21.15.163:0/3675781297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec04103fb0 msgr2=0x7fec041a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:00:52.385 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.382+0000 7fec08b32700 1 --2- 172.21.15.163:0/3675781297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec04103fb0 0x7fec041a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:00:52.385 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.382+0000 7fec08b32700 1 -- 172.21.15.163:0/3675781297 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec041a13d0 con 0x7fec04102410 2024-05-08T06:00:52.385 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.382+0000 7fec08b32700 1 --2- 172.21.15.163:0/3675781297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec04102410 0x7fec04199770 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7febf800c970 tx=0x7febf800cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:00:52.386 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.383+0000 7fec01ffb700 1 -- 172.21.15.163:0/3675781297 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf800ce60 con 0x7fec04102410 2024-05-08T06:00:52.386 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.383+0000 7fec01ffb700 1 -- 172.21.15.163:0/3675781297 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febf80078c0 con 0x7fec04102410 2024-05-08T06:00:52.386 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.383+0000 7fec0ad96700 1 -- 172.21.15.163:0/3675781297 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec041a16c0 con 0x7fec04102410 2024-05-08T06:00:52.387 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.383+0000 7fec01ffb700 1 -- 172.21.15.163:0/3675781297 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf8004d70 con 0x7fec04102410 2024-05-08T06:00:52.387 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.383+0000 7fec0ad96700 1 -- 172.21.15.163:0/3675781297 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec041a1bb0 con 0x7fec04102410 2024-05-08T06:00:52.387 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.384+0000 7fec0ad96700 1 -- 172.21.15.163:0/3675781297 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec0406a190 con 0x7fec04102410 2024-05-08T06:00:52.387 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.385+0000 7fec01ffb700 1 -- 172.21.15.163:0/3675781297 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7febf8007a70 con 0x7fec04102410 2024-05-08T06:00:52.387 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.385+0000 7fec01ffb700 1 --2- 172.21.15.163:0/3675781297 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7febf40b2880 0x7febf40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:00:52.387 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.386+0000 7fec01ffb700 1 -- 172.21.15.163:0/3675781297 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3669+0+0 (secure 0 0 0) 0x7febf80e0ec0 con 0x7fec04102410 2024-05-08T06:00:52.387 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.386+0000 7fec03fff700 1 --2- 172.21.15.163:0/3675781297 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7febf40b2880 0x7febf40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:00:52.387 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.387+0000 7fec03fff700 1 --2- 172.21.15.163:0/3675781297 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7febf40b2880 0x7febf40b4d40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7febf0009f50 tx=0x7febf0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:00:52.390 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.390+0000 7fec01ffb700 1 -- 172.21.15.163:0/3675781297 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7febf8014030 con 0x7fec04102410 2024-05-08T06:00:52.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:52 smithi192 ceph-mon[34664]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T06:00:52.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:52 smithi192 ceph-mon[34664]: osdmap e29: 4 total, 4 up, 4 in 2024-05-08T06:00:52.601 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:00:52.601+0000 7fec0ad96700 1 -- 172.21.15.163:0/3675781297 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fec0406c370 con 0x7febf40b2880 2024-05-08T06:00:52.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:52 smithi123 ceph-mon[29787]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T06:00:52.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:52 smithi123 ceph-mon[29787]: osdmap e29: 4 total, 4 up, 4 in 2024-05-08T06:00:53.471 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:53 smithi163 ceph-mon[34520]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:00:53.472 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:53 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T06:00:53.472 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:53 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T06:00:53.472 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:53 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:00:53.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:53 smithi192 ceph-mon[34664]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:00:53.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:53 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T06:00:53.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:53 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T06:00:53.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:53 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:00:53.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:53 smithi123 ceph-mon[29787]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:00:53.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:53 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T06:00:53.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:53 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T06:00:53.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:53 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:00:54.420 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:54 smithi163 ceph-mon[34520]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-05-08T06:00:54.420 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:54 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:54.420 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:54 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:54.420 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:54 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:00:54.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:54 smithi123 ceph-mon[29787]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-05-08T06:00:54.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:54 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:54.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:54 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:54.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:54 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:00:54.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:54 smithi192 ceph-mon[34664]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-05-08T06:00:54.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:54 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:54.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:54 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:54.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:54 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:00:55.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:55 smithi163 ceph-mon[34520]: Detected new or changed devices on smithi123 2024-05-08T06:00:55.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:55 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:00:55.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:55 smithi163 ceph-mon[34520]: Adjusting osd_memory_target on smithi123 to 8544M 2024-05-08T06:00:55.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:55 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:55.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:55 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:00:55.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:55 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:00:55.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:55 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:55.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:55 smithi192 ceph-mon[34664]: Detected new or changed devices on smithi123 2024-05-08T06:00:55.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:55 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:00:55.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:55 smithi192 ceph-mon[34664]: Adjusting osd_memory_target on smithi123 to 8544M 2024-05-08T06:00:55.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:55 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:55.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:55 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:00:55.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:55 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:00:55.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:55 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:55.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:55 smithi123 ceph-mon[29787]: Detected new or changed devices on smithi123 2024-05-08T06:00:55.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:55 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:00:55.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:55 smithi123 ceph-mon[29787]: Adjusting osd_memory_target on smithi123 to 8544M 2024-05-08T06:00:55.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:55 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:55.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:55 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:00:55.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:55 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:00:55.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:55 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:00:56.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:56 smithi163 ceph-mon[34520]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T06:00:56.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:56 smithi163 ceph-mon[34520]: from='client.? 172.21.15.163:0/3267776003' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a7a3462-e74f-4d15-a85c-709b5d423745"}]: dispatch 2024-05-08T06:00:56.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:56 smithi163 ceph-mon[34520]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a7a3462-e74f-4d15-a85c-709b5d423745"}]: dispatch 2024-05-08T06:00:56.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:56 smithi163 ceph-mon[34520]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a7a3462-e74f-4d15-a85c-709b5d423745"}]': finished 2024-05-08T06:00:56.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:56 smithi163 ceph-mon[34520]: osdmap e30: 5 total, 4 up, 5 in 2024-05-08T06:00:56.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:56 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T06:00:56.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:56 smithi163 ceph-mon[34520]: from='client.? 172.21.15.163:0/1023628061' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T06:00:56.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:56 smithi192 ceph-mon[34664]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T06:00:56.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:56 smithi192 ceph-mon[34664]: from='client.? 172.21.15.163:0/3267776003' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a7a3462-e74f-4d15-a85c-709b5d423745"}]: dispatch 2024-05-08T06:00:56.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:56 smithi192 ceph-mon[34664]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a7a3462-e74f-4d15-a85c-709b5d423745"}]: dispatch 2024-05-08T06:00:56.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:56 smithi192 ceph-mon[34664]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a7a3462-e74f-4d15-a85c-709b5d423745"}]': finished 2024-05-08T06:00:56.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:56 smithi192 ceph-mon[34664]: osdmap e30: 5 total, 4 up, 5 in 2024-05-08T06:00:56.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:56 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T06:00:56.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:56 smithi192 ceph-mon[34664]: from='client.? 172.21.15.163:0/1023628061' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T06:00:56.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:56 smithi123 ceph-mon[29787]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T06:00:56.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:56 smithi123 ceph-mon[29787]: from='client.? 172.21.15.163:0/3267776003' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a7a3462-e74f-4d15-a85c-709b5d423745"}]: dispatch 2024-05-08T06:00:56.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:56 smithi123 ceph-mon[29787]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a7a3462-e74f-4d15-a85c-709b5d423745"}]: dispatch 2024-05-08T06:00:56.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:56 smithi123 ceph-mon[29787]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a7a3462-e74f-4d15-a85c-709b5d423745"}]': finished 2024-05-08T06:00:56.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:56 smithi123 ceph-mon[29787]: osdmap e30: 5 total, 4 up, 5 in 2024-05-08T06:00:56.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:56 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T06:00:56.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:56 smithi123 ceph-mon[29787]: from='client.? 172.21.15.163:0/1023628061' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T06:00:58.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:00:58 smithi192 ceph-mon[34664]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-08T06:00:58.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:00:58 smithi123 ceph-mon[29787]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-08T06:00:58.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:00:58 smithi163 ceph-mon[34520]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-08T06:01:00.519 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:00 smithi163 ceph-mon[34520]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T06:01:00.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:00 smithi123 ceph-mon[29787]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T06:01:00.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:00 smithi192 ceph-mon[34664]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T06:01:02.463 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:02 smithi163 ceph-mon[34520]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-08T06:01:02.607 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:02 smithi123 ceph-mon[29787]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-08T06:01:02.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:02 smithi192 ceph-mon[34664]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-08T06:01:03.409 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:03 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-08T06:01:03.409 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:03 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:01:03.409 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:03 smithi163 ceph-mon[34520]: Deploying daemon osd.4 on smithi163 2024-05-08T06:01:03.630 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:03 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-08T06:01:03.630 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:03 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:01:03.630 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:03 smithi123 ceph-mon[29787]: Deploying daemon osd.4 on smithi163 2024-05-08T06:01:03.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:03 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-08T06:01:03.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:03 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:01:03.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:03 smithi192 ceph-mon[34664]: Deploying daemon osd.4 on smithi163 2024-05-08T06:01:04.614 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:04 smithi163 ceph-mon[34520]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T06:01:04.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:04 smithi123 ceph-mon[29787]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T06:01:04.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:04 smithi192 ceph-mon[34664]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T06:01:06.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:06 smithi163 ceph-mon[34520]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T06:01:06.602 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:06 smithi192 ceph-mon[34664]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T06:01:06.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:06 smithi123 ceph-mon[29787]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T06:01:08.619 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:08 smithi163 ceph-mon[34520]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T06:01:08.619 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:08 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:08.619 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:08 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:08.619 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:08 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:01:08.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:08 smithi123 ceph-mon[29787]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T06:01:08.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:08 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:08.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:08 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:08.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:08 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:01:08.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:08 smithi192 ceph-mon[34664]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T06:01:08.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:08 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:08.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:08 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:08.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:08 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:01:10.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:10 smithi123 ceph-mon[29787]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T06:01:10.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:10 smithi163 ceph-mon[34520]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T06:01:10.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:10 smithi192 ceph-mon[34664]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T06:01:11.632 INFO:teuthology.orchestra.run.smithi163.stdout:Created osd(s) 4 on host 'smithi163' 2024-05-08T06:01:11.632 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:01:11.632+0000 7fec01ffb700 1 -- 172.21.15.163:0/3675781297 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fec0406c370 con 0x7febf40b2880 2024-05-08T06:01:11.634 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:01:11.634+0000 7fec0ad96700 1 -- 172.21.15.163:0/3675781297 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7febf40b2880 msgr2=0x7febf40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:11.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:01:11.634+0000 7fec0ad96700 1 --2- 172.21.15.163:0/3675781297 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7febf40b2880 0x7febf40b4d40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7febf0009f50 tx=0x7febf0005fb0 comp rx=0 tx=0).stop 2024-05-08T06:01:11.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:01:11.635+0000 7fec0ad96700 1 -- 172.21.15.163:0/3675781297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec04102410 msgr2=0x7fec04199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:11.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:01:11.635+0000 7fec0ad96700 1 --2- 172.21.15.163:0/3675781297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec04102410 0x7fec04199770 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7febf800c970 tx=0x7febf800cc80 comp rx=0 tx=0).stop 2024-05-08T06:01:11.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:01:11.635+0000 7fec0ad96700 1 -- 172.21.15.163:0/3675781297 shutdown_connections 2024-05-08T06:01:11.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:01:11.635+0000 7fec0ad96700 1 --2- 172.21.15.163:0/3675781297 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7febf40b2880 0x7febf40b4d40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:11.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:01:11.635+0000 7fec0ad96700 1 --2- 172.21.15.163:0/3675781297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec04103fb0 0x7fec041a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:11.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:01:11.635+0000 7fec0ad96700 1 --2- 172.21.15.163:0/3675781297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec04102410 0x7fec04199770 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:11.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:01:11.635+0000 7fec0ad96700 1 --2- 172.21.15.163:0/3675781297 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec04103610 0x7fec04199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:11.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:01:11.635+0000 7fec0ad96700 1 -- 172.21.15.163:0/3675781297 >> 172.21.15.163:0/3675781297 conn(0x7fec040fd9d0 msgr2=0x7fec0406bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:11.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:01:11.635+0000 7fec0ad96700 1 -- 172.21.15.163:0/3675781297 shutdown_connections 2024-05-08T06:01:11.635 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:01:11.635+0000 7fec0ad96700 1 -- 172.21.15.163:0/3675781297 wait complete. 2024-05-08T06:01:11.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:11 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:11.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:11 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:11.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:11 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:01:11.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:11 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:01:11.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:11 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:11.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:11 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:11.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:11 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:11.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:11 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:01:11.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:11 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:11.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:11 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:11.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:11 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:01:11.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:11 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:01:11.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:11 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:11.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:11 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:11.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:11 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:11.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:11 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:01:12.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:11 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:12.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:11 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:12.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:11 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:01:12.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:11 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:01:12.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:11 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:12.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:11 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:12.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:11 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:12.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:11 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:01:12.290 DEBUG:teuthology.orchestra.run.smithi163:osd.4> sudo journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@osd.4.service 2024-05-08T06:01:12.292 INFO:tasks.cephadm:Deploying osd.5 on smithi192 with /dev/vg_nvme/lv_3... 2024-05-08T06:01:12.292 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-08T06:01:12.537 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T06:01:12.912 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:12 smithi192 ceph-mon[34664]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T06:01:12.912 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:12 smithi192 ceph-mon[34664]: from='osd.4 [v2:172.21.15.163:6808/591680273,v1:172.21.15.163:6809/591680273]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T06:01:12.912 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:12 smithi192 ceph-mon[34664]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T06:01:12.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:12 smithi123 ceph-mon[29787]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T06:01:12.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:12 smithi123 ceph-mon[29787]: from='osd.4 [v2:172.21.15.163:6808/591680273,v1:172.21.15.163:6809/591680273]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T06:01:12.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:12 smithi123 ceph-mon[29787]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T06:01:13.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:12 smithi163 ceph-mon[34520]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T06:01:13.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:12 smithi163 ceph-mon[34520]: from='osd.4 [v2:172.21.15.163:6808/591680273,v1:172.21.15.163:6809/591680273]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T06:01:13.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:12 smithi163 ceph-mon[34520]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T06:01:13.992 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:13 smithi163 ceph-mon[34520]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-08T06:01:13.992 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:13 smithi163 ceph-mon[34520]: from='osd.4 [v2:172.21.15.163:6808/591680273,v1:172.21.15.163:6809/591680273]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-05-08T06:01:13.992 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:13 smithi163 ceph-mon[34520]: osdmap e31: 5 total, 4 up, 5 in 2024-05-08T06:01:13.992 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:13 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T06:01:13.992 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:13 smithi163 ceph-mon[34520]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-05-08T06:01:13.992 INFO:journalctl@ceph.osd.4.smithi163.stdout:May 08 06:01:13 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-4[46064]: 2024-05-08T06:01:13.704+0000 7f16e3e03700 -1 osd.4 0 waiting for initial osdmap 2024-05-08T06:01:13.992 INFO:journalctl@ceph.osd.4.smithi163.stdout:May 08 06:01:13 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-4[46064]: 2024-05-08T06:01:13.709+0000 7f16dfbf6700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T06:01:14.011 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:13 smithi192 ceph-mon[34664]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-08T06:01:14.011 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:13 smithi192 ceph-mon[34664]: from='osd.4 [v2:172.21.15.163:6808/591680273,v1:172.21.15.163:6809/591680273]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-05-08T06:01:14.011 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:13 smithi192 ceph-mon[34664]: osdmap e31: 5 total, 4 up, 5 in 2024-05-08T06:01:14.012 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:13 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T06:01:14.012 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:13 smithi192 ceph-mon[34664]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-05-08T06:01:14.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:13 smithi123 ceph-mon[29787]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-08T06:01:14.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:13 smithi123 ceph-mon[29787]: from='osd.4 [v2:172.21.15.163:6808/591680273,v1:172.21.15.163:6809/591680273]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-05-08T06:01:14.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:13 smithi123 ceph-mon[29787]: osdmap e31: 5 total, 4 up, 5 in 2024-05-08T06:01:14.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:13 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T06:01:14.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:13 smithi123 ceph-mon[29787]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-05-08T06:01:14.586 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-08T06:01:14.587 INFO:teuthology.orchestra.run.smithi192.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-08T06:01:14.587 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10+0 records in 2024-05-08T06:01:14.587 INFO:teuthology.orchestra.run.smithi192.stderr:10+0 records out 2024-05-08T06:01:14.587 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126723 s, 827 MB/s 2024-05-08T06:01:14.587 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping successful for: 2024-05-08T06:01:14.853 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:14 smithi163 ceph-mon[34520]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T06:01:14.853 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:14 smithi163 ceph-mon[34520]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-05-08T06:01:14.853 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:14 smithi163 ceph-mon[34520]: osdmap e32: 5 total, 4 up, 5 in 2024-05-08T06:01:14.853 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:14 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T06:01:14.853 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:14 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T06:01:14.903 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:14 smithi192 ceph-mon[34664]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T06:01:14.903 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:14 smithi192 ceph-mon[34664]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-05-08T06:01:14.903 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:14 smithi192 ceph-mon[34664]: osdmap e32: 5 total, 4 up, 5 in 2024-05-08T06:01:14.903 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:14 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T06:01:14.903 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:14 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T06:01:14.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:14 smithi123 ceph-mon[29787]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T06:01:14.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:14 smithi123 ceph-mon[29787]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-05-08T06:01:14.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:14 smithi123 ceph-mon[29787]: osdmap e32: 5 total, 4 up, 5 in 2024-05-08T06:01:14.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:14 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T06:01:14.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:14 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T06:01:15.234 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi192:vg_nvme/lv_3 2024-05-08T06:01:15.468 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T06:01:15.932 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:15 smithi192 ceph-mon[34664]: purged_snaps scrub starts 2024-05-08T06:01:15.932 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:15 smithi192 ceph-mon[34664]: purged_snaps scrub ok 2024-05-08T06:01:15.932 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:15 smithi192 ceph-mon[34664]: osd.4 [v2:172.21.15.163:6808/591680273,v1:172.21.15.163:6809/591680273] boot 2024-05-08T06:01:15.933 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:15 smithi192 ceph-mon[34664]: osdmap e33: 5 total, 5 up, 5 in 2024-05-08T06:01:15.933 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:15 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T06:01:15.933 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:15 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:15.933 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:15 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:15.933 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:15 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:01:15.933 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:15 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:01:15.933 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:15 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:15.933 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:15 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:01:15.933 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:15 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:01:15.933 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:15 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:16.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:15 smithi163 ceph-mon[34520]: purged_snaps scrub starts 2024-05-08T06:01:16.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:15 smithi163 ceph-mon[34520]: purged_snaps scrub ok 2024-05-08T06:01:16.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:15 smithi163 ceph-mon[34520]: osd.4 [v2:172.21.15.163:6808/591680273,v1:172.21.15.163:6809/591680273] boot 2024-05-08T06:01:16.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:15 smithi163 ceph-mon[34520]: osdmap e33: 5 total, 5 up, 5 in 2024-05-08T06:01:16.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:15 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T06:01:16.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:15 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:16.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:15 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:16.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:15 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:01:16.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:15 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:01:16.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:15 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:16.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:15 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:01:16.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:15 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:01:16.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:15 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:16.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:15 smithi123 ceph-mon[29787]: purged_snaps scrub starts 2024-05-08T06:01:16.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:15 smithi123 ceph-mon[29787]: purged_snaps scrub ok 2024-05-08T06:01:16.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:15 smithi123 ceph-mon[29787]: osd.4 [v2:172.21.15.163:6808/591680273,v1:172.21.15.163:6809/591680273] boot 2024-05-08T06:01:16.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:15 smithi123 ceph-mon[29787]: osdmap e33: 5 total, 5 up, 5 in 2024-05-08T06:01:16.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:15 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T06:01:16.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:15 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:16.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:15 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:16.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:15 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:01:16.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:15 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:01:16.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:15 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:16.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:15 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:01:16.207 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:15 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:01:16.207 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:15 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:16.876 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:16 smithi192 ceph-mon[34664]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T06:01:16.876 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:16 smithi192 ceph-mon[34664]: Detected new or changed devices on smithi163 2024-05-08T06:01:16.876 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:16 smithi192 ceph-mon[34664]: Adjusting osd_memory_target on smithi163 to 8544M 2024-05-08T06:01:16.876 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:16 smithi192 ceph-mon[34664]: osdmap e34: 5 total, 5 up, 5 in 2024-05-08T06:01:17.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:16 smithi163 ceph-mon[34520]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T06:01:17.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:16 smithi163 ceph-mon[34520]: Detected new or changed devices on smithi163 2024-05-08T06:01:17.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:16 smithi163 ceph-mon[34520]: Adjusting osd_memory_target on smithi163 to 8544M 2024-05-08T06:01:17.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:16 smithi163 ceph-mon[34520]: osdmap e34: 5 total, 5 up, 5 in 2024-05-08T06:01:17.121 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.121+0000 7f65d64cf700 1 -- 172.21.15.192:0/1534493971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65d0102d90 msgr2=0x7f65d01031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:17.122 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.121+0000 7f65d64cf700 1 --2- 172.21.15.192:0/1534493971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65d0102d90 0x7f65d01031f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f65c4009990 tx=0x7f65c4009ca0 comp rx=0 tx=0).stop 2024-05-08T06:01:17.122 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.121+0000 7f65d64cf700 1 -- 172.21.15.192:0/1534493971 shutdown_connections 2024-05-08T06:01:17.122 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.121+0000 7f65d64cf700 1 --2- 172.21.15.192:0/1534493971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65d0101b90 0x7f65d0101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:17.122 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.121+0000 7f65d64cf700 1 --2- 172.21.15.192:0/1534493971 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65d0103730 0x7f65d0109fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:17.122 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.121+0000 7f65d64cf700 1 --2- 172.21.15.192:0/1534493971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65d0102d90 0x7f65d01031f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:17.122 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.121+0000 7f65d64cf700 1 -- 172.21.15.192:0/1534493971 >> 172.21.15.192:0/1534493971 conn(0x7f65d00fd110 msgr2=0x7f65d00ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:17.122 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.121+0000 7f65d64cf700 1 -- 172.21.15.192:0/1534493971 shutdown_connections 2024-05-08T06:01:17.122 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.121+0000 7f65d64cf700 1 -- 172.21.15.192:0/1534493971 wait complete. 2024-05-08T06:01:17.122 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.122+0000 7f65d64cf700 1 Processor -- start 2024-05-08T06:01:17.122 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.122+0000 7f65d64cf700 1 -- start start 2024-05-08T06:01:17.124 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.122+0000 7f65d64cf700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65d0101b90 0x7f65d0198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:17.124 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.122+0000 7f65d64cf700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65d0102d90 0x7f65d0198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:17.124 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.122+0000 7f65d64cf700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65d0103730 0x7f65d019fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:17.124 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.122+0000 7f65d64cf700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f65d010df40 con 0x7f65d0102d90 2024-05-08T06:01:17.124 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.122+0000 7f65d64cf700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f65d010ddc0 con 0x7f65d0103730 2024-05-08T06:01:17.125 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.122+0000 7f65d64cf700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f65d010e0c0 con 0x7f65d0101b90 2024-05-08T06:01:17.125 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.122+0000 7f65cffff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65d0101b90 0x7f65d0198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:17.125 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.122+0000 7f65cffff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65d0101b90 0x7f65d0198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:37542/0 (socket says 172.21.15.192:37542) 2024-05-08T06:01:17.125 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.122+0000 7f65cffff700 1 -- 172.21.15.192:0/2211287849 learned_addr learned my addr 172.21.15.192:0/2211287849 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-08T06:01:17.125 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.122+0000 7f65d4a6c700 1 --2- 172.21.15.192:0/2211287849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65d0103730 0x7f65d019fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:17.125 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.122+0000 7f65cffff700 1 -- 172.21.15.192:0/2211287849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65d0103730 msgr2=0x7f65d019fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:17.125 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.122+0000 7f65cffff700 1 --2- 172.21.15.192:0/2211287849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65d0103730 0x7f65d019fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:17.125 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.122+0000 7f65cffff700 1 -- 172.21.15.192:0/2211287849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65d0102d90 msgr2=0x7f65d0198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:17.125 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.122+0000 7f65cffff700 1 --2- 172.21.15.192:0/2211287849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65d0102d90 0x7f65d0198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:17.125 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.122+0000 7f65cffff700 1 -- 172.21.15.192:0/2211287849 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d01a01e0 con 0x7f65d0101b90 2024-05-08T06:01:17.125 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.123+0000 7f65cffff700 1 --2- 172.21.15.192:0/2211287849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65d0101b90 0x7f65d0198580 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f65c000d9e0 tx=0x7f65c000dcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:17.125 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.123+0000 7f65cd7fa700 1 -- 172.21.15.192:0/2211287849 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c001b070 con 0x7f65d0101b90 2024-05-08T06:01:17.125 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.123+0000 7f65cd7fa700 1 -- 172.21.15.192:0/2211287849 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65c000ebd0 con 0x7f65d0101b90 2024-05-08T06:01:17.126 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.123+0000 7f65d64cf700 1 -- 172.21.15.192:0/2211287849 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65d010aa50 con 0x7f65d0101b90 2024-05-08T06:01:17.126 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.123+0000 7f65cd7fa700 1 -- 172.21.15.192:0/2211287849 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c00145a0 con 0x7f65d0101b90 2024-05-08T06:01:17.126 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.123+0000 7f65d64cf700 1 -- 172.21.15.192:0/2211287849 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65d010af90 con 0x7f65d0101b90 2024-05-08T06:01:17.126 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.124+0000 7f65d64cf700 1 -- 172.21.15.192:0/2211287849 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65d006a190 con 0x7f65d0101b90 2024-05-08T06:01:17.126 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.125+0000 7f65cd7fa700 1 -- 172.21.15.192:0/2211287849 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f65c001e070 con 0x7f65d0101b90 2024-05-08T06:01:17.127 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.126+0000 7f65cd7fa700 1 --2- 172.21.15.192:0/2211287849 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f65b80b2880 0x7f65b80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:17.127 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.126+0000 7f65cd7fa700 1 -- 172.21.15.192:0/2211287849 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7f65c00e2f50 con 0x7f65d0101b90 2024-05-08T06:01:17.127 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.126+0000 7f65cf7fe700 1 --2- 172.21.15.192:0/2211287849 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f65b80b2880 0x7f65b80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:17.127 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.127+0000 7f65cf7fe700 1 --2- 172.21.15.192:0/2211287849 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f65b80b2880 0x7f65b80b4d40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f65c400b5c0 tx=0x7f65c4003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:17.129 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.129+0000 7f65cd7fa700 1 -- 172.21.15.192:0/2211287849 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f65c0025070 con 0x7f65d0101b90 2024-05-08T06:01:17.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:16 smithi123 ceph-mon[29787]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T06:01:17.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:16 smithi123 ceph-mon[29787]: Detected new or changed devices on smithi163 2024-05-08T06:01:17.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:16 smithi123 ceph-mon[29787]: Adjusting osd_memory_target on smithi163 to 8544M 2024-05-08T06:01:17.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:16 smithi123 ceph-mon[29787]: osdmap e34: 5 total, 5 up, 5 in 2024-05-08T06:01:17.339 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:17.339+0000 7f65d64cf700 1 -- 172.21.15.192:0/2211287849 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f65d010b2b0 con 0x7f65b80b2880 2024-05-08T06:01:18.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:17 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T06:01:18.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:17 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T06:01:18.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:17 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:01:18.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:17 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T06:01:18.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:17 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T06:01:18.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:17 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:01:18.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:17 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T06:01:18.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:17 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T06:01:18.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:17 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:01:19.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:18 smithi163 ceph-mon[34520]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T06:01:19.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:18 smithi163 ceph-mon[34520]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:01:19.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:18 smithi192 ceph-mon[34664]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T06:01:19.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:18 smithi192 ceph-mon[34664]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:01:19.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:18 smithi123 ceph-mon[29787]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T06:01:19.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:18 smithi123 ceph-mon[29787]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:01:20.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:19 smithi123 ceph-mon[29787]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T06:01:20.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:19 smithi163 ceph-mon[34520]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T06:01:20.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:19 smithi192 ceph-mon[34664]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T06:01:21.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:20 smithi123 ceph-mon[29787]: from='client.? 172.21.15.192:0/2003715989' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85e3f1ab-f8c6-4025-bab9-9ddb6bd3e2dd"}]: dispatch 2024-05-08T06:01:21.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:20 smithi123 ceph-mon[29787]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85e3f1ab-f8c6-4025-bab9-9ddb6bd3e2dd"}]: dispatch 2024-05-08T06:01:21.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:20 smithi123 ceph-mon[29787]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "85e3f1ab-f8c6-4025-bab9-9ddb6bd3e2dd"}]': finished 2024-05-08T06:01:21.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:20 smithi123 ceph-mon[29787]: osdmap e35: 6 total, 5 up, 6 in 2024-05-08T06:01:21.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:20 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T06:01:21.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:20 smithi123 ceph-mon[29787]: from='client.? 172.21.15.192:0/1492509088' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T06:01:21.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:20 smithi163 ceph-mon[34520]: from='client.? 172.21.15.192:0/2003715989' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85e3f1ab-f8c6-4025-bab9-9ddb6bd3e2dd"}]: dispatch 2024-05-08T06:01:21.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:20 smithi163 ceph-mon[34520]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85e3f1ab-f8c6-4025-bab9-9ddb6bd3e2dd"}]: dispatch 2024-05-08T06:01:21.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:20 smithi163 ceph-mon[34520]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "85e3f1ab-f8c6-4025-bab9-9ddb6bd3e2dd"}]': finished 2024-05-08T06:01:21.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:20 smithi163 ceph-mon[34520]: osdmap e35: 6 total, 5 up, 6 in 2024-05-08T06:01:21.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:20 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T06:01:21.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:20 smithi163 ceph-mon[34520]: from='client.? 172.21.15.192:0/1492509088' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T06:01:21.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:20 smithi192 ceph-mon[34664]: from='client.? 172.21.15.192:0/2003715989' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85e3f1ab-f8c6-4025-bab9-9ddb6bd3e2dd"}]: dispatch 2024-05-08T06:01:21.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:20 smithi192 ceph-mon[34664]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85e3f1ab-f8c6-4025-bab9-9ddb6bd3e2dd"}]: dispatch 2024-05-08T06:01:21.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:20 smithi192 ceph-mon[34664]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "85e3f1ab-f8c6-4025-bab9-9ddb6bd3e2dd"}]': finished 2024-05-08T06:01:21.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:20 smithi192 ceph-mon[34664]: osdmap e35: 6 total, 5 up, 6 in 2024-05-08T06:01:21.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:20 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T06:01:21.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:20 smithi192 ceph-mon[34664]: from='client.? 172.21.15.192:0/1492509088' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T06:01:22.189 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:21 smithi163 ceph-mon[34520]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T06:01:22.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:21 smithi123 ceph-mon[29787]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T06:01:22.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:21 smithi192 ceph-mon[34664]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T06:01:24.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:24 smithi192 ceph-mon[34664]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T06:01:24.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:24 smithi163 ceph-mon[34520]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T06:01:24.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:24 smithi123 ceph-mon[29787]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T06:01:26.241 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:26 smithi192 ceph-mon[34664]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T06:01:26.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:26 smithi163 ceph-mon[34520]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T06:01:26.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:26 smithi123 ceph-mon[29787]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T06:01:28.191 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:28 smithi192 ceph-mon[34664]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T06:01:28.192 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:28 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-08T06:01:28.192 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:28 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:01:28.192 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:28 smithi192 ceph-mon[34664]: Deploying daemon osd.5 on smithi192 2024-05-08T06:01:28.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:28 smithi163 ceph-mon[34520]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T06:01:28.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:28 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-08T06:01:28.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:28 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:01:28.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:28 smithi163 ceph-mon[34520]: Deploying daemon osd.5 on smithi192 2024-05-08T06:01:28.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:28 smithi123 ceph-mon[29787]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T06:01:28.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:28 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-08T06:01:28.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:28 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:01:28.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:28 smithi123 ceph-mon[29787]: Deploying daemon osd.5 on smithi192 2024-05-08T06:01:30.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:30 smithi163 ceph-mon[34520]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T06:01:30.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:30 smithi192 ceph-mon[34664]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T06:01:30.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:30 smithi123 ceph-mon[29787]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T06:01:32.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:32 smithi163 ceph-mon[34520]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T06:01:32.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:32 smithi192 ceph-mon[34664]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T06:01:32.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:32 smithi123 ceph-mon[29787]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T06:01:33.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:33 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:01:33.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:33 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:33.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:33 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:33.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:33 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:01:33.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:33 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:33.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:33 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:34.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:33 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:01:34.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:33 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:34.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:33 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:34.901 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:34 smithi192 ceph-mon[34664]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T06:01:34.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:34 smithi123 ceph-mon[29787]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T06:01:35.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:34 smithi163 ceph-mon[34520]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T06:01:36.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:36 smithi192 ceph-mon[34664]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T06:01:36.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:36 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:36.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:36 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:36.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:36 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:01:36.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:36 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:01:36.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:36 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:36.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:36 smithi123 ceph-mon[29787]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T06:01:36.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:36 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:36.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:36 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:36.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:36 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:01:36.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:36 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:01:36.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:36 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:36.470 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:36.469+0000 7f65cd7fa700 1 -- 172.21.15.192:0/2211287849 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f65d010b2b0 con 0x7f65b80b2880 2024-05-08T06:01:36.473 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 5 on host 'smithi192' 2024-05-08T06:01:36.474 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:36.471+0000 7f65d64cf700 1 -- 172.21.15.192:0/2211287849 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f65b80b2880 msgr2=0x7f65b80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:36.474 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:36.471+0000 7f65d64cf700 1 --2- 172.21.15.192:0/2211287849 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f65b80b2880 0x7f65b80b4d40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f65c400b5c0 tx=0x7f65c4003860 comp rx=0 tx=0).stop 2024-05-08T06:01:36.474 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:36.471+0000 7f65d64cf700 1 -- 172.21.15.192:0/2211287849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65d0101b90 msgr2=0x7f65d0198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:36.474 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:36.471+0000 7f65d64cf700 1 --2- 172.21.15.192:0/2211287849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65d0101b90 0x7f65d0198580 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f65c000d9e0 tx=0x7f65c000dcf0 comp rx=0 tx=0).stop 2024-05-08T06:01:36.474 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:36.472+0000 7f65d64cf700 1 -- 172.21.15.192:0/2211287849 shutdown_connections 2024-05-08T06:01:36.475 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:36.472+0000 7f65d64cf700 1 --2- 172.21.15.192:0/2211287849 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f65b80b2880 0x7f65b80b4d40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:36.475 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:36.472+0000 7f65d64cf700 1 --2- 172.21.15.192:0/2211287849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65d0103730 0x7f65d019fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:36.475 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:36.472+0000 7f65d64cf700 1 --2- 172.21.15.192:0/2211287849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65d0101b90 0x7f65d0198580 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:36.475 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:36.472+0000 7f65d64cf700 1 --2- 172.21.15.192:0/2211287849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65d0102d90 0x7f65d0198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:36.475 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:36.472+0000 7f65d64cf700 1 -- 172.21.15.192:0/2211287849 >> 172.21.15.192:0/2211287849 conn(0x7f65d00fd110 msgr2=0x7f65d0106090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:36.475 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:36.472+0000 7f65d64cf700 1 -- 172.21.15.192:0/2211287849 shutdown_connections 2024-05-08T06:01:36.475 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:01:36.472+0000 7f65d64cf700 1 -- 172.21.15.192:0/2211287849 wait complete. 2024-05-08T06:01:36.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:36 smithi163 ceph-mon[34520]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T06:01:36.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:36.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:36.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:01:36.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:01:36.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:37.132 DEBUG:teuthology.orchestra.run.smithi192:osd.5> sudo journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@osd.5.service 2024-05-08T06:01:37.134 INFO:tasks.cephadm:Deploying osd.6 on smithi123 with /dev/vg_nvme/lv_2... 2024-05-08T06:01:37.134 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-08T06:01:37.569 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:01:37.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:37 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:37.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:37 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:37.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:37 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:01:37.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:37 smithi123 ceph-mon[29787]: from='osd.5 [v2:172.21.15.192:6808/1734821180,v1:172.21.15.192:6809/1734821180]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T06:01:37.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:37 smithi123 ceph-mon[29787]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T06:01:37.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:37 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:37.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:37 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:37.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:37 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:01:37.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:37 smithi163 ceph-mon[34520]: from='osd.5 [v2:172.21.15.192:6808/1734821180,v1:172.21.15.192:6809/1734821180]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T06:01:37.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:37 smithi163 ceph-mon[34520]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T06:01:37.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:37 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:37.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:37 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:37.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:37 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:01:37.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:37 smithi192 ceph-mon[34664]: from='osd.5 [v2:172.21.15.192:6808/1734821180,v1:172.21.15.192:6809/1734821180]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T06:01:37.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:37 smithi192 ceph-mon[34664]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T06:01:38.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:38 smithi192 ceph-mon[34664]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T06:01:38.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:38 smithi192 ceph-mon[34664]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-08T06:01:38.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:38 smithi192 ceph-mon[34664]: osdmap e36: 6 total, 5 up, 6 in 2024-05-08T06:01:38.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:38 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T06:01:38.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:38 smithi192 ceph-mon[34664]: from='osd.5 [v2:172.21.15.192:6808/1734821180,v1:172.21.15.192:6809/1734821180]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-08T06:01:38.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:38 smithi192 ceph-mon[34664]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-08T06:01:38.618 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:01:38 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:01:38.457+0000 7f4689ab0700 -1 osd.5 0 waiting for initial osdmap 2024-05-08T06:01:38.619 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:01:38 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:01:38.464+0000 7f46858a3700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T06:01:38.757 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:38 smithi123 ceph-mon[29787]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T06:01:38.757 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:38 smithi123 ceph-mon[29787]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-08T06:01:38.757 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:38 smithi123 ceph-mon[29787]: osdmap e36: 6 total, 5 up, 6 in 2024-05-08T06:01:38.757 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:38 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T06:01:38.757 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:38 smithi123 ceph-mon[29787]: from='osd.5 [v2:172.21.15.192:6808/1734821180,v1:172.21.15.192:6809/1734821180]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-08T06:01:38.757 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:38 smithi123 ceph-mon[29787]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-08T06:01:38.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:38 smithi163 ceph-mon[34520]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T06:01:38.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:38 smithi163 ceph-mon[34520]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-08T06:01:38.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:38 smithi163 ceph-mon[34520]: osdmap e36: 6 total, 5 up, 6 in 2024-05-08T06:01:38.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:38 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T06:01:38.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:38 smithi163 ceph-mon[34520]: from='osd.5 [v2:172.21.15.192:6808/1734821180,v1:172.21.15.192:6809/1734821180]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-08T06:01:38.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:38 smithi163 ceph-mon[34520]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-08T06:01:39.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:39 smithi123 ceph-mon[29787]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-05-08T06:01:39.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:39 smithi123 ceph-mon[29787]: osdmap e37: 6 total, 5 up, 6 in 2024-05-08T06:01:39.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:39 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T06:01:39.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:39 smithi123 ceph-mon[29787]: osd.5 [v2:172.21.15.192:6808/1734821180,v1:172.21.15.192:6809/1734821180] boot 2024-05-08T06:01:39.738 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:39 smithi192 ceph-mon[34664]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-05-08T06:01:39.738 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:39 smithi192 ceph-mon[34664]: osdmap e37: 6 total, 5 up, 6 in 2024-05-08T06:01:39.738 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:39 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T06:01:39.738 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:39 smithi192 ceph-mon[34664]: osd.5 [v2:172.21.15.192:6808/1734821180,v1:172.21.15.192:6809/1734821180] boot 2024-05-08T06:01:39.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:39 smithi163 ceph-mon[34520]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-05-08T06:01:39.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:39 smithi163 ceph-mon[34520]: osdmap e37: 6 total, 5 up, 6 in 2024-05-08T06:01:39.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:39 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T06:01:39.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:39 smithi163 ceph-mon[34520]: osd.5 [v2:172.21.15.192:6808/1734821180,v1:172.21.15.192:6809/1734821180] boot 2024-05-08T06:01:40.704 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-08T06:01:40.705 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-08T06:01:40.705 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2024-05-08T06:01:40.705 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2024-05-08T06:01:40.705 INFO:teuthology.orchestra.run.smithi123.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0324702 s, 323 MB/s 2024-05-08T06:01:40.705 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2024-05-08T06:01:40.745 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:40 smithi123 ceph-mon[29787]: purged_snaps scrub starts 2024-05-08T06:01:40.745 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:40 smithi123 ceph-mon[29787]: purged_snaps scrub ok 2024-05-08T06:01:40.745 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:40 smithi123 ceph-mon[29787]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T06:01:40.745 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:40 smithi123 ceph-mon[29787]: osdmap e38: 6 total, 6 up, 6 in 2024-05-08T06:01:40.745 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:40 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T06:01:40.745 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:40 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:40.745 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:40 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:40.746 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:40 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:01:40.746 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:40 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:01:40.746 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:40 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:40.746 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:40 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:01:40.746 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:40 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:01:40.746 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:40 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:40.746 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:40 smithi123 ceph-mon[29787]: osdmap e39: 6 total, 6 up, 6 in 2024-05-08T06:01:40.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:40 smithi163 ceph-mon[34520]: purged_snaps scrub starts 2024-05-08T06:01:40.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:40 smithi163 ceph-mon[34520]: purged_snaps scrub ok 2024-05-08T06:01:40.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:40 smithi163 ceph-mon[34520]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T06:01:40.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:40 smithi163 ceph-mon[34520]: osdmap e38: 6 total, 6 up, 6 in 2024-05-08T06:01:40.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:40 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T06:01:40.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:40 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:40.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:40 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:40.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:40 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:01:40.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:40 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:01:40.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:40 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:40.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:40 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:01:40.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:40 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:01:40.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:40 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:40.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:40 smithi163 ceph-mon[34520]: osdmap e39: 6 total, 6 up, 6 in 2024-05-08T06:01:40.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:40 smithi192 ceph-mon[34664]: purged_snaps scrub starts 2024-05-08T06:01:40.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:40 smithi192 ceph-mon[34664]: purged_snaps scrub ok 2024-05-08T06:01:40.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:40 smithi192 ceph-mon[34664]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T06:01:40.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:40 smithi192 ceph-mon[34664]: osdmap e38: 6 total, 6 up, 6 in 2024-05-08T06:01:40.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:40 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T06:01:40.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:40 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:40.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:40 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:40.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:40 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:01:40.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:40 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:01:40.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:40 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:40.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:40 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:01:40.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:40 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:01:40.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:40 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:01:40.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:40 smithi192 ceph-mon[34664]: osdmap e39: 6 total, 6 up, 6 in 2024-05-08T06:01:41.690 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi123:vg_nvme/lv_2 2024-05-08T06:01:41.760 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:41 smithi123 ceph-mon[29787]: Detected new or changed devices on smithi192 2024-05-08T06:01:41.760 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:41 smithi123 ceph-mon[29787]: Adjusting osd_memory_target on smithi192 to 8544M 2024-05-08T06:01:41.760 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:41 smithi123 ceph-mon[29787]: osdmap e40: 6 total, 6 up, 6 in 2024-05-08T06:01:41.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:41 smithi163 ceph-mon[34520]: Detected new or changed devices on smithi192 2024-05-08T06:01:41.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:41 smithi163 ceph-mon[34520]: Adjusting osd_memory_target on smithi192 to 8544M 2024-05-08T06:01:41.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:41 smithi163 ceph-mon[34520]: osdmap e40: 6 total, 6 up, 6 in 2024-05-08T06:01:41.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:41 smithi192 ceph-mon[34664]: Detected new or changed devices on smithi192 2024-05-08T06:01:41.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:41 smithi192 ceph-mon[34664]: Adjusting osd_memory_target on smithi192 to 8544M 2024-05-08T06:01:41.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:41 smithi192 ceph-mon[34664]: osdmap e40: 6 total, 6 up, 6 in 2024-05-08T06:01:42.117 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:01:42.530 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:42 smithi123 ceph-mon[29787]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-05-08T06:01:42.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:42 smithi163 ceph-mon[34520]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-05-08T06:01:42.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:42 smithi192 ceph-mon[34664]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-05-08T06:01:43.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:43 smithi192 ceph-mon[34664]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-08T06:01:43.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:43 smithi123 ceph-mon[29787]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-08T06:01:44.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:43 smithi163 ceph-mon[34520]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-08T06:01:44.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.533+0000 7fe2cd800700 1 -- 172.21.15.123:0/3589881055 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2c81050a0 msgr2=0x7fe2c810b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:44.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.533+0000 7fe2cd800700 1 --2- 172.21.15.123:0/3589881055 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2c81050a0 0x7fe2c810b950 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0009990 tx=0x7fe2c0009ca0 comp rx=0 tx=0).stop 2024-05-08T06:01:44.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.533+0000 7fe2cd800700 1 -- 172.21.15.123:0/3589881055 shutdown_connections 2024-05-08T06:01:44.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.533+0000 7fe2cd800700 1 --2- 172.21.15.123:0/3589881055 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2c81050a0 0x7fe2c810b950 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:44.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.533+0000 7fe2cd800700 1 --2- 172.21.15.123:0/3589881055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2c8103500 0x7fe2c8103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:44.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.533+0000 7fe2cd800700 1 --2- 172.21.15.123:0/3589881055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2c8104700 0x7fe2c8104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:44.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.533+0000 7fe2cd800700 1 -- 172.21.15.123:0/3589881055 >> 172.21.15.123:0/3589881055 conn(0x7fe2c80feaa0 msgr2=0x7fe2c8100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:01:44.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.533+0000 7fe2cd800700 1 -- 172.21.15.123:0/3589881055 shutdown_connections 2024-05-08T06:01:44.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.533+0000 7fe2cd800700 1 -- 172.21.15.123:0/3589881055 wait complete. 2024-05-08T06:01:44.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.534+0000 7fe2cd800700 1 Processor -- start 2024-05-08T06:01:44.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.535+0000 7fe2cd800700 1 -- start start 2024-05-08T06:01:44.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.535+0000 7fe2cd800700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2c8103500 0x7fe2c819b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:44.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.535+0000 7fe2cd800700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2c8104700 0x7fe2c819bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:44.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.535+0000 7fe2cd800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2c81050a0 0x7fe2c81a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:44.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.535+0000 7fe2cd800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c810eff0 con 0x7fe2c81050a0 2024-05-08T06:01:44.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.535+0000 7fe2cd800700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c810ee70 con 0x7fe2c8104700 2024-05-08T06:01:44.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.535+0000 7fe2cd800700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c810f170 con 0x7fe2c8103500 2024-05-08T06:01:44.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.535+0000 7fe2be000700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2c8104700 0x7fe2c819bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:44.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.535+0000 7fe2c7400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2c81050a0 0x7fe2c81a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:44.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.536+0000 7fe2be000700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2c8104700 0x7fe2c819bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:51576/0 (socket says 172.21.15.123:51576) 2024-05-08T06:01:44.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.536+0000 7fe2be000700 1 -- 172.21.15.123:0/2832799273 learned_addr learned my addr 172.21.15.123:0/2832799273 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:01:44.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.536+0000 7fe2c6a00700 1 --2- 172.21.15.123:0/2832799273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2c8103500 0x7fe2c819b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:44.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.536+0000 7fe2be000700 1 -- 172.21.15.123:0/2832799273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2c8103500 msgr2=0x7fe2c819b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:44.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.536+0000 7fe2be000700 1 --2- 172.21.15.123:0/2832799273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2c8103500 0x7fe2c819b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:44.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.536+0000 7fe2be000700 1 -- 172.21.15.123:0/2832799273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2c81050a0 msgr2=0x7fe2c81a2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:01:44.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.536+0000 7fe2be000700 1 --2- 172.21.15.123:0/2832799273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2c81050a0 0x7fe2c81a2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:01:44.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.536+0000 7fe2be000700 1 -- 172.21.15.123:0/2832799273 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c81a34a0 con 0x7fe2c8104700 2024-05-08T06:01:44.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.536+0000 7fe2c7400700 1 --2- 172.21.15.123:0/2832799273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2c81050a0 0x7fe2c81a2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:01:44.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.536+0000 7fe2be000700 1 --2- 172.21.15.123:0/2832799273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2c8104700 0x7fe2c819bd80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe2b000e9a0 tx=0x7fe2b000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:44.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.537+0000 7fe2bfe00700 1 -- 172.21.15.123:0/2832799273 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b000ca10 con 0x7fe2c8104700 2024-05-08T06:01:44.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.537+0000 7fe2bfe00700 1 -- 172.21.15.123:0/2832799273 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2b000cbb0 con 0x7fe2c8104700 2024-05-08T06:01:44.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.537+0000 7fe2bfe00700 1 -- 172.21.15.123:0/2832799273 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0018680 con 0x7fe2c8104700 2024-05-08T06:01:44.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.537+0000 7fe2cd800700 1 -- 172.21.15.123:0/2832799273 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2c81a3790 con 0x7fe2c8104700 2024-05-08T06:01:44.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.537+0000 7fe2cd800700 1 -- 172.21.15.123:0/2832799273 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2c81a3c80 con 0x7fe2c8104700 2024-05-08T06:01:44.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.538+0000 7fe2cd800700 1 -- 172.21.15.123:0/2832799273 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2c806a190 con 0x7fe2c8104700 2024-05-08T06:01:44.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.540+0000 7fe2bfe00700 1 -- 172.21.15.123:0/2832799273 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fe2b004aab0 con 0x7fe2c8104700 2024-05-08T06:01:44.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.542+0000 7fe2bfe00700 1 --2- 172.21.15.123:0/2832799273 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe2ac0b2900 0x7fe2ac0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:01:44.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.542+0000 7fe2bfe00700 1 -- 172.21.15.123:0/2832799273 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4533+0+0 (secure 0 0 0) 0x7fe2b00e14e0 con 0x7fe2c8104700 2024-05-08T06:01:44.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.542+0000 7fe2c6a00700 1 --2- 172.21.15.123:0/2832799273 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe2ac0b2900 0x7fe2ac0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:01:44.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.543+0000 7fe2c6a00700 1 --2- 172.21.15.123:0/2832799273 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe2ac0b2900 0x7fe2ac0b4dc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8104560 tx=0x7fe2b800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:01:44.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.544+0000 7fe2bfe00700 1 -- 172.21.15.123:0/2832799273 <== mon.2 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe2b00aa020 con 0x7fe2c8104700 2024-05-08T06:01:44.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:44 smithi123 ceph-mon[29787]: pgmap v156: 1 pgs: 1 activating+degraded; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-08T06:01:44.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:01:44.788+0000 7fe2cd800700 1 -- 172.21.15.123:0/2832799273 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fe2c8076df0 con 0x7fe2ac0b2900 2024-05-08T06:01:44.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:44 smithi192 ceph-mon[34664]: pgmap v156: 1 pgs: 1 activating+degraded; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-08T06:01:45.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:44 smithi163 ceph-mon[34520]: pgmap v156: 1 pgs: 1 activating+degraded; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-08T06:01:45.924 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:45 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T06:01:45.924 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:45 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T06:01:45.924 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:45 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:01:46.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:45 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T06:01:46.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:45 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T06:01:46.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:45 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:01:46.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:45 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T06:01:46.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:45 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T06:01:46.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:45 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:01:46.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:46 smithi123 ceph-mon[29787]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:01:46.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:46 smithi123 ceph-mon[29787]: pgmap v157: 1 pgs: 1 activating+degraded; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-08T06:01:47.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:46 smithi163 ceph-mon[34520]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:01:47.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:46 smithi163 ceph-mon[34520]: pgmap v157: 1 pgs: 1 activating+degraded; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-08T06:01:47.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:46 smithi192 ceph-mon[34664]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:01:47.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:46 smithi192 ceph-mon[34664]: pgmap v157: 1 pgs: 1 activating+degraded; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-08T06:01:48.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:48 smithi192 ceph-mon[34664]: pgmap v158: 1 pgs: 1 activating+degraded; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-08T06:01:48.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:48 smithi123 ceph-mon[29787]: pgmap v158: 1 pgs: 1 activating+degraded; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-08T06:01:49.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:48 smithi163 ceph-mon[34520]: pgmap v158: 1 pgs: 1 activating+degraded; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-08T06:01:49.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:49 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/423183461' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f21a5624-6f8f-48be-bc6d-0a883d6b7b32"}]: dispatch 2024-05-08T06:01:49.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:49 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/423183461' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f21a5624-6f8f-48be-bc6d-0a883d6b7b32"}]': finished 2024-05-08T06:01:49.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:49 smithi123 ceph-mon[29787]: osdmap e41: 7 total, 6 up, 7 in 2024-05-08T06:01:49.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:49 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T06:01:50.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:49 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/423183461' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f21a5624-6f8f-48be-bc6d-0a883d6b7b32"}]: dispatch 2024-05-08T06:01:50.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:49 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/423183461' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f21a5624-6f8f-48be-bc6d-0a883d6b7b32"}]': finished 2024-05-08T06:01:50.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:49 smithi163 ceph-mon[34520]: osdmap e41: 7 total, 6 up, 7 in 2024-05-08T06:01:50.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:49 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T06:01:50.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:49 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/423183461' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f21a5624-6f8f-48be-bc6d-0a883d6b7b32"}]: dispatch 2024-05-08T06:01:50.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:49 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/423183461' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f21a5624-6f8f-48be-bc6d-0a883d6b7b32"}]': finished 2024-05-08T06:01:50.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:49 smithi192 ceph-mon[34664]: osdmap e41: 7 total, 6 up, 7 in 2024-05-08T06:01:50.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:49 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T06:01:50.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:50 smithi123 ceph-mon[29787]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-08T06:01:50.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:50 smithi123 ceph-mon[29787]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-08T06:01:50.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:50 smithi123 ceph-mon[29787]: Cluster is now healthy 2024-05-08T06:01:50.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:50 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/2122700886' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T06:01:51.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:50 smithi163 ceph-mon[34520]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-08T06:01:51.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:50 smithi163 ceph-mon[34520]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-08T06:01:51.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:50 smithi163 ceph-mon[34520]: Cluster is now healthy 2024-05-08T06:01:51.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:50 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/2122700886' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T06:01:51.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:50 smithi192 ceph-mon[34664]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-08T06:01:51.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:50 smithi192 ceph-mon[34664]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-08T06:01:51.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:50 smithi192 ceph-mon[34664]: Cluster is now healthy 2024-05-08T06:01:51.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:50 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/2122700886' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T06:01:52.840 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:52 smithi163 ceph-mon[34520]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-08T06:01:52.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:52 smithi123 ceph-mon[29787]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-08T06:01:53.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:52 smithi192 ceph-mon[34664]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-08T06:01:54.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:54 smithi123 ceph-mon[29787]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T06:01:55.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:54 smithi163 ceph-mon[34520]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T06:01:55.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:54 smithi192 ceph-mon[34664]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T06:01:56.940 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:56 smithi123 ceph-mon[29787]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T06:01:57.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:56 smithi163 ceph-mon[34520]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T06:01:57.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:56 smithi192 ceph-mon[34664]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T06:01:58.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:58 smithi123 ceph-mon[29787]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T06:01:59.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:58 smithi163 ceph-mon[34520]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T06:01:59.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:58 smithi192 ceph-mon[34664]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T06:01:59.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:59 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-08T06:01:59.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:01:59 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:02:00.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:59 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-08T06:02:00.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:01:59 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:02:00.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:59 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-08T06:02:00.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:01:59 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:02:00.959 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:00 smithi123 ceph-mon[29787]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-08T06:02:00.959 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:00 smithi123 ceph-mon[29787]: Deploying daemon osd.6 on smithi123 2024-05-08T06:02:01.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:00 smithi163 ceph-mon[34520]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-08T06:02:01.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:00 smithi163 ceph-mon[34520]: Deploying daemon osd.6 on smithi123 2024-05-08T06:02:01.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:00 smithi192 ceph-mon[34664]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-08T06:02:01.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:00 smithi192 ceph-mon[34664]: Deploying daemon osd.6 on smithi123 2024-05-08T06:02:02.628 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:02 smithi123 ceph-mon[29787]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-08T06:02:03.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:02 smithi163 ceph-mon[34520]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-08T06:02:03.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:02 smithi192 ceph-mon[34664]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-08T06:02:04.883 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:04 smithi123 ceph-mon[29787]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-08T06:02:05.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:04 smithi163 ceph-mon[34520]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-08T06:02:05.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:04 smithi192 ceph-mon[34664]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-08T06:02:06.871 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:06 smithi192 ceph-mon[34664]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-08T06:02:06.886 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:06 smithi123 ceph-mon[29787]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-08T06:02:07.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:06 smithi163 ceph-mon[34520]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-08T06:02:08.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:08 smithi123 ceph-mon[29787]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-08T06:02:08.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:08 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:08.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:08 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:08.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:08 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:02:09.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:08 smithi163 ceph-mon[34520]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-08T06:02:09.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:08 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:09.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:08 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:09.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:08 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:02:09.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:08 smithi192 ceph-mon[34664]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-08T06:02:09.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:08 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:09.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:08 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:09.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:08 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:02:10.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:10 smithi123 ceph-mon[29787]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-08T06:02:11.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:10 smithi163 ceph-mon[34520]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-08T06:02:11.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:10 smithi192 ceph-mon[34664]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-08T06:02:12.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:12 smithi123 ceph-mon[29787]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-08T06:02:12.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:12 smithi123 ceph-mon[29787]: from='osd.6 [v2:172.21.15.123:6818/1002138878,v1:172.21.15.123:6819/1002138878]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T06:02:12.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:12 smithi123 ceph-mon[29787]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T06:02:13.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:12 smithi163 ceph-mon[34520]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-08T06:02:13.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:12 smithi163 ceph-mon[34520]: from='osd.6 [v2:172.21.15.123:6818/1002138878,v1:172.21.15.123:6819/1002138878]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T06:02:13.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:12 smithi163 ceph-mon[34520]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T06:02:13.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:12 smithi192 ceph-mon[34664]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-08T06:02:13.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:12 smithi192 ceph-mon[34664]: from='osd.6 [v2:172.21.15.123:6818/1002138878,v1:172.21.15.123:6819/1002138878]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T06:02:13.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:12 smithi192 ceph-mon[34664]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T06:02:13.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:02:13.233+0000 7fe2bfe00700 1 -- 172.21.15.123:0/2832799273 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe2c8076df0 con 0x7fe2ac0b2900 2024-05-08T06:02:13.234 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 6 on host 'smithi123' 2024-05-08T06:02:13.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:02:13.237+0000 7fe2cd800700 1 -- 172.21.15.123:0/2832799273 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe2ac0b2900 msgr2=0x7fe2ac0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:13.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:02:13.237+0000 7fe2cd800700 1 --2- 172.21.15.123:0/2832799273 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe2ac0b2900 0x7fe2ac0b4dc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8104560 tx=0x7fe2b800b040 comp rx=0 tx=0).stop 2024-05-08T06:02:13.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:02:13.237+0000 7fe2cd800700 1 -- 172.21.15.123:0/2832799273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2c8104700 msgr2=0x7fe2c819bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:13.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:02:13.237+0000 7fe2cd800700 1 --2- 172.21.15.123:0/2832799273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2c8104700 0x7fe2c819bd80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe2b000e9a0 tx=0x7fe2b000ecb0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:02:13.237+0000 7fe2cd800700 1 -- 172.21.15.123:0/2832799273 shutdown_connections 2024-05-08T06:02:13.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:02:13.237+0000 7fe2cd800700 1 --2- 172.21.15.123:0/2832799273 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe2ac0b2900 0x7fe2ac0b4dc0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:02:13.237+0000 7fe2cd800700 1 --2- 172.21.15.123:0/2832799273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2c81050a0 0x7fe2c81a2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:02:13.237+0000 7fe2cd800700 1 --2- 172.21.15.123:0/2832799273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2c8103500 0x7fe2c819b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:02:13.237+0000 7fe2cd800700 1 --2- 172.21.15.123:0/2832799273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2c8104700 0x7fe2c819bd80 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:13.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:02:13.237+0000 7fe2cd800700 1 -- 172.21.15.123:0/2832799273 >> 172.21.15.123:0/2832799273 conn(0x7fe2c80feaa0 msgr2=0x7fe2c80756d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:13.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:02:13.237+0000 7fe2cd800700 1 -- 172.21.15.123:0/2832799273 shutdown_connections 2024-05-08T06:02:13.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:02:13.237+0000 7fe2cd800700 1 -- 172.21.15.123:0/2832799273 wait complete. 2024-05-08T06:02:13.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:13 smithi123 ceph-mon[29787]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-08T06:02:13.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:13 smithi123 ceph-mon[29787]: osdmap e42: 7 total, 6 up, 7 in 2024-05-08T06:02:13.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:13 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T06:02:13.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:13 smithi123 ceph-mon[29787]: from='osd.6 [v2:172.21.15.123:6818/1002138878,v1:172.21.15.123:6819/1002138878]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-08T06:02:13.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:13 smithi123 ceph-mon[29787]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-08T06:02:13.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:13 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:13.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:13 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:13.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:13 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:13.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:13 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:14.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:13 smithi163 ceph-mon[34520]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-08T06:02:14.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:13 smithi163 ceph-mon[34520]: osdmap e42: 7 total, 6 up, 7 in 2024-05-08T06:02:14.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:13 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T06:02:14.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:13 smithi163 ceph-mon[34520]: from='osd.6 [v2:172.21.15.123:6818/1002138878,v1:172.21.15.123:6819/1002138878]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-08T06:02:14.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:13 smithi163 ceph-mon[34520]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-08T06:02:14.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:13 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:14.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:13 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:14.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:13 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:14.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:13 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:14.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:13 smithi192 ceph-mon[34664]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-08T06:02:14.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:13 smithi192 ceph-mon[34664]: osdmap e42: 7 total, 6 up, 7 in 2024-05-08T06:02:14.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:13 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T06:02:14.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:13 smithi192 ceph-mon[34664]: from='osd.6 [v2:172.21.15.123:6818/1002138878,v1:172.21.15.123:6819/1002138878]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-08T06:02:14.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:13 smithi192 ceph-mon[34664]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-08T06:02:14.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:13 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:14.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:13 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:14.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:13 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:14.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:13 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:14.217 DEBUG:teuthology.orchestra.run.smithi123:osd.6> sudo journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@osd.6.service 2024-05-08T06:02:14.221 INFO:tasks.cephadm:Deploying osd.7 on smithi163 with /dev/vg_nvme/lv_2... 2024-05-08T06:02:14.221 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-08T06:02:14.456 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:02:14 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:02:14.304+0000 7fd762e00700 -1 osd.6 0 waiting for initial osdmap 2024-05-08T06:02:14.456 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:02:14 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:02:14.327+0000 7fd758200700 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T06:02:14.466 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T06:02:14.910 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:14 smithi163 ceph-mon[34520]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-08T06:02:14.911 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:14 smithi163 ceph-mon[34520]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-05-08T06:02:14.911 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:14 smithi163 ceph-mon[34520]: osdmap e43: 7 total, 6 up, 7 in 2024-05-08T06:02:14.911 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:14 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T06:02:14.911 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:14 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T06:02:14.911 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:14 smithi163 ceph-mon[34520]: from='osd.6 ' entity='osd.6' 2024-05-08T06:02:14.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:14 smithi123 ceph-mon[29787]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-08T06:02:14.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:14 smithi123 ceph-mon[29787]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-05-08T06:02:14.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:14 smithi123 ceph-mon[29787]: osdmap e43: 7 total, 6 up, 7 in 2024-05-08T06:02:14.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:14 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T06:02:14.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:14 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T06:02:14.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:14 smithi123 ceph-mon[29787]: from='osd.6 ' entity='osd.6' 2024-05-08T06:02:15.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:14 smithi192 ceph-mon[34664]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-08T06:02:15.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:14 smithi192 ceph-mon[34664]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-05-08T06:02:15.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:14 smithi192 ceph-mon[34664]: osdmap e43: 7 total, 6 up, 7 in 2024-05-08T06:02:15.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:14 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T06:02:15.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:14 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T06:02:15.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:14 smithi192 ceph-mon[34664]: from='osd.6 ' entity='osd.6' 2024-05-08T06:02:15.845 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:15 smithi163 ceph-mon[34520]: purged_snaps scrub starts 2024-05-08T06:02:15.845 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:15 smithi163 ceph-mon[34520]: purged_snaps scrub ok 2024-05-08T06:02:15.845 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:15 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T06:02:15.845 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:15 smithi163 ceph-mon[34520]: osd.6 [v2:172.21.15.123:6818/1002138878,v1:172.21.15.123:6819/1002138878] boot 2024-05-08T06:02:15.845 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:15 smithi163 ceph-mon[34520]: osdmap e44: 7 total, 7 up, 7 in 2024-05-08T06:02:15.845 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:15 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T06:02:15.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:15 smithi123 ceph-mon[29787]: purged_snaps scrub starts 2024-05-08T06:02:15.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:15 smithi123 ceph-mon[29787]: purged_snaps scrub ok 2024-05-08T06:02:15.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:15 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T06:02:15.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:15 smithi123 ceph-mon[29787]: osd.6 [v2:172.21.15.123:6818/1002138878,v1:172.21.15.123:6819/1002138878] boot 2024-05-08T06:02:15.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:15 smithi123 ceph-mon[29787]: osdmap e44: 7 total, 7 up, 7 in 2024-05-08T06:02:15.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:15 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T06:02:16.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:15 smithi192 ceph-mon[34664]: purged_snaps scrub starts 2024-05-08T06:02:16.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:15 smithi192 ceph-mon[34664]: purged_snaps scrub ok 2024-05-08T06:02:16.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:15 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T06:02:16.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:15 smithi192 ceph-mon[34664]: osd.6 [v2:172.21.15.123:6818/1002138878,v1:172.21.15.123:6819/1002138878] boot 2024-05-08T06:02:16.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:15 smithi192 ceph-mon[34664]: osdmap e44: 7 total, 7 up, 7 in 2024-05-08T06:02:16.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:15 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T06:02:16.407 INFO:teuthology.orchestra.run.smithi163.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-08T06:02:16.407 INFO:teuthology.orchestra.run.smithi163.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-08T06:02:16.407 INFO:teuthology.orchestra.run.smithi163.stderr: stderr: 10+0 records in 2024-05-08T06:02:16.407 INFO:teuthology.orchestra.run.smithi163.stderr:10+0 records out 2024-05-08T06:02:16.407 INFO:teuthology.orchestra.run.smithi163.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0131034 s, 800 MB/s 2024-05-08T06:02:16.407 INFO:teuthology.orchestra.run.smithi163.stderr:--> Zapping successful for: 2024-05-08T06:02:16.930 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:16 smithi123 ceph-mon[29787]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-08T06:02:17.021 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:16 smithi163 ceph-mon[34520]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-08T06:02:17.050 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi163:vg_nvme/lv_2 2024-05-08T06:02:17.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:16 smithi192 ceph-mon[34664]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-05-08T06:02:17.292 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T06:02:17.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:17 smithi123 ceph-mon[29787]: osdmap e45: 7 total, 7 up, 7 in 2024-05-08T06:02:18.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:17 smithi192 ceph-mon[34664]: osdmap e45: 7 total, 7 up, 7 in 2024-05-08T06:02:18.092 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:17 smithi163 ceph-mon[34520]: osdmap e45: 7 total, 7 up, 7 in 2024-05-08T06:02:18.780 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.777+0000 7f3d82cc5700 1 -- 172.21.15.163:0/497599041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d7c109260 msgr2=0x7f3d7c10b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:18.780 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.777+0000 7f3d82cc5700 1 --2- 172.21.15.163:0/497599041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d7c109260 0x7f3d7c10b670 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3d7400b0a0 tx=0x7f3d7400b3b0 comp rx=0 tx=0).stop 2024-05-08T06:02:18.780 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.777+0000 7f3d82cc5700 1 -- 172.21.15.163:0/497599041 shutdown_connections 2024-05-08T06:02:18.780 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.777+0000 7f3d82cc5700 1 --2- 172.21.15.163:0/497599041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d7c109260 0x7f3d7c10b670 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:18.780 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.777+0000 7f3d82cc5700 1 --2- 172.21.15.163:0/497599041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d7c104000 0x7f3d7c1063f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:18.780 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.777+0000 7f3d82cc5700 1 --2- 172.21.15.163:0/497599041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d7c106930 0x7f3d7c108d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:18.780 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.777+0000 7f3d82cc5700 1 -- 172.21.15.163:0/497599041 >> 172.21.15.163:0/497599041 conn(0x7f3d7c0fd9d0 msgr2=0x7f3d7c0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:18.780 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.777+0000 7f3d82cc5700 1 -- 172.21.15.163:0/497599041 shutdown_connections 2024-05-08T06:02:18.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.777+0000 7f3d82cc5700 1 -- 172.21.15.163:0/497599041 wait complete. 2024-05-08T06:02:18.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.777+0000 7f3d82cc5700 1 Processor -- start 2024-05-08T06:02:18.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.777+0000 7f3d82cc5700 1 -- start start 2024-05-08T06:02:18.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.777+0000 7f3d82cc5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d7c104000 0x7f3d7c199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:18.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.777+0000 7f3d82cc5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d7c106930 0x7f3d7c199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:18.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.778+0000 7f3d82cc5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d7c109260 0x7f3d7c1a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:18.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.778+0000 7f3d82cc5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7c10efd0 con 0x7f3d7c106930 2024-05-08T06:02:18.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.778+0000 7f3d82cc5700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7c10ee50 con 0x7f3d7c109260 2024-05-08T06:02:18.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.778+0000 7f3d82cc5700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7c10f150 con 0x7f3d7c104000 2024-05-08T06:02:18.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.778+0000 7f3d81262700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d7c109260 0x7f3d7c1a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:18.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.778+0000 7f3d81262700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d7c109260 0x7f3d7c1a0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39104/0 (socket says 172.21.15.163:39104) 2024-05-08T06:02:18.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.778+0000 7f3d81262700 1 -- 172.21.15.163:0/2502060305 learned_addr learned my addr 172.21.15.163:0/2502060305 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:02:18.781 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.778+0000 7f3d80a61700 1 --2- 172.21.15.163:0/2502060305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d7c104000 0x7f3d7c199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:18.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.778+0000 7f3d7bfff700 1 --2- 172.21.15.163:0/2502060305 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d7c106930 0x7f3d7c199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:18.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.778+0000 7f3d81262700 1 -- 172.21.15.163:0/2502060305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d7c104000 msgr2=0x7f3d7c199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:18.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.778+0000 7f3d81262700 1 --2- 172.21.15.163:0/2502060305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d7c104000 0x7f3d7c199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:18.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.778+0000 7f3d81262700 1 -- 172.21.15.163:0/2502060305 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d7c106930 msgr2=0x7f3d7c199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:18.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.778+0000 7f3d81262700 1 --2- 172.21.15.163:0/2502060305 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d7c106930 0x7f3d7c199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:18.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.778+0000 7f3d81262700 1 -- 172.21.15.163:0/2502060305 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d7c1a12a0 con 0x7f3d7c109260 2024-05-08T06:02:18.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.778+0000 7f3d81262700 1 --2- 172.21.15.163:0/2502060305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d7c109260 0x7f3d7c1a0c00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3d7400b070 tx=0x7f3d7400ced0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:18.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.778+0000 7f3d79ffb700 1 -- 172.21.15.163:0/2502060305 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d74021070 con 0x7f3d7c109260 2024-05-08T06:02:18.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.778+0000 7f3d79ffb700 1 -- 172.21.15.163:0/2502060305 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d74007d00 con 0x7f3d7c109260 2024-05-08T06:02:18.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.778+0000 7f3d82cc5700 1 -- 172.21.15.163:0/2502060305 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d7c1a1530 con 0x7f3d7c109260 2024-05-08T06:02:18.782 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.778+0000 7f3d79ffb700 1 -- 172.21.15.163:0/2502060305 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7401d450 con 0x7f3d7c109260 2024-05-08T06:02:18.783 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.779+0000 7f3d82cc5700 1 -- 172.21.15.163:0/2502060305 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d7c1a1990 con 0x7f3d7c109260 2024-05-08T06:02:18.783 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.780+0000 7f3d82cc5700 1 -- 172.21.15.163:0/2502060305 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d7c06a190 con 0x7f3d7c109260 2024-05-08T06:02:18.784 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.781+0000 7f3d79ffb700 1 -- 172.21.15.163:0/2502060305 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f3d74003a00 con 0x7f3d7c109260 2024-05-08T06:02:18.784 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.782+0000 7f3d79ffb700 1 --2- 172.21.15.163:0/2502060305 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f3d6c0b2880 0x7f3d6c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:18.784 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.782+0000 7f3d79ffb700 1 -- 172.21.15.163:0/2502060305 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 4965+0+0 (secure 0 0 0) 0x7f3d740e27c0 con 0x7f3d7c109260 2024-05-08T06:02:18.784 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.782+0000 7f3d80a61700 1 --2- 172.21.15.163:0/2502060305 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f3d6c0b2880 0x7f3d6c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:18.784 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.783+0000 7f3d80a61700 1 --2- 172.21.15.163:0/2502060305 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f3d6c0b2880 0x7f3d6c0b4d40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3d70005950 tx=0x7f3d7000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:18.786 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.786+0000 7f3d79ffb700 1 -- 172.21.15.163:0/2502060305 <== mon.2 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3d740ab1e0 con 0x7f3d7c109260 2024-05-08T06:02:18.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:18 smithi123 ceph-mon[29787]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:18.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:18.997+0000 7f3d82cc5700 1 -- 172.21.15.163:0/2502060305 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f3d7c0638e0 con 0x7f3d6c0b2880 2024-05-08T06:02:18.998 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:18 smithi163 ceph-mon[34520]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:19.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:18 smithi192 ceph-mon[34664]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:19.926 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:19 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T06:02:19.926 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:19 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T06:02:19.926 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:19 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:02:19.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:19 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T06:02:19.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:19 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T06:02:19.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:19 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:02:20.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:19 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T06:02:20.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:19 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T06:02:20.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:19 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:02:20.928 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:20 smithi163 ceph-mon[34520]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:20.928 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:20 smithi163 ceph-mon[34520]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:02:20.928 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:20 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:20.928 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:20 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:20.929 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:20 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:02:20.929 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:20 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:02:20.929 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:20 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:02:20.929 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:20 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:20.929 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:20 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:02:20.929 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:20 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:02:20.929 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:20 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:20.929 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:20 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:02:20.929 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:20 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:02:20.929 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:20 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:02:20.929 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:20 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:21.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:20 smithi192 ceph-mon[34664]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:21.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:20 smithi192 ceph-mon[34664]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:02:21.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:20 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:21.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:20 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:21.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:20 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:02:21.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:20 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:02:21.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:20 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:02:21.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:20 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:21.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:20 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:02:21.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:20 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:02:21.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:20 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:21.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:20 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:02:21.052 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:20 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:02:21.052 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:20 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:02:21.052 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:20 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:21.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:20 smithi123 ceph-mon[29787]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:21.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:20 smithi123 ceph-mon[29787]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:02:21.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:20 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:21.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:20 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:21.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:20 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:02:21.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:20 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:02:21.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:20 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:02:21.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:20 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:21.207 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:20 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:02:21.207 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:20 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:02:21.207 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:20 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:21.207 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:20 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:02:21.207 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:20 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:02:21.207 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:20 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:02:21.207 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:20 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:22.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:21 smithi163 ceph-mon[34520]: Detected new or changed devices on smithi123 2024-05-08T06:02:22.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:21 smithi163 ceph-mon[34520]: Adjusting osd_memory_target on smithi123 to 5696M 2024-05-08T06:02:22.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:21 smithi192 ceph-mon[34664]: Detected new or changed devices on smithi123 2024-05-08T06:02:22.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:21 smithi192 ceph-mon[34664]: Adjusting osd_memory_target on smithi123 to 5696M 2024-05-08T06:02:22.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:21 smithi123 ceph-mon[29787]: Detected new or changed devices on smithi123 2024-05-08T06:02:22.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:21 smithi123 ceph-mon[29787]: Adjusting osd_memory_target on smithi123 to 5696M 2024-05-08T06:02:23.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:22 smithi163 ceph-mon[34520]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:23.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:22 smithi163 ceph-mon[34520]: from='client.? 172.21.15.163:0/2370730737' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6eb16729-d260-4041-9829-3e868795b111"}]: dispatch 2024-05-08T06:02:23.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:22 smithi163 ceph-mon[34520]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6eb16729-d260-4041-9829-3e868795b111"}]: dispatch 2024-05-08T06:02:23.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:22 smithi163 ceph-mon[34520]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6eb16729-d260-4041-9829-3e868795b111"}]': finished 2024-05-08T06:02:23.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:22 smithi163 ceph-mon[34520]: osdmap e46: 8 total, 7 up, 8 in 2024-05-08T06:02:23.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:22 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T06:02:23.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:22 smithi163 ceph-mon[34520]: from='client.? 172.21.15.163:0/2896702871' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T06:02:23.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:22 smithi192 ceph-mon[34664]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:23.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:22 smithi192 ceph-mon[34664]: from='client.? 172.21.15.163:0/2370730737' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6eb16729-d260-4041-9829-3e868795b111"}]: dispatch 2024-05-08T06:02:23.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:22 smithi192 ceph-mon[34664]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6eb16729-d260-4041-9829-3e868795b111"}]: dispatch 2024-05-08T06:02:23.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:22 smithi192 ceph-mon[34664]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6eb16729-d260-4041-9829-3e868795b111"}]': finished 2024-05-08T06:02:23.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:22 smithi192 ceph-mon[34664]: osdmap e46: 8 total, 7 up, 8 in 2024-05-08T06:02:23.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:22 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T06:02:23.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:22 smithi192 ceph-mon[34664]: from='client.? 172.21.15.163:0/2896702871' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T06:02:23.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:22 smithi123 ceph-mon[29787]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:23.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:22 smithi123 ceph-mon[29787]: from='client.? 172.21.15.163:0/2370730737' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6eb16729-d260-4041-9829-3e868795b111"}]: dispatch 2024-05-08T06:02:23.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:22 smithi123 ceph-mon[29787]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6eb16729-d260-4041-9829-3e868795b111"}]: dispatch 2024-05-08T06:02:23.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:22 smithi123 ceph-mon[29787]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6eb16729-d260-4041-9829-3e868795b111"}]': finished 2024-05-08T06:02:23.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:22 smithi123 ceph-mon[29787]: osdmap e46: 8 total, 7 up, 8 in 2024-05-08T06:02:23.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:22 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T06:02:23.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:22 smithi123 ceph-mon[29787]: from='client.? 172.21.15.163:0/2896702871' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T06:02:25.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:24 smithi163 ceph-mon[34520]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:25.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:24 smithi192 ceph-mon[34664]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:25.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:24 smithi123 ceph-mon[29787]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:26.904 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:26 smithi163 ceph-mon[34520]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:27.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:26 smithi192 ceph-mon[34664]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:27.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:26 smithi123 ceph-mon[29787]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:28.946 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:28 smithi163 ceph-mon[34520]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:29.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:28 smithi192 ceph-mon[34664]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:29.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:28 smithi123 ceph-mon[29787]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:29.922 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:29 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-08T06:02:29.922 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:29 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:02:30.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:29 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-08T06:02:30.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:29 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:02:30.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:29 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-08T06:02:30.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:29 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:02:30.894 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:30 smithi163 ceph-mon[34520]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:30.895 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:30 smithi163 ceph-mon[34520]: Deploying daemon osd.7 on smithi163 2024-05-08T06:02:31.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:30 smithi192 ceph-mon[34664]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:31.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:30 smithi192 ceph-mon[34664]: Deploying daemon osd.7 on smithi163 2024-05-08T06:02:31.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:30 smithi123 ceph-mon[29787]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:31.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:30 smithi123 ceph-mon[29787]: Deploying daemon osd.7 on smithi163 2024-05-08T06:02:32.895 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:32 smithi163 ceph-mon[34520]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:33.008 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:32 smithi123 ceph-mon[29787]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:33.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:32 smithi192 ceph-mon[34664]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:34.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:34 smithi123 ceph-mon[29787]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:34.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:34 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:34.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:34 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:34.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:34 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:02:35.014 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:34 smithi163 ceph-mon[34520]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:35.014 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:34 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:35.014 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:34 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:35.014 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:34 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:02:35.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:34 smithi192 ceph-mon[34664]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:35.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:34 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:35.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:34 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:35.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:34 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:02:36.960 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:36 smithi163 ceph-mon[34520]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:37.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:36 smithi192 ceph-mon[34664]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:37.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:36 smithi123 ceph-mon[29787]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:38.382 INFO:teuthology.orchestra.run.smithi163.stdout:Created osd(s) 7 on host 'smithi163' 2024-05-08T06:02:38.383 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:38.382+0000 7f3d79ffb700 1 -- 172.21.15.163:0/2502060305 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3d7c0638e0 con 0x7f3d6c0b2880 2024-05-08T06:02:38.385 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:38.385+0000 7f3d82cc5700 1 -- 172.21.15.163:0/2502060305 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f3d6c0b2880 msgr2=0x7f3d6c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:38.385 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:38.385+0000 7f3d82cc5700 1 --2- 172.21.15.163:0/2502060305 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f3d6c0b2880 0x7f3d6c0b4d40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3d70005950 tx=0x7f3d7000a300 comp rx=0 tx=0).stop 2024-05-08T06:02:38.385 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:38.385+0000 7f3d82cc5700 1 -- 172.21.15.163:0/2502060305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d7c109260 msgr2=0x7f3d7c1a0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:38.385 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:38.385+0000 7f3d82cc5700 1 --2- 172.21.15.163:0/2502060305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d7c109260 0x7f3d7c1a0c00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3d7400b070 tx=0x7f3d7400ced0 comp rx=0 tx=0).stop 2024-05-08T06:02:38.385 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:38.385+0000 7f3d82cc5700 1 -- 172.21.15.163:0/2502060305 shutdown_connections 2024-05-08T06:02:38.385 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:38.385+0000 7f3d82cc5700 1 --2- 172.21.15.163:0/2502060305 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f3d6c0b2880 0x7f3d6c0b4d40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:38.385 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:38.385+0000 7f3d82cc5700 1 --2- 172.21.15.163:0/2502060305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d7c109260 0x7f3d7c1a0c00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:38.385 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:38.385+0000 7f3d82cc5700 1 --2- 172.21.15.163:0/2502060305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d7c104000 0x7f3d7c199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:38.386 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:38.385+0000 7f3d82cc5700 1 --2- 172.21.15.163:0/2502060305 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d7c106930 0x7f3d7c199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:38.386 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:38.385+0000 7f3d82cc5700 1 -- 172.21.15.163:0/2502060305 >> 172.21.15.163:0/2502060305 conn(0x7f3d7c0fd9d0 msgr2=0x7f3d7c104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:38.386 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:38.385+0000 7f3d82cc5700 1 -- 172.21.15.163:0/2502060305 shutdown_connections 2024-05-08T06:02:38.386 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:02:38.385+0000 7f3d82cc5700 1 -- 172.21.15.163:0/2502060305 wait complete. 2024-05-08T06:02:38.681 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:38 smithi163 ceph-mon[34520]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:38.682 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:38 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:38.682 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:38 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:38.682 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:38 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:02:38.682 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:38 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:02:38.682 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:38 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:38.682 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:38 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:38.682 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:38 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:38.682 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:38 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:02:38.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:38 smithi192 ceph-mon[34664]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:38.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:38 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:38.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:38 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:38.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:38 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:02:38.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:38 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:02:38.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:38 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:38.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:38 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:38.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:38 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:38.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:38 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:02:38.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:38 smithi123 ceph-mon[29787]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:38.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:38 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:38.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:38 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:38.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:38 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:02:38.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:38 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:02:38.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:38 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:38.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:38 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:38.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:38 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:38.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:38 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:02:39.000 DEBUG:teuthology.orchestra.run.smithi163:osd.7> sudo journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@osd.7.service 2024-05-08T06:02:39.003 INFO:tasks.cephadm:Deploying osd.8 on smithi192 with /dev/vg_nvme/lv_2... 2024-05-08T06:02:39.003 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-08T06:02:39.234 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T06:02:39.666 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:39 smithi192 ceph-mon[34664]: from='osd.7 [v2:172.21.15.163:6816/354411978,v1:172.21.15.163:6817/354411978]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T06:02:39.666 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:39 smithi192 ceph-mon[34664]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T06:02:39.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:39 smithi163 ceph-mon[34520]: from='osd.7 [v2:172.21.15.163:6816/354411978,v1:172.21.15.163:6817/354411978]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T06:02:39.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:39 smithi163 ceph-mon[34520]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T06:02:39.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:39 smithi123 ceph-mon[29787]: from='osd.7 [v2:172.21.15.163:6816/354411978,v1:172.21.15.163:6817/354411978]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T06:02:39.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:39 smithi123 ceph-mon[29787]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T06:02:40.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:40 smithi163 ceph-mon[34520]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:40.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:40 smithi163 ceph-mon[34520]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-08T06:02:40.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:40 smithi163 ceph-mon[34520]: from='osd.7 [v2:172.21.15.163:6816/354411978,v1:172.21.15.163:6817/354411978]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-05-08T06:02:40.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:40 smithi163 ceph-mon[34520]: osdmap e47: 8 total, 7 up, 8 in 2024-05-08T06:02:40.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:40 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T06:02:40.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:40 smithi163 ceph-mon[34520]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-05-08T06:02:40.789 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:02:40 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:02:40.511+0000 7f73e57b4700 -1 osd.7 0 waiting for initial osdmap 2024-05-08T06:02:40.789 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:02:40 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:02:40.524+0000 7f73e15a7700 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T06:02:40.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:40 smithi192 ceph-mon[34664]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:40.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:40 smithi192 ceph-mon[34664]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-08T06:02:40.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:40 smithi192 ceph-mon[34664]: from='osd.7 [v2:172.21.15.163:6816/354411978,v1:172.21.15.163:6817/354411978]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-05-08T06:02:40.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:40 smithi192 ceph-mon[34664]: osdmap e47: 8 total, 7 up, 8 in 2024-05-08T06:02:40.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:40 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T06:02:40.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:40 smithi192 ceph-mon[34664]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-05-08T06:02:40.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:40 smithi123 ceph-mon[29787]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:40.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:40 smithi123 ceph-mon[29787]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-08T06:02:40.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:40 smithi123 ceph-mon[29787]: from='osd.7 [v2:172.21.15.163:6816/354411978,v1:172.21.15.163:6817/354411978]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-05-08T06:02:40.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:40 smithi123 ceph-mon[29787]: osdmap e47: 8 total, 7 up, 8 in 2024-05-08T06:02:40.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:40 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T06:02:40.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:40 smithi123 ceph-mon[29787]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-05-08T06:02:41.052 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-08T06:02:41.052 INFO:teuthology.orchestra.run.smithi192.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-08T06:02:41.052 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10+0 records in 2024-05-08T06:02:41.052 INFO:teuthology.orchestra.run.smithi192.stderr:10+0 records out 2024-05-08T06:02:41.052 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112885 s, 929 MB/s 2024-05-08T06:02:41.052 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping successful for: 2024-05-08T06:02:41.563 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi192:vg_nvme/lv_2 2024-05-08T06:02:41.700 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:41 smithi163 ceph-mon[34520]: purged_snaps scrub starts 2024-05-08T06:02:41.700 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:41 smithi163 ceph-mon[34520]: purged_snaps scrub ok 2024-05-08T06:02:41.700 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:41 smithi163 ceph-mon[34520]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-05-08T06:02:41.700 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:41 smithi163 ceph-mon[34520]: osdmap e48: 8 total, 7 up, 8 in 2024-05-08T06:02:41.700 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:41 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T06:02:41.700 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:41 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T06:02:41.795 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T06:02:41.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:41 smithi192 ceph-mon[34664]: purged_snaps scrub starts 2024-05-08T06:02:41.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:41 smithi192 ceph-mon[34664]: purged_snaps scrub ok 2024-05-08T06:02:41.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:41 smithi192 ceph-mon[34664]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-05-08T06:02:41.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:41 smithi192 ceph-mon[34664]: osdmap e48: 8 total, 7 up, 8 in 2024-05-08T06:02:41.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:41 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T06:02:41.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:41 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T06:02:41.958 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:41 smithi123 ceph-mon[29787]: purged_snaps scrub starts 2024-05-08T06:02:41.958 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:41 smithi123 ceph-mon[29787]: purged_snaps scrub ok 2024-05-08T06:02:41.958 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:41 smithi123 ceph-mon[29787]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-05-08T06:02:41.958 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:41 smithi123 ceph-mon[29787]: osdmap e48: 8 total, 7 up, 8 in 2024-05-08T06:02:41.958 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:41 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T06:02:41.958 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:41 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T06:02:42.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:42 smithi192 ceph-mon[34664]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:42.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:42 smithi192 ceph-mon[34664]: osd.7 [v2:172.21.15.163:6816/354411978,v1:172.21.15.163:6817/354411978] boot 2024-05-08T06:02:42.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:42 smithi192 ceph-mon[34664]: osdmap e49: 8 total, 8 up, 8 in 2024-05-08T06:02:42.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:42 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T06:02:42.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:42 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:42.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:42 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:42.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:42 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:02:42.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:42 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:02:42.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:42 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:02:42.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:42 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:42.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:42 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:02:42.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:42 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:02:42.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:42 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:42.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:42 smithi123 ceph-mon[29787]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:42.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:42 smithi123 ceph-mon[29787]: osd.7 [v2:172.21.15.163:6816/354411978,v1:172.21.15.163:6817/354411978] boot 2024-05-08T06:02:42.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:42 smithi123 ceph-mon[29787]: osdmap e49: 8 total, 8 up, 8 in 2024-05-08T06:02:42.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:42 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T06:02:42.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:42 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:42.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:42 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:42.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:42 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:02:42.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:42 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:02:42.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:42 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:02:42.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:42 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:42.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:42 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:02:42.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:42 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:02:42.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:42 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:43.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:42 smithi163 ceph-mon[34520]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T06:02:43.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:42 smithi163 ceph-mon[34520]: osd.7 [v2:172.21.15.163:6816/354411978,v1:172.21.15.163:6817/354411978] boot 2024-05-08T06:02:43.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:42 smithi163 ceph-mon[34520]: osdmap e49: 8 total, 8 up, 8 in 2024-05-08T06:02:43.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:42 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T06:02:43.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:42 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:43.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:42 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:43.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:42 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:02:43.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:42 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:02:43.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:42 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:02:43.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:42 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:43.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:42 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:02:43.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:42 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:02:43.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:42 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:43.343 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.341+0000 7fcaa32ac700 1 -- 172.21.15.192:0/1372688553 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca9c1068e0 msgr2=0x7fca9c108cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:43.343 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.341+0000 7fcaa32ac700 1 --2- 172.21.15.192:0/1372688553 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca9c1068e0 0x7fca9c108cd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fca8c009990 tx=0x7fca8c009ca0 comp rx=0 tx=0).stop 2024-05-08T06:02:43.343 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.341+0000 7fcaa32ac700 1 -- 172.21.15.192:0/1372688553 shutdown_connections 2024-05-08T06:02:43.343 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.341+0000 7fcaa32ac700 1 --2- 172.21.15.192:0/1372688553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca9c103fb0 0x7fca9c1063a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:43.343 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.341+0000 7fcaa32ac700 1 --2- 172.21.15.192:0/1372688553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca9c109210 0x7fca9c10b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:43.343 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.341+0000 7fcaa32ac700 1 --2- 172.21.15.192:0/1372688553 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca9c1068e0 0x7fca9c108cd0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:43.343 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.341+0000 7fcaa32ac700 1 -- 172.21.15.192:0/1372688553 >> 172.21.15.192:0/1372688553 conn(0x7fca9c0fd9c0 msgr2=0x7fca9c0ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:02:43.343 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.341+0000 7fcaa32ac700 1 -- 172.21.15.192:0/1372688553 shutdown_connections 2024-05-08T06:02:43.344 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.341+0000 7fcaa32ac700 1 -- 172.21.15.192:0/1372688553 wait complete. 2024-05-08T06:02:43.344 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.341+0000 7fcaa32ac700 1 Processor -- start 2024-05-08T06:02:43.344 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.341+0000 7fcaa32ac700 1 -- start start 2024-05-08T06:02:43.344 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.341+0000 7fcaa32ac700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca9c103fb0 0x7fca9c199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:43.344 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.341+0000 7fcaa32ac700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca9c1068e0 0x7fca9c199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:43.344 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.342+0000 7fcaa32ac700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca9c109210 0x7fca9c1a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:43.344 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.342+0000 7fcaa32ac700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c10f040 con 0x7fca9c1068e0 2024-05-08T06:02:43.344 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.342+0000 7fcaa32ac700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c10eec0 con 0x7fca9c109210 2024-05-08T06:02:43.344 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.342+0000 7fcaa32ac700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c10f1c0 con 0x7fca9c103fb0 2024-05-08T06:02:43.344 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.342+0000 7fcaa1048700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca9c103fb0 0x7fca9c199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:43.344 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.342+0000 7fcaa1048700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca9c103fb0 0x7fca9c199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:58196/0 (socket says 172.21.15.192:58196) 2024-05-08T06:02:43.344 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.342+0000 7fcaa1048700 1 -- 172.21.15.192:0/3177733286 learned_addr learned my addr 172.21.15.192:0/3177733286 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-08T06:02:43.344 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.342+0000 7fcaa1048700 1 -- 172.21.15.192:0/3177733286 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca9c109210 msgr2=0x7fca9c1a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:43.344 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.342+0000 7fcaa1048700 1 --2- 172.21.15.192:0/3177733286 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca9c109210 0x7fca9c1a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:43.345 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.342+0000 7fcaa1048700 1 -- 172.21.15.192:0/3177733286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca9c1068e0 msgr2=0x7fca9c199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:02:43.345 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.342+0000 7fcaa1048700 1 --2- 172.21.15.192:0/3177733286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca9c1068e0 0x7fca9c199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:02:43.345 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.342+0000 7fcaa1048700 1 -- 172.21.15.192:0/3177733286 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca9c1a12e0 con 0x7fca9c103fb0 2024-05-08T06:02:43.345 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.342+0000 7fcaa1048700 1 --2- 172.21.15.192:0/3177733286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca9c103fb0 0x7fca9c199680 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fca9800b560 tx=0x7fca9800b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:43.345 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.342+0000 7fca927fc700 1 -- 172.21.15.192:0/3177733286 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca98015020 con 0x7fca9c103fb0 2024-05-08T06:02:43.345 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.342+0000 7fca927fc700 1 -- 172.21.15.192:0/3177733286 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca98011ce0 con 0x7fca9c103fb0 2024-05-08T06:02:43.345 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.342+0000 7fcaa32ac700 1 -- 172.21.15.192:0/3177733286 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca9c1a15d0 con 0x7fca9c103fb0 2024-05-08T06:02:43.345 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.342+0000 7fca927fc700 1 -- 172.21.15.192:0/3177733286 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca98010da0 con 0x7fca9c103fb0 2024-05-08T06:02:43.345 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.342+0000 7fcaa32ac700 1 -- 172.21.15.192:0/3177733286 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca9c1a1be0 con 0x7fca9c103fb0 2024-05-08T06:02:43.346 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.343+0000 7fcaa32ac700 1 -- 172.21.15.192:0/3177733286 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca9c06a190 con 0x7fca9c103fb0 2024-05-08T06:02:43.346 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.345+0000 7fca927fc700 1 -- 172.21.15.192:0/3177733286 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fca98018020 con 0x7fca9c103fb0 2024-05-08T06:02:43.346 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.345+0000 7fca927fc700 1 --2- 172.21.15.192:0/3177733286 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fca880b2900 0x7fca880b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:02:43.346 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.345+0000 7fca927fc700 1 -- 172.21.15.192:0/3177733286 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5397+0+0 (secure 0 0 0) 0x7fca980e13c0 con 0x7fca9c103fb0 2024-05-08T06:02:43.346 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.346+0000 7fcaa0847700 1 --2- 172.21.15.192:0/3177733286 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fca880b2900 0x7fca880b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:02:43.347 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.347+0000 7fcaa0847700 1 --2- 172.21.15.192:0/3177733286 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fca880b2900 0x7fca880b4dc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00b5c0 tx=0x7fca8c003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:02:43.349 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.349+0000 7fca927fc700 1 -- 172.21.15.192:0/3177733286 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fca9801f020 con 0x7fca9c103fb0 2024-05-08T06:02:43.563 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:02:43.563+0000 7fcaa32ac700 1 -- 172.21.15.192:0/3177733286 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fca9c0638e0 con 0x7fca880b2900 2024-05-08T06:02:43.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:43 smithi192 ceph-mon[34664]: Detected new or changed devices on smithi163 2024-05-08T06:02:43.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:43 smithi192 ceph-mon[34664]: Adjusting osd_memory_target on smithi163 to 5696M 2024-05-08T06:02:43.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:43 smithi192 ceph-mon[34664]: osdmap e50: 8 total, 8 up, 8 in 2024-05-08T06:02:43.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:43 smithi123 ceph-mon[29787]: Detected new or changed devices on smithi163 2024-05-08T06:02:43.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:43 smithi123 ceph-mon[29787]: Adjusting osd_memory_target on smithi163 to 5696M 2024-05-08T06:02:43.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:43 smithi123 ceph-mon[29787]: osdmap e50: 8 total, 8 up, 8 in 2024-05-08T06:02:44.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:43 smithi163 ceph-mon[34520]: Detected new or changed devices on smithi163 2024-05-08T06:02:44.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:43 smithi163 ceph-mon[34520]: Adjusting osd_memory_target on smithi163 to 5696M 2024-05-08T06:02:44.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:43 smithi163 ceph-mon[34520]: osdmap e50: 8 total, 8 up, 8 in 2024-05-08T06:02:44.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:44 smithi192 ceph-mon[34664]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T06:02:44.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:44 smithi192 ceph-mon[34664]: osdmap e51: 8 total, 8 up, 8 in 2024-05-08T06:02:44.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:44 smithi192 ceph-mon[34664]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:02:44.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:44 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T06:02:44.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:44 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T06:02:44.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:44 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:02:44.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:44 smithi123 ceph-mon[29787]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T06:02:44.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:44 smithi123 ceph-mon[29787]: osdmap e51: 8 total, 8 up, 8 in 2024-05-08T06:02:44.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:44 smithi123 ceph-mon[29787]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:02:44.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:44 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T06:02:44.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:44 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T06:02:44.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:44 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:02:45.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:44 smithi163 ceph-mon[34520]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T06:02:45.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:44 smithi163 ceph-mon[34520]: osdmap e51: 8 total, 8 up, 8 in 2024-05-08T06:02:45.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:44 smithi163 ceph-mon[34520]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:02:45.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:44 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T06:02:45.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:44 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T06:02:45.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:44 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:02:46.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:46 smithi123 ceph-mon[29787]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T06:02:46.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:46 smithi123 ceph-mon[29787]: from='client.? 172.21.15.192:0/1507575006' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "122a281e-18b8-4728-97db-458e7432b2b2"}]: dispatch 2024-05-08T06:02:46.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:46 smithi123 ceph-mon[29787]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "122a281e-18b8-4728-97db-458e7432b2b2"}]: dispatch 2024-05-08T06:02:46.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:46 smithi123 ceph-mon[29787]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "122a281e-18b8-4728-97db-458e7432b2b2"}]': finished 2024-05-08T06:02:46.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:46 smithi123 ceph-mon[29787]: osdmap e52: 9 total, 8 up, 9 in 2024-05-08T06:02:46.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:46 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T06:02:47.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:46 smithi163 ceph-mon[34520]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T06:02:47.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:46 smithi163 ceph-mon[34520]: from='client.? 172.21.15.192:0/1507575006' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "122a281e-18b8-4728-97db-458e7432b2b2"}]: dispatch 2024-05-08T06:02:47.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:46 smithi163 ceph-mon[34520]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "122a281e-18b8-4728-97db-458e7432b2b2"}]: dispatch 2024-05-08T06:02:47.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:46 smithi163 ceph-mon[34520]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "122a281e-18b8-4728-97db-458e7432b2b2"}]': finished 2024-05-08T06:02:47.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:46 smithi163 ceph-mon[34520]: osdmap e52: 9 total, 8 up, 9 in 2024-05-08T06:02:47.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:46 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T06:02:47.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:46 smithi192 ceph-mon[34664]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T06:02:47.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:46 smithi192 ceph-mon[34664]: from='client.? 172.21.15.192:0/1507575006' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "122a281e-18b8-4728-97db-458e7432b2b2"}]: dispatch 2024-05-08T06:02:47.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:46 smithi192 ceph-mon[34664]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "122a281e-18b8-4728-97db-458e7432b2b2"}]: dispatch 2024-05-08T06:02:47.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:46 smithi192 ceph-mon[34664]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "122a281e-18b8-4728-97db-458e7432b2b2"}]': finished 2024-05-08T06:02:47.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:46 smithi192 ceph-mon[34664]: osdmap e52: 9 total, 8 up, 9 in 2024-05-08T06:02:47.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:46 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T06:02:47.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:47 smithi123 ceph-mon[29787]: from='client.? 172.21.15.192:0/3141774462' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T06:02:48.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:47 smithi163 ceph-mon[34520]: from='client.? 172.21.15.192:0/3141774462' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T06:02:48.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:47 smithi192 ceph-mon[34664]: from='client.? 172.21.15.192:0/3141774462' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T06:02:48.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:48 smithi123 ceph-mon[29787]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T06:02:49.076 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:48 smithi192 ceph-mon[34664]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T06:02:49.077 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:48 smithi163 ceph-mon[34520]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T06:02:50.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:50 smithi192 ceph-mon[34664]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-05-08T06:02:50.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:50 smithi123 ceph-mon[29787]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-05-08T06:02:51.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:50 smithi163 ceph-mon[34520]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-05-08T06:02:52.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:52 smithi192 ceph-mon[34664]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T06:02:52.840 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:52 smithi163 ceph-mon[34520]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T06:02:52.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:52 smithi123 ceph-mon[29787]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T06:02:53.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:53 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-08T06:02:53.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:53 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:02:53.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:53 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-08T06:02:53.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:53 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:02:54.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:53 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-08T06:02:54.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:53 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:02:54.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:54 smithi192 ceph-mon[34664]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-08T06:02:54.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:54 smithi192 ceph-mon[34664]: Deploying daemon osd.8 on smithi192 2024-05-08T06:02:54.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:54 smithi123 ceph-mon[29787]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-08T06:02:54.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:54 smithi123 ceph-mon[29787]: Deploying daemon osd.8 on smithi192 2024-05-08T06:02:55.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:54 smithi163 ceph-mon[34520]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-08T06:02:55.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:54 smithi163 ceph-mon[34520]: Deploying daemon osd.8 on smithi192 2024-05-08T06:02:56.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:56 smithi192 ceph-mon[34664]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T06:02:56.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:56 smithi123 ceph-mon[29787]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T06:02:57.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:56 smithi163 ceph-mon[34520]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T06:02:58.892 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:58 smithi192 ceph-mon[34664]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-05-08T06:02:58.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:58 smithi123 ceph-mon[29787]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-05-08T06:02:59.042 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:58 smithi163 ceph-mon[34520]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-05-08T06:02:59.913 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:59 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:02:59.913 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:59 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:59.913 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:02:59 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:59.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:59 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:02:59.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:59 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:02:59.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:02:59 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:00.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:59 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:03:00.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:59 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:00.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:02:59 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:00.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:00 smithi192 ceph-mon[34664]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-08T06:03:00.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:00 smithi123 ceph-mon[29787]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-08T06:03:01.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:00 smithi163 ceph-mon[34520]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-08T06:03:02.898 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:03:02.898+0000 7fca927fc700 1 -- 172.21.15.192:0/3177733286 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fca9c0638e0 con 0x7fca880b2900 2024-05-08T06:03:02.898 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 8 on host 'smithi192' 2024-05-08T06:03:02.899 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:02 smithi192 ceph-mon[34664]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T06:03:02.899 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:02 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:02.899 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:02 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:02.899 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:02 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:03:02.899 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:02 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:03:02.899 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:02 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:02.900 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:03:02.900+0000 7fcaa32ac700 1 -- 172.21.15.192:0/3177733286 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fca880b2900 msgr2=0x7fca880b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:02.900 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:03:02.900+0000 7fcaa32ac700 1 --2- 172.21.15.192:0/3177733286 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fca880b2900 0x7fca880b4dc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00b5c0 tx=0x7fca8c003860 comp rx=0 tx=0).stop 2024-05-08T06:03:02.900 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:03:02.900+0000 7fcaa32ac700 1 -- 172.21.15.192:0/3177733286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca9c103fb0 msgr2=0x7fca9c199680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:02.901 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:03:02.900+0000 7fcaa32ac700 1 --2- 172.21.15.192:0/3177733286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca9c103fb0 0x7fca9c199680 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fca9800b560 tx=0x7fca9800b920 comp rx=0 tx=0).stop 2024-05-08T06:03:02.901 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:03:02.900+0000 7fcaa32ac700 1 -- 172.21.15.192:0/3177733286 shutdown_connections 2024-05-08T06:03:02.901 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:03:02.900+0000 7fcaa32ac700 1 --2- 172.21.15.192:0/3177733286 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fca880b2900 0x7fca880b4dc0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:02.901 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:03:02.900+0000 7fcaa32ac700 1 --2- 172.21.15.192:0/3177733286 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca9c109210 0x7fca9c1a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:02.901 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:03:02.900+0000 7fcaa32ac700 1 --2- 172.21.15.192:0/3177733286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca9c103fb0 0x7fca9c199680 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:02.901 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:03:02.900+0000 7fcaa32ac700 1 --2- 172.21.15.192:0/3177733286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca9c1068e0 0x7fca9c199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:02.901 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:03:02.900+0000 7fcaa32ac700 1 -- 172.21.15.192:0/3177733286 >> 172.21.15.192:0/3177733286 conn(0x7fca9c0fd9c0 msgr2=0x7fca9c1049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:02.901 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:03:02.901+0000 7fcaa32ac700 1 -- 172.21.15.192:0/3177733286 shutdown_connections 2024-05-08T06:03:02.901 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:03:02.901+0000 7fcaa32ac700 1 -- 172.21.15.192:0/3177733286 wait complete. 2024-05-08T06:03:02.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:02 smithi123 ceph-mon[29787]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T06:03:02.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:02 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:02.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:02 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:02.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:02 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:03:02.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:02 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:03:02.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:02 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:03.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:02 smithi163 ceph-mon[34520]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T06:03:03.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:02 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:03.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:02 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:03.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:02 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:03:03.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:02 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:03:03.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:02 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:03.502 DEBUG:teuthology.orchestra.run.smithi192:osd.8> sudo journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@osd.8.service 2024-05-08T06:03:03.504 INFO:tasks.cephadm:Deploying osd.9 on smithi123 with /dev/vg_nvme/lv_1... 2024-05-08T06:03:03.504 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-08T06:03:03.795 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:03 smithi192 ceph-mon[34664]: from='osd.8 [v2:172.21.15.192:6816/3669753689,v1:172.21.15.192:6817/3669753689]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T06:03:03.795 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:03 smithi192 ceph-mon[34664]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T06:03:03.795 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:03 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:03.795 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:03 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:03.796 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:03 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:03:03.856 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:03 smithi123 ceph-mon[29787]: from='osd.8 [v2:172.21.15.192:6816/3669753689,v1:172.21.15.192:6817/3669753689]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T06:03:03.856 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:03 smithi123 ceph-mon[29787]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T06:03:03.856 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:03 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:03.856 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:03 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:03.857 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:03 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:03:03.934 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:03:04.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:03 smithi163 ceph-mon[34520]: from='osd.8 [v2:172.21.15.192:6816/3669753689,v1:172.21.15.192:6817/3669753689]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T06:03:04.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:03 smithi163 ceph-mon[34520]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T06:03:04.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:03 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:04.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:03 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:04.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:03 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:03:04.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:04 smithi123 ceph-mon[29787]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T06:03:04.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:04 smithi123 ceph-mon[29787]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-08T06:03:04.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:04 smithi123 ceph-mon[29787]: from='osd.8 [v2:172.21.15.192:6816/3669753689,v1:172.21.15.192:6817/3669753689]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-08T06:03:04.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:04 smithi123 ceph-mon[29787]: osdmap e53: 9 total, 8 up, 9 in 2024-05-08T06:03:04.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:04 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T06:03:04.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:04 smithi123 ceph-mon[29787]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-08T06:03:04.957 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:04 smithi192 ceph-mon[34664]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T06:03:04.957 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:04 smithi192 ceph-mon[34664]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-08T06:03:04.958 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:04 smithi192 ceph-mon[34664]: from='osd.8 [v2:172.21.15.192:6816/3669753689,v1:172.21.15.192:6817/3669753689]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-08T06:03:04.958 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:04 smithi192 ceph-mon[34664]: osdmap e53: 9 total, 8 up, 9 in 2024-05-08T06:03:04.958 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:04 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T06:03:04.958 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:04 smithi192 ceph-mon[34664]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-08T06:03:05.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:04 smithi163 ceph-mon[34520]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T06:03:05.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:04 smithi163 ceph-mon[34520]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-08T06:03:05.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:04 smithi163 ceph-mon[34520]: from='osd.8 [v2:172.21.15.192:6816/3669753689,v1:172.21.15.192:6817/3669753689]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-08T06:03:05.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:04 smithi163 ceph-mon[34520]: osdmap e53: 9 total, 8 up, 9 in 2024-05-08T06:03:05.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:04 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T06:03:05.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:04 smithi163 ceph-mon[34520]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-08T06:03:05.301 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:03:05 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:03:05.091+0000 7fd028fd2700 -1 osd.8 0 waiting for initial osdmap 2024-05-08T06:03:05.301 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:03:05 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:03:05.103+0000 7fd020dbd700 -1 osd.8 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T06:03:05.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:05 smithi123 ceph-mon[29787]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-05-08T06:03:05.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:05 smithi123 ceph-mon[29787]: osdmap e54: 9 total, 8 up, 9 in 2024-05-08T06:03:05.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:05 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T06:03:05.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:05 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T06:03:05.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:05 smithi123 ceph-mon[29787]: from='osd.8 ' entity='osd.8' 2024-05-08T06:03:06.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:05 smithi163 ceph-mon[34520]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-05-08T06:03:06.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:05 smithi163 ceph-mon[34520]: osdmap e54: 9 total, 8 up, 9 in 2024-05-08T06:03:06.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:05 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T06:03:06.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:05 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T06:03:06.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:05 smithi163 ceph-mon[34520]: from='osd.8 ' entity='osd.8' 2024-05-08T06:03:06.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:05 smithi192 ceph-mon[34664]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-05-08T06:03:06.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:05 smithi192 ceph-mon[34664]: osdmap e54: 9 total, 8 up, 9 in 2024-05-08T06:03:06.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:05 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T06:03:06.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:05 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T06:03:06.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:05 smithi192 ceph-mon[34664]: from='osd.8 ' entity='osd.8' 2024-05-08T06:03:06.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:06 smithi192 ceph-mon[34664]: purged_snaps scrub starts 2024-05-08T06:03:06.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:06 smithi192 ceph-mon[34664]: purged_snaps scrub ok 2024-05-08T06:03:06.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:06 smithi192 ceph-mon[34664]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T06:03:06.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:06 smithi192 ceph-mon[34664]: osd.8 [v2:172.21.15.192:6816/3669753689,v1:172.21.15.192:6817/3669753689] boot 2024-05-08T06:03:06.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:06 smithi192 ceph-mon[34664]: osdmap e55: 9 total, 9 up, 9 in 2024-05-08T06:03:06.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:06 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T06:03:06.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:06 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:06.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:06 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:06.803 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:06 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:03:06.803 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:06 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:03:06.803 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:06 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:03:06.803 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:06 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:06.803 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:06 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:03:06.803 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:06 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:03:06.803 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:06 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:06.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:06 smithi123 ceph-mon[29787]: purged_snaps scrub starts 2024-05-08T06:03:06.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:06 smithi123 ceph-mon[29787]: purged_snaps scrub ok 2024-05-08T06:03:06.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:06 smithi123 ceph-mon[29787]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T06:03:06.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:06 smithi123 ceph-mon[29787]: osd.8 [v2:172.21.15.192:6816/3669753689,v1:172.21.15.192:6817/3669753689] boot 2024-05-08T06:03:06.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:06 smithi123 ceph-mon[29787]: osdmap e55: 9 total, 9 up, 9 in 2024-05-08T06:03:06.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:06 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T06:03:06.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:06 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:06.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:06 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:06.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:06 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:03:06.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:06 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:03:06.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:06 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:03:06.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:06 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:06.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:06 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:03:06.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:06 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:03:06.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:06 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:07.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:06 smithi163 ceph-mon[34520]: purged_snaps scrub starts 2024-05-08T06:03:07.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:06 smithi163 ceph-mon[34520]: purged_snaps scrub ok 2024-05-08T06:03:07.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:06 smithi163 ceph-mon[34520]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T06:03:07.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:06 smithi163 ceph-mon[34520]: osd.8 [v2:172.21.15.192:6816/3669753689,v1:172.21.15.192:6817/3669753689] boot 2024-05-08T06:03:07.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:06 smithi163 ceph-mon[34520]: osdmap e55: 9 total, 9 up, 9 in 2024-05-08T06:03:07.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:06 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T06:03:07.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:06 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:07.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:06 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:07.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:06 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:03:07.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:06 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:03:07.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:06 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:03:07.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:06 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:07.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:06 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:03:07.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:06 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:03:07.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:06 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:07.282 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-08T06:03:07.282 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-08T06:03:07.283 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2024-05-08T06:03:07.283 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2024-05-08T06:03:07.283 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0371357 s, 282 MB/s 2024-05-08T06:03:07.283 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2024-05-08T06:03:07.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:07 smithi123 ceph-mon[29787]: Detected new or changed devices on smithi192 2024-05-08T06:03:07.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:07 smithi123 ceph-mon[29787]: Adjusting osd_memory_target on smithi192 to 5696M 2024-05-08T06:03:07.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:07 smithi123 ceph-mon[29787]: osdmap e56: 9 total, 9 up, 9 in 2024-05-08T06:03:08.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:07 smithi163 ceph-mon[34520]: Detected new or changed devices on smithi192 2024-05-08T06:03:08.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:07 smithi163 ceph-mon[34520]: Adjusting osd_memory_target on smithi192 to 5696M 2024-05-08T06:03:08.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:07 smithi163 ceph-mon[34520]: osdmap e56: 9 total, 9 up, 9 in 2024-05-08T06:03:08.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:07 smithi192 ceph-mon[34664]: Detected new or changed devices on smithi192 2024-05-08T06:03:08.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:07 smithi192 ceph-mon[34664]: Adjusting osd_memory_target on smithi192 to 5696M 2024-05-08T06:03:08.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:07 smithi192 ceph-mon[34664]: osdmap e56: 9 total, 9 up, 9 in 2024-05-08T06:03:08.264 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi123:vg_nvme/lv_1 2024-05-08T06:03:08.695 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:03:08.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:08 smithi123 ceph-mon[29787]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:08.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:08 smithi123 ceph-mon[29787]: osdmap e57: 9 total, 9 up, 9 in 2024-05-08T06:03:08.969 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:08 smithi192 ceph-mon[34664]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:08.969 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:08 smithi192 ceph-mon[34664]: osdmap e57: 9 total, 9 up, 9 in 2024-05-08T06:03:09.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:08 smithi163 ceph-mon[34520]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:09.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:08 smithi163 ceph-mon[34520]: osdmap e57: 9 total, 9 up, 9 in 2024-05-08T06:03:10.878 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:10 smithi123 ceph-mon[29787]: pgmap v216: 1 pgs: 1 peering; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:11.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:10 smithi163 ceph-mon[34520]: pgmap v216: 1 pgs: 1 peering; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:11.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:10 smithi192 ceph-mon[34664]: pgmap v216: 1 pgs: 1 peering; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:11.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.251+0000 7f3aebe00700 1 -- 172.21.15.123:0/2403244063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aec10b320 msgr2=0x7f3aec10d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:11.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.251+0000 7f3aebe00700 1 --2- 172.21.15.123:0/2403244063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aec10b320 0x7f3aec10d790 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3ae400b0a0 tx=0x7f3ae400b3b0 comp rx=0 tx=0).stop 2024-05-08T06:03:11.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.251+0000 7f3aebe00700 1 -- 172.21.15.123:0/2403244063 shutdown_connections 2024-05-08T06:03:11.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.251+0000 7f3aebe00700 1 --2- 172.21.15.123:0/2403244063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aec10b320 0x7f3aec10d790 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:11.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.251+0000 7f3aebe00700 1 --2- 172.21.15.123:0/2403244063 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3aec077450 0x7f3aec0758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:11.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.251+0000 7f3aebe00700 1 --2- 172.21.15.123:0/2403244063 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3aec075e80 0x7f3aec076300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:11.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.251+0000 7f3aebe00700 1 -- 172.21.15.123:0/2403244063 >> 172.21.15.123:0/2403244063 conn(0x7f3aec0fe9e0 msgr2=0x7f3aec100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:11.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.251+0000 7f3aebe00700 1 -- 172.21.15.123:0/2403244063 shutdown_connections 2024-05-08T06:03:11.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.251+0000 7f3aebe00700 1 -- 172.21.15.123:0/2403244063 wait complete. 2024-05-08T06:03:11.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.252+0000 7f3aebe00700 1 Processor -- start 2024-05-08T06:03:11.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.253+0000 7f3aebe00700 1 -- start start 2024-05-08T06:03:11.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.253+0000 7f3aebe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aec075e80 0x7f3aec19b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:11.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.253+0000 7f3aebe00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3aec077450 0x7f3aec19bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:11.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.253+0000 7f3aebe00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3aec10b320 0x7f3aec1a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:11.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.253+0000 7f3aebe00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3aec1110f0 con 0x7f3aec075e80 2024-05-08T06:03:11.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.253+0000 7f3aebe00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f3aec110f70 con 0x7f3aec10b320 2024-05-08T06:03:11.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.253+0000 7f3aebe00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3aec111270 con 0x7f3aec077450 2024-05-08T06:03:11.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.253+0000 7f3aeaa00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aec075e80 0x7f3aec19b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:11.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.253+0000 7f3aea000700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3aec077450 0x7f3aec19bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:11.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.253+0000 7f3aea000700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3aec077450 0x7f3aec19bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:57282/0 (socket says 172.21.15.123:57282) 2024-05-08T06:03:11.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.253+0000 7f3aeaa00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aec075e80 0x7f3aec19b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51566/0 (socket says 172.21.15.123:51566) 2024-05-08T06:03:11.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.253+0000 7f3aea000700 1 -- 172.21.15.123:0/5978166 learned_addr learned my addr 172.21.15.123:0/5978166 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:03:11.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.254+0000 7f3aeb400700 1 --2- 172.21.15.123:0/5978166 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3aec10b320 0x7f3aec1a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:11.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.254+0000 7f3aea000700 1 -- 172.21.15.123:0/5978166 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3aec10b320 msgr2=0x7f3aec1a2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:11.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.254+0000 7f3aea000700 1 --2- 172.21.15.123:0/5978166 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3aec10b320 0x7f3aec1a2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:11.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.254+0000 7f3aea000700 1 -- 172.21.15.123:0/5978166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aec075e80 msgr2=0x7f3aec19b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:11.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.254+0000 7f3aea000700 1 --2- 172.21.15.123:0/5978166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aec075e80 0x7f3aec19b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:11.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.254+0000 7f3aea000700 1 -- 172.21.15.123:0/5978166 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aec1a3310 con 0x7f3aec077450 2024-05-08T06:03:11.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.254+0000 7f3aea000700 1 --2- 172.21.15.123:0/5978166 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3aec077450 0x7f3aec19bbf0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3ad800e9f0 tx=0x7f3ad800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:11.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.255+0000 7f3ae3400700 1 -- 172.21.15.123:0/5978166 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad800cbd0 con 0x7f3aec077450 2024-05-08T06:03:11.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.255+0000 7f3aebe00700 1 -- 172.21.15.123:0/5978166 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aec1a3600 con 0x7f3aec077450 2024-05-08T06:03:11.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.255+0000 7f3ae3400700 1 -- 172.21.15.123:0/5978166 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ad800ee90 con 0x7f3aec077450 2024-05-08T06:03:11.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.255+0000 7f3ae3400700 1 -- 172.21.15.123:0/5978166 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8005590 con 0x7f3aec077450 2024-05-08T06:03:11.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.255+0000 7f3aebe00700 1 -- 172.21.15.123:0/5978166 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aec1a3b40 con 0x7f3aec077450 2024-05-08T06:03:11.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.256+0000 7f3aebe00700 1 -- 172.21.15.123:0/5978166 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3aec06a190 con 0x7f3aec077450 2024-05-08T06:03:11.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.260+0000 7f3ae3400700 1 -- 172.21.15.123:0/5978166 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f3ad8047960 con 0x7f3aec077450 2024-05-08T06:03:11.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.262+0000 7f3ae3400700 1 --2- 172.21.15.123:0/5978166 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f3ad00b2900 0x7f3ad00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:11.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.262+0000 7f3ae3400700 1 -- 172.21.15.123:0/5978166 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5829+0+0 (secure 0 0 0) 0x7f3ad80e1aa0 con 0x7f3aec077450 2024-05-08T06:03:11.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.262+0000 7f3aeaa00700 1 --2- 172.21.15.123:0/5978166 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f3ad00b2900 0x7f3ad00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:11.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.262+0000 7f3ae3400700 1 -- 172.21.15.123:0/5978166 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3ad80aa0d0 con 0x7f3aec077450 2024-05-08T06:03:11.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.262+0000 7f3aeaa00700 1 --2- 172.21.15.123:0/5978166 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f3ad00b2900 0x7f3ad00b4dc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3adc007be0 tx=0x7f3adc0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:11.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:11.511+0000 7f3aebe00700 1 -- 172.21.15.123:0/5978166 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f3aec0638e0 con 0x7f3ad00b2900 2024-05-08T06:03:11.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:11 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T06:03:11.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:11 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T06:03:11.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:11 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:03:12.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:11 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T06:03:12.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:11 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T06:03:12.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:11 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:03:12.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:11 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T06:03:12.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:11 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T06:03:12.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:11 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:03:12.726 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:12 smithi123 ceph-mon[29787]: pgmap v217: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:12.726 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:12 smithi123 ceph-mon[29787]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:03:13.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:12 smithi163 ceph-mon[34520]: pgmap v217: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:13.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:12 smithi163 ceph-mon[34520]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:03:13.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:12 smithi192 ceph-mon[34664]: pgmap v217: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:13.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:12 smithi192 ceph-mon[34664]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:03:14.727 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:14 smithi123 ceph-mon[29787]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-05-08T06:03:15.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:14 smithi163 ceph-mon[34520]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-05-08T06:03:15.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:14 smithi192 ceph-mon[34664]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-05-08T06:03:16.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:16 smithi123 ceph-mon[29787]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-08T06:03:16.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:16 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/2365247004' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9fa022a9-b873-4e07-806c-641a706a150f"}]: dispatch 2024-05-08T06:03:16.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:16 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/2365247004' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9fa022a9-b873-4e07-806c-641a706a150f"}]': finished 2024-05-08T06:03:16.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:16 smithi123 ceph-mon[29787]: osdmap e58: 10 total, 9 up, 10 in 2024-05-08T06:03:16.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:16 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T06:03:17.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:16 smithi163 ceph-mon[34520]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-08T06:03:17.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:16 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/2365247004' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9fa022a9-b873-4e07-806c-641a706a150f"}]: dispatch 2024-05-08T06:03:17.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:16 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/2365247004' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9fa022a9-b873-4e07-806c-641a706a150f"}]': finished 2024-05-08T06:03:17.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:16 smithi163 ceph-mon[34520]: osdmap e58: 10 total, 9 up, 10 in 2024-05-08T06:03:17.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:16 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T06:03:17.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:16 smithi192 ceph-mon[34664]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-08T06:03:17.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:16 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/2365247004' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9fa022a9-b873-4e07-806c-641a706a150f"}]: dispatch 2024-05-08T06:03:17.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:16 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/2365247004' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9fa022a9-b873-4e07-806c-641a706a150f"}]': finished 2024-05-08T06:03:17.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:16 smithi192 ceph-mon[34664]: osdmap e58: 10 total, 9 up, 10 in 2024-05-08T06:03:17.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:16 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T06:03:18.011 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:17 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/2237631699' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T06:03:18.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:17 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/2237631699' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T06:03:18.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:17 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/2237631699' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T06:03:19.021 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:18 smithi192 ceph-mon[34664]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-08T06:03:19.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:18 smithi163 ceph-mon[34520]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-08T06:03:19.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:18 smithi123 ceph-mon[29787]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-08T06:03:20.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:20 smithi123 ceph-mon[29787]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T06:03:21.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:20 smithi163 ceph-mon[34520]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T06:03:21.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:20 smithi192 ceph-mon[34664]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T06:03:22.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:22 smithi123 ceph-mon[29787]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T06:03:23.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:22 smithi163 ceph-mon[34520]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T06:03:23.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:22 smithi192 ceph-mon[34664]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T06:03:24.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:24 smithi123 ceph-mon[29787]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:25.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:24 smithi163 ceph-mon[34520]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:25.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:24 smithi192 ceph-mon[34664]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:25.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:25 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-08T06:03:25.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:25 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:03:26.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:25 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-08T06:03:26.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:25 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:03:26.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:25 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-08T06:03:26.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:25 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:03:26.954 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:26 smithi123 ceph-mon[29787]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:26.954 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:26 smithi123 ceph-mon[29787]: Deploying daemon osd.9 on smithi123 2024-05-08T06:03:27.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:26 smithi163 ceph-mon[34520]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:27.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:26 smithi163 ceph-mon[34520]: Deploying daemon osd.9 on smithi123 2024-05-08T06:03:27.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:26 smithi192 ceph-mon[34664]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:27.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:26 smithi192 ceph-mon[34664]: Deploying daemon osd.9 on smithi123 2024-05-08T06:03:28.828 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:28 smithi123 ceph-mon[29787]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:29.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:28 smithi163 ceph-mon[34520]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:29.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:28 smithi192 ceph-mon[34664]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:31.027 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:30 smithi123 ceph-mon[29787]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:31.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:30 smithi163 ceph-mon[34520]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:31.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:30 smithi192 ceph-mon[34664]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:32.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:32 smithi123 ceph-mon[29787]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:33.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:32 smithi163 ceph-mon[34520]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:33.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:32 smithi192 ceph-mon[34664]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:35.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:34 smithi163 ceph-mon[34520]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:35.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:34 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:35.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:34 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:35.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:34 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:03:35.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:34 smithi192 ceph-mon[34664]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:35.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:34 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:35.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:34 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:35.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:34 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:03:35.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:34 smithi123 ceph-mon[29787]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:35.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:34 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:35.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:34 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:35.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:34 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:03:36.952 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:36 smithi123 ceph-mon[29787]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:37.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:36 smithi163 ceph-mon[34520]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:37.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:36 smithi192 ceph-mon[34664]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:39.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:38 smithi163 ceph-mon[34520]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:39.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:38 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:39.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:38 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:39.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:38 smithi163 ceph-mon[34520]: from='osd.9 [v2:172.21.15.123:6826/3104436473,v1:172.21.15.123:6827/3104436473]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T06:03:39.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:38 smithi192 ceph-mon[34664]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:39.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:38 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:39.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:38 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:39.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:38 smithi192 ceph-mon[34664]: from='osd.9 [v2:172.21.15.123:6826/3104436473,v1:172.21.15.123:6827/3104436473]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T06:03:39.154 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:38 smithi123 ceph-mon[29787]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:39.155 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:38 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:39.155 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:38 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:39.155 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:38 smithi123 ceph-mon[29787]: from='osd.9 [v2:172.21.15.123:6826/3104436473,v1:172.21.15.123:6827/3104436473]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T06:03:39.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:39.660+0000 7f3ae3400700 1 -- 172.21.15.123:0/5978166 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3aec0638e0 con 0x7f3ad00b2900 2024-05-08T06:03:39.661 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 9 on host 'smithi123' 2024-05-08T06:03:39.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:39.664+0000 7f3aebe00700 1 -- 172.21.15.123:0/5978166 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f3ad00b2900 msgr2=0x7f3ad00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:39.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:39.664+0000 7f3aebe00700 1 --2- 172.21.15.123:0/5978166 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f3ad00b2900 0x7f3ad00b4dc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3adc007be0 tx=0x7f3adc0073d0 comp rx=0 tx=0).stop 2024-05-08T06:03:39.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:39.664+0000 7f3aebe00700 1 -- 172.21.15.123:0/5978166 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3aec077450 msgr2=0x7f3aec19bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:39.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:39.664+0000 7f3aebe00700 1 --2- 172.21.15.123:0/5978166 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3aec077450 0x7f3aec19bbf0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3ad800e9f0 tx=0x7f3ad800ed00 comp rx=0 tx=0).stop 2024-05-08T06:03:39.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:39.664+0000 7f3aebe00700 1 -- 172.21.15.123:0/5978166 shutdown_connections 2024-05-08T06:03:39.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:39.664+0000 7f3aebe00700 1 --2- 172.21.15.123:0/5978166 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f3ad00b2900 0x7f3ad00b4dc0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:39.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:39.664+0000 7f3aebe00700 1 --2- 172.21.15.123:0/5978166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aec075e80 0x7f3aec19b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:39.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:39.664+0000 7f3aebe00700 1 --2- 172.21.15.123:0/5978166 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3aec10b320 0x7f3aec1a2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:39.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:39.664+0000 7f3aebe00700 1 --2- 172.21.15.123:0/5978166 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3aec077450 0x7f3aec19bbf0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:39.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:39.664+0000 7f3aebe00700 1 -- 172.21.15.123:0/5978166 >> 172.21.15.123:0/5978166 conn(0x7f3aec0fe9e0 msgr2=0x7f3aec107270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:39.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:39.664+0000 7f3aebe00700 1 -- 172.21.15.123:0/5978166 shutdown_connections 2024-05-08T06:03:39.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:03:39.664+0000 7f3aebe00700 1 -- 172.21.15.123:0/5978166 wait complete. 2024-05-08T06:03:39.907 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:39 smithi123 ceph-mon[29787]: from='osd.9 [v2:172.21.15.123:6826/3104436473,v1:172.21.15.123:6827/3104436473]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-08T06:03:39.907 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:39 smithi123 ceph-mon[29787]: osdmap e59: 10 total, 9 up, 10 in 2024-05-08T06:03:39.907 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:39 smithi123 ceph-mon[29787]: from='osd.9 [v2:172.21.15.123:6826/3104436473,v1:172.21.15.123:6827/3104436473]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-08T06:03:39.907 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:39 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T06:03:39.907 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:39 smithi123 ceph-mon[29787]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:39.907 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:39 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:03:39.908 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:39 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:03:39.908 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:39 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:39.908 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:39 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:39.908 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:39 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:39.908 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:39 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:03:40.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:39 smithi163 ceph-mon[34520]: from='osd.9 [v2:172.21.15.123:6826/3104436473,v1:172.21.15.123:6827/3104436473]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-08T06:03:40.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:39 smithi163 ceph-mon[34520]: osdmap e59: 10 total, 9 up, 10 in 2024-05-08T06:03:40.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:39 smithi163 ceph-mon[34520]: from='osd.9 [v2:172.21.15.123:6826/3104436473,v1:172.21.15.123:6827/3104436473]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-08T06:03:40.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:39 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T06:03:40.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:39 smithi163 ceph-mon[34520]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:40.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:39 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:03:40.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:39 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:03:40.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:39 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:40.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:39 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:40.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:39 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:40.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:39 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:03:40.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:39 smithi192 ceph-mon[34664]: from='osd.9 [v2:172.21.15.123:6826/3104436473,v1:172.21.15.123:6827/3104436473]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-08T06:03:40.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:39 smithi192 ceph-mon[34664]: osdmap e59: 10 total, 9 up, 10 in 2024-05-08T06:03:40.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:39 smithi192 ceph-mon[34664]: from='osd.9 [v2:172.21.15.123:6826/3104436473,v1:172.21.15.123:6827/3104436473]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-08T06:03:40.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:39 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T06:03:40.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:39 smithi192 ceph-mon[34664]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:40.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:39 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:03:40.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:39 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:03:40.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:39 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:40.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:39 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:40.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:39 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:40.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:39 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:03:40.691 DEBUG:teuthology.orchestra.run.smithi123:osd.9> sudo journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@osd.9.service 2024-05-08T06:03:40.696 INFO:tasks.cephadm:Deploying osd.10 on smithi163 with /dev/vg_nvme/lv_1... 2024-05-08T06:03:40.696 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-08T06:03:40.942 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T06:03:41.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:40 smithi163 ceph-mon[34520]: from='osd.9 [v2:172.21.15.123:6826/3104436473,v1:172.21.15.123:6827/3104436473]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-05-08T06:03:41.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:40 smithi163 ceph-mon[34520]: osdmap e60: 10 total, 9 up, 10 in 2024-05-08T06:03:41.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:40 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T06:03:41.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:40 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T06:03:41.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:40 smithi163 ceph-mon[34520]: from='osd.9 [v2:172.21.15.123:6826/3104436473,v1:172.21.15.123:6827/3104436473]' entity='osd.9' 2024-05-08T06:03:41.165 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:40 smithi123 ceph-mon[29787]: from='osd.9 [v2:172.21.15.123:6826/3104436473,v1:172.21.15.123:6827/3104436473]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-05-08T06:03:41.165 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:40 smithi123 ceph-mon[29787]: osdmap e60: 10 total, 9 up, 10 in 2024-05-08T06:03:41.165 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:40 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T06:03:41.166 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:40 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T06:03:41.166 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:40 smithi123 ceph-mon[29787]: from='osd.9 [v2:172.21.15.123:6826/3104436473,v1:172.21.15.123:6827/3104436473]' entity='osd.9' 2024-05-08T06:03:41.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:40 smithi192 ceph-mon[34664]: from='osd.9 [v2:172.21.15.123:6826/3104436473,v1:172.21.15.123:6827/3104436473]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-05-08T06:03:41.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:40 smithi192 ceph-mon[34664]: osdmap e60: 10 total, 9 up, 10 in 2024-05-08T06:03:41.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:40 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T06:03:41.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:40 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T06:03:41.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:40 smithi192 ceph-mon[34664]: from='osd.9 [v2:172.21.15.123:6826/3104436473,v1:172.21.15.123:6827/3104436473]' entity='osd.9' 2024-05-08T06:03:42.204 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:41 smithi163 ceph-mon[34520]: purged_snaps scrub starts 2024-05-08T06:03:42.205 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:41 smithi163 ceph-mon[34520]: purged_snaps scrub ok 2024-05-08T06:03:42.205 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:41 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T06:03:42.205 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:41 smithi163 ceph-mon[34520]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:42.205 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:41 smithi163 ceph-mon[34520]: osd.9 [v2:172.21.15.123:6826/3104436473,v1:172.21.15.123:6827/3104436473] boot 2024-05-08T06:03:42.205 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:41 smithi163 ceph-mon[34520]: osdmap e61: 10 total, 10 up, 10 in 2024-05-08T06:03:42.205 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:41 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T06:03:42.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:41 smithi123 ceph-mon[29787]: purged_snaps scrub starts 2024-05-08T06:03:42.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:41 smithi123 ceph-mon[29787]: purged_snaps scrub ok 2024-05-08T06:03:42.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:41 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T06:03:42.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:41 smithi123 ceph-mon[29787]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:42.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:41 smithi123 ceph-mon[29787]: osd.9 [v2:172.21.15.123:6826/3104436473,v1:172.21.15.123:6827/3104436473] boot 2024-05-08T06:03:42.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:41 smithi123 ceph-mon[29787]: osdmap e61: 10 total, 10 up, 10 in 2024-05-08T06:03:42.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:41 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T06:03:42.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:41 smithi192 ceph-mon[34664]: purged_snaps scrub starts 2024-05-08T06:03:42.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:41 smithi192 ceph-mon[34664]: purged_snaps scrub ok 2024-05-08T06:03:42.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:41 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T06:03:42.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:41 smithi192 ceph-mon[34664]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T06:03:42.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:41 smithi192 ceph-mon[34664]: osd.9 [v2:172.21.15.123:6826/3104436473,v1:172.21.15.123:6827/3104436473] boot 2024-05-08T06:03:42.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:41 smithi192 ceph-mon[34664]: osdmap e61: 10 total, 10 up, 10 in 2024-05-08T06:03:42.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:41 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T06:03:42.950 INFO:teuthology.orchestra.run.smithi163.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-08T06:03:42.950 INFO:teuthology.orchestra.run.smithi163.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-08T06:03:42.950 INFO:teuthology.orchestra.run.smithi163.stderr: stderr: 10+0 records in 2024-05-08T06:03:42.950 INFO:teuthology.orchestra.run.smithi163.stderr:10+0 records out 2024-05-08T06:03:42.950 INFO:teuthology.orchestra.run.smithi163.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113875 s, 921 MB/s 2024-05-08T06:03:42.950 INFO:teuthology.orchestra.run.smithi163.stderr:--> Zapping successful for: 2024-05-08T06:03:43.565 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi163:vg_nvme/lv_1 2024-05-08T06:03:43.809 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T06:03:44.149 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:43 smithi163 ceph-mon[34520]: osdmap e62: 10 total, 10 up, 10 in 2024-05-08T06:03:44.149 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:43 smithi163 ceph-mon[34520]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:03:44.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:43 smithi123 ceph-mon[29787]: osdmap e62: 10 total, 10 up, 10 in 2024-05-08T06:03:44.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:43 smithi123 ceph-mon[29787]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:03:44.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:43 smithi192 ceph-mon[34664]: osdmap e62: 10 total, 10 up, 10 in 2024-05-08T06:03:44.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:43 smithi192 ceph-mon[34664]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:03:45.452 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.450+0000 7f54e8f57700 1 -- 172.21.15.163:0/1591192938 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54e4103500 msgr2=0x7f54e4103920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:45.453 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.450+0000 7f54e8f57700 1 --2- 172.21.15.163:0/1591192938 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54e4103500 0x7f54e4103920 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f54d400b0d0 tx=0x7f54d400b3e0 comp rx=0 tx=0).stop 2024-05-08T06:03:45.453 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.450+0000 7f54e8f57700 1 -- 172.21.15.163:0/1591192938 shutdown_connections 2024-05-08T06:03:45.453 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.450+0000 7f54e8f57700 1 --2- 172.21.15.163:0/1591192938 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54e41050a0 0x7f54e410b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:45.453 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.450+0000 7f54e8f57700 1 --2- 172.21.15.163:0/1591192938 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54e4103500 0x7f54e4103920 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:45.453 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.450+0000 7f54e8f57700 1 --2- 172.21.15.163:0/1591192938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54e4104700 0x7f54e4104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:45.453 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.450+0000 7f54e8f57700 1 -- 172.21.15.163:0/1591192938 >> 172.21.15.163:0/1591192938 conn(0x7f54e40feaa0 msgr2=0x7f54e4100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:45.453 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.450+0000 7f54e8f57700 1 -- 172.21.15.163:0/1591192938 shutdown_connections 2024-05-08T06:03:45.453 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.450+0000 7f54e8f57700 1 -- 172.21.15.163:0/1591192938 wait complete. 2024-05-08T06:03:45.453 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.451+0000 7f54e8f57700 1 Processor -- start 2024-05-08T06:03:45.453 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.451+0000 7f54e8f57700 1 -- start start 2024-05-08T06:03:45.454 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.451+0000 7f54e8f57700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54e4103500 0x7f54e4077240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:45.454 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.451+0000 7f54e8f57700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54e4104700 0x7f54e4075890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:45.454 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.451+0000 7f54e8f57700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54e41050a0 0x7f54e4075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:45.454 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.451+0000 7f54e8f57700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f54e410f020 con 0x7f54e41050a0 2024-05-08T06:03:45.454 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.451+0000 7f54e8f57700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f54e410eea0 con 0x7f54e4104700 2024-05-08T06:03:45.454 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.451+0000 7f54e8f57700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f54e410f1a0 con 0x7f54e4103500 2024-05-08T06:03:45.454 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.451+0000 7f54e1d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54e4104700 0x7f54e4075890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:45.454 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.451+0000 7f54e259c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54e4103500 0x7f54e4077240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:45.454 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.451+0000 7f54e1d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54e4104700 0x7f54e4075890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51128/0 (socket says 172.21.15.163:51128) 2024-05-08T06:03:45.454 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.451+0000 7f54e1d9b700 1 -- 172.21.15.163:0/698223314 learned_addr learned my addr 172.21.15.163:0/698223314 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-08T06:03:45.454 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.451+0000 7f54e1d9b700 1 -- 172.21.15.163:0/698223314 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54e4103500 msgr2=0x7f54e4077240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:45.454 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.451+0000 7f54e1d9b700 1 --2- 172.21.15.163:0/698223314 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54e4103500 0x7f54e4077240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:45.454 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.451+0000 7f54e1d9b700 1 -- 172.21.15.163:0/698223314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54e41050a0 msgr2=0x7f54e4075dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:45.455 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.451+0000 7f54e1d9b700 1 --2- 172.21.15.163:0/698223314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54e41050a0 0x7f54e4075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:45.455 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.451+0000 7f54e1d9b700 1 -- 172.21.15.163:0/698223314 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e4076680 con 0x7f54e4104700 2024-05-08T06:03:45.455 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.452+0000 7f54e1d9b700 1 --2- 172.21.15.163:0/698223314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54e4104700 0x7f54e4075890 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f54d800b530 tx=0x7f54d800b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:45.455 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.452+0000 7f54d37fe700 1 -- 172.21.15.163:0/698223314 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d8015020 con 0x7f54e4104700 2024-05-08T06:03:45.455 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.452+0000 7f54d37fe700 1 -- 172.21.15.163:0/698223314 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54d8011ce0 con 0x7f54e4104700 2024-05-08T06:03:45.455 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.452+0000 7f54e8f57700 1 -- 172.21.15.163:0/698223314 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54e41a38b0 con 0x7f54e4104700 2024-05-08T06:03:45.455 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.452+0000 7f54d37fe700 1 -- 172.21.15.163:0/698223314 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d8010e40 con 0x7f54e4104700 2024-05-08T06:03:45.455 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.452+0000 7f54e8f57700 1 -- 172.21.15.163:0/698223314 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54e41a3e40 con 0x7f54e4104700 2024-05-08T06:03:45.456 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.453+0000 7f54e8f57700 1 -- 172.21.15.163:0/698223314 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54e406a190 con 0x7f54e4104700 2024-05-08T06:03:45.456 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.454+0000 7f54d37fe700 1 -- 172.21.15.163:0/698223314 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f54d8018020 con 0x7f54e4104700 2024-05-08T06:03:45.456 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.455+0000 7f54d37fe700 1 --2- 172.21.15.163:0/698223314 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f54cc0b2880 0x7f54cc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:45.456 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.455+0000 7f54d37fe700 1 -- 172.21.15.163:0/698223314 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6261+0+0 (secure 0 0 0) 0x7f54d80e17a0 con 0x7f54e4104700 2024-05-08T06:03:45.456 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.455+0000 7f54e259c700 1 --2- 172.21.15.163:0/698223314 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f54cc0b2880 0x7f54cc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:45.457 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.456+0000 7f54e259c700 1 --2- 172.21.15.163:0/698223314 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f54cc0b2880 0x7f54cc0b4d40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f54d4009040 tx=0x7f54d400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:45.459 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.458+0000 7f54d37fe700 1 -- 172.21.15.163:0/698223314 <== mon.2 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f54d801f020 con 0x7f54e4104700 2024-05-08T06:03:45.670 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:03:45.669+0000 7f54e8f57700 1 -- 172.21.15.163:0/698223314 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f54e40638e0 con 0x7f54cc0b2880 2024-05-08T06:03:46.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:46 smithi123 ceph-mon[29787]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:03:46.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:46 smithi123 ceph-mon[29787]: Detected new or changed devices on smithi123 2024-05-08T06:03:46.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:46 smithi123 ceph-mon[29787]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:03:46.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:46 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:46.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:46 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T06:03:46.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:46 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:46.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:46 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T06:03:46.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:46 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:03:46.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:46 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:03:46.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:46 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:03:46.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:46 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:03:46.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:46 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:03:46.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:46 smithi123 ceph-mon[29787]: Adjusting osd_memory_target on smithi123 to 4272M 2024-05-08T06:03:46.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:46 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:46.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:46 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:03:46.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:46 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:03:46.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:46 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:47.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:46 smithi163 ceph-mon[34520]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:03:47.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:46 smithi163 ceph-mon[34520]: Detected new or changed devices on smithi123 2024-05-08T06:03:47.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:46 smithi163 ceph-mon[34520]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:03:47.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:46 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:47.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:46 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T06:03:47.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:46 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:47.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:46 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T06:03:47.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:46 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:03:47.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:46 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:03:47.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:46 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:03:47.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:46 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:03:47.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:46 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:03:47.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:46 smithi163 ceph-mon[34520]: Adjusting osd_memory_target on smithi123 to 4272M 2024-05-08T06:03:47.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:46 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:47.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:46 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:03:47.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:46 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:03:47.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:46 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:47.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:46 smithi192 ceph-mon[34664]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:03:47.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:46 smithi192 ceph-mon[34664]: Detected new or changed devices on smithi123 2024-05-08T06:03:47.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:46 smithi192 ceph-mon[34664]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:03:47.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:46 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:47.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:46 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T06:03:47.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:46 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:47.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:46 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T06:03:47.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:46 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:03:47.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:46 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:03:47.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:46 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:03:47.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:46 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:03:47.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:46 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:03:47.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:46 smithi192 ceph-mon[34664]: Adjusting osd_memory_target on smithi123 to 4272M 2024-05-08T06:03:47.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:46 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:47.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:46 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:03:47.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:46 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:03:47.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:46 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:03:48.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:48 smithi123 ceph-mon[29787]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:03:48.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:48 smithi123 ceph-mon[29787]: from='client.? 172.21.15.163:0/1878668446' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb59339b-e11b-4ffc-b202-7f67cb21da23"}]: dispatch 2024-05-08T06:03:48.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:48 smithi123 ceph-mon[29787]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb59339b-e11b-4ffc-b202-7f67cb21da23"}]: dispatch 2024-05-08T06:03:48.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:48 smithi123 ceph-mon[29787]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb59339b-e11b-4ffc-b202-7f67cb21da23"}]': finished 2024-05-08T06:03:48.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:48 smithi123 ceph-mon[29787]: osdmap e63: 11 total, 10 up, 11 in 2024-05-08T06:03:48.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:48 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T06:03:49.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:48 smithi163 ceph-mon[34520]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:03:49.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:48 smithi163 ceph-mon[34520]: from='client.? 172.21.15.163:0/1878668446' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb59339b-e11b-4ffc-b202-7f67cb21da23"}]: dispatch 2024-05-08T06:03:49.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:48 smithi163 ceph-mon[34520]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb59339b-e11b-4ffc-b202-7f67cb21da23"}]: dispatch 2024-05-08T06:03:49.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:48 smithi163 ceph-mon[34520]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb59339b-e11b-4ffc-b202-7f67cb21da23"}]': finished 2024-05-08T06:03:49.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:48 smithi163 ceph-mon[34520]: osdmap e63: 11 total, 10 up, 11 in 2024-05-08T06:03:49.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:48 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T06:03:49.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:48 smithi192 ceph-mon[34664]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:03:49.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:48 smithi192 ceph-mon[34664]: from='client.? 172.21.15.163:0/1878668446' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb59339b-e11b-4ffc-b202-7f67cb21da23"}]: dispatch 2024-05-08T06:03:49.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:48 smithi192 ceph-mon[34664]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb59339b-e11b-4ffc-b202-7f67cb21da23"}]: dispatch 2024-05-08T06:03:49.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:48 smithi192 ceph-mon[34664]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb59339b-e11b-4ffc-b202-7f67cb21da23"}]': finished 2024-05-08T06:03:49.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:48 smithi192 ceph-mon[34664]: osdmap e63: 11 total, 10 up, 11 in 2024-05-08T06:03:49.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:48 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T06:03:49.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:49 smithi123 ceph-mon[29787]: from='client.? 172.21.15.163:0/2143461631' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T06:03:50.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:49 smithi163 ceph-mon[34520]: from='client.? 172.21.15.163:0/2143461631' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T06:03:50.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:49 smithi192 ceph-mon[34664]: from='client.? 172.21.15.163:0/2143461631' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T06:03:50.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:50 smithi123 ceph-mon[29787]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:03:51.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:50 smithi163 ceph-mon[34520]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:03:51.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:50 smithi192 ceph-mon[34664]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:03:52.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:52 smithi123 ceph-mon[29787]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:03:52.971 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:52 smithi163 ceph-mon[34520]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:03:53.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:52 smithi192 ceph-mon[34664]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:03:54.951 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:54 smithi163 ceph-mon[34520]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:03:54.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:54 smithi123 ceph-mon[29787]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:03:55.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:54 smithi192 ceph-mon[34664]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:03:56.941 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:56 smithi163 ceph-mon[34520]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:03:56.941 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:56 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-08T06:03:56.941 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:56 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:03:57.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:56 smithi192 ceph-mon[34664]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:03:57.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:56 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-08T06:03:57.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:56 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:03:57.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:56 smithi123 ceph-mon[29787]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:03:57.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:56 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-08T06:03:57.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:56 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:03:57.989 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:57 smithi163 ceph-mon[34520]: Deploying daemon osd.10 on smithi163 2024-05-08T06:03:58.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:57 smithi192 ceph-mon[34664]: Deploying daemon osd.10 on smithi163 2024-05-08T06:03:58.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:57 smithi123 ceph-mon[29787]: Deploying daemon osd.10 on smithi163 2024-05-08T06:03:59.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:03:58 smithi163 ceph-mon[34520]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:03:59.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:03:58 smithi192 ceph-mon[34664]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:03:59.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:03:58 smithi123 ceph-mon[29787]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:04:00.844 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:00 smithi163 ceph-mon[34520]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:04:01.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:00 smithi192 ceph-mon[34664]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:04:01.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:00 smithi123 ceph-mon[29787]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:04:02.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:02 smithi163 ceph-mon[34520]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:04:02.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:02 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:02.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:02 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:02.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:02 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:04:02.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:02 smithi192 ceph-mon[34664]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:04:02.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:02 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:02.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:02 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:02.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:02 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:04:02.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:02 smithi123 ceph-mon[29787]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:04:02.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:02 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:02.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:02 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:02.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:02 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:04:04.759 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:04 smithi123 ceph-mon[29787]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:04:04.759 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:04 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:04.760 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:04 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:04.760 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:04 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:04.760 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:04 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:04:04.760 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:04 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:04.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:04 smithi163 ceph-mon[34520]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:04:04.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:04 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:04.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:04 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:04.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:04 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:04.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:04 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:04:04.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:04 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:04.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:04 smithi192 ceph-mon[34664]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:04:04.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:04 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:04.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:04 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:04.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:04 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:04.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:04 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:04:04.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:04 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:05.162 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:04:05.162+0000 7f54d37fe700 1 -- 172.21.15.163:0/698223314 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f54e40638e0 con 0x7f54cc0b2880 2024-05-08T06:04:05.163 INFO:teuthology.orchestra.run.smithi163.stdout:Created osd(s) 10 on host 'smithi163' 2024-05-08T06:04:05.165 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:04:05.165+0000 7f54e8f57700 1 -- 172.21.15.163:0/698223314 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f54cc0b2880 msgr2=0x7f54cc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:05.165 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:04:05.165+0000 7f54e8f57700 1 --2- 172.21.15.163:0/698223314 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f54cc0b2880 0x7f54cc0b4d40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f54d4009040 tx=0x7f54d400b040 comp rx=0 tx=0).stop 2024-05-08T06:04:05.165 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:04:05.165+0000 7f54e8f57700 1 -- 172.21.15.163:0/698223314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54e4104700 msgr2=0x7f54e4075890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:05.165 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:04:05.165+0000 7f54e8f57700 1 --2- 172.21.15.163:0/698223314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54e4104700 0x7f54e4075890 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f54d800b530 tx=0x7f54d800b840 comp rx=0 tx=0).stop 2024-05-08T06:04:05.165 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:04:05.165+0000 7f54e8f57700 1 -- 172.21.15.163:0/698223314 shutdown_connections 2024-05-08T06:04:05.165 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:04:05.165+0000 7f54e8f57700 1 --2- 172.21.15.163:0/698223314 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f54cc0b2880 0x7f54cc0b4d40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:05.165 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:04:05.165+0000 7f54e8f57700 1 --2- 172.21.15.163:0/698223314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54e41050a0 0x7f54e4075dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:05.166 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:04:05.165+0000 7f54e8f57700 1 --2- 172.21.15.163:0/698223314 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54e4103500 0x7f54e4077240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:05.166 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:04:05.165+0000 7f54e8f57700 1 --2- 172.21.15.163:0/698223314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54e4104700 0x7f54e4075890 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:05.166 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:04:05.165+0000 7f54e8f57700 1 -- 172.21.15.163:0/698223314 >> 172.21.15.163:0/698223314 conn(0x7f54e40feaa0 msgr2=0x7f54e4107b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:05.166 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:04:05.165+0000 7f54e8f57700 1 -- 172.21.15.163:0/698223314 shutdown_connections 2024-05-08T06:04:05.166 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-08T06:04:05.165+0000 7f54e8f57700 1 -- 172.21.15.163:0/698223314 wait complete. 2024-05-08T06:04:05.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:05 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:04:05.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:05 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:05.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:05 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:05.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:05 smithi123 ceph-mon[29787]: from='osd.10 [v2:172.21.15.163:6824/2628487883,v1:172.21.15.163:6825/2628487883]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T06:04:05.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:05 smithi123 ceph-mon[29787]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T06:04:05.771 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:05 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:04:05.772 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:05 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:05.772 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:05 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:05.772 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:05 smithi163 ceph-mon[34520]: from='osd.10 [v2:172.21.15.163:6824/2628487883,v1:172.21.15.163:6825/2628487883]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T06:04:05.772 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:05 smithi163 ceph-mon[34520]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T06:04:05.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:05 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:04:05.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:05 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:05.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:05 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:05.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:05 smithi192 ceph-mon[34664]: from='osd.10 [v2:172.21.15.163:6824/2628487883,v1:172.21.15.163:6825/2628487883]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T06:04:05.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:05 smithi192 ceph-mon[34664]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T06:04:05.804 DEBUG:teuthology.orchestra.run.smithi163:osd.10> sudo journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@osd.10.service 2024-05-08T06:04:05.807 INFO:tasks.cephadm:Deploying osd.11 on smithi192 with /dev/vg_nvme/lv_1... 2024-05-08T06:04:05.807 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-08T06:04:06.056 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T06:04:06.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:06 smithi123 ceph-mon[29787]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:04:06.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:06 smithi123 ceph-mon[29787]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-08T06:04:06.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:06 smithi123 ceph-mon[29787]: from='osd.10 [v2:172.21.15.163:6824/2628487883,v1:172.21.15.163:6825/2628487883]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-05-08T06:04:06.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:06 smithi123 ceph-mon[29787]: osdmap e64: 11 total, 10 up, 11 in 2024-05-08T06:04:06.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:06 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T06:04:06.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:06 smithi123 ceph-mon[29787]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-05-08T06:04:06.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:06 smithi163 ceph-mon[34520]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:04:06.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:06 smithi163 ceph-mon[34520]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-08T06:04:06.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:06 smithi163 ceph-mon[34520]: from='osd.10 [v2:172.21.15.163:6824/2628487883,v1:172.21.15.163:6825/2628487883]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-05-08T06:04:06.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:06 smithi163 ceph-mon[34520]: osdmap e64: 11 total, 10 up, 11 in 2024-05-08T06:04:06.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:06 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T06:04:06.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:06 smithi163 ceph-mon[34520]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-05-08T06:04:06.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:06 smithi192 ceph-mon[34664]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:04:06.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:06 smithi192 ceph-mon[34664]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-08T06:04:06.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:06 smithi192 ceph-mon[34664]: from='osd.10 [v2:172.21.15.163:6824/2628487883,v1:172.21.15.163:6825/2628487883]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-05-08T06:04:06.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:06 smithi192 ceph-mon[34664]: osdmap e64: 11 total, 10 up, 11 in 2024-05-08T06:04:06.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:06 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T06:04:06.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:06 smithi192 ceph-mon[34664]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-05-08T06:04:08.009 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:04:07 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:04:07.615+0000 7f931a4cc700 -1 osd.10 0 waiting for initial osdmap 2024-05-08T06:04:08.010 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:04:07 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:04:07.629+0000 7f9311ab6700 -1 osd.10 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T06:04:08.101 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-08T06:04:08.101 INFO:teuthology.orchestra.run.smithi192.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-08T06:04:08.101 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10+0 records in 2024-05-08T06:04:08.101 INFO:teuthology.orchestra.run.smithi192.stderr:10+0 records out 2024-05-08T06:04:08.102 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112958 s, 928 MB/s 2024-05-08T06:04:08.102 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping successful for: 2024-05-08T06:04:08.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:08 smithi163 ceph-mon[34520]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:04:08.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:08 smithi163 ceph-mon[34520]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-05-08T06:04:08.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:08 smithi163 ceph-mon[34520]: osdmap e65: 11 total, 10 up, 11 in 2024-05-08T06:04:08.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:08 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T06:04:08.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:08 smithi163 ceph-mon[34520]: from='osd.10 ' entity='osd.10' 2024-05-08T06:04:08.405 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:08 smithi192 ceph-mon[34664]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:04:08.406 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:08 smithi192 ceph-mon[34664]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-05-08T06:04:08.406 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:08 smithi192 ceph-mon[34664]: osdmap e65: 11 total, 10 up, 11 in 2024-05-08T06:04:08.406 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:08 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T06:04:08.406 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:08 smithi192 ceph-mon[34664]: from='osd.10 ' entity='osd.10' 2024-05-08T06:04:08.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:08 smithi123 ceph-mon[29787]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-05-08T06:04:08.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:08 smithi123 ceph-mon[29787]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-05-08T06:04:08.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:08 smithi123 ceph-mon[29787]: osdmap e65: 11 total, 10 up, 11 in 2024-05-08T06:04:08.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:08 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T06:04:08.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:08 smithi123 ceph-mon[29787]: from='osd.10 ' entity='osd.10' 2024-05-08T06:04:08.751 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi192:vg_nvme/lv_1 2024-05-08T06:04:09.005 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T06:04:09.387 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:09 smithi192 ceph-mon[34664]: purged_snaps scrub starts 2024-05-08T06:04:09.387 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:09 smithi192 ceph-mon[34664]: purged_snaps scrub ok 2024-05-08T06:04:09.387 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:09 smithi192 ceph-mon[34664]: osd.10 [v2:172.21.15.163:6824/2628487883,v1:172.21.15.163:6825/2628487883] boot 2024-05-08T06:04:09.387 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:09 smithi192 ceph-mon[34664]: osdmap e66: 11 total, 11 up, 11 in 2024-05-08T06:04:09.387 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:09 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T06:04:09.387 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:09 smithi192 ceph-mon[34664]: osdmap e67: 11 total, 11 up, 11 in 2024-05-08T06:04:09.387 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:09 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:09.387 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:09 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:09.388 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:09 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:09.388 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:09 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:09.388 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:09 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:09.388 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:09 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:09.388 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:09 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:09.388 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:09 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:09.388 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:09 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:04:09.388 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:09 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:09.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:09 smithi123 ceph-mon[29787]: purged_snaps scrub starts 2024-05-08T06:04:09.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:09 smithi123 ceph-mon[29787]: purged_snaps scrub ok 2024-05-08T06:04:09.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:09 smithi123 ceph-mon[29787]: osd.10 [v2:172.21.15.163:6824/2628487883,v1:172.21.15.163:6825/2628487883] boot 2024-05-08T06:04:09.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:09 smithi123 ceph-mon[29787]: osdmap e66: 11 total, 11 up, 11 in 2024-05-08T06:04:09.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:09 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T06:04:09.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:09 smithi123 ceph-mon[29787]: osdmap e67: 11 total, 11 up, 11 in 2024-05-08T06:04:09.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:09 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:09.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:09 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:09.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:09 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:09.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:09 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:09.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:09 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:09.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:09 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:09.457 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:09 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:09.457 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:09 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:09.457 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:09 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:04:09.457 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:09 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:09.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:09 smithi163 ceph-mon[34520]: purged_snaps scrub starts 2024-05-08T06:04:09.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:09 smithi163 ceph-mon[34520]: purged_snaps scrub ok 2024-05-08T06:04:09.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:09 smithi163 ceph-mon[34520]: osd.10 [v2:172.21.15.163:6824/2628487883,v1:172.21.15.163:6825/2628487883] boot 2024-05-08T06:04:09.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:09 smithi163 ceph-mon[34520]: osdmap e66: 11 total, 11 up, 11 in 2024-05-08T06:04:09.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:09 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T06:04:09.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:09 smithi163 ceph-mon[34520]: osdmap e67: 11 total, 11 up, 11 in 2024-05-08T06:04:09.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:09 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:09.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:09 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:09.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:09 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:09.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:09 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:09.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:09 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:09.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:09 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:09.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:09 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:09.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:09 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:09.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:09 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:04:09.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:09 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:10.519 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:10 smithi192 ceph-mon[34664]: Detected new or changed devices on smithi163 2024-05-08T06:04:10.519 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:10 smithi192 ceph-mon[34664]: Adjusting osd_memory_target on smithi163 to 4272M 2024-05-08T06:04:10.519 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:10 smithi192 ceph-mon[34664]: pgmap v256: 1 pgs: 1 peering; 577 KiB data, 351 MiB used, 983 GiB / 983 GiB avail 2024-05-08T06:04:10.520 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:10 smithi192 ceph-mon[34664]: osdmap e68: 11 total, 11 up, 11 in 2024-05-08T06:04:10.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:10 smithi123 ceph-mon[29787]: Detected new or changed devices on smithi163 2024-05-08T06:04:10.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:10 smithi123 ceph-mon[29787]: Adjusting osd_memory_target on smithi163 to 4272M 2024-05-08T06:04:10.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:10 smithi123 ceph-mon[29787]: pgmap v256: 1 pgs: 1 peering; 577 KiB data, 351 MiB used, 983 GiB / 983 GiB avail 2024-05-08T06:04:10.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:10 smithi123 ceph-mon[29787]: osdmap e68: 11 total, 11 up, 11 in 2024-05-08T06:04:10.740 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.738+0000 7f4e19cd4700 1 -- 172.21.15.192:0/4058760050 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e141050a0 msgr2=0x7f4e1410b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:10.740 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.738+0000 7f4e19cd4700 1 --2- 172.21.15.192:0/4058760050 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e141050a0 0x7f4e1410b950 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c00b0a0 tx=0x7f4e0c00b3b0 comp rx=0 tx=0).stop 2024-05-08T06:04:10.740 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.738+0000 7f4e19cd4700 1 -- 172.21.15.192:0/4058760050 shutdown_connections 2024-05-08T06:04:10.740 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.738+0000 7f4e19cd4700 1 --2- 172.21.15.192:0/4058760050 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e141050a0 0x7f4e1410b950 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:10.740 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.738+0000 7f4e19cd4700 1 --2- 172.21.15.192:0/4058760050 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e14103500 0x7f4e14103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:10.740 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.738+0000 7f4e19cd4700 1 --2- 172.21.15.192:0/4058760050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e14104700 0x7f4e14104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:10.741 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.738+0000 7f4e19cd4700 1 -- 172.21.15.192:0/4058760050 >> 172.21.15.192:0/4058760050 conn(0x7f4e140feaa0 msgr2=0x7f4e14100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:10.741 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.738+0000 7f4e19cd4700 1 -- 172.21.15.192:0/4058760050 shutdown_connections 2024-05-08T06:04:10.741 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.738+0000 7f4e19cd4700 1 -- 172.21.15.192:0/4058760050 wait complete. 2024-05-08T06:04:10.741 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.739+0000 7f4e19cd4700 1 Processor -- start 2024-05-08T06:04:10.741 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.739+0000 7f4e19cd4700 1 -- start start 2024-05-08T06:04:10.741 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.739+0000 7f4e19cd4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e14103500 0x7f4e14077240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:10.741 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.739+0000 7f4e19cd4700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e14104700 0x7f4e14075890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:10.741 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.739+0000 7f4e19cd4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e141050a0 0x7f4e14075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:10.741 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.739+0000 7f4e19cd4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1410eff0 con 0x7f4e14103500 2024-05-08T06:04:10.741 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.739+0000 7f4e19cd4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1410ee70 con 0x7f4e141050a0 2024-05-08T06:04:10.741 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.739+0000 7f4e19cd4700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1410f170 con 0x7f4e14104700 2024-05-08T06:04:10.741 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.739+0000 7f4e12ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e14104700 0x7f4e14075890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:10.741 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.739+0000 7f4e12ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e14104700 0x7f4e14075890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:57670/0 (socket says 172.21.15.192:57670) 2024-05-08T06:04:10.742 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.739+0000 7f4e12ffd700 1 -- 172.21.15.192:0/435711595 learned_addr learned my addr 172.21.15.192:0/435711595 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-08T06:04:10.742 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.739+0000 7f4e13fff700 1 --2- 172.21.15.192:0/435711595 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e141050a0 0x7f4e14075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:10.742 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.740+0000 7f4e12ffd700 1 -- 172.21.15.192:0/435711595 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e141050a0 msgr2=0x7f4e14075dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:10.742 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.740+0000 7f4e12ffd700 1 --2- 172.21.15.192:0/435711595 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e141050a0 0x7f4e14075dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:10.742 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.740+0000 7f4e12ffd700 1 -- 172.21.15.192:0/435711595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e14103500 msgr2=0x7f4e14077240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:10.742 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.740+0000 7f4e12ffd700 1 --2- 172.21.15.192:0/435711595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e14103500 0x7f4e14077240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:10.742 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.740+0000 7f4e12ffd700 1 -- 172.21.15.192:0/435711595 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e14076680 con 0x7f4e14104700 2024-05-08T06:04:10.742 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.740+0000 7f4e12ffd700 1 --2- 172.21.15.192:0/435711595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e14104700 0x7f4e14075890 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc00b530 tx=0x7f4dfc00b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:10.742 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.740+0000 7f4e10ff9700 1 -- 172.21.15.192:0/435711595 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc015020 con 0x7f4e14104700 2024-05-08T06:04:10.742 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.740+0000 7f4e10ff9700 1 -- 172.21.15.192:0/435711595 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dfc011ce0 con 0x7f4e14104700 2024-05-08T06:04:10.742 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.740+0000 7f4e19cd4700 1 -- 172.21.15.192:0/435711595 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e141a37f0 con 0x7f4e14104700 2024-05-08T06:04:10.742 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.740+0000 7f4e10ff9700 1 -- 172.21.15.192:0/435711595 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc010e40 con 0x7f4e14104700 2024-05-08T06:04:10.742 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.740+0000 7f4e19cd4700 1 -- 172.21.15.192:0/435711595 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e141a3d80 con 0x7f4e14104700 2024-05-08T06:04:10.743 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.741+0000 7f4e19cd4700 1 -- 172.21.15.192:0/435711595 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e1406a190 con 0x7f4e14104700 2024-05-08T06:04:10.743 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.742+0000 7f4e10ff9700 1 -- 172.21.15.192:0/435711595 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f4dfc018020 con 0x7f4e14104700 2024-05-08T06:04:10.743 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.743+0000 7f4e10ff9700 1 --2- 172.21.15.192:0/435711595 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f4e000b2880 0x7f4e000b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:10.743 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.743+0000 7f4e10ff9700 1 -- 172.21.15.192:0/435711595 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6694+0+0 (secure 0 0 0) 0x7f4dfc0e19a0 con 0x7f4e14104700 2024-05-08T06:04:10.744 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.744+0000 7f4e137fe700 1 --2- 172.21.15.192:0/435711595 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f4e000b2880 0x7f4e000b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:10.744 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.744+0000 7f4e137fe700 1 --2- 172.21.15.192:0/435711595 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f4e000b2880 0x7f4e000b4d40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4e14104560 tx=0x7f4e04005ca0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:10.746 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.747+0000 7f4e10ff9700 1 -- 172.21.15.192:0/435711595 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4dfc01f020 con 0x7f4e14104700 2024-05-08T06:04:10.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:10 smithi163 ceph-mon[34520]: Detected new or changed devices on smithi163 2024-05-08T06:04:10.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:10 smithi163 ceph-mon[34520]: Adjusting osd_memory_target on smithi163 to 4272M 2024-05-08T06:04:10.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:10 smithi163 ceph-mon[34520]: pgmap v256: 1 pgs: 1 peering; 577 KiB data, 351 MiB used, 983 GiB / 983 GiB avail 2024-05-08T06:04:10.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:10 smithi163 ceph-mon[34520]: osdmap e68: 11 total, 11 up, 11 in 2024-05-08T06:04:10.958 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:10.958+0000 7f4e19cd4700 1 -- 172.21.15.192:0/435711595 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f4e140638e0 con 0x7f4e000b2880 2024-05-08T06:04:11.613 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:11 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T06:04:11.614 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:11 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T06:04:11.614 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:11 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:11.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:11 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T06:04:11.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:11 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T06:04:11.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:11 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:11.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:11 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T06:04:11.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:11 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T06:04:11.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:11 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:12.625 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:12 smithi192 ceph-mon[34664]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:04:12.625 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:12 smithi192 ceph-mon[34664]: pgmap v258: 1 pgs: 1 peering; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail 2024-05-08T06:04:12.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:12 smithi123 ceph-mon[29787]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:04:12.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:12 smithi123 ceph-mon[29787]: pgmap v258: 1 pgs: 1 peering; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail 2024-05-08T06:04:12.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:12 smithi163 ceph-mon[34520]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:04:12.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:12 smithi163 ceph-mon[34520]: pgmap v258: 1 pgs: 1 peering; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail 2024-05-08T06:04:14.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:14 smithi192 ceph-mon[34664]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-05-08T06:04:14.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:14 smithi192 ceph-mon[34664]: from='client.? 172.21.15.192:0/3121729608' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be7a5f36-c1be-41d7-80ee-004d0546eb9e"}]: dispatch 2024-05-08T06:04:14.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:14 smithi192 ceph-mon[34664]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be7a5f36-c1be-41d7-80ee-004d0546eb9e"}]: dispatch 2024-05-08T06:04:14.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:14 smithi192 ceph-mon[34664]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "be7a5f36-c1be-41d7-80ee-004d0546eb9e"}]': finished 2024-05-08T06:04:14.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:14 smithi192 ceph-mon[34664]: osdmap e69: 12 total, 11 up, 12 in 2024-05-08T06:04:14.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:14 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T06:04:14.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:14 smithi123 ceph-mon[29787]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-05-08T06:04:14.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:14 smithi123 ceph-mon[29787]: from='client.? 172.21.15.192:0/3121729608' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be7a5f36-c1be-41d7-80ee-004d0546eb9e"}]: dispatch 2024-05-08T06:04:14.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:14 smithi123 ceph-mon[29787]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be7a5f36-c1be-41d7-80ee-004d0546eb9e"}]: dispatch 2024-05-08T06:04:14.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:14 smithi123 ceph-mon[29787]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "be7a5f36-c1be-41d7-80ee-004d0546eb9e"}]': finished 2024-05-08T06:04:14.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:14 smithi123 ceph-mon[29787]: osdmap e69: 12 total, 11 up, 12 in 2024-05-08T06:04:14.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:14 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T06:04:14.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:14 smithi163 ceph-mon[34520]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-05-08T06:04:14.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:14 smithi163 ceph-mon[34520]: from='client.? 172.21.15.192:0/3121729608' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be7a5f36-c1be-41d7-80ee-004d0546eb9e"}]: dispatch 2024-05-08T06:04:14.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:14 smithi163 ceph-mon[34520]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be7a5f36-c1be-41d7-80ee-004d0546eb9e"}]: dispatch 2024-05-08T06:04:14.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:14 smithi163 ceph-mon[34520]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "be7a5f36-c1be-41d7-80ee-004d0546eb9e"}]': finished 2024-05-08T06:04:14.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:14 smithi163 ceph-mon[34520]: osdmap e69: 12 total, 11 up, 12 in 2024-05-08T06:04:14.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:14 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T06:04:15.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:15 smithi123 ceph-mon[29787]: from='client.? 172.21.15.192:0/3296071201' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T06:04:15.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:15 smithi163 ceph-mon[34520]: from='client.? 172.21.15.192:0/3296071201' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T06:04:15.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:15 smithi192 ceph-mon[34664]: from='client.? 172.21.15.192:0/3296071201' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T06:04:16.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:16 smithi123 ceph-mon[29787]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-08T06:04:16.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:16 smithi163 ceph-mon[34520]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-08T06:04:16.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:16 smithi192 ceph-mon[34664]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-08T06:04:18.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:18 smithi123 ceph-mon[29787]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T06:04:18.760 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:18 smithi192 ceph-mon[34664]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T06:04:18.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:18 smithi163 ceph-mon[34520]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T06:04:20.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:20 smithi123 ceph-mon[29787]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-08T06:04:20.775 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:20 smithi192 ceph-mon[34664]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-08T06:04:20.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:20 smithi163 ceph-mon[34520]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-08T06:04:21.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:21 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-08T06:04:21.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:21 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:21.758 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:21 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-08T06:04:21.758 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:21 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:21.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:21 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-08T06:04:21.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:21 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:22.688 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:22 smithi163 ceph-mon[34520]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T06:04:22.689 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:22 smithi163 ceph-mon[34520]: Deploying daemon osd.11 on smithi192 2024-05-08T06:04:22.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:22 smithi123 ceph-mon[29787]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T06:04:22.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:22 smithi123 ceph-mon[29787]: Deploying daemon osd.11 on smithi192 2024-05-08T06:04:22.789 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:22 smithi192 ceph-mon[34664]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T06:04:22.789 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:22 smithi192 ceph-mon[34664]: Deploying daemon osd.11 on smithi192 2024-05-08T06:04:24.494 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:24 smithi192 ceph-mon[34664]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-08T06:04:24.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:24 smithi123 ceph-mon[29787]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-08T06:04:24.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:24 smithi163 ceph-mon[34520]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-08T06:04:26.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:26 smithi192 ceph-mon[34664]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-08T06:04:26.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:26 smithi123 ceph-mon[29787]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-08T06:04:26.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:26 smithi163 ceph-mon[34520]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-08T06:04:28.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:27 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:28.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:27 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:28.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:27 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:04:28.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:27 smithi123 ceph-mon[29787]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-08T06:04:28.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:27 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:28.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:27 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:28.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:27 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:04:28.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:27 smithi163 ceph-mon[34520]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-08T06:04:28.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:28.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:28.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:04:28.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:27 smithi192 ceph-mon[34664]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-08T06:04:30.164 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:30 smithi192 ceph-mon[34664]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-08T06:04:30.244 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:30.244+0000 7f4e10ff9700 1 -- 172.21.15.192:0/435711595 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f4e140638e0 con 0x7f4e000b2880 2024-05-08T06:04:30.244 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 11 on host 'smithi192' 2024-05-08T06:04:30.247 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:30.247+0000 7f4e19cd4700 1 -- 172.21.15.192:0/435711595 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f4e000b2880 msgr2=0x7f4e000b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:30.247 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:30.247+0000 7f4e19cd4700 1 --2- 172.21.15.192:0/435711595 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f4e000b2880 0x7f4e000b4d40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4e14104560 tx=0x7f4e04005ca0 comp rx=0 tx=0).stop 2024-05-08T06:04:30.247 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:30.247+0000 7f4e19cd4700 1 -- 172.21.15.192:0/435711595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e14104700 msgr2=0x7f4e14075890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:30.247 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:30.247+0000 7f4e19cd4700 1 --2- 172.21.15.192:0/435711595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e14104700 0x7f4e14075890 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc00b530 tx=0x7f4dfc00b840 comp rx=0 tx=0).stop 2024-05-08T06:04:30.247 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:30.247+0000 7f4e19cd4700 1 -- 172.21.15.192:0/435711595 shutdown_connections 2024-05-08T06:04:30.247 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:30.247+0000 7f4e19cd4700 1 --2- 172.21.15.192:0/435711595 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f4e000b2880 0x7f4e000b4d40 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:30.247 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:30.247+0000 7f4e19cd4700 1 --2- 172.21.15.192:0/435711595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e14103500 0x7f4e14077240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:30.247 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:30.247+0000 7f4e19cd4700 1 --2- 172.21.15.192:0/435711595 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e141050a0 0x7f4e14075dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:30.247 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:30.247+0000 7f4e19cd4700 1 --2- 172.21.15.192:0/435711595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e14104700 0x7f4e14075890 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:30.247 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:30.247+0000 7f4e19cd4700 1 -- 172.21.15.192:0/435711595 >> 172.21.15.192:0/435711595 conn(0x7f4e140feaa0 msgr2=0x7f4e14107a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:30.247 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:30.247+0000 7f4e19cd4700 1 -- 172.21.15.192:0/435711595 shutdown_connections 2024-05-08T06:04:30.248 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:30.247+0000 7f4e19cd4700 1 -- 172.21.15.192:0/435711595 wait complete. 2024-05-08T06:04:30.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:30 smithi123 ceph-mon[29787]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-08T06:04:30.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:30 smithi163 ceph-mon[34520]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-08T06:04:30.903 DEBUG:teuthology.orchestra.run.smithi192:osd.11> sudo journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@osd.11.service 2024-05-08T06:04:30.905 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-08T06:04:30.906 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-08T06:04:31.341 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:04:31.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:31 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:31.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:31 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:31.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:31 smithi123 ceph-mon[29787]: from='osd.11 [v2:172.21.15.192:6824/1339159347,v1:172.21.15.192:6825/1339159347]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T06:04:31.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:31 smithi123 ceph-mon[29787]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T06:04:31.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:31 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:31.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:31 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:31.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:31 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:31.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:31 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:31.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:31 smithi163 ceph-mon[34520]: from='osd.11 [v2:172.21.15.192:6824/1339159347,v1:172.21.15.192:6825/1339159347]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T06:04:31.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:31 smithi163 ceph-mon[34520]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T06:04:31.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:31 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:31.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:31 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:31.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:31 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:31.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:31 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:31.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:31 smithi192 ceph-mon[34664]: from='osd.11 [v2:172.21.15.192:6824/1339159347,v1:172.21.15.192:6825/1339159347]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T06:04:31.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:31 smithi192 ceph-mon[34664]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T06:04:31.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:31 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:31.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:31 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:32.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:32 smithi192 ceph-mon[34664]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-08T06:04:32.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:32 smithi192 ceph-mon[34664]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-08T06:04:32.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:32 smithi192 ceph-mon[34664]: osdmap e70: 12 total, 11 up, 12 in 2024-05-08T06:04:32.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:32 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T06:04:32.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:32 smithi192 ceph-mon[34664]: from='osd.11 [v2:172.21.15.192:6824/1339159347,v1:172.21.15.192:6825/1339159347]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-08T06:04:32.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:32 smithi192 ceph-mon[34664]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-08T06:04:32.419 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:32 smithi123 ceph-mon[29787]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-08T06:04:32.419 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:32 smithi123 ceph-mon[29787]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-08T06:04:32.419 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:32 smithi123 ceph-mon[29787]: osdmap e70: 12 total, 11 up, 12 in 2024-05-08T06:04:32.419 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:32 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T06:04:32.419 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:32 smithi123 ceph-mon[29787]: from='osd.11 [v2:172.21.15.192:6824/1339159347,v1:172.21.15.192:6825/1339159347]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-08T06:04:32.419 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:32 smithi123 ceph-mon[29787]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-08T06:04:32.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:32 smithi163 ceph-mon[34520]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-08T06:04:32.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:32 smithi163 ceph-mon[34520]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-08T06:04:32.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:32 smithi163 ceph-mon[34520]: osdmap e70: 12 total, 11 up, 12 in 2024-05-08T06:04:32.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:32 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T06:04:32.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:32 smithi163 ceph-mon[34520]: from='osd.11 [v2:172.21.15.192:6824/1339159347,v1:172.21.15.192:6825/1339159347]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-08T06:04:32.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:32 smithi163 ceph-mon[34520]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-05-08T06:04:32.912 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:04:32 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:04:32.649+0000 7f06d589d700 -1 osd.11 0 waiting for initial osdmap 2024-05-08T06:04:32.913 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:04:32 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:04:32.661+0000 7f06cd688700 -1 osd.11 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T06:04:33.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:33 smithi123 ceph-mon[29787]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-05-08T06:04:33.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:33 smithi123 ceph-mon[29787]: osdmap e71: 12 total, 11 up, 12 in 2024-05-08T06:04:33.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:33 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T06:04:33.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:33 smithi123 ceph-mon[29787]: from='osd.11 ' entity='osd.11' 2024-05-08T06:04:33.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:33 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T06:04:33.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:33 smithi163 ceph-mon[34520]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-05-08T06:04:33.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:33 smithi163 ceph-mon[34520]: osdmap e71: 12 total, 11 up, 12 in 2024-05-08T06:04:33.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:33 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T06:04:33.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:33 smithi163 ceph-mon[34520]: from='osd.11 ' entity='osd.11' 2024-05-08T06:04:33.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:33 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T06:04:33.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:33 smithi192 ceph-mon[34664]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-05-08T06:04:33.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:33 smithi192 ceph-mon[34664]: osdmap e71: 12 total, 11 up, 12 in 2024-05-08T06:04:33.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:33 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T06:04:33.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:33 smithi192 ceph-mon[34664]: from='osd.11 ' entity='osd.11' 2024-05-08T06:04:33.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:33 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T06:04:33.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.727+0000 7fbd16400700 1 -- 172.21.15.123:0/3334375841 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd101068c0 msgr2=0x7fbd10108cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:33.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.727+0000 7fbd16400700 1 --2- 172.21.15.123:0/3334375841 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd101068c0 0x7fbd10108cb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc009990 tx=0x7fbcfc009ca0 comp rx=0 tx=0).stop 2024-05-08T06:04:33.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.727+0000 7fbd16400700 1 -- 172.21.15.123:0/3334375841 shutdown_connections 2024-05-08T06:04:33.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.727+0000 7fbd16400700 1 --2- 172.21.15.123:0/3334375841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd10103f90 0x7fbd10106380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:33.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.727+0000 7fbd16400700 1 --2- 172.21.15.123:0/3334375841 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd101091f0 0x7fbd1010b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:33.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.727+0000 7fbd16400700 1 --2- 172.21.15.123:0/3334375841 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd101068c0 0x7fbd10108cb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:33.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.727+0000 7fbd16400700 1 -- 172.21.15.123:0/3334375841 >> 172.21.15.123:0/3334375841 conn(0x7fbd100fd9a0 msgr2=0x7fbd100ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:33.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.727+0000 7fbd16400700 1 -- 172.21.15.123:0/3334375841 shutdown_connections 2024-05-08T06:04:33.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.727+0000 7fbd16400700 1 -- 172.21.15.123:0/3334375841 wait complete. 2024-05-08T06:04:33.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.728+0000 7fbd16400700 1 Processor -- start 2024-05-08T06:04:33.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.728+0000 7fbd16400700 1 -- start start 2024-05-08T06:04:33.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.728+0000 7fbd16400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd10103f90 0x7fbd101996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:33.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.728+0000 7fbd16400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd101068c0 0x7fbd10199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:33.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.729+0000 7fbd16400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd101091f0 0x7fbd101a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:33.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.729+0000 7fbd16400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1010f010 con 0x7fbd10103f90 2024-05-08T06:04:33.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.729+0000 7fbd16400700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1010ee90 con 0x7fbd101091f0 2024-05-08T06:04:33.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.729+0000 7fbd16400700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1010f190 con 0x7fbd101068c0 2024-05-08T06:04:33.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.729+0000 7fbd0f400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd10103f90 0x7fbd101996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:33.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.729+0000 7fbd0f400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd10103f90 0x7fbd101996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39234/0 (socket says 172.21.15.123:39234) 2024-05-08T06:04:33.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.729+0000 7fbd0f400700 1 -- 172.21.15.123:0/2939696022 learned_addr learned my addr 172.21.15.123:0/2939696022 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:04:33.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.729+0000 7fbd0ea00700 1 --2- 172.21.15.123:0/2939696022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd101068c0 0x7fbd10199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:33.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.729+0000 7fbd0fe00700 1 --2- 172.21.15.123:0/2939696022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd101091f0 0x7fbd101a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:33.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.729+0000 7fbd0ea00700 1 -- 172.21.15.123:0/2939696022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd101091f0 msgr2=0x7fbd101a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:33.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.729+0000 7fbd0ea00700 1 --2- 172.21.15.123:0/2939696022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd101091f0 0x7fbd101a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:33.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.729+0000 7fbd0ea00700 1 -- 172.21.15.123:0/2939696022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd10103f90 msgr2=0x7fbd101996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:33.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.729+0000 7fbd0ea00700 1 --2- 172.21.15.123:0/2939696022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd10103f90 0x7fbd101996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:33.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.729+0000 7fbd0ea00700 1 -- 172.21.15.123:0/2939696022 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd101a1330 con 0x7fbd101068c0 2024-05-08T06:04:33.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.729+0000 7fbd0f400700 1 --2- 172.21.15.123:0/2939696022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd10103f90 0x7fbd101996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:04:33.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.729+0000 7fbd0fe00700 1 --2- 172.21.15.123:0/2939696022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd101091f0 0x7fbd101a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:04:33.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.730+0000 7fbd0ea00700 1 --2- 172.21.15.123:0/2939696022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd101068c0 0x7fbd10199c10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc0098e0 tx=0x7fbcfc009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:33.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.730+0000 7fbd07e00700 1 -- 172.21.15.123:0/2939696022 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc026070 con 0x7fbd101068c0 2024-05-08T06:04:33.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.730+0000 7fbd07e00700 1 -- 172.21.15.123:0/2939696022 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcfc021410 con 0x7fbd101068c0 2024-05-08T06:04:33.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.730+0000 7fbd16400700 1 -- 172.21.15.123:0/2939696022 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd101a15c0 con 0x7fbd101068c0 2024-05-08T06:04:33.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.730+0000 7fbd07e00700 1 -- 172.21.15.123:0/2939696022 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc017650 con 0x7fbd101068c0 2024-05-08T06:04:33.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.730+0000 7fbd16400700 1 -- 172.21.15.123:0/2939696022 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd101a1a70 con 0x7fbd101068c0 2024-05-08T06:04:33.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.731+0000 7fbd16400700 1 -- 172.21.15.123:0/2939696022 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd1006a190 con 0x7fbd101068c0 2024-05-08T06:04:33.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.733+0000 7fbd07e00700 1 -- 172.21.15.123:0/2939696022 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fbcfc029070 con 0x7fbd101068c0 2024-05-08T06:04:33.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.735+0000 7fbd07e00700 1 --2- 172.21.15.123:0/2939696022 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fbcf40b2900 0x7fbcf40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:33.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.735+0000 7fbd07e00700 1 -- 172.21.15.123:0/2939696022 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fbcfc0e2cc0 con 0x7fbd101068c0 2024-05-08T06:04:33.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.735+0000 7fbd0f400700 1 --2- 172.21.15.123:0/2939696022 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fbcf40b2900 0x7fbcf40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:33.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.736+0000 7fbd0f400700 1 --2- 172.21.15.123:0/2939696022 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fbcf40b2900 0x7fbcf40b4dc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbd00011550 tx=0x7fbd00009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:33.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.741+0000 7fbd07e00700 1 -- 172.21.15.123:0/2939696022 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbcfc013070 con 0x7fbd101068c0 2024-05-08T06:04:33.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.978+0000 7fbd16400700 1 -- 172.21.15.123:0/2939696022 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fbd100640b0 con 0x7fbd101068c0 2024-05-08T06:04:33.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.979+0000 7fbd07e00700 1 -- 172.21.15.123:0/2939696022 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7fbcfc0aae70 con 0x7fbd101068c0 2024-05-08T06:04:33.980 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:04:33.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.981+0000 7fbd16400700 1 -- 172.21.15.123:0/2939696022 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fbcf40b2900 msgr2=0x7fbcf40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:33.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.981+0000 7fbd16400700 1 --2- 172.21.15.123:0/2939696022 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fbcf40b2900 0x7fbcf40b4dc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbd00011550 tx=0x7fbd00009260 comp rx=0 tx=0).stop 2024-05-08T06:04:33.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.981+0000 7fbd16400700 1 -- 172.21.15.123:0/2939696022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd101068c0 msgr2=0x7fbd10199c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:33.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.981+0000 7fbd16400700 1 --2- 172.21.15.123:0/2939696022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd101068c0 0x7fbd10199c10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc0098e0 tx=0x7fbcfc009580 comp rx=0 tx=0).stop 2024-05-08T06:04:33.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.982+0000 7fbd16400700 1 -- 172.21.15.123:0/2939696022 shutdown_connections 2024-05-08T06:04:33.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.982+0000 7fbd16400700 1 --2- 172.21.15.123:0/2939696022 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fbcf40b2900 0x7fbcf40b4dc0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:33.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.982+0000 7fbd16400700 1 --2- 172.21.15.123:0/2939696022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd10103f90 0x7fbd101996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:33.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.982+0000 7fbd16400700 1 --2- 172.21.15.123:0/2939696022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd101091f0 0x7fbd101a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:33.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.982+0000 7fbd16400700 1 --2- 172.21.15.123:0/2939696022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd101068c0 0x7fbd10199c10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:33.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.982+0000 7fbd16400700 1 -- 172.21.15.123:0/2939696022 >> 172.21.15.123:0/2939696022 conn(0x7fbd100fd9a0 msgr2=0x7fbd10105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:33.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.982+0000 7fbd16400700 1 -- 172.21.15.123:0/2939696022 shutdown_connections 2024-05-08T06:04:33.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:33.982+0000 7fbd16400700 1 -- 172.21.15.123:0/2939696022 wait complete. 2024-05-08T06:04:34.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:34 smithi192 ceph-mon[34664]: purged_snaps scrub starts 2024-05-08T06:04:34.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:34 smithi192 ceph-mon[34664]: purged_snaps scrub ok 2024-05-08T06:04:34.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:34 smithi192 ceph-mon[34664]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-08T06:04:34.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:34 smithi192 ceph-mon[34664]: osd.11 [v2:172.21.15.192:6824/1339159347,v1:172.21.15.192:6825/1339159347] boot 2024-05-08T06:04:34.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:34 smithi192 ceph-mon[34664]: osdmap e72: 12 total, 12 up, 12 in 2024-05-08T06:04:34.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:34 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T06:04:34.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:34 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/2939696022' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T06:04:34.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:34 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:34.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:34 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:34.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:34 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:34.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:34 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:34.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:34 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:34.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:34 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:34.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:34 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:34.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:34 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:34.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:34 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:04:34.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:34 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:34.803 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":72,"num_osds":12,"num_up_osds":12,"osd_up_since":1715148273,"num_in_osds":12,"osd_in_since":1715148253,"num_remapped_pgs":0} 2024-05-08T06:04:34.803 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T06:04:34.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:34 smithi123 ceph-mon[29787]: purged_snaps scrub starts 2024-05-08T06:04:34.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:34 smithi123 ceph-mon[29787]: purged_snaps scrub ok 2024-05-08T06:04:34.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:34 smithi123 ceph-mon[29787]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-08T06:04:34.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:34 smithi123 ceph-mon[29787]: osd.11 [v2:172.21.15.192:6824/1339159347,v1:172.21.15.192:6825/1339159347] boot 2024-05-08T06:04:34.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:34 smithi123 ceph-mon[29787]: osdmap e72: 12 total, 12 up, 12 in 2024-05-08T06:04:34.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:34 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T06:04:34.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:34 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/2939696022' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T06:04:34.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:34 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:34.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:34 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:34.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:34 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:34.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:34 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:34.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:34 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:34.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:34 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:34.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:34 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:34.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:34 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:34.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:34 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:04:34.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:34 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:34.981 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:34 smithi163 ceph-mon[34520]: purged_snaps scrub starts 2024-05-08T06:04:34.981 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:34 smithi163 ceph-mon[34520]: purged_snaps scrub ok 2024-05-08T06:04:34.981 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:34 smithi163 ceph-mon[34520]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-08T06:04:34.981 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:34 smithi163 ceph-mon[34520]: osd.11 [v2:172.21.15.192:6824/1339159347,v1:172.21.15.192:6825/1339159347] boot 2024-05-08T06:04:34.981 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:34 smithi163 ceph-mon[34520]: osdmap e72: 12 total, 12 up, 12 in 2024-05-08T06:04:34.981 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:34 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T06:04:34.981 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:34 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/2939696022' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T06:04:34.981 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:34 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:34.981 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:34 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:34.981 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:34 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:34.981 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:34 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:34.982 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:34 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:34.982 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:34 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:34.982 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:34 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:34.982 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:34 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:34.982 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:34 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:04:34.982 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:34 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:35.201 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:04:35.893 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:35 smithi123 ceph-mon[29787]: Detected new or changed devices on smithi192 2024-05-08T06:04:35.894 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:35 smithi123 ceph-mon[29787]: Adjusting osd_memory_target on smithi192 to 4272M 2024-05-08T06:04:35.894 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:35 smithi123 ceph-mon[29787]: osdmap e73: 12 total, 12 up, 12 in 2024-05-08T06:04:35.894 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:35 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:04:35.894 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:35 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:35.894 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:35 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:04:35.894 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:35 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:36.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:35 smithi163 ceph-mon[34520]: Detected new or changed devices on smithi192 2024-05-08T06:04:36.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:35 smithi163 ceph-mon[34520]: Adjusting osd_memory_target on smithi192 to 4272M 2024-05-08T06:04:36.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:35 smithi163 ceph-mon[34520]: osdmap e73: 12 total, 12 up, 12 in 2024-05-08T06:04:36.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:35 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:04:36.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:35 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:36.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:35 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:04:36.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:35 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:36.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:35 smithi192 ceph-mon[34664]: Detected new or changed devices on smithi192 2024-05-08T06:04:36.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:35 smithi192 ceph-mon[34664]: Adjusting osd_memory_target on smithi192 to 4272M 2024-05-08T06:04:36.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:35 smithi192 ceph-mon[34664]: osdmap e73: 12 total, 12 up, 12 in 2024-05-08T06:04:36.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:35 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:04:36.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:35 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:36.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:35 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:04:36.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:35 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:36.952 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:36 smithi192 ceph-mon[34664]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:36.954 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:36 smithi123 ceph-mon[29787]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:37.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:36 smithi163 ceph-mon[34520]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:37.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.560+0000 7f3168c00700 1 -- 172.21.15.123:0/2381193978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f316410a4d0 msgr2=0x7f3164116990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:37.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.560+0000 7f3168c00700 1 --2- 172.21.15.123:0/2381193978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f316410a4d0 0x7f3164116990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f315c00b0a0 tx=0x7f315c00b3b0 comp rx=0 tx=0).stop 2024-05-08T06:04:37.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.560+0000 7f3168c00700 1 -- 172.21.15.123:0/2381193978 shutdown_connections 2024-05-08T06:04:37.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.560+0000 7f3168c00700 1 --2- 172.21.15.123:0/2381193978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f316410fac0 0x7f316410fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:37.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.560+0000 7f3168c00700 1 --2- 172.21.15.123:0/2381193978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f316410a4d0 0x7f3164116990 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:37.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.560+0000 7f3168c00700 1 --2- 172.21.15.123:0/2381193978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3164109b10 0x7f3164109f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:37.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.560+0000 7f3168c00700 1 -- 172.21.15.123:0/2381193978 >> 172.21.15.123:0/2381193978 conn(0x7f3164076c70 msgr2=0x7f3164079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:37.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.560+0000 7f3168c00700 1 -- 172.21.15.123:0/2381193978 shutdown_connections 2024-05-08T06:04:37.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.560+0000 7f3168c00700 1 -- 172.21.15.123:0/2381193978 wait complete. 2024-05-08T06:04:37.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.561+0000 7f3168c00700 1 Processor -- start 2024-05-08T06:04:37.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.562+0000 7f3168c00700 1 -- start start 2024-05-08T06:04:37.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.562+0000 7f3168c00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3164109b10 0x7f31641a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:37.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.562+0000 7f3168c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f316410a4d0 0x7f31641a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:37.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.562+0000 7f3168c00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f316410fac0 0x7f31641a4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:37.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.562+0000 7f3168c00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3164119d10 con 0x7f316410a4d0 2024-05-08T06:04:37.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.562+0000 7f3168c00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f3164119b90 con 0x7f316410fac0 2024-05-08T06:04:37.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.562+0000 7f3168c00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3164119e90 con 0x7f3164109b10 2024-05-08T06:04:37.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.562+0000 7f3161400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f316410a4d0 0x7f31641a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:37.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.562+0000 7f3161e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3164109b10 0x7f31641a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:37.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.562+0000 7f3161e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3164109b10 0x7f31641a01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:47018/0 (socket says 172.21.15.123:47018) 2024-05-08T06:04:37.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.562+0000 7f3161400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f316410a4d0 0x7f31641a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48168/0 (socket says 172.21.15.123:48168) 2024-05-08T06:04:37.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.562+0000 7f3161e00700 1 -- 172.21.15.123:0/3005424072 learned_addr learned my addr 172.21.15.123:0/3005424072 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:04:37.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.563+0000 7f3162800700 1 --2- 172.21.15.123:0/3005424072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f316410fac0 0x7f31641a4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:37.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.563+0000 7f3161e00700 1 -- 172.21.15.123:0/3005424072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f316410fac0 msgr2=0x7f31641a4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:37.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.563+0000 7f3161e00700 1 --2- 172.21.15.123:0/3005424072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f316410fac0 0x7f31641a4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:37.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.563+0000 7f3161e00700 1 -- 172.21.15.123:0/3005424072 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f316410a4d0 msgr2=0x7f31641a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:37.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.563+0000 7f3161e00700 1 --2- 172.21.15.123:0/3005424072 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f316410a4d0 0x7f31641a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:37.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.563+0000 7f3161e00700 1 -- 172.21.15.123:0/3005424072 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31641a5160 con 0x7f3164109b10 2024-05-08T06:04:37.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.563+0000 7f3161400700 1 --2- 172.21.15.123:0/3005424072 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f316410a4d0 0x7f31641a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:04:37.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.563+0000 7f3161e00700 1 --2- 172.21.15.123:0/3005424072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3164109b10 0x7f31641a01a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f315400e9a0 tx=0x7f315400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:37.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.564+0000 7f315aa00700 1 -- 172.21.15.123:0/3005424072 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315400ca10 con 0x7f3164109b10 2024-05-08T06:04:37.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.564+0000 7f3168c00700 1 -- 172.21.15.123:0/3005424072 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31641a5450 con 0x7f3164109b10 2024-05-08T06:04:37.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.564+0000 7f315aa00700 1 -- 172.21.15.123:0/3005424072 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f315400cbb0 con 0x7f3164109b10 2024-05-08T06:04:37.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.564+0000 7f3168c00700 1 -- 172.21.15.123:0/3005424072 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31641accd0 con 0x7f3164109b10 2024-05-08T06:04:37.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.564+0000 7f315aa00700 1 -- 172.21.15.123:0/3005424072 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3154018620 con 0x7f3164109b10 2024-05-08T06:04:37.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.565+0000 7f3168c00700 1 -- 172.21.15.123:0/3005424072 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f316406a190 con 0x7f3164109b10 2024-05-08T06:04:37.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.567+0000 7f315aa00700 1 -- 172.21.15.123:0/3005424072 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f315404aab0 con 0x7f3164109b10 2024-05-08T06:04:37.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.569+0000 7f315aa00700 1 --2- 172.21.15.123:0/3005424072 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f31480b2880 0x7f31480b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:37.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.569+0000 7f315aa00700 1 -- 172.21.15.123:0/3005424072 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f31540e1fb0 con 0x7f3164109b10 2024-05-08T06:04:37.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.569+0000 7f3161400700 1 --2- 172.21.15.123:0/3005424072 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f31480b2880 0x7f31480b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:37.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.570+0000 7f3161400700 1 --2- 172.21.15.123:0/3005424072 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f31480b2880 0x7f31480b4d40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f316410f0c0 tx=0x7f3150009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:37.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.571+0000 7f315aa00700 1 -- 172.21.15.123:0/3005424072 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f31540aa0d0 con 0x7f3164109b10 2024-05-08T06:04:37.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.805+0000 7f3168c00700 1 -- 172.21.15.123:0/3005424072 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f316402e510 con 0x7f3164109b10 2024-05-08T06:04:37.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.807+0000 7f315aa00700 1 -- 172.21.15.123:0/3005424072 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+18525 (secure 0 0 0) 0x7f31540a99d0 con 0x7f3164109b10 2024-05-08T06:04:37.808 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:04:37.808 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":73,"fsid":"85cfb2e6-0cff-11ef-bc97-c7b262605968","created":"2024-05-08T05:56:12.544750+0000","modified":"2024-05-08T06:04:34.664405+0000","last_up_change":"2024-05-08T06:04:33.654727+0000","last_in_change":"2024-05-08T06:04:13.809050+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T06:00:17.050518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a14fde9b-bfb1-40a7-a105-305e8981a778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6803","nonce":3608618058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6805","nonce":3608618058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6809","nonce":3608618058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6807","nonce":3608618058}]},"public_addr":"172.21.15.123:6803/3608618058","cluster_addr":"172.21.15.123:6805/3608618058","heartbeat_back_addr":"172.21.15.123:6809/3608618058","heartbeat_front_addr":"172.21.15.123:6807/3608618058","state":["exists","up"]},{"osd":1,"uuid":"e023ae3d-a82e-4bb6-8fef-889dd07fe183","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6801","nonce":1884753449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6803","nonce":1884753449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6807","nonce":1884753449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6805","nonce":1884753449}]},"public_addr":"172.21.15.163:6801/1884753449","cluster_addr":"172.21.15.163:6803/1884753449","heartbeat_back_addr":"172.21.15.163:6807/1884753449","heartbeat_front_addr":"172.21.15.163:6805/1884753449","state":["exists","up"]},{"osd":2,"uuid":"69f4cfca-63cb-4862-9480-d79891b0f338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6801","nonce":4161570404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6803","nonce":4161570404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6807","nonce":4161570404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6805","nonce":4161570404}]},"public_addr":"172.21.15.192:6801/4161570404","cluster_addr":"172.21.15.192:6803/4161570404","heartbeat_back_addr":"172.21.15.192:6807/4161570404","heartbeat_front_addr":"172.21.15.192:6805/4161570404","state":["exists","up"]},{"osd":3,"uuid":"71bb5ecd-c88c-45fb-8b51-0eefb53941c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6811","nonce":1243003258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6813","nonce":1243003258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6817","nonce":1243003258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6815","nonce":1243003258}]},"public_addr":"172.21.15.123:6811/1243003258","cluster_addr":"172.21.15.123:6813/1243003258","heartbeat_back_addr":"172.21.15.123:6817/1243003258","heartbeat_front_addr":"172.21.15.123:6815/1243003258","state":["exists","up"]},{"osd":4,"uuid":"5a7a3462-e74f-4d15-a85c-709b5d423745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6809","nonce":591680273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6811","nonce":591680273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6815","nonce":591680273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6813","nonce":591680273}]},"public_addr":"172.21.15.163:6809/591680273","cluster_addr":"172.21.15.163:6811/591680273","heartbeat_back_addr":"172.21.15.163:6815/591680273","heartbeat_front_addr":"172.21.15.163:6813/591680273","state":["exists","up"]},{"osd":5,"uuid":"85e3f1ab-f8c6-4025-bab9-9ddb6bd3e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6809","nonce":1734821180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6811","nonce":1734821180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6815","nonce":1734821180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6813","nonce":1734821180}]},"public_addr":"172.21.15.192:6809/1734821180","cluster_addr":"172.21.15.192:6811/1734821180","heartbeat_back_addr":"172.21.15.192:6815/1734821180","heartbeat_front_addr":"172.21.15.192:6813/1734821180","state":["exists","up"]},{"osd":6,"uuid":"f21a5624-6f8f-48be-bc6d-0a883d6b7b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6819","nonce":1002138878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6821","nonce":1002138878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6825","nonce":1002138878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6823","nonce":1002138878}]},"public_addr":"172.21.15.123:6819/1002138878","cluster_addr":"172.21.15.123:6821/1002138878","heartbeat_back_addr":"172.21.15.123:6825/1002138878","heartbeat_front_addr":"172.21.15.123:6823/1002138878","state":["exists","up"]},{"osd":7,"uuid":"6eb16729-d260-4041-9829-3e868795b111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6817","nonce":354411978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6819","nonce":354411978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6823","nonce":354411978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6821","nonce":354411978}]},"public_addr":"172.21.15.163:6817/354411978","cluster_addr":"172.21.15.163:6819/354411978","heartbeat_back_addr":"172.21.15.163:6823/354411978","heartbeat_front_addr":"172.21.15.163:6821/354411978","state":["exists","up"]},{"osd":8,"uuid":"122a281e-18b8-4728-97db-458e7432b2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6817","nonce":3669753689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6819","nonce":3669753689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6823","nonce":3669753689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6821","nonce":3669753689}]},"public_addr":"172.21.15.192:6817/3669753689","cluster_addr":"172.21.15.192:6819/3669753689","heartbeat_back_addr":"172.21.15.192:6823/3669753689","heartbeat_front_addr":"172.21.15.192:6821/3669753689","state":["exists","up"]},{"osd":9,"uuid":"9fa022a9-b873-4e07-806c-641a706a150f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6827","nonce":3104436473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6829","nonce":3104436473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6833","nonce":3104436473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6831","nonce":3104436473}]},"public_addr":"172.21.15.123:6827/3104436473","cluster_addr":"172.21.15.123:6829/3104436473","heartbeat_back_addr":"172.21.15.123:6833/3104436473","heartbeat_front_addr":"172.21.15.123:6831/3104436473","state":["exists","up"]},{"osd":10,"uuid":"fb59339b-e11b-4ffc-b202-7f67cb21da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6825","nonce":2628487883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6827","nonce":2628487883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6831","nonce":2628487883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6829","nonce":2628487883}]},"public_addr":"172.21.15.163:6825/2628487883","cluster_addr":"172.21.15.163:6827/2628487883","heartbeat_back_addr":"172.21.15.163:6831/2628487883","heartbeat_front_addr":"172.21.15.163:6829/2628487883","state":["exists","up"]},{"osd":11,"uuid":"be7a5f36-c1be-41d7-80ee-004d0546eb9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6825","nonce":1339159347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6827","nonce":1339159347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6831","nonce":1339159347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6829","nonce":1339159347}]},"public_addr":"172.21.15.192:6825/1339159347","cluster_addr":"172.21.15.192:6827/1339159347","heartbeat_back_addr":"172.21.15.192:6831/1339159347","heartbeat_front_addr":"172.21.15.192:6829/1339159347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:59:24.923234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:59:49.867071+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:00:13.454062+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:00:46.983594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:01:12.820452+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:01:37.451382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:02:13.325181+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:02:39.453436+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:03:03.837509+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:03:39.460579+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:04:06.314082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:04:31.723154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6801/3057688649":"2024-05-09T05:57:30.887936+0000","172.21.15.123:6800/3057688649":"2024-05-09T05:57:30.887936+0000","172.21.15.123:6800/2769724312":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/399884043":"2024-05-09T05:57:30.887936+0000","172.21.15.123:0/3406061897":"2024-05-09T05:57:30.887936+0000","172.21.15.123:0/644457268":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/3055092197":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/343390357":"2024-05-09T05:57:30.887936+0000","172.21.15.123:6801/2769724312":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/2263171562":"2024-05-09T05:56:55.849036+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T06:04:37.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.810+0000 7f3168c00700 1 -- 172.21.15.123:0/3005424072 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f31480b2880 msgr2=0x7f31480b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:37.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.810+0000 7f3168c00700 1 --2- 172.21.15.123:0/3005424072 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f31480b2880 0x7f31480b4d40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f316410f0c0 tx=0x7f3150009260 comp rx=0 tx=0).stop 2024-05-08T06:04:37.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.810+0000 7f3168c00700 1 -- 172.21.15.123:0/3005424072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3164109b10 msgr2=0x7f31641a01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:37.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.810+0000 7f3168c00700 1 --2- 172.21.15.123:0/3005424072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3164109b10 0x7f31641a01a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f315400e9a0 tx=0x7f315400ecb0 comp rx=0 tx=0).stop 2024-05-08T06:04:37.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.810+0000 7f3168c00700 1 -- 172.21.15.123:0/3005424072 shutdown_connections 2024-05-08T06:04:37.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.810+0000 7f3168c00700 1 --2- 172.21.15.123:0/3005424072 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f31480b2880 0x7f31480b4d40 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:37.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.810+0000 7f3168c00700 1 --2- 172.21.15.123:0/3005424072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f316410fac0 0x7f31641a4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:37.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.810+0000 7f3168c00700 1 --2- 172.21.15.123:0/3005424072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3164109b10 0x7f31641a01a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:37.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.810+0000 7f3168c00700 1 --2- 172.21.15.123:0/3005424072 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f316410a4d0 0x7f31641a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:37.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.810+0000 7f3168c00700 1 -- 172.21.15.123:0/3005424072 >> 172.21.15.123:0/3005424072 conn(0x7f3164076c70 msgr2=0x7f3164077250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:37.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.810+0000 7f3168c00700 1 -- 172.21.15.123:0/3005424072 shutdown_connections 2024-05-08T06:04:37.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:37.810+0000 7f3168c00700 1 -- 172.21.15.123:0/3005424072 wait complete. 2024-05-08T06:04:38.690 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T06:00:17.050518+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T06:04:38.690 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-08T06:04:38.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:38 smithi123 ceph-mon[29787]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:38.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:38 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/3005424072' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:04:39.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:38 smithi163 ceph-mon[34520]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:39.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:38 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/3005424072' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:04:39.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:38 smithi192 ceph-mon[34664]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:39.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:38 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/3005424072' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:04:39.133 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:04:40.927 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:40 smithi123 ceph-mon[29787]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:41.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:40 smithi163 ceph-mon[34520]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:41.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:40 smithi192 ceph-mon[34664]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:41.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.467+0000 7faf49e00700 1 -- 172.21.15.123:0/2787921526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf4410b6c0 msgr2=0x7faf4410baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:41.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.467+0000 7faf49e00700 1 --2- 172.21.15.123:0/2787921526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf4410b6c0 0x7faf4410baa0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7faf34009990 tx=0x7faf34009ca0 comp rx=0 tx=0).stop 2024-05-08T06:04:41.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.467+0000 7faf49e00700 1 -- 172.21.15.123:0/2787921526 shutdown_connections 2024-05-08T06:04:41.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.467+0000 7faf49e00700 1 --2- 172.21.15.123:0/2787921526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf4410b6c0 0x7faf4410baa0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:41.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.467+0000 7faf49e00700 1 --2- 172.21.15.123:0/2787921526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf44106080 0x7faf44112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:41.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.467+0000 7faf49e00700 1 --2- 172.21.15.123:0/2787921526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf441056c0 0x7faf44105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:41.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.467+0000 7faf49e00700 1 -- 172.21.15.123:0/2787921526 >> 172.21.15.123:0/2787921526 conn(0x7faf441011b0 msgr2=0x7faf441035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:41.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.467+0000 7faf49e00700 1 -- 172.21.15.123:0/2787921526 shutdown_connections 2024-05-08T06:04:41.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.467+0000 7faf49e00700 1 -- 172.21.15.123:0/2787921526 wait complete. 2024-05-08T06:04:41.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.468+0000 7faf49e00700 1 Processor -- start 2024-05-08T06:04:41.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.468+0000 7faf49e00700 1 -- start start 2024-05-08T06:04:41.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.469+0000 7faf49e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf441056c0 0x7faf4419bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:41.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.469+0000 7faf49e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf44106080 0x7faf4419c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:41.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.469+0000 7faf49e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf4410b6c0 0x7faf441a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:41.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.469+0000 7faf49e00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faf44115900 con 0x7faf4410b6c0 2024-05-08T06:04:41.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.469+0000 7faf49e00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7faf44115780 con 0x7faf44106080 2024-05-08T06:04:41.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.469+0000 7faf49e00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7faf44115a80 con 0x7faf441056c0 2024-05-08T06:04:41.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.469+0000 7faf3be00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf44106080 0x7faf4419c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:41.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.469+0000 7faf42a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf441056c0 0x7faf4419bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:41.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.469+0000 7faf3be00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf44106080 0x7faf4419c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:39128/0 (socket says 172.21.15.123:39128) 2024-05-08T06:04:41.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.469+0000 7faf3be00700 1 -- 172.21.15.123:0/1804820077 learned_addr learned my addr 172.21.15.123:0/1804820077 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:04:41.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.469+0000 7faf43400700 1 --2- 172.21.15.123:0/1804820077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf4410b6c0 0x7faf441a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:41.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.470+0000 7faf3be00700 1 -- 172.21.15.123:0/1804820077 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf441056c0 msgr2=0x7faf4419bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:41.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.470+0000 7faf3be00700 1 --2- 172.21.15.123:0/1804820077 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf441056c0 0x7faf4419bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:41.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.470+0000 7faf3be00700 1 -- 172.21.15.123:0/1804820077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf4410b6c0 msgr2=0x7faf441a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:41.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.470+0000 7faf3be00700 1 --2- 172.21.15.123:0/1804820077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf4410b6c0 0x7faf441a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:41.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.470+0000 7faf3be00700 1 -- 172.21.15.123:0/1804820077 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf441a0de0 con 0x7faf44106080 2024-05-08T06:04:41.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.470+0000 7faf3be00700 1 --2- 172.21.15.123:0/1804820077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf44106080 0x7faf4419c310 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faf2c00d950 tx=0x7faf2c00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:41.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.470+0000 7faf48a00700 1 -- 172.21.15.123:0/1804820077 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2c016070 con 0x7faf44106080 2024-05-08T06:04:41.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.470+0000 7faf49e00700 1 -- 172.21.15.123:0/1804820077 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf441a10d0 con 0x7faf44106080 2024-05-08T06:04:41.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.470+0000 7faf48a00700 1 -- 172.21.15.123:0/1804820077 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf2c009de0 con 0x7faf44106080 2024-05-08T06:04:41.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.470+0000 7faf48a00700 1 -- 172.21.15.123:0/1804820077 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2c020060 con 0x7faf44106080 2024-05-08T06:04:41.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.471+0000 7faf49e00700 1 -- 172.21.15.123:0/1804820077 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf441a8950 con 0x7faf44106080 2024-05-08T06:04:41.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.472+0000 7faf49e00700 1 -- 172.21.15.123:0/1804820077 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf4406a190 con 0x7faf44106080 2024-05-08T06:04:41.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.474+0000 7faf48a00700 1 -- 172.21.15.123:0/1804820077 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7faf2c019070 con 0x7faf44106080 2024-05-08T06:04:41.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.475+0000 7faf48a00700 1 --2- 172.21.15.123:0/1804820077 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7faf280b2900 0x7faf280b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:41.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.476+0000 7faf48a00700 1 -- 172.21.15.123:0/1804820077 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7faf2c0e18e0 con 0x7faf44106080 2024-05-08T06:04:41.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.476+0000 7faf42a00700 1 --2- 172.21.15.123:0/1804820077 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7faf280b2900 0x7faf280b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:41.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.476+0000 7faf42a00700 1 --2- 172.21.15.123:0/1804820077 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7faf280b2900 0x7faf280b4dc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7faf34006010 tx=0x7faf3400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:41.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.481+0000 7faf48a00700 1 -- 172.21.15.123:0/1804820077 <== mon.2 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faf2c0a9a00 con 0x7faf44106080 2024-05-08T06:04:41.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.710+0000 7faf49e00700 1 -- 172.21.15.123:0/1804820077 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faf4402e510 con 0x7faf44106080 2024-05-08T06:04:41.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.711+0000 7faf48a00700 1 -- 172.21.15.123:0/1804820077 <== mon.2 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) ==== 93+0+10 (secure 0 0 0) 0x7faf2c0a9300 con 0x7faf44106080 2024-05-08T06:04:41.712 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-05-08T06:04:41.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.715+0000 7faf49e00700 1 -- 172.21.15.123:0/1804820077 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7faf280b2900 msgr2=0x7faf280b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:41.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.715+0000 7faf49e00700 1 --2- 172.21.15.123:0/1804820077 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7faf280b2900 0x7faf280b4dc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7faf34006010 tx=0x7faf3400b260 comp rx=0 tx=0).stop 2024-05-08T06:04:41.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.715+0000 7faf49e00700 1 -- 172.21.15.123:0/1804820077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf44106080 msgr2=0x7faf4419c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:41.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.715+0000 7faf49e00700 1 --2- 172.21.15.123:0/1804820077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf44106080 0x7faf4419c310 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faf2c00d950 tx=0x7faf2c00dc60 comp rx=0 tx=0).stop 2024-05-08T06:04:41.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.715+0000 7faf49e00700 1 -- 172.21.15.123:0/1804820077 shutdown_connections 2024-05-08T06:04:41.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.715+0000 7faf49e00700 1 --2- 172.21.15.123:0/1804820077 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7faf280b2900 0x7faf280b4dc0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:41.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.715+0000 7faf49e00700 1 --2- 172.21.15.123:0/1804820077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf4410b6c0 0x7faf441a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:41.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.715+0000 7faf49e00700 1 --2- 172.21.15.123:0/1804820077 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf441056c0 0x7faf4419bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:41.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.715+0000 7faf49e00700 1 --2- 172.21.15.123:0/1804820077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf44106080 0x7faf4419c310 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:41.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.715+0000 7faf49e00700 1 -- 172.21.15.123:0/1804820077 >> 172.21.15.123:0/1804820077 conn(0x7faf441011b0 msgr2=0x7faf44101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:41.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.715+0000 7faf49e00700 1 -- 172.21.15.123:0/1804820077 shutdown_connections 2024-05-08T06:04:41.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:41.715+0000 7faf49e00700 1 -- 172.21.15.123:0/1804820077 wait complete. 2024-05-08T06:04:42.580 INFO:tasks.cephadm:Adding mds.a on smithi123 2024-05-08T06:04:42.581 INFO:tasks.cephadm:Adding mds.d on smithi123 2024-05-08T06:04:42.581 INFO:tasks.cephadm:Adding mds.g on smithi123 2024-05-08T06:04:42.581 INFO:tasks.cephadm:Adding mds.j on smithi123 2024-05-08T06:04:42.581 INFO:tasks.cephadm:Adding mds.b on smithi163 2024-05-08T06:04:42.581 INFO:tasks.cephadm:Adding mds.e on smithi163 2024-05-08T06:04:42.581 INFO:tasks.cephadm:Adding mds.h on smithi163 2024-05-08T06:04:42.581 INFO:tasks.cephadm:Adding mds.k on smithi163 2024-05-08T06:04:42.581 INFO:tasks.cephadm:Adding mds.c on smithi192 2024-05-08T06:04:42.581 INFO:tasks.cephadm:Adding mds.f on smithi192 2024-05-08T06:04:42.581 INFO:tasks.cephadm:Adding mds.i on smithi192 2024-05-08T06:04:42.581 INFO:tasks.cephadm:Adding mds.l on smithi192 2024-05-08T06:04:42.581 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph orch apply mds all '12;smithi123=a;smithi123=d;smithi123=g;smithi123=j;smithi163=b;smithi163=e;smithi163=h;smithi163=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l' 2024-05-08T06:04:42.831 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T06:04:42.950 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:42 smithi192 ceph-mon[34664]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:42.950 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:42 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/1804820077' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T06:04:42.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:42 smithi123 ceph-mon[29787]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:42.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:42 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/1804820077' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T06:04:43.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:42 smithi163 ceph-mon[34520]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:43.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:42 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/1804820077' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T06:04:44.436 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.434+0000 7f75d4524700 1 -- 172.21.15.192:0/3220046786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75cc103500 msgr2=0x7f75cc103920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:44.436 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.434+0000 7f75d4524700 1 --2- 172.21.15.192:0/3220046786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75cc103500 0x7f75cc103920 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f75bc009990 tx=0x7f75bc009ca0 comp rx=0 tx=0).stop 2024-05-08T06:04:44.436 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.434+0000 7f75d4524700 1 -- 172.21.15.192:0/3220046786 shutdown_connections 2024-05-08T06:04:44.437 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.434+0000 7f75d4524700 1 --2- 172.21.15.192:0/3220046786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75cc1050a0 0x7f75cc10b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:44.437 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.434+0000 7f75d4524700 1 --2- 172.21.15.192:0/3220046786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75cc103500 0x7f75cc103920 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:44.437 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.434+0000 7f75d4524700 1 --2- 172.21.15.192:0/3220046786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75cc104700 0x7f75cc104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:44.437 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.434+0000 7f75d4524700 1 -- 172.21.15.192:0/3220046786 >> 172.21.15.192:0/3220046786 conn(0x7f75cc0feaa0 msgr2=0x7f75cc100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:44.437 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.434+0000 7f75d4524700 1 -- 172.21.15.192:0/3220046786 shutdown_connections 2024-05-08T06:04:44.437 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.434+0000 7f75d4524700 1 -- 172.21.15.192:0/3220046786 wait complete. 2024-05-08T06:04:44.437 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.435+0000 7f75d4524700 1 Processor -- start 2024-05-08T06:04:44.437 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.435+0000 7f75d4524700 1 -- start start 2024-05-08T06:04:44.437 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.435+0000 7f75d4524700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75cc103500 0x7f75cc199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:44.437 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.435+0000 7f75d4524700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75cc104700 0x7f75cc199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:44.437 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.435+0000 7f75d4524700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75cc1050a0 0x7f75cc1a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:44.437 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.435+0000 7f75d4524700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f75cc10f070 con 0x7f75cc103500 2024-05-08T06:04:44.437 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.435+0000 7f75d4524700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f75cc10eef0 con 0x7f75cc1050a0 2024-05-08T06:04:44.437 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.435+0000 7f75d4524700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f75cc10f1f0 con 0x7f75cc104700 2024-05-08T06:04:44.438 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.435+0000 7f75d1abf700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75cc104700 0x7f75cc199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:44.438 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.435+0000 7f75d1abf700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75cc104700 0x7f75cc199cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:53946/0 (socket says 172.21.15.192:53946) 2024-05-08T06:04:44.438 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.435+0000 7f75d1abf700 1 -- 172.21.15.192:0/1947666021 learned_addr learned my addr 172.21.15.192:0/1947666021 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-08T06:04:44.438 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.435+0000 7f75d2ac1700 1 --2- 172.21.15.192:0/1947666021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75cc1050a0 0x7f75cc1a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:44.438 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.436+0000 7f75d1abf700 1 -- 172.21.15.192:0/1947666021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75cc1050a0 msgr2=0x7f75cc1a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:44.438 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.436+0000 7f75d1abf700 1 --2- 172.21.15.192:0/1947666021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75cc1050a0 0x7f75cc1a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:44.438 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.436+0000 7f75d1abf700 1 -- 172.21.15.192:0/1947666021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75cc103500 msgr2=0x7f75cc199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:44.438 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.436+0000 7f75d1abf700 1 --2- 172.21.15.192:0/1947666021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75cc103500 0x7f75cc199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:44.438 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.436+0000 7f75d1abf700 1 -- 172.21.15.192:0/1947666021 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75cc1a13f0 con 0x7f75cc104700 2024-05-08T06:04:44.438 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.436+0000 7f75d1abf700 1 --2- 172.21.15.192:0/1947666021 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75cc104700 0x7f75cc199cd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f75c800b560 tx=0x7f75c800b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:44.438 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.436+0000 7f75c37fe700 1 -- 172.21.15.192:0/1947666021 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8015020 con 0x7f75cc104700 2024-05-08T06:04:44.438 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.436+0000 7f75c37fe700 1 -- 172.21.15.192:0/1947666021 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75c8011ce0 con 0x7f75cc104700 2024-05-08T06:04:44.438 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.436+0000 7f75d4524700 1 -- 172.21.15.192:0/1947666021 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75cc1a16e0 con 0x7f75cc104700 2024-05-08T06:04:44.438 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.436+0000 7f75c37fe700 1 -- 172.21.15.192:0/1947666021 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8010d20 con 0x7f75cc104700 2024-05-08T06:04:44.438 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.436+0000 7f75d4524700 1 -- 172.21.15.192:0/1947666021 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75cc1a1cf0 con 0x7f75cc104700 2024-05-08T06:04:44.439 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.437+0000 7f75d4524700 1 -- 172.21.15.192:0/1947666021 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75cc06a190 con 0x7f75cc104700 2024-05-08T06:04:44.439 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.438+0000 7f75c37fe700 1 -- 172.21.15.192:0/1947666021 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f75c8018020 con 0x7f75cc104700 2024-05-08T06:04:44.440 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.439+0000 7f75c37fe700 1 --2- 172.21.15.192:0/1947666021 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f75b80b2900 0x7f75b80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:44.440 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.439+0000 7f75c37fe700 1 -- 172.21.15.192:0/1947666021 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f75c80e19c0 con 0x7f75cc104700 2024-05-08T06:04:44.440 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.439+0000 7f75d22c0700 1 --2- 172.21.15.192:0/1947666021 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f75b80b2900 0x7f75b80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:44.440 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.440+0000 7f75d22c0700 1 --2- 172.21.15.192:0/1947666021 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f75b80b2900 0x7f75b80b4dc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f75bc0098e0 tx=0x7f75bc003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:44.442 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.442+0000 7f75c37fe700 1 -- 172.21.15.192:0/1947666021 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f75c801f020 con 0x7f75cc104700 2024-05-08T06:04:44.655 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.654+0000 7f75d4524700 1 -- 172.21.15.192:0/1947666021 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi123=a;smithi123=d;smithi123=g;smithi123=j;smithi163=b;smithi163=e;smithi163=h;smithi163=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l", "target": ["mon-mgr", ""]}) -- 0x7f75cc063ae0 con 0x7f75b80b2900 2024-05-08T06:04:44.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.672+0000 7f75c37fe700 1 -- 172.21.15.192:0/1947666021 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f75cc063ae0 con 0x7f75b80b2900 2024-05-08T06:04:44.672 INFO:teuthology.orchestra.run.smithi192.stdout:Scheduled mds.all update... 2024-05-08T06:04:44.674 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.674+0000 7f75d4524700 1 -- 172.21.15.192:0/1947666021 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f75b80b2900 msgr2=0x7f75b80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:44.674 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.674+0000 7f75d4524700 1 --2- 172.21.15.192:0/1947666021 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f75b80b2900 0x7f75b80b4dc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f75bc0098e0 tx=0x7f75bc003860 comp rx=0 tx=0).stop 2024-05-08T06:04:44.674 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.674+0000 7f75d4524700 1 -- 172.21.15.192:0/1947666021 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75cc104700 msgr2=0x7f75cc199cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:44.674 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.674+0000 7f75d4524700 1 --2- 172.21.15.192:0/1947666021 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75cc104700 0x7f75cc199cd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f75c800b560 tx=0x7f75c800b870 comp rx=0 tx=0).stop 2024-05-08T06:04:44.674 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.674+0000 7f75d4524700 1 -- 172.21.15.192:0/1947666021 shutdown_connections 2024-05-08T06:04:44.674 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.674+0000 7f75d4524700 1 --2- 172.21.15.192:0/1947666021 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f75b80b2900 0x7f75b80b4dc0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:44.675 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.674+0000 7f75d4524700 1 --2- 172.21.15.192:0/1947666021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75cc103500 0x7f75cc199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:44.675 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.674+0000 7f75d4524700 1 --2- 172.21.15.192:0/1947666021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75cc1050a0 0x7f75cc1a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:44.675 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.674+0000 7f75d4524700 1 --2- 172.21.15.192:0/1947666021 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75cc104700 0x7f75cc199cd0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:44.675 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.674+0000 7f75d4524700 1 -- 172.21.15.192:0/1947666021 >> 172.21.15.192:0/1947666021 conn(0x7f75cc0feaa0 msgr2=0x7f75cc107b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:44.675 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.674+0000 7f75d4524700 1 -- 172.21.15.192:0/1947666021 shutdown_connections 2024-05-08T06:04:44.675 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-08T06:04:44.674+0000 7f75d4524700 1 -- 172.21.15.192:0/1947666021 wait complete. 2024-05-08T06:04:44.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:44 smithi123 ceph-mon[29787]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:44.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:44 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:45.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:44 smithi163 ceph-mon[34520]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:45.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:44 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:45.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:44 smithi192 ceph-mon[34664]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:45.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:44 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:45.363 DEBUG:teuthology.orchestra.run.smithi123:mds.a> sudo journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mds.a.service 2024-05-08T06:04:45.374 DEBUG:teuthology.orchestra.run.smithi123:mds.d> sudo journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mds.d.service 2024-05-08T06:04:45.418 DEBUG:teuthology.orchestra.run.smithi123:mds.g> sudo journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mds.g.service 2024-05-08T06:04:45.464 DEBUG:teuthology.orchestra.run.smithi123:mds.j> sudo journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mds.j.service 2024-05-08T06:04:45.509 DEBUG:teuthology.orchestra.run.smithi163:mds.b> sudo journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mds.b.service 2024-05-08T06:04:45.511 DEBUG:teuthology.orchestra.run.smithi163:mds.e> sudo journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mds.e.service 2024-05-08T06:04:45.554 DEBUG:teuthology.orchestra.run.smithi163:mds.h> sudo journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mds.h.service 2024-05-08T06:04:45.597 DEBUG:teuthology.orchestra.run.smithi163:mds.k> sudo journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mds.k.service 2024-05-08T06:04:45.639 DEBUG:teuthology.orchestra.run.smithi192:mds.c> sudo journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mds.c.service 2024-05-08T06:04:45.642 DEBUG:teuthology.orchestra.run.smithi192:mds.f> sudo journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mds.f.service 2024-05-08T06:04:45.685 DEBUG:teuthology.orchestra.run.smithi192:mds.i> sudo journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mds.i.service 2024-05-08T06:04:45.727 DEBUG:teuthology.orchestra.run.smithi192:mds.l> sudo journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mds.l.service 2024-05-08T06:04:45.770 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-08T06:04:45.771 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T06:04:45.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:45 smithi163 ceph-mon[34520]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi123=a;smithi123=d;smithi123=g;smithi123=j;smithi163=b;smithi163=e;smithi163=h;smithi163=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:04:45.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:45 smithi163 ceph-mon[34520]: Saving service mds.all spec with placement smithi123=a;smithi123=d;smithi123=g;smithi123=j;smithi163=b;smithi163=e;smithi163=h;smithi163=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l;count:12 2024-05-08T06:04:45.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:45 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:04:45.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:45 smithi123 ceph-mon[29787]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi123=a;smithi123=d;smithi123=g;smithi123=j;smithi163=b;smithi163=e;smithi163=h;smithi163=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:04:45.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:45 smithi123 ceph-mon[29787]: Saving service mds.all spec with placement smithi123=a;smithi123=d;smithi123=g;smithi123=j;smithi163=b;smithi163=e;smithi163=h;smithi163=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l;count:12 2024-05-08T06:04:45.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:45 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:04:46.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:45 smithi192 ceph-mon[34664]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi123=a;smithi123=d;smithi123=g;smithi123=j;smithi163=b;smithi163=e;smithi163=h;smithi163=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:04:46.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:45 smithi192 ceph-mon[34664]: Saving service mds.all spec with placement smithi123=a;smithi123=d;smithi123=g;smithi123=j;smithi163=b;smithi163=e;smithi163=h;smithi163=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l;count:12 2024-05-08T06:04:46.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:45 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:04:46.197 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:04:46.794 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:46 smithi192 ceph-mon[34664]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:46.794 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:46 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:46.795 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:46 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:04:46.795 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:46 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:46.795 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:46 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T06:04:46.795 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:46 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T06:04:46.795 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:46 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:46.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:46 smithi123 ceph-mon[29787]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:46.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:46 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:46.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:46 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:04:46.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:46 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:46.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:46 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T06:04:46.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:46 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T06:04:46.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:46 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:47.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:46 smithi163 ceph-mon[34520]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:47.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:46 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:47.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:46 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:04:47.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:46 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:47.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:46 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T06:04:47.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:46 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T06:04:47.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:46 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:47.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:47 smithi123 ceph-mon[29787]: Deploying daemon mds.l on smithi192 2024-05-08T06:04:47.976 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:47 smithi192 ceph-mon[34664]: Deploying daemon mds.l on smithi192 2024-05-08T06:04:48.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:47 smithi163 ceph-mon[34520]: Deploying daemon mds.l on smithi192 2024-05-08T06:04:48.301 INFO:journalctl@ceph.mds.l.smithi192.stdout:May 08 06:04:48 smithi192 systemd[1]: Starting Ceph mds.l for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T06:04:48.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.511+0000 7f8e3b200700 1 -- 172.21.15.123:0/201557858 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e3c103fc0 msgr2=0x7f8e3c108010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:48.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.511+0000 7f8e3b200700 1 --2- 172.21.15.123:0/201557858 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e3c103fc0 0x7f8e3c108010 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8e280099c0 tx=0x7f8e28009cd0 comp rx=0 tx=0).stop 2024-05-08T06:04:48.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.511+0000 7f8e3b200700 1 -- 172.21.15.123:0/201557858 shutdown_connections 2024-05-08T06:04:48.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.511+0000 7f8e3b200700 1 --2- 172.21.15.123:0/201557858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e3c1086f0 0x7f8e3c1102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:48.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.511+0000 7f8e3b200700 1 --2- 172.21.15.123:0/201557858 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e3c103610 0x7f8e3c1039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:48.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.511+0000 7f8e3b200700 1 --2- 172.21.15.123:0/201557858 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e3c103fc0 0x7f8e3c108010 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:48.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.511+0000 7f8e3b200700 1 -- 172.21.15.123:0/201557858 >> 172.21.15.123:0/201557858 conn(0x7f8e3c0feec0 msgr2=0x7f8e3c1012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:48.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.511+0000 7f8e3b200700 1 -- 172.21.15.123:0/201557858 shutdown_connections 2024-05-08T06:04:48.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.511+0000 7f8e3b200700 1 -- 172.21.15.123:0/201557858 wait complete. 2024-05-08T06:04:48.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.512+0000 7f8e3b200700 1 Processor -- start 2024-05-08T06:04:48.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.512+0000 7f8e3b200700 1 -- start start 2024-05-08T06:04:48.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.513+0000 7f8e3b200700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e3c103610 0x7f8e3c19f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:48.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.513+0000 7f8e3b200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e3c103fc0 0x7f8e3c19fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:48.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.513+0000 7f8e3b200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e3c1086f0 0x7f8e3c199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:48.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.513+0000 7f8e3b200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3c113560 con 0x7f8e3c103fc0 2024-05-08T06:04:48.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.513+0000 7f8e3b200700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3c1133e0 con 0x7f8e3c1086f0 2024-05-08T06:04:48.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.513+0000 7f8e3b200700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3c1136e0 con 0x7f8e3c103610 2024-05-08T06:04:48.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.513+0000 7f8e39e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e3c103610 0x7f8e3c19f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:48.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.513+0000 7f8e39e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e3c103610 0x7f8e3c19f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:38650/0 (socket says 172.21.15.123:38650) 2024-05-08T06:04:48.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.513+0000 7f8e39e00700 1 -- 172.21.15.123:0/3128892584 learned_addr learned my addr 172.21.15.123:0/3128892584 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:04:48.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.513+0000 7f8e39400700 1 --2- 172.21.15.123:0/3128892584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e3c103fc0 0x7f8e3c19fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:48.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.513+0000 7f8e3a800700 1 --2- 172.21.15.123:0/3128892584 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e3c1086f0 0x7f8e3c199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:48.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.513+0000 7f8e39e00700 1 -- 172.21.15.123:0/3128892584 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e3c1086f0 msgr2=0x7f8e3c199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:48.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.513+0000 7f8e39e00700 1 --2- 172.21.15.123:0/3128892584 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e3c1086f0 0x7f8e3c199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:48.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.513+0000 7f8e39e00700 1 -- 172.21.15.123:0/3128892584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e3c103fc0 msgr2=0x7f8e3c19fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:48.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.513+0000 7f8e39e00700 1 --2- 172.21.15.123:0/3128892584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e3c103fc0 0x7f8e3c19fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:48.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.513+0000 7f8e39e00700 1 -- 172.21.15.123:0/3128892584 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e3c19a060 con 0x7f8e3c103610 2024-05-08T06:04:48.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.513+0000 7f8e39400700 1 --2- 172.21.15.123:0/3128892584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e3c103fc0 0x7f8e3c19fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:04:48.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.514+0000 7f8e39e00700 1 --2- 172.21.15.123:0/3128892584 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e3c103610 0x7f8e3c19f6f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c00d950 tx=0x7f8e2c00dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:48.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.514+0000 7f8e32a00700 1 -- 172.21.15.123:0/3128892584 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c016070 con 0x7f8e3c103610 2024-05-08T06:04:48.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.514+0000 7f8e32a00700 1 -- 172.21.15.123:0/3128892584 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e2c010460 con 0x7f8e3c103610 2024-05-08T06:04:48.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.514+0000 7f8e3b200700 1 -- 172.21.15.123:0/3128892584 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e3c19a350 con 0x7f8e3c103610 2024-05-08T06:04:48.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.514+0000 7f8e32a00700 1 -- 172.21.15.123:0/3128892584 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c00f620 con 0x7f8e3c103610 2024-05-08T06:04:48.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.514+0000 7f8e3b200700 1 -- 172.21.15.123:0/3128892584 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e3c1a65c0 con 0x7f8e3c103610 2024-05-08T06:04:48.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.515+0000 7f8e3b200700 1 -- 172.21.15.123:0/3128892584 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e3c06a190 con 0x7f8e3c103610 2024-05-08T06:04:48.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.517+0000 7f8e32a00700 1 -- 172.21.15.123:0/3128892584 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f8e2c019070 con 0x7f8e3c103610 2024-05-08T06:04:48.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.519+0000 7f8e32a00700 1 --2- 172.21.15.123:0/3128892584 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f8e200b2900 0x7f8e200b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:48.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.519+0000 7f8e32a00700 1 -- 172.21.15.123:0/3128892584 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f8e2c0e1a00 con 0x7f8e3c103610 2024-05-08T06:04:48.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.519+0000 7f8e39400700 1 --2- 172.21.15.123:0/3128892584 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f8e200b2900 0x7f8e200b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:48.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.520+0000 7f8e39400700 1 --2- 172.21.15.123:0/3128892584 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f8e200b2900 0x7f8e200b4dc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8e28000c00 tx=0x7f8e28005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:48.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.522+0000 7f8e32a00700 1 -- 172.21.15.123:0/3128892584 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8e2c0a9b20 con 0x7f8e3c103610 2024-05-08T06:04:48.652 INFO:journalctl@ceph.mds.l.smithi192.stdout:May 08 06:04:48 smithi192 podman[64807]: 2024-05-08 06:04:48.259701371 +0000 UTC m=+0.017997614 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:04:48.652 INFO:journalctl@ceph.mds.l.smithi192.stdout:May 08 06:04:48 smithi192 podman[64807]: 2024-05-08 06:04:48.382361336 +0000 UTC m=+0.140657564 container create 9f56e97b86fe48a424c5af5d83b46338e4c82ba64f856a35248c1c0da853edd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-l, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-08T06:04:48.652 INFO:journalctl@ceph.mds.l.smithi192.stdout:May 08 06:04:48 smithi192 podman[64807]: 2024-05-08 06:04:48.576820201 +0000 UTC m=+0.335116430 container init 9f56e97b86fe48a424c5af5d83b46338e4c82ba64f856a35248c1c0da853edd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-l, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True) 2024-05-08T06:04:48.652 INFO:journalctl@ceph.mds.l.smithi192.stdout:May 08 06:04:48 smithi192 podman[64807]: 2024-05-08 06:04:48.579572366 +0000 UTC m=+0.337868603 container start 9f56e97b86fe48a424c5af5d83b46338e4c82ba64f856a35248c1c0da853edd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507) 2024-05-08T06:04:48.652 INFO:journalctl@ceph.mds.l.smithi192.stdout:May 08 06:04:48 smithi192 ceph-mds[64826]: -- 172.21.15.192:0/3988118585 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56265fbcba40 con 0x56265fb9e800 2024-05-08T06:04:48.652 INFO:journalctl@ceph.mds.l.smithi192.stdout:May 08 06:04:48 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-l[64822]: starting mds.l at 2024-05-08T06:04:48.652 INFO:journalctl@ceph.mds.l.smithi192.stdout:May 08 06:04:48 smithi192 bash[64807]: 9f56e97b86fe48a424c5af5d83b46338e4c82ba64f856a35248c1c0da853edd3 2024-05-08T06:04:48.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.821+0000 7f8e3b200700 1 -- 172.21.15.123:0/3128892584 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e3c02e510 con 0x7f8e3c103610 2024-05-08T06:04:48.822 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:48 smithi123 ceph-mon[29787]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:48.822 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:48 smithi123 ceph-mon[29787]: mds.? [v2:172.21.15.192:6832/175423115,v1:172.21.15.192:6833/175423115] up:boot 2024-05-08T06:04:48.822 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:48 smithi123 ceph-mon[29787]: fsmap 1 up:standby 2024-05-08T06:04:48.822 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:48 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T06:04:48.822 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:48 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:48.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.822+0000 7f8e32a00700 1 -- 172.21.15.123:0/3128892584 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1393 (secure 0 0 0) 0x7f8e2c0a9420 con 0x7f8e3c103610 2024-05-08T06:04:48.823 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:04:48.823 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/175423115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":175423115},{"type":"v1","addr":"172.21.15.192:6833","nonce":175423115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-08T06:04:48.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.825+0000 7f8e3b200700 1 -- 172.21.15.123:0/3128892584 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f8e200b2900 msgr2=0x7f8e200b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:48.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.825+0000 7f8e3b200700 1 --2- 172.21.15.123:0/3128892584 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f8e200b2900 0x7f8e200b4dc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8e28000c00 tx=0x7f8e28005ab0 comp rx=0 tx=0).stop 2024-05-08T06:04:48.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.825+0000 7f8e3b200700 1 -- 172.21.15.123:0/3128892584 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e3c103610 msgr2=0x7f8e3c19f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:48.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.825+0000 7f8e3b200700 1 --2- 172.21.15.123:0/3128892584 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e3c103610 0x7f8e3c19f6f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c00d950 tx=0x7f8e2c00dd10 comp rx=0 tx=0).stop 2024-05-08T06:04:48.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.825+0000 7f8e3b200700 1 -- 172.21.15.123:0/3128892584 shutdown_connections 2024-05-08T06:04:48.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.825+0000 7f8e3b200700 1 --2- 172.21.15.123:0/3128892584 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f8e200b2900 0x7f8e200b4dc0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:48.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.825+0000 7f8e3b200700 1 --2- 172.21.15.123:0/3128892584 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e3c1086f0 0x7f8e3c199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:48.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.825+0000 7f8e3b200700 1 --2- 172.21.15.123:0/3128892584 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e3c103610 0x7f8e3c19f6f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:48.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.825+0000 7f8e3b200700 1 --2- 172.21.15.123:0/3128892584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e3c103fc0 0x7f8e3c19fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:48.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.825+0000 7f8e3b200700 1 -- 172.21.15.123:0/3128892584 >> 172.21.15.123:0/3128892584 conn(0x7f8e3c0feec0 msgr2=0x7f8e3c101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:48.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.825+0000 7f8e3b200700 1 -- 172.21.15.123:0/3128892584 shutdown_connections 2024-05-08T06:04:48.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:48.826+0000 7f8e3b200700 1 -- 172.21.15.123:0/3128892584 wait complete. 2024-05-08T06:04:48.827 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 2 2024-05-08T06:04:49.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:48 smithi163 ceph-mon[34520]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:49.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:48 smithi163 ceph-mon[34520]: mds.? [v2:172.21.15.192:6832/175423115,v1:172.21.15.192:6833/175423115] up:boot 2024-05-08T06:04:49.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:48 smithi163 ceph-mon[34520]: fsmap 1 up:standby 2024-05-08T06:04:49.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:48 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T06:04:49.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:48 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:49.050 INFO:journalctl@ceph.mds.l.smithi192.stdout:May 08 06:04:48 smithi192 systemd[1]: Started Ceph mds.l for 85cfb2e6-0cff-11ef-bc97-c7b262605968. 2024-05-08T06:04:49.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:48 smithi192 ceph-mon[34664]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:49.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:48 smithi192 ceph-mon[34664]: mds.? [v2:172.21.15.192:6832/175423115,v1:172.21.15.192:6833/175423115] up:boot 2024-05-08T06:04:49.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:48 smithi192 ceph-mon[34664]: fsmap 1 up:standby 2024-05-08T06:04:49.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:48 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T06:04:49.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:48 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:49.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:49 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:49.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:49 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:49.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:49 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T06:04:49.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:49 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T06:04:49.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:49 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:49.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:49 smithi123 ceph-mon[29787]: Deploying daemon mds.k on smithi163 2024-05-08T06:04:49.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:49 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/3128892584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:04:50.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:49 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:50.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:49 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:50.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:49 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T06:04:50.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:49 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T06:04:50.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:49 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:50.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:49 smithi163 ceph-mon[34520]: Deploying daemon mds.k on smithi163 2024-05-08T06:04:50.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:49 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/3128892584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:04:50.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:49 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:50.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:49 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:50.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:49 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T06:04:50.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:49 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T06:04:50.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:49 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:50.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:49 smithi192 ceph-mon[34664]: Deploying daemon mds.k on smithi163 2024-05-08T06:04:50.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:49 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/3128892584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:04:51.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:50 smithi163 ceph-mon[34520]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:51.038 INFO:journalctl@ceph.mds.k.smithi163.stdout:May 08 06:04:51 smithi163 systemd[1]: Starting Ceph mds.k for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T06:04:51.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:50 smithi192 ceph-mon[34664]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:51.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:50 smithi123 ceph-mon[29787]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:51.474 INFO:journalctl@ceph.mds.k.smithi163.stdout:May 08 06:04:51 smithi163 podman[64083]: 2024-05-08 06:04:51.212740456 +0000 UTC m=+0.098403486 container create aafe0eeceb55efafd793cde31ed4c999b597055d96977065ebf905adb10c3e4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507) 2024-05-08T06:04:51.474 INFO:journalctl@ceph.mds.k.smithi163.stdout:May 08 06:04:51 smithi163 podman[64083]: 2024-05-08 06:04:51.132924418 +0000 UTC m=+0.018587468 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:04:51.474 INFO:journalctl@ceph.mds.k.smithi163.stdout:May 08 06:04:51 smithi163 podman[64083]: 2024-05-08 06:04:51.448647195 +0000 UTC m=+0.334310226 container init aafe0eeceb55efafd793cde31ed4c999b597055d96977065ebf905adb10c3e4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-k, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2) 2024-05-08T06:04:51.474 INFO:journalctl@ceph.mds.k.smithi163.stdout:May 08 06:04:51 smithi163 podman[64083]: 2024-05-08 06:04:51.451142478 +0000 UTC m=+0.336805505 container start aafe0eeceb55efafd793cde31ed4c999b597055d96977065ebf905adb10c3e4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-08T06:04:51.743 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T06:04:51.788 INFO:journalctl@ceph.mds.k.smithi163.stdout:May 08 06:04:51 smithi163 ceph-mds[64102]: -- 172.21.15.163:0/682685517 <== mon.2 v2:172.21.15.163:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556f8a403a40 con 0x556f8a3d6800 2024-05-08T06:04:51.788 INFO:journalctl@ceph.mds.k.smithi163.stdout:May 08 06:04:51 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-k[64098]: starting mds.k at 2024-05-08T06:04:51.788 INFO:journalctl@ceph.mds.k.smithi163.stdout:May 08 06:04:51 smithi163 bash[64083]: aafe0eeceb55efafd793cde31ed4c999b597055d96977065ebf905adb10c3e4e 2024-05-08T06:04:51.788 INFO:journalctl@ceph.mds.k.smithi163.stdout:May 08 06:04:51 smithi163 systemd[1]: Started Ceph mds.k for 85cfb2e6-0cff-11ef-bc97-c7b262605968. 2024-05-08T06:04:52.159 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:04:52.877 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:52 smithi163 ceph-mon[34520]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:52.878 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:52 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:52.878 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:52 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:52.878 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:52 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:52.878 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:52 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T06:04:52.878 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:52 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T06:04:52.878 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:52 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:52.878 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:52 smithi163 ceph-mon[34520]: Deploying daemon mds.j on smithi123 2024-05-08T06:04:52.878 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:52 smithi163 ceph-mon[34520]: mds.? [v2:172.21.15.163:6832/3263009472,v1:172.21.15.163:6833/3263009472] up:boot 2024-05-08T06:04:52.878 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:52 smithi163 ceph-mon[34520]: fsmap 2 up:standby 2024-05-08T06:04:52.878 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:52 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T06:04:52.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:52 smithi123 ceph-mon[29787]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:52.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:52 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:52.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:52 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:52.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:52 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:52.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:52 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T06:04:52.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:52 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T06:04:52.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:52 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:52.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:52 smithi123 ceph-mon[29787]: Deploying daemon mds.j on smithi123 2024-05-08T06:04:52.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:52 smithi123 ceph-mon[29787]: mds.? [v2:172.21.15.163:6832/3263009472,v1:172.21.15.163:6833/3263009472] up:boot 2024-05-08T06:04:52.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:52 smithi123 ceph-mon[29787]: fsmap 2 up:standby 2024-05-08T06:04:52.957 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:52 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T06:04:53.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:52 smithi192 ceph-mon[34664]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:53.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:52 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:53.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:52 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:53.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:52 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:53.052 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:52 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T06:04:53.052 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:52 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T06:04:53.052 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:52 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:53.052 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:52 smithi192 ceph-mon[34664]: Deploying daemon mds.j on smithi123 2024-05-08T06:04:53.052 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:52 smithi192 ceph-mon[34664]: mds.? [v2:172.21.15.163:6832/3263009472,v1:172.21.15.163:6833/3263009472] up:boot 2024-05-08T06:04:53.052 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:52 smithi192 ceph-mon[34664]: fsmap 2 up:standby 2024-05-08T06:04:53.052 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:52 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T06:04:54.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:54 smithi123 ceph-mon[29787]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:55.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:54 smithi163 ceph-mon[34520]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:55.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:54 smithi192 ceph-mon[34664]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:55.516 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:04:55.517 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/175423115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":175423115},{"type":"v1","addr":"172.21.15.192:6833","nonce":175423115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/3263009472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3263009472},{"type":"v1","addr":"172.21.15.163:6833","nonce":3263009472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-08T06:04:55.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.075+0000 7f893a600700 1 -- 172.21.15.123:0/2386095950 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f893410b6e0 msgr2=0x7f893410bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:55.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.075+0000 7f893a600700 1 --2- 172.21.15.123:0/2386095950 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f893410b6e0 0x7f893410bac0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8924009990 tx=0x7f8924009ca0 comp rx=0 tx=0).stop 2024-05-08T06:04:55.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.075+0000 7f893a600700 1 -- 172.21.15.123:0/2386095950 shutdown_connections 2024-05-08T06:04:55.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.075+0000 7f893a600700 1 --2- 172.21.15.123:0/2386095950 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89341060a0 0x7f89341125b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:55.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.075+0000 7f893a600700 1 --2- 172.21.15.123:0/2386095950 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f893410b6e0 0x7f893410bac0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:55.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.075+0000 7f893a600700 1 --2- 172.21.15.123:0/2386095950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89341056e0 0x7f8934105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:55.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.075+0000 7f893a600700 1 -- 172.21.15.123:0/2386095950 >> 172.21.15.123:0/2386095950 conn(0x7f89341011d0 msgr2=0x7f89341035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:55.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.075+0000 7f893a600700 1 -- 172.21.15.123:0/2386095950 shutdown_connections 2024-05-08T06:04:55.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.075+0000 7f893a600700 1 -- 172.21.15.123:0/2386095950 wait complete. 2024-05-08T06:04:55.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.076+0000 7f893a600700 1 Processor -- start 2024-05-08T06:04:55.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.077+0000 7f893a600700 1 -- start start 2024-05-08T06:04:55.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.077+0000 7f893a600700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89341056e0 0x7f893419bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:55.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.077+0000 7f893a600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89341060a0 0x7f893419c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:55.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.077+0000 7f893a600700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f893410b6e0 0x7f89341a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:55.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.077+0000 7f893a600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f89341158e0 con 0x7f89341060a0 2024-05-08T06:04:55.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.077+0000 7f893a600700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8934115760 con 0x7f893410b6e0 2024-05-08T06:04:55.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.077+0000 7f893a600700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8934115a60 con 0x7f89341056e0 2024-05-08T06:04:55.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.078+0000 7f892be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89341060a0 0x7f893419c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:55.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.078+0000 7f8933e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f893410b6e0 0x7f89341a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:55.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.078+0000 7f8933400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89341056e0 0x7f893419bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:55.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.078+0000 7f8933e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f893410b6e0 0x7f89341a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:49246/0 (socket says 172.21.15.123:49246) 2024-05-08T06:04:55.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.078+0000 7f8933e00700 1 -- 172.21.15.123:0/3862843431 learned_addr learned my addr 172.21.15.123:0/3862843431 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:04:55.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.078+0000 7f8933400700 1 -- 172.21.15.123:0/3862843431 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f893410b6e0 msgr2=0x7f89341a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:55.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.078+0000 7f8933400700 1 --2- 172.21.15.123:0/3862843431 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f893410b6e0 0x7f89341a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:55.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.078+0000 7f8933400700 1 -- 172.21.15.123:0/3862843431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89341060a0 msgr2=0x7f893419c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:55.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.078+0000 7f8933400700 1 --2- 172.21.15.123:0/3862843431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89341060a0 0x7f893419c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:55.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.078+0000 7f8933400700 1 -- 172.21.15.123:0/3862843431 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89341a0d80 con 0x7f89341056e0 2024-05-08T06:04:55.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.078+0000 7f892be00700 1 --2- 172.21.15.123:0/3862843431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89341060a0 0x7f893419c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:04:55.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.078+0000 7f8933400700 1 --2- 172.21.15.123:0/3862843431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89341056e0 0x7f893419bd70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f89240059d0 tx=0x7f89240047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:55.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.079+0000 7f8930c00700 1 -- 172.21.15.123:0/3862843431 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8924027070 con 0x7f89341056e0 2024-05-08T06:04:55.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.079+0000 7f893a600700 1 -- 172.21.15.123:0/3862843431 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89341a1010 con 0x7f89341056e0 2024-05-08T06:04:55.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.079+0000 7f893a600700 1 -- 172.21.15.123:0/3862843431 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89341a88f0 con 0x7f89341056e0 2024-05-08T06:04:55.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.079+0000 7f8930c00700 1 -- 172.21.15.123:0/3862843431 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8924022470 con 0x7f89341056e0 2024-05-08T06:04:55.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.079+0000 7f8930c00700 1 -- 172.21.15.123:0/3862843431 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8924017650 con 0x7f89341056e0 2024-05-08T06:04:55.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.080+0000 7f893a600700 1 -- 172.21.15.123:0/3862843431 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8910005330 con 0x7f89341056e0 2024-05-08T06:04:55.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.082+0000 7f8930c00700 1 -- 172.21.15.123:0/3862843431 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f892402a070 con 0x7f89341056e0 2024-05-08T06:04:55.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.084+0000 7f8930c00700 1 --2- 172.21.15.123:0/3862843431 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f89180b2900 0x7f89180b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:55.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.084+0000 7f8930c00700 1 -- 172.21.15.123:0/3862843431 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f89240e2910 con 0x7f89341056e0 2024-05-08T06:04:55.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.084+0000 7f892be00700 1 --2- 172.21.15.123:0/3862843431 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f89180b2900 0x7f89180b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:55.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.084+0000 7f892be00700 1 --2- 172.21.15.123:0/3862843431 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f89180b2900 0x7f89180b4dc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8934103880 tx=0x7f891c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:55.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.088+0000 7f8930c00700 1 -- 172.21.15.123:0/3862843431 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8924013070 con 0x7f89341056e0 2024-05-08T06:04:55.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.374+0000 7f893a600700 1 -- 172.21.15.123:0/3862843431 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8910005620 con 0x7f89341056e0 2024-05-08T06:04:55.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.375+0000 7f8930c00700 1 -- 172.21.15.123:0/3862843431 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f89240aaac0 con 0x7f89341056e0 2024-05-08T06:04:55.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.378+0000 7f893a600700 1 -- 172.21.15.123:0/3862843431 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f89180b2900 msgr2=0x7f89180b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:55.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.378+0000 7f893a600700 1 --2- 172.21.15.123:0/3862843431 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f89180b2900 0x7f89180b4dc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8934103880 tx=0x7f891c009260 comp rx=0 tx=0).stop 2024-05-08T06:04:55.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.378+0000 7f893a600700 1 -- 172.21.15.123:0/3862843431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89341056e0 msgr2=0x7f893419bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:55.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.378+0000 7f893a600700 1 --2- 172.21.15.123:0/3862843431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89341056e0 0x7f893419bd70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f89240059d0 tx=0x7f89240047b0 comp rx=0 tx=0).stop 2024-05-08T06:04:55.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.378+0000 7f893a600700 1 -- 172.21.15.123:0/3862843431 shutdown_connections 2024-05-08T06:04:55.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.378+0000 7f893a600700 1 --2- 172.21.15.123:0/3862843431 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f89180b2900 0x7f89180b4dc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:55.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.378+0000 7f893a600700 1 --2- 172.21.15.123:0/3862843431 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f893410b6e0 0x7f89341a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:55.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.378+0000 7f893a600700 1 --2- 172.21.15.123:0/3862843431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89341056e0 0x7f893419bd70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:55.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.378+0000 7f893a600700 1 --2- 172.21.15.123:0/3862843431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89341060a0 0x7f893419c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:55.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.378+0000 7f893a600700 1 -- 172.21.15.123:0/3862843431 >> 172.21.15.123:0/3862843431 conn(0x7f89341011d0 msgr2=0x7f8934101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:55.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.379+0000 7f893a600700 1 -- 172.21.15.123:0/3862843431 shutdown_connections 2024-05-08T06:04:55.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:04:55.379+0000 7f893a600700 1 -- 172.21.15.123:0/3862843431 wait complete. 2024-05-08T06:04:55.526 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 3 2024-05-08T06:04:55.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:55 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/3862843431' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:04:55.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:55 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/3862843431' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:04:56.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:55 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/3862843431' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:04:56.919 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:56 smithi123 ceph-mon[29787]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:57.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:56 smithi163 ceph-mon[34520]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:57.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:56 smithi192 ceph-mon[34664]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:57.206 INFO:journalctl@ceph.mds.j.smithi123.stdout:May 08 06:04:56 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-j[69685]: starting mds.j at 2024-05-08T06:04:57.705 INFO:journalctl@ceph.mds.j.smithi123.stdout:May 08 06:04:57 smithi123 bash[69670]: 4b34666c1f3629a84af127336806c575b608c4d522179b618676b8077ddc1961 2024-05-08T06:04:57.705 INFO:journalctl@ceph.mds.j.smithi123.stdout:May 08 06:04:57 smithi123 systemd[1]: Started Ceph mds.j for 85cfb2e6-0cff-11ef-bc97-c7b262605968. 2024-05-08T06:04:58.313 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T06:04:58.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:58 smithi123 ceph-mon[29787]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:58.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:58 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:58.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:58 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:58.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:58 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:58.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:58 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T06:04:58.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:58 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T06:04:58.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:58 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:58.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:58 smithi123 ceph-mon[29787]: Deploying daemon mds.d on smithi123 2024-05-08T06:04:58.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:58 smithi123 ceph-mon[29787]: mds.? [v2:172.21.15.123:6834/3646476522,v1:172.21.15.123:6835/3646476522] up:boot 2024-05-08T06:04:58.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:58 smithi123 ceph-mon[29787]: fsmap 3 up:standby 2024-05-08T06:04:58.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:04:58 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T06:04:58.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:58 smithi163 ceph-mon[34520]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:58.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:58 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:58.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:58 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:58.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:58 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:58.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:58 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T06:04:58.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:58 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T06:04:58.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:58 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:58.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:58 smithi163 ceph-mon[34520]: Deploying daemon mds.d on smithi123 2024-05-08T06:04:58.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:58 smithi163 ceph-mon[34520]: mds.? [v2:172.21.15.123:6834/3646476522,v1:172.21.15.123:6835/3646476522] up:boot 2024-05-08T06:04:58.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:58 smithi163 ceph-mon[34520]: fsmap 3 up:standby 2024-05-08T06:04:58.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:04:58 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T06:04:58.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:58 smithi192 ceph-mon[34664]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:58.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:58 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:58.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:58 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:58.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:58 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:04:58.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:58 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T06:04:58.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:58 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T06:04:58.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:58 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:58.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:58 smithi192 ceph-mon[34664]: Deploying daemon mds.d on smithi123 2024-05-08T06:04:58.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:58 smithi192 ceph-mon[34664]: mds.? [v2:172.21.15.123:6834/3646476522,v1:172.21.15.123:6835/3646476522] up:boot 2024-05-08T06:04:58.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:58 smithi192 ceph-mon[34664]: fsmap 3 up:standby 2024-05-08T06:04:58.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:04:58 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T06:04:59.675 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:05:00.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:00 smithi123 ceph-mon[29787]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:00.706 INFO:journalctl@ceph.mds.d.smithi123.stdout:May 08 06:05:00 smithi123 systemd[1]: Starting Ceph mds.d for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T06:05:00.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:00 smithi163 ceph-mon[34520]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:00.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:00 smithi192 ceph-mon[34664]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:01.706 INFO:journalctl@ceph.mds.d.smithi123.stdout:May 08 06:05:01 smithi123 podman[70067]: 2024-05-08 06:05:01.336085966 +0000 UTC m=+0.035189508 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:05:01.707 INFO:journalctl@ceph.mds.d.smithi123.stdout:May 08 06:05:01 smithi123 podman[70067]: 2024-05-08 06:05:01.460333474 +0000 UTC m=+0.159437057 container create 781cf0787d16ef22888fed6f87d22dce3c3cfb75687b6f76a3ddcaa9f69c761c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-08T06:05:02.002 INFO:journalctl@ceph.mds.d.smithi123.stdout:May 08 06:05:01 smithi123 podman[70067]: 2024-05-08 06:05:01.949060804 +0000 UTC m=+0.648164343 container init 781cf0787d16ef22888fed6f87d22dce3c3cfb75687b6f76a3ddcaa9f69c761c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T06:05:02.002 INFO:journalctl@ceph.mds.d.smithi123.stdout:May 08 06:05:01 smithi123 podman[70067]: 2024-05-08 06:05:01.958341786 +0000 UTC m=+0.657445327 container start 781cf0787d16ef22888fed6f87d22dce3c3cfb75687b6f76a3ddcaa9f69c761c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T06:05:02.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.214+0000 7fe317e00700 1 -- 172.21.15.123:0/1570336014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3181033d0 msgr2=0x7fe318103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:02.334 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.214+0000 7fe317e00700 1 --2- 172.21.15.123:0/1570336014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3181033d0 0x7fe318103850 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe300009990 tx=0x7fe300009ca0 comp rx=0 tx=0).stop 2024-05-08T06:05:02.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.214+0000 7fe317e00700 1 -- 172.21.15.123:0/1570336014 shutdown_connections 2024-05-08T06:05:02.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.214+0000 7fe317e00700 1 --2- 172.21.15.123:0/1570336014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe318109460 0x7fe318109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:02.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.214+0000 7fe317e00700 1 --2- 172.21.15.123:0/1570336014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe318103d90 0x7fe3181102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:02.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.214+0000 7fe317e00700 1 --2- 172.21.15.123:0/1570336014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3181033d0 0x7fe318103850 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:02.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.214+0000 7fe317e00700 1 -- 172.21.15.123:0/1570336014 >> 172.21.15.123:0/1570336014 conn(0x7fe3180fef20 msgr2=0x7fe318101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:05:02.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.214+0000 7fe317e00700 1 -- 172.21.15.123:0/1570336014 shutdown_connections 2024-05-08T06:05:02.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.214+0000 7fe317e00700 1 -- 172.21.15.123:0/1570336014 wait complete. 2024-05-08T06:05:02.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.216+0000 7fe317e00700 1 Processor -- start 2024-05-08T06:05:02.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.216+0000 7fe317e00700 1 -- start start 2024-05-08T06:05:02.335 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.216+0000 7fe317e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3181033d0 0x7fe31819bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:02.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.216+0000 7fe317e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe318103d90 0x7fe31819c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:02.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.216+0000 7fe317e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe318109460 0x7fe3181a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:02.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.216+0000 7fe317e00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe3181135c0 con 0x7fe3181033d0 2024-05-08T06:05:02.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.216+0000 7fe317e00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe318113440 con 0x7fe318103d90 2024-05-08T06:05:02.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.216+0000 7fe317e00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe318113740 con 0x7fe318109460 2024-05-08T06:05:02.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.217+0000 7fe30fe00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe318103d90 0x7fe31819c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:02.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.217+0000 7fe317400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe318109460 0x7fe3181a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:02.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.217+0000 7fe30fe00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe318103d90 0x7fe31819c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:49312/0 (socket says 172.21.15.123:49312) 2024-05-08T06:05:02.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.217+0000 7fe317400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe318109460 0x7fe3181a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:50186/0 (socket says 172.21.15.123:50186) 2024-05-08T06:05:02.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.217+0000 7fe30fe00700 1 -- 172.21.15.123:0/2747280726 learned_addr learned my addr 172.21.15.123:0/2747280726 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:05:02.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.217+0000 7fe316a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3181033d0 0x7fe31819bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:02.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.217+0000 7fe317400700 1 -- 172.21.15.123:0/2747280726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe318103d90 msgr2=0x7fe31819c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:02.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.217+0000 7fe317400700 1 --2- 172.21.15.123:0/2747280726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe318103d90 0x7fe31819c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:02.336 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.217+0000 7fe317400700 1 -- 172.21.15.123:0/2747280726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3181033d0 msgr2=0x7fe31819bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:02.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.217+0000 7fe317400700 1 --2- 172.21.15.123:0/2747280726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3181033d0 0x7fe31819bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:02.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.217+0000 7fe317400700 1 -- 172.21.15.123:0/2747280726 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3181a0d20 con 0x7fe318109460 2024-05-08T06:05:02.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.217+0000 7fe317400700 1 --2- 172.21.15.123:0/2747280726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe318109460 0x7fe3181a05e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe31000e900 tx=0x7fe31000ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:05:02.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.218+0000 7fe31ca00700 1 -- 172.21.15.123:0/2747280726 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe310004220 con 0x7fe318109460 2024-05-08T06:05:02.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.218+0000 7fe31ca00700 1 -- 172.21.15.123:0/2747280726 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3100043c0 con 0x7fe318109460 2024-05-08T06:05:02.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.218+0000 7fe317e00700 1 -- 172.21.15.123:0/2747280726 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3181a1010 con 0x7fe318109460 2024-05-08T06:05:02.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.218+0000 7fe31ca00700 1 -- 172.21.15.123:0/2747280726 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe310003d50 con 0x7fe318109460 2024-05-08T06:05:02.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.218+0000 7fe317e00700 1 -- 172.21.15.123:0/2747280726 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3181a8890 con 0x7fe318109460 2024-05-08T06:05:02.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.219+0000 7fe317e00700 1 -- 172.21.15.123:0/2747280726 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe31806a190 con 0x7fe318109460 2024-05-08T06:05:02.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.221+0000 7fe31ca00700 1 -- 172.21.15.123:0/2747280726 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fe31000c780 con 0x7fe318109460 2024-05-08T06:05:02.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.223+0000 7fe31ca00700 1 --2- 172.21.15.123:0/2747280726 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe2fc0b2900 0x7fe2fc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:02.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.223+0000 7fe31ca00700 1 -- 172.21.15.123:0/2747280726 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fe3100e1f90 con 0x7fe318109460 2024-05-08T06:05:02.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.223+0000 7fe316a00700 1 --2- 172.21.15.123:0/2747280726 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe2fc0b2900 0x7fe2fc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:02.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.223+0000 7fe316a00700 1 --2- 172.21.15.123:0/2747280726 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe2fc0b2900 0x7fe2fc0b4dc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe30800f540 tx=0x7fe30800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:05:02.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.225+0000 7fe31ca00700 1 -- 172.21.15.123:0/2747280726 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe3100aa0b0 con 0x7fe318109460 2024-05-08T06:05:02.338 INFO:journalctl@ceph.mds.d.smithi123.stdout:May 08 06:05:02 smithi123 ceph-mds[70105]: -- 172.21.15.123:0/3014937057 <== mon.2 v2:172.21.15.163:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56342cd8ba40 con 0x56342cd5f000 2024-05-08T06:05:02.338 INFO:journalctl@ceph.mds.d.smithi123.stdout:May 08 06:05:02 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-d[70097]: starting mds.d at 2024-05-08T06:05:02.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.511+0000 7fe317e00700 1 -- 172.21.15.123:0/2747280726 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3181a8a60 con 0x7fe318109460 2024-05-08T06:05:02.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.512+0000 7fe31ca00700 1 -- 172.21.15.123:0/2747280726 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3877 (secure 0 0 0) 0x7fe3100a99b0 con 0x7fe318109460 2024-05-08T06:05:02.512 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:05:02.512 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3646476522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3646476522},{"type":"v1","addr":"172.21.15.123:6835","nonce":3646476522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2763706497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2763706497},{"type":"v1","addr":"172.21.15.123:6837","nonce":2763706497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/175423115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":175423115},{"type":"v1","addr":"172.21.15.192:6833","nonce":175423115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/3263009472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3263009472},{"type":"v1","addr":"172.21.15.163:6833","nonce":3263009472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-08T06:05:02.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.515+0000 7fe317e00700 1 -- 172.21.15.123:0/2747280726 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe2fc0b2900 msgr2=0x7fe2fc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:02.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.515+0000 7fe317e00700 1 --2- 172.21.15.123:0/2747280726 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe2fc0b2900 0x7fe2fc0b4dc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe30800f540 tx=0x7fe30800b040 comp rx=0 tx=0).stop 2024-05-08T06:05:02.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.515+0000 7fe317e00700 1 -- 172.21.15.123:0/2747280726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe318109460 msgr2=0x7fe3181a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:02.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.515+0000 7fe317e00700 1 --2- 172.21.15.123:0/2747280726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe318109460 0x7fe3181a05e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe31000e900 tx=0x7fe31000ec10 comp rx=0 tx=0).stop 2024-05-08T06:05:02.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.515+0000 7fe317e00700 1 -- 172.21.15.123:0/2747280726 shutdown_connections 2024-05-08T06:05:02.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.515+0000 7fe317e00700 1 --2- 172.21.15.123:0/2747280726 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe2fc0b2900 0x7fe2fc0b4dc0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:02.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.515+0000 7fe317e00700 1 --2- 172.21.15.123:0/2747280726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3181033d0 0x7fe31819bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:02.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.515+0000 7fe317e00700 1 --2- 172.21.15.123:0/2747280726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe318109460 0x7fe3181a05e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:02.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.515+0000 7fe317e00700 1 --2- 172.21.15.123:0/2747280726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe318103d90 0x7fe31819c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:02.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.515+0000 7fe317e00700 1 -- 172.21.15.123:0/2747280726 >> 172.21.15.123:0/2747280726 conn(0x7fe3180fef20 msgr2=0x7fe31810eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:05:02.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.515+0000 7fe317e00700 1 -- 172.21.15.123:0/2747280726 shutdown_connections 2024-05-08T06:05:02.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:02.515+0000 7fe317e00700 1 -- 172.21.15.123:0/2747280726 wait complete. 2024-05-08T06:05:02.516 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 5 2024-05-08T06:05:02.705 INFO:journalctl@ceph.mds.d.smithi123.stdout:May 08 06:05:02 smithi123 bash[70067]: 781cf0787d16ef22888fed6f87d22dce3c3cfb75687b6f76a3ddcaa9f69c761c 2024-05-08T06:05:02.705 INFO:journalctl@ceph.mds.d.smithi123.stdout:May 08 06:05:02 smithi123 systemd[1]: Started Ceph mds.d for 85cfb2e6-0cff-11ef-bc97-c7b262605968. 2024-05-08T06:05:02.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:02 smithi123 ceph-mon[29787]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:02.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:02 smithi163 ceph-mon[34520]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:02.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:02 smithi192 ceph-mon[34664]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:03.618 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:03 smithi163 ceph-mon[34520]: mds.? [v2:172.21.15.123:6836/2763706497,v1:172.21.15.123:6837/2763706497] up:boot 2024-05-08T06:05:03.618 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:03 smithi163 ceph-mon[34520]: fsmap 4 up:standby 2024-05-08T06:05:03.618 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:03 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T06:05:03.618 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:03 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/2747280726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:05:03.619 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:03 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:03.619 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:03 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:03.619 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:03 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:03.619 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:03 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T06:05:03.619 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:03 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T06:05:03.619 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:03 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:05:03.619 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:03 smithi163 ceph-mon[34520]: Deploying daemon mds.e on smithi163 2024-05-08T06:05:03.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:03 smithi123 ceph-mon[29787]: mds.? [v2:172.21.15.123:6836/2763706497,v1:172.21.15.123:6837/2763706497] up:boot 2024-05-08T06:05:03.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:03 smithi123 ceph-mon[29787]: fsmap 4 up:standby 2024-05-08T06:05:03.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:03 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T06:05:03.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:03 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/2747280726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:05:03.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:03 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:03.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:03 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:03.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:03 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:03.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:03 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T06:05:03.707 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:03 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T06:05:03.707 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:03 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:05:03.707 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:03 smithi123 ceph-mon[29787]: Deploying daemon mds.e on smithi163 2024-05-08T06:05:03.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:03 smithi192 ceph-mon[34664]: mds.? [v2:172.21.15.123:6836/2763706497,v1:172.21.15.123:6837/2763706497] up:boot 2024-05-08T06:05:03.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:03 smithi192 ceph-mon[34664]: fsmap 4 up:standby 2024-05-08T06:05:03.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:03 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T06:05:03.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:03 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/2747280726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:05:03.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:03 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:03.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:03 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:03.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:03 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:03.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:03 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T06:05:03.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:03 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T06:05:03.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:03 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:05:03.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:03 smithi192 ceph-mon[34664]: Deploying daemon mds.e on smithi163 2024-05-08T06:05:04.807 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:04 smithi123 ceph-mon[29787]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:04.973 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:04 smithi163 ceph-mon[34520]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:04.974 INFO:journalctl@ceph.mds.e.smithi163.stdout:May 08 06:05:04 smithi163 systemd[1]: Starting Ceph mds.e for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T06:05:05.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:04 smithi192 ceph-mon[34664]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:05.226 INFO:journalctl@ceph.mds.e.smithi163.stdout:May 08 06:05:04 smithi163 podman[64400]: 2024-05-08 06:05:04.877241427 +0000 UTC m=+0.017971089 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:05:05.226 INFO:journalctl@ceph.mds.e.smithi163.stdout:May 08 06:05:05 smithi163 podman[64400]: 2024-05-08 06:05:05.007540897 +0000 UTC m=+0.148270556 container create 7203fdc45ce121a996826bbc71bd21e6a8f510438d3dd0d663567f9e9da08cd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-e, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-08T06:05:05.226 INFO:journalctl@ceph.mds.e.smithi163.stdout:May 08 06:05:05 smithi163 podman[64400]: 2024-05-08 06:05:05.22679424 +0000 UTC m=+0.367523896 container init 7203fdc45ce121a996826bbc71bd21e6a8f510438d3dd0d663567f9e9da08cd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507) 2024-05-08T06:05:05.430 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T06:05:05.538 INFO:journalctl@ceph.mds.e.smithi163.stdout:May 08 06:05:05 smithi163 podman[64400]: 2024-05-08 06:05:05.229631309 +0000 UTC m=+0.370360974 container start 7203fdc45ce121a996826bbc71bd21e6a8f510438d3dd0d663567f9e9da08cd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2) 2024-05-08T06:05:05.538 INFO:journalctl@ceph.mds.e.smithi163.stdout:May 08 06:05:05 smithi163 ceph-mds[64419]: -- 172.21.15.163:0/1934064172 <== mon.2 v2:172.21.15.163:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d4b97d7a40 con 0x55d4b97aa800 2024-05-08T06:05:05.538 INFO:journalctl@ceph.mds.e.smithi163.stdout:May 08 06:05:05 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-e[64415]: starting mds.e at 2024-05-08T06:05:05.539 INFO:journalctl@ceph.mds.e.smithi163.stdout:May 08 06:05:05 smithi163 bash[64400]: 7203fdc45ce121a996826bbc71bd21e6a8f510438d3dd0d663567f9e9da08cd5 2024-05-08T06:05:05.539 INFO:journalctl@ceph.mds.e.smithi163.stdout:May 08 06:05:05 smithi163 systemd[1]: Started Ceph mds.e for 85cfb2e6-0cff-11ef-bc97-c7b262605968. 2024-05-08T06:05:05.838 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:05:06.504 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:06 smithi123 ceph-mon[29787]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:06.505 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:06 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:06.505 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:06 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:06.505 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:06 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:06.505 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:06 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T06:05:06.505 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:06 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T06:05:06.505 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:06 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:05:06.505 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:06 smithi123 ceph-mon[29787]: Deploying daemon mds.f on smithi192 2024-05-08T06:05:06.505 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:06 smithi123 ceph-mon[29787]: mds.? [v2:172.21.15.163:6834/3308612826,v1:172.21.15.163:6835/3308612826] up:boot 2024-05-08T06:05:06.505 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:06 smithi123 ceph-mon[29787]: fsmap 5 up:standby 2024-05-08T06:05:06.505 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:06 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T06:05:06.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:06 smithi163 ceph-mon[34520]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:06.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:06 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:06.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:06 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:06.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:06 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:06.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:06 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T06:05:06.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:06 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T06:05:06.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:06 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:05:06.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:06 smithi163 ceph-mon[34520]: Deploying daemon mds.f on smithi192 2024-05-08T06:05:06.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:06 smithi163 ceph-mon[34520]: mds.? [v2:172.21.15.163:6834/3308612826,v1:172.21.15.163:6835/3308612826] up:boot 2024-05-08T06:05:06.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:06 smithi163 ceph-mon[34520]: fsmap 5 up:standby 2024-05-08T06:05:06.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:06 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T06:05:06.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:06 smithi192 ceph-mon[34664]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:06.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:06 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:06.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:06 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:06.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:06 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:06.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:06 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T06:05:06.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:06 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T06:05:06.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:06 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:05:06.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:06 smithi192 ceph-mon[34664]: Deploying daemon mds.f on smithi192 2024-05-08T06:05:06.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:06 smithi192 ceph-mon[34664]: mds.? [v2:172.21.15.163:6834/3308612826,v1:172.21.15.163:6835/3308612826] up:boot 2024-05-08T06:05:06.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:06 smithi192 ceph-mon[34664]: fsmap 5 up:standby 2024-05-08T06:05:06.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:06 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T06:05:07.801 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 08 06:05:07 smithi192 systemd[1]: Starting Ceph mds.f for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T06:05:08.158 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 08 06:05:07 smithi192 podman[65120]: 2024-05-08 06:05:07.747091248 +0000 UTC m=+0.017980385 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:05:08.158 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 08 06:05:07 smithi192 podman[65120]: 2024-05-08 06:05:07.863335975 +0000 UTC m=+0.134225095 container create 67904393a6f4cd214ebcbcd064a173ad5316ccc7a474f8d4fdcee2507f9de675 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f) 2024-05-08T06:05:08.158 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 08 06:05:08 smithi192 podman[65120]: 2024-05-08 06:05:08.066008695 +0000 UTC m=+0.336897815 container init 67904393a6f4cd214ebcbcd064a173ad5316ccc7a474f8d4fdcee2507f9de675 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-f, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-05-08T06:05:08.158 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 08 06:05:08 smithi192 podman[65120]: 2024-05-08 06:05:08.068561161 +0000 UTC m=+0.339450289 container start 67904393a6f4cd214ebcbcd064a173ad5316ccc7a474f8d4fdcee2507f9de675 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-f, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-08T06:05:08.158 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 08 06:05:08 smithi192 ceph-mds[65139]: -- 172.21.15.192:0/4191775123 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dd42569a40 con 0x55dd4253c800 2024-05-08T06:05:08.158 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 08 06:05:08 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-f[65135]: starting mds.f at 2024-05-08T06:05:08.158 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 08 06:05:08 smithi192 bash[65120]: 67904393a6f4cd214ebcbcd064a173ad5316ccc7a474f8d4fdcee2507f9de675 2024-05-08T06:05:08.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.302+0000 7f22d7800700 1 -- 172.21.15.123:0/3646257902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22d006bb40 msgr2=0x7f22d0110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:08.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.302+0000 7f22d7800700 1 --2- 172.21.15.123:0/3646257902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22d006bb40 0x7f22d0110330 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f22c800b0a0 tx=0x7f22c800b3b0 comp rx=0 tx=0).stop 2024-05-08T06:05:08.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.302+0000 7f22d7800700 1 -- 172.21.15.123:0/3646257902 shutdown_connections 2024-05-08T06:05:08.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.302+0000 7f22d7800700 1 --2- 172.21.15.123:0/3646257902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22d006bb40 0x7f22d0110330 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:08.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.302+0000 7f22d7800700 1 --2- 172.21.15.123:0/3646257902 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22d0109510 0x7f22d01098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:08.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.302+0000 7f22d7800700 1 --2- 172.21.15.123:0/3646257902 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22d006b180 0x7f22d006b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:08.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.302+0000 7f22d7800700 1 -- 172.21.15.123:0/3646257902 >> 172.21.15.123:0/3646257902 conn(0x7f22d0079350 msgr2=0x7f22d0079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:05:08.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.302+0000 7f22d7800700 1 -- 172.21.15.123:0/3646257902 shutdown_connections 2024-05-08T06:05:08.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.303+0000 7f22d7800700 1 -- 172.21.15.123:0/3646257902 wait complete. 2024-05-08T06:05:08.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.304+0000 7f22d7800700 1 Processor -- start 2024-05-08T06:05:08.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.304+0000 7f22d7800700 1 -- start start 2024-05-08T06:05:08.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.304+0000 7f22d7800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22d006b180 0x7f22d019bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:08.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.304+0000 7f22d7800700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22d006bb40 0x7f22d019c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:08.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.304+0000 7f22d7800700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22d0109510 0x7f22d01a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:08.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.304+0000 7f22d7800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f22d0113670 con 0x7f22d006b180 2024-05-08T06:05:08.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.304+0000 7f22d7800700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f22d01134f0 con 0x7f22d006bb40 2024-05-08T06:05:08.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.304+0000 7f22d7800700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f22d01137f0 con 0x7f22d0109510 2024-05-08T06:05:08.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.304+0000 7f22d4e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22d006b180 0x7f22d019bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:08.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.304+0000 7f22d4e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22d006b180 0x7f22d019bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50212/0 (socket says 172.21.15.123:50212) 2024-05-08T06:05:08.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.305+0000 7f22d4e00700 1 -- 172.21.15.123:0/1674247024 learned_addr learned my addr 172.21.15.123:0/1674247024 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:05:08.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.304+0000 7f22d5800700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22d0109510 0x7f22d01a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:08.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.305+0000 7f22cfe00700 1 --2- 172.21.15.123:0/1674247024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22d006bb40 0x7f22d019c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:08.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.305+0000 7f22d4e00700 1 -- 172.21.15.123:0/1674247024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22d0109510 msgr2=0x7f22d01a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:08.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.305+0000 7f22d4e00700 1 --2- 172.21.15.123:0/1674247024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22d0109510 0x7f22d01a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:08.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.305+0000 7f22d4e00700 1 -- 172.21.15.123:0/1674247024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22d006bb40 msgr2=0x7f22d019c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:08.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.305+0000 7f22d4e00700 1 --2- 172.21.15.123:0/1674247024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22d006bb40 0x7f22d019c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:08.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.305+0000 7f22d4e00700 1 -- 172.21.15.123:0/1674247024 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22d01a0d40 con 0x7f22d006b180 2024-05-08T06:05:08.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.305+0000 7f22d5800700 1 --2- 172.21.15.123:0/1674247024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22d0109510 0x7f22d01a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:05:08.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.305+0000 7f22cfe00700 1 --2- 172.21.15.123:0/1674247024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22d006bb40 0x7f22d019c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:05:08.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.305+0000 7f22d4e00700 1 --2- 172.21.15.123:0/1674247024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22d006b180 0x7f22d019bd30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f22c400e9a0 tx=0x7f22c400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:05:08.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.306+0000 7f22cd600700 1 -- 172.21.15.123:0/1674247024 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c400ca10 con 0x7f22d006b180 2024-05-08T06:05:08.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.306+0000 7f22cd600700 1 -- 172.21.15.123:0/1674247024 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22c400cbb0 con 0x7f22d006b180 2024-05-08T06:05:08.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.306+0000 7f22cd600700 1 -- 172.21.15.123:0/1674247024 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c4018680 con 0x7f22d006b180 2024-05-08T06:05:08.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.306+0000 7f22d7800700 1 -- 172.21.15.123:0/1674247024 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22d01a1030 con 0x7f22d006b180 2024-05-08T06:05:08.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.306+0000 7f22d7800700 1 -- 172.21.15.123:0/1674247024 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22d01a88b0 con 0x7f22d006b180 2024-05-08T06:05:08.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.307+0000 7f22d7800700 1 -- 172.21.15.123:0/1674247024 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22d006a190 con 0x7f22d006b180 2024-05-08T06:05:08.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.309+0000 7f22cd600700 1 -- 172.21.15.123:0/1674247024 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f22c4010430 con 0x7f22d006b180 2024-05-08T06:05:08.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.311+0000 7f22cd600700 1 --2- 172.21.15.123:0/1674247024 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f22b80b2900 0x7f22b80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:08.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.311+0000 7f22cd600700 1 -- 172.21.15.123:0/1674247024 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f22c40e1df0 con 0x7f22d006b180 2024-05-08T06:05:08.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.311+0000 7f22cfe00700 1 --2- 172.21.15.123:0/1674247024 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f22b80b2900 0x7f22b80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:08.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.312+0000 7f22cfe00700 1 --2- 172.21.15.123:0/1674247024 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f22b80b2900 0x7f22b80b4dc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f22d0104460 tx=0x7f22bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:05:08.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.316+0000 7f22cd600700 1 -- 172.21.15.123:0/1674247024 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f22c40a9fb0 con 0x7f22d006b180 2024-05-08T06:05:08.443 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:08 smithi192 ceph-mon[34664]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:08.443 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:08 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:08.443 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:08 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:08.444 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:08 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:08.444 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:08 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T06:05:08.444 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:08 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T06:05:08.444 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:08 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:05:08.444 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 08 06:05:08 smithi192 systemd[1]: Started Ceph mds.f for 85cfb2e6-0cff-11ef-bc97-c7b262605968. 2024-05-08T06:05:08.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.597+0000 7f22d7800700 1 -- 172.21.15.123:0/1674247024 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22d01a8ab0 con 0x7f22d006b180 2024-05-08T06:05:08.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.598+0000 7f22cd600700 1 -- 172.21.15.123:0/1674247024 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5530 (secure 0 0 0) 0x7f22c40a98b0 con 0x7f22d006b180 2024-05-08T06:05:08.599 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:05:08.599 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3646476522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3646476522},{"type":"v1","addr":"172.21.15.123:6835","nonce":3646476522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2763706497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2763706497},{"type":"v1","addr":"172.21.15.123:6837","nonce":2763706497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/175423115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":175423115},{"type":"v1","addr":"172.21.15.192:6833","nonce":175423115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/3263009472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3263009472},{"type":"v1","addr":"172.21.15.163:6833","nonce":3263009472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3308612826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3308612826},{"type":"v1","addr":"172.21.15.163:6835","nonce":3308612826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/202513680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":202513680},{"type":"v1","addr":"172.21.15.192:6835","nonce":202513680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-05-08T06:05:08.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.601+0000 7f22d7800700 1 -- 172.21.15.123:0/1674247024 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f22b80b2900 msgr2=0x7f22b80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:08.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.601+0000 7f22d7800700 1 --2- 172.21.15.123:0/1674247024 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f22b80b2900 0x7f22b80b4dc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f22d0104460 tx=0x7f22bc009260 comp rx=0 tx=0).stop 2024-05-08T06:05:08.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.601+0000 7f22d7800700 1 -- 172.21.15.123:0/1674247024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22d006b180 msgr2=0x7f22d019bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:08.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.601+0000 7f22d7800700 1 --2- 172.21.15.123:0/1674247024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22d006b180 0x7f22d019bd30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f22c400e9a0 tx=0x7f22c400ecb0 comp rx=0 tx=0).stop 2024-05-08T06:05:08.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.602+0000 7f22d7800700 1 -- 172.21.15.123:0/1674247024 shutdown_connections 2024-05-08T06:05:08.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.602+0000 7f22d7800700 1 --2- 172.21.15.123:0/1674247024 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f22b80b2900 0x7f22b80b4dc0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:08.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.602+0000 7f22d7800700 1 --2- 172.21.15.123:0/1674247024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22d006b180 0x7f22d019bd30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:08.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.602+0000 7f22d7800700 1 --2- 172.21.15.123:0/1674247024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22d0109510 0x7f22d01a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:08.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.602+0000 7f22d7800700 1 --2- 172.21.15.123:0/1674247024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22d006bb40 0x7f22d019c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:08.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.602+0000 7f22d7800700 1 -- 172.21.15.123:0/1674247024 >> 172.21.15.123:0/1674247024 conn(0x7f22d0079350 msgr2=0x7f22d010e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:05:08.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.602+0000 7f22d7800700 1 -- 172.21.15.123:0/1674247024 shutdown_connections 2024-05-08T06:05:08.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:08.602+0000 7f22d7800700 1 -- 172.21.15.123:0/1674247024 wait complete. 2024-05-08T06:05:08.603 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 7 2024-05-08T06:05:08.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:08 smithi123 ceph-mon[29787]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:08.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:08 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:08.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:08 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:08.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:08 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:08.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:08 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T06:05:08.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:08 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T06:05:08.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:08 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:05:08.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:08 smithi163 ceph-mon[34520]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:08.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:08 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:08.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:08 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:08.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:08 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:08.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:08 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T06:05:08.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:08 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T06:05:08.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:08 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:05:09.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:09 smithi192 ceph-mon[34664]: Deploying daemon mds.c on smithi192 2024-05-08T06:05:09.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:09 smithi192 ceph-mon[34664]: mds.? [v2:172.21.15.192:6834/202513680,v1:172.21.15.192:6835/202513680] up:boot 2024-05-08T06:05:09.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:09 smithi192 ceph-mon[34664]: fsmap 6 up:standby 2024-05-08T06:05:09.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:09 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T06:05:09.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:09 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/1674247024' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:05:09.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:09 smithi123 ceph-mon[29787]: Deploying daemon mds.c on smithi192 2024-05-08T06:05:09.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:09 smithi123 ceph-mon[29787]: mds.? [v2:172.21.15.192:6834/202513680,v1:172.21.15.192:6835/202513680] up:boot 2024-05-08T06:05:09.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:09 smithi123 ceph-mon[29787]: fsmap 6 up:standby 2024-05-08T06:05:09.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:09 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T06:05:09.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:09 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/1674247024' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:05:09.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:09 smithi163 ceph-mon[34520]: Deploying daemon mds.c on smithi192 2024-05-08T06:05:09.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:09 smithi163 ceph-mon[34520]: mds.? [v2:172.21.15.192:6834/202513680,v1:172.21.15.192:6835/202513680] up:boot 2024-05-08T06:05:09.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:09 smithi163 ceph-mon[34520]: fsmap 6 up:standby 2024-05-08T06:05:09.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:09 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T06:05:09.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:09 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/1674247024' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:05:10.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:10 smithi123 ceph-mon[29787]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:10.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:10 smithi163 ceph-mon[34520]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:10.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:10 smithi192 ceph-mon[34664]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:10.801 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 08 06:05:10 smithi192 systemd[1]: Starting Ceph mds.c for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T06:05:11.081 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 08 06:05:10 smithi192 podman[65442]: 2024-05-08 06:05:10.71189663 +0000 UTC m=+0.018032716 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:05:11.081 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 08 06:05:10 smithi192 podman[65442]: 2024-05-08 06:05:10.820144717 +0000 UTC m=+0.126298332 container create 61baa468f6d082634a12e77df3d294e5a779b9de9505af78c9c28408fa70034d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux ) 2024-05-08T06:05:11.081 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 08 06:05:11 smithi192 podman[65442]: 2024-05-08 06:05:11.056557002 +0000 UTC m=+0.362693073 container init 61baa468f6d082634a12e77df3d294e5a779b9de9505af78c9c28408fa70034d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-c, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True) 2024-05-08T06:05:11.082 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 08 06:05:11 smithi192 podman[65442]: 2024-05-08 06:05:11.05907126 +0000 UTC m=+0.365207342 container start 61baa468f6d082634a12e77df3d294e5a779b9de9505af78c9c28408fa70034d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-c, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-08T06:05:11.550 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 08 06:05:11 smithi192 ceph-mds[65461]: -- 172.21.15.192:0/3382919050 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556eb8df1a40 con 0x556eb8dc5000 2024-05-08T06:05:11.551 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 08 06:05:11 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-c[65457]: starting mds.c at 2024-05-08T06:05:11.551 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 08 06:05:11 smithi192 bash[65442]: 61baa468f6d082634a12e77df3d294e5a779b9de9505af78c9c28408fa70034d 2024-05-08T06:05:11.551 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 08 06:05:11 smithi192 systemd[1]: Started Ceph mds.c for 85cfb2e6-0cff-11ef-bc97-c7b262605968. 2024-05-08T06:05:11.568 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T06:05:12.006 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:05:12.407 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:12 smithi123 ceph-mon[29787]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:12.407 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:12 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:12.407 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:12 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:12.407 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:12 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:12.407 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:12 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T06:05:12.407 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:12 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T06:05:12.407 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:12 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:05:12.407 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:12 smithi123 ceph-mon[29787]: Deploying daemon mds.h on smithi163 2024-05-08T06:05:12.407 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:12 smithi123 ceph-mon[29787]: mds.? [v2:172.21.15.192:6836/2483545969,v1:172.21.15.192:6837/2483545969] up:boot 2024-05-08T06:05:12.407 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:12 smithi123 ceph-mon[29787]: fsmap 7 up:standby 2024-05-08T06:05:12.407 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:12 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T06:05:12.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:12 smithi163 ceph-mon[34520]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:12.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:12 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:12.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:12 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:12.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:12 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:12.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:12 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T06:05:12.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:12 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T06:05:12.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:12 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:05:12.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:12 smithi163 ceph-mon[34520]: Deploying daemon mds.h on smithi163 2024-05-08T06:05:12.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:12 smithi163 ceph-mon[34520]: mds.? [v2:172.21.15.192:6836/2483545969,v1:172.21.15.192:6837/2483545969] up:boot 2024-05-08T06:05:12.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:12 smithi163 ceph-mon[34520]: fsmap 7 up:standby 2024-05-08T06:05:12.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:12 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T06:05:12.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:12 smithi192 ceph-mon[34664]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:12.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:12 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:12.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:12 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:12.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:12 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:12.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:12 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T06:05:12.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:12 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T06:05:12.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:12 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:05:12.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:12 smithi192 ceph-mon[34664]: Deploying daemon mds.h on smithi163 2024-05-08T06:05:12.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:12 smithi192 ceph-mon[34664]: mds.? [v2:172.21.15.192:6836/2483545969,v1:172.21.15.192:6837/2483545969] up:boot 2024-05-08T06:05:12.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:12 smithi192 ceph-mon[34664]: fsmap 7 up:standby 2024-05-08T06:05:12.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:12 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T06:05:13.510 INFO:journalctl@ceph.mds.h.smithi163.stdout:May 08 06:05:13 smithi163 systemd[1]: Starting Ceph mds.h for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T06:05:13.768 INFO:journalctl@ceph.mds.h.smithi163.stdout:May 08 06:05:13 smithi163 podman[64707]: 2024-05-08 06:05:13.413164406 +0000 UTC m=+0.018245653 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:05:13.768 INFO:journalctl@ceph.mds.h.smithi163.stdout:May 08 06:05:13 smithi163 podman[64707]: 2024-05-08 06:05:13.515859705 +0000 UTC m=+0.120940981 container create ae165ce0973ef5b6f757325a4c249793fb30007b6633d3c114b68954f5a5bb62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-h, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-08T06:05:13.768 INFO:journalctl@ceph.mds.h.smithi163.stdout:May 08 06:05:13 smithi163 podman[64707]: 2024-05-08 06:05:13.768997222 +0000 UTC m=+0.374078456 container init ae165ce0973ef5b6f757325a4c249793fb30007b6633d3c114b68954f5a5bb62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-h, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-08T06:05:14.038 INFO:journalctl@ceph.mds.h.smithi163.stdout:May 08 06:05:13 smithi163 podman[64707]: 2024-05-08 06:05:13.771486522 +0000 UTC m=+0.376567759 container start ae165ce0973ef5b6f757325a4c249793fb30007b6633d3c114b68954f5a5bb62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T06:05:14.038 INFO:journalctl@ceph.mds.h.smithi163.stdout:May 08 06:05:13 smithi163 ceph-mds[64727]: -- 172.21.15.163:0/4222827978 <== mon.2 v2:172.21.15.163:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5631cee2fa40 con 0x5631cee02800 2024-05-08T06:05:14.038 INFO:journalctl@ceph.mds.h.smithi163.stdout:May 08 06:05:13 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-h[64723]: starting mds.h at 2024-05-08T06:05:14.039 INFO:journalctl@ceph.mds.h.smithi163.stdout:May 08 06:05:13 smithi163 bash[64707]: ae165ce0973ef5b6f757325a4c249793fb30007b6633d3c114b68954f5a5bb62 2024-05-08T06:05:14.039 INFO:journalctl@ceph.mds.h.smithi163.stdout:May 08 06:05:13 smithi163 systemd[1]: Started Ceph mds.h for 85cfb2e6-0cff-11ef-bc97-c7b262605968. 2024-05-08T06:05:14.337 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:14 smithi123 ceph-mon[29787]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:14.337 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:14 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:14.337 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:14 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:14.337 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:14 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:14.337 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:14 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T06:05:14.337 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:14 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T06:05:14.337 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:14 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:05:14.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.504+0000 7f9178a00700 1 -- 172.21.15.123:0/4063934359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9174106080 msgr2=0x7f9174112590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:14.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.504+0000 7f9178a00700 1 --2- 172.21.15.123:0/4063934359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9174106080 0x7f9174112590 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f916c009f90 tx=0x7f916c00b330 comp rx=0 tx=0).stop 2024-05-08T06:05:14.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.504+0000 7f9178a00700 1 -- 172.21.15.123:0/4063934359 shutdown_connections 2024-05-08T06:05:14.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.504+0000 7f9178a00700 1 --2- 172.21.15.123:0/4063934359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f917410b6c0 0x7f917410baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:14.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.504+0000 7f9178a00700 1 --2- 172.21.15.123:0/4063934359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9174106080 0x7f9174112590 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:14.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.504+0000 7f9178a00700 1 --2- 172.21.15.123:0/4063934359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91741056c0 0x7f9174105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:14.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.504+0000 7f9178a00700 1 -- 172.21.15.123:0/4063934359 >> 172.21.15.123:0/4063934359 conn(0x7f91741011b0 msgr2=0x7f91741035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:05:14.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.504+0000 7f9178a00700 1 -- 172.21.15.123:0/4063934359 shutdown_connections 2024-05-08T06:05:14.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.504+0000 7f9178a00700 1 -- 172.21.15.123:0/4063934359 wait complete. 2024-05-08T06:05:14.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.506+0000 7f9178a00700 1 Processor -- start 2024-05-08T06:05:14.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.506+0000 7f9178a00700 1 -- start start 2024-05-08T06:05:14.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.507+0000 7f9178a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91741056c0 0x7f917419bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:14.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.507+0000 7f9178a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9174106080 0x7f917419c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:14.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.507+0000 7f9178a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f917410b6c0 0x7f91741a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:14.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.507+0000 7f9178a00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9174115900 con 0x7f917410b6c0 2024-05-08T06:05:14.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.507+0000 7f9178a00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9174115780 con 0x7f91741056c0 2024-05-08T06:05:14.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.507+0000 7f9178a00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9174115a80 con 0x7f9174106080 2024-05-08T06:05:14.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.507+0000 7f9171e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91741056c0 0x7f917419bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:14.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.507+0000 7f9171e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91741056c0 0x7f917419bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:47652/0 (socket says 172.21.15.123:47652) 2024-05-08T06:05:14.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.507+0000 7f9171400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9174106080 0x7f917419c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:14.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.507+0000 7f9171e00700 1 -- 172.21.15.123:0/1480130653 learned_addr learned my addr 172.21.15.123:0/1480130653 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:05:14.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.507+0000 7f9172800700 1 --2- 172.21.15.123:0/1480130653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f917410b6c0 0x7f91741a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:14.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.507+0000 7f9171400700 1 -- 172.21.15.123:0/1480130653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91741056c0 msgr2=0x7f917419bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:14.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.507+0000 7f9171400700 1 --2- 172.21.15.123:0/1480130653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91741056c0 0x7f917419bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:14.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.507+0000 7f9171400700 1 -- 172.21.15.123:0/1480130653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f917410b6c0 msgr2=0x7f91741a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:14.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.507+0000 7f9171400700 1 --2- 172.21.15.123:0/1480130653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f917410b6c0 0x7f91741a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:14.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.507+0000 7f9171400700 1 -- 172.21.15.123:0/1480130653 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91741a0d60 con 0x7f9174106080 2024-05-08T06:05:14.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.507+0000 7f9171e00700 1 --2- 172.21.15.123:0/1480130653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91741056c0 0x7f917419bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:05:14.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.508+0000 7f9171400700 1 --2- 172.21.15.123:0/1480130653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9174106080 0x7f917419c290 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f916000e9f0 tx=0x7f916000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:05:14.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.508+0000 7f916aa00700 1 -- 172.21.15.123:0/1480130653 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916000c9c0 con 0x7f9174106080 2024-05-08T06:05:14.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.508+0000 7f9178a00700 1 -- 172.21.15.123:0/1480130653 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91741a1050 con 0x7f9174106080 2024-05-08T06:05:14.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.508+0000 7f916aa00700 1 -- 172.21.15.123:0/1480130653 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f916000cb60 con 0x7f9174106080 2024-05-08T06:05:14.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.508+0000 7f916aa00700 1 -- 172.21.15.123:0/1480130653 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9160018680 con 0x7f9174106080 2024-05-08T06:05:14.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.508+0000 7f9178a00700 1 -- 172.21.15.123:0/1480130653 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91741a88d0 con 0x7f9174106080 2024-05-08T06:05:14.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.509+0000 7f9178a00700 1 -- 172.21.15.123:0/1480130653 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f917406a190 con 0x7f9174106080 2024-05-08T06:05:14.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.512+0000 7f916aa00700 1 -- 172.21.15.123:0/1480130653 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f9160010430 con 0x7f9174106080 2024-05-08T06:05:14.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.513+0000 7f916aa00700 1 --2- 172.21.15.123:0/1480130653 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f91580b2880 0x7f91580b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:14.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.514+0000 7f916aa00700 1 -- 172.21.15.123:0/1480130653 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f91600e1f60 con 0x7f9174106080 2024-05-08T06:05:14.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.514+0000 7f9171e00700 1 --2- 172.21.15.123:0/1480130653 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f91580b2880 0x7f91580b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:14.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.514+0000 7f9171e00700 1 --2- 172.21.15.123:0/1480130653 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f91580b2880 0x7f91580b4d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9164009750 tx=0x7f9164009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:05:14.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.518+0000 7f916aa00700 1 -- 172.21.15.123:0/1480130653 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f91600aa080 con 0x7f9174106080 2024-05-08T06:05:14.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:14 smithi163 ceph-mon[34520]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:14.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:14 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:14.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:14 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:14.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:14 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:14.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:14 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T06:05:14.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:14 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T06:05:14.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:14 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:05:14.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:14 smithi192 ceph-mon[34664]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:14.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:14 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:14.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:14 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:14.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:14 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:14.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:14 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T06:05:14.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:14 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T06:05:14.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:14 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:05:14.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.797+0000 7f9178a00700 1 -- 172.21.15.123:0/1480130653 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f917402e510 con 0x7f9174106080 2024-05-08T06:05:14.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.798+0000 7f916aa00700 1 -- 172.21.15.123:0/1480130653 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7183 (secure 0 0 0) 0x7f91600b2100 con 0x7f9174106080 2024-05-08T06:05:14.798 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:05:14.798 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3646476522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3646476522},{"type":"v1","addr":"172.21.15.123:6835","nonce":3646476522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2763706497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2763706497},{"type":"v1","addr":"172.21.15.123:6837","nonce":2763706497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/175423115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":175423115},{"type":"v1","addr":"172.21.15.192:6833","nonce":175423115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/3263009472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3263009472},{"type":"v1","addr":"172.21.15.163:6833","nonce":3263009472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3308612826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3308612826},{"type":"v1","addr":"172.21.15.163:6835","nonce":3308612826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/202513680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":202513680},{"type":"v1","addr":"172.21.15.192:6835","nonce":202513680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/2483545969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2483545969},{"type":"v1","addr":"172.21.15.192:6837","nonce":2483545969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/660511838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":660511838},{"type":"v1","addr":"172.21.15.163:6837","nonce":660511838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-08T06:05:14.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.801+0000 7f9178a00700 1 -- 172.21.15.123:0/1480130653 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f91580b2880 msgr2=0x7f91580b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:14.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.801+0000 7f9178a00700 1 --2- 172.21.15.123:0/1480130653 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f91580b2880 0x7f91580b4d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9164009750 tx=0x7f9164009310 comp rx=0 tx=0).stop 2024-05-08T06:05:14.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.801+0000 7f9178a00700 1 -- 172.21.15.123:0/1480130653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9174106080 msgr2=0x7f917419c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:14.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.801+0000 7f9178a00700 1 --2- 172.21.15.123:0/1480130653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9174106080 0x7f917419c290 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f916000e9f0 tx=0x7f916000ed00 comp rx=0 tx=0).stop 2024-05-08T06:05:14.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.801+0000 7f9178a00700 1 -- 172.21.15.123:0/1480130653 shutdown_connections 2024-05-08T06:05:14.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.801+0000 7f9178a00700 1 --2- 172.21.15.123:0/1480130653 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f91580b2880 0x7f91580b4d40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:14.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.801+0000 7f9178a00700 1 --2- 172.21.15.123:0/1480130653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f917410b6c0 0x7f91741a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:14.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.801+0000 7f9178a00700 1 --2- 172.21.15.123:0/1480130653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91741056c0 0x7f917419bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:14.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.801+0000 7f9178a00700 1 --2- 172.21.15.123:0/1480130653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9174106080 0x7f917419c290 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:14.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.801+0000 7f9178a00700 1 -- 172.21.15.123:0/1480130653 >> 172.21.15.123:0/1480130653 conn(0x7f91741011b0 msgr2=0x7f91741016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:05:14.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.802+0000 7f9178a00700 1 -- 172.21.15.123:0/1480130653 shutdown_connections 2024-05-08T06:05:14.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:14.802+0000 7f9178a00700 1 -- 172.21.15.123:0/1480130653 wait complete. 2024-05-08T06:05:14.803 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 9 2024-05-08T06:05:15.502 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:15 smithi123 ceph-mon[29787]: Deploying daemon mds.i on smithi192 2024-05-08T06:05:15.503 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:15 smithi123 ceph-mon[29787]: mds.? [v2:172.21.15.163:6836/660511838,v1:172.21.15.163:6837/660511838] up:boot 2024-05-08T06:05:15.503 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:15 smithi123 ceph-mon[29787]: fsmap 8 up:standby 2024-05-08T06:05:15.503 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:15 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T06:05:15.503 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:15 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/1480130653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:05:15.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:15 smithi163 ceph-mon[34520]: Deploying daemon mds.i on smithi192 2024-05-08T06:05:15.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:15 smithi163 ceph-mon[34520]: mds.? [v2:172.21.15.163:6836/660511838,v1:172.21.15.163:6837/660511838] up:boot 2024-05-08T06:05:15.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:15 smithi163 ceph-mon[34520]: fsmap 8 up:standby 2024-05-08T06:05:15.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:15 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T06:05:15.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:15 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/1480130653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:05:15.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:15 smithi192 ceph-mon[34664]: Deploying daemon mds.i on smithi192 2024-05-08T06:05:15.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:15 smithi192 ceph-mon[34664]: mds.? [v2:172.21.15.163:6836/660511838,v1:172.21.15.163:6837/660511838] up:boot 2024-05-08T06:05:15.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:15 smithi192 ceph-mon[34664]: fsmap 8 up:standby 2024-05-08T06:05:15.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:15 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T06:05:15.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:15 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/1480130653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:05:16.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:16 smithi192 ceph-mon[34664]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:16.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:16 smithi123 ceph-mon[29787]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:16.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:16 smithi163 ceph-mon[34520]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:17.050 INFO:journalctl@ceph.mds.i.smithi192.stdout:May 08 06:05:16 smithi192 systemd[1]: Started Ceph mds.i for 85cfb2e6-0cff-11ef-bc97-c7b262605968. 2024-05-08T06:05:17.735 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T06:05:18.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:17 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:18.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:17 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:18.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:17 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:18.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:17 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T06:05:18.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:17 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T06:05:18.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:17 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:05:18.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:17 smithi123 ceph-mon[29787]: Deploying daemon mds.g on smithi123 2024-05-08T06:05:18.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:17 smithi123 ceph-mon[29787]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:18.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:17 smithi123 ceph-mon[29787]: mds.? [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] up:boot 2024-05-08T06:05:18.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:17 smithi123 ceph-mon[29787]: fsmap 9 up:standby 2024-05-08T06:05:18.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:17 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T06:05:18.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:17 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:18.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:17 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:18.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:17 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:18.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:17 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T06:05:18.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:17 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T06:05:18.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:17 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:05:18.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:17 smithi163 ceph-mon[34520]: Deploying daemon mds.g on smithi123 2024-05-08T06:05:18.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:17 smithi163 ceph-mon[34520]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:18.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:17 smithi163 ceph-mon[34520]: mds.? [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] up:boot 2024-05-08T06:05:18.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:17 smithi163 ceph-mon[34520]: fsmap 9 up:standby 2024-05-08T06:05:18.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:17 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T06:05:18.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:17 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:18.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:17 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:18.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:17 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:18.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:17 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T06:05:18.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:17 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T06:05:18.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:17 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:05:18.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:17 smithi192 ceph-mon[34664]: Deploying daemon mds.g on smithi123 2024-05-08T06:05:18.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:17 smithi192 ceph-mon[34664]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:18.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:17 smithi192 ceph-mon[34664]: mds.? [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] up:boot 2024-05-08T06:05:18.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:17 smithi192 ceph-mon[34664]: fsmap 9 up:standby 2024-05-08T06:05:18.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:17 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T06:05:19.574 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:05:20.235 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:20 smithi123 ceph-mon[29787]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:20.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:20 smithi163 ceph-mon[34520]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:20.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:20 smithi192 ceph-mon[34664]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:21.295 INFO:journalctl@ceph.mds.g.smithi123.stdout:May 08 06:05:21 smithi123 podman[70843]: 2024-05-08 06:05:21.158228458 +0000 UTC m=+0.035654155 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:05:21.295 INFO:journalctl@ceph.mds.g.smithi123.stdout:May 08 06:05:21 smithi123 podman[70843]: 2024-05-08 06:05:21.294340763 +0000 UTC m=+0.171766498 container create 18d9e4aac5d989c69b945c509096468800ed63b867bef9622a1e6c3903954993 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-g, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T06:05:21.891 INFO:journalctl@ceph.mds.g.smithi123.stdout:May 08 06:05:21 smithi123 podman[70843]: 2024-05-08 06:05:21.620750481 +0000 UTC m=+0.498176260 container init 18d9e4aac5d989c69b945c509096468800ed63b867bef9622a1e6c3903954993 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-g, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-08T06:05:21.891 INFO:journalctl@ceph.mds.g.smithi123.stdout:May 08 06:05:21 smithi123 podman[70843]: 2024-05-08 06:05:21.631943526 +0000 UTC m=+0.509369247 container start 18d9e4aac5d989c69b945c509096468800ed63b867bef9622a1e6c3903954993 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-g, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-08T06:05:21.891 INFO:journalctl@ceph.mds.g.smithi123.stdout:May 08 06:05:21 smithi123 ceph-mds[70878]: -- 172.21.15.123:0/2311050054 <== mon.2 v2:172.21.15.163:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55666ed65a40 con 0x55666ed38c00 2024-05-08T06:05:21.891 INFO:journalctl@ceph.mds.g.smithi123.stdout:May 08 06:05:21 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-g[70874]: starting mds.g at 2024-05-08T06:05:21.892 INFO:journalctl@ceph.mds.g.smithi123.stdout:May 08 06:05:21 smithi123 bash[70843]: 18d9e4aac5d989c69b945c509096468800ed63b867bef9622a1e6c3903954993 2024-05-08T06:05:21.892 INFO:journalctl@ceph.mds.g.smithi123.stdout:May 08 06:05:21 smithi123 systemd[1]: Started Ceph mds.g for 85cfb2e6-0cff-11ef-bc97-c7b262605968. 2024-05-08T06:05:22.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:22 smithi123 ceph-mon[29787]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:22.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:22 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:22.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:22 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:22.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:22 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:22.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:22 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T06:05:22.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:22 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T06:05:22.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:22 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:05:22.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.394+0000 7f72eee00700 1 -- 172.21.15.123:0/259038357 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72e8103750 msgr2=0x7f72e8103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:22.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.394+0000 7f72eee00700 1 --2- 172.21.15.123:0/259038357 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72e8103750 0x7f72e8103b30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f72d8009990 tx=0x7f72d8009ca0 comp rx=0 tx=0).stop 2024-05-08T06:05:22.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.394+0000 7f72eee00700 1 -- 172.21.15.123:0/259038357 shutdown_connections 2024-05-08T06:05:22.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.394+0000 7f72eee00700 1 --2- 172.21.15.123:0/259038357 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72e8108850 0x7f72e81103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:22.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.394+0000 7f72eee00700 1 --2- 172.21.15.123:0/259038357 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72e8103750 0x7f72e8103b30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:22.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.394+0000 7f72eee00700 1 --2- 172.21.15.123:0/259038357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72e8104100 0x7f72e8108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:22.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.394+0000 7f72eee00700 1 -- 172.21.15.123:0/259038357 >> 172.21.15.123:0/259038357 conn(0x7f72e807a600 msgr2=0x7f72e807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:05:22.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.394+0000 7f72eee00700 1 -- 172.21.15.123:0/259038357 shutdown_connections 2024-05-08T06:05:22.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.394+0000 7f72eee00700 1 -- 172.21.15.123:0/259038357 wait complete. 2024-05-08T06:05:22.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.396+0000 7f72eee00700 1 Processor -- start 2024-05-08T06:05:22.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.396+0000 7f72eee00700 1 -- start start 2024-05-08T06:05:22.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.396+0000 7f72eee00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72e8103750 0x7f72e819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:22.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.396+0000 7f72eee00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72e8104100 0x7f72e819fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:22.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.396+0000 7f72eee00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72e8108850 0x7f72e8199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:22.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.396+0000 7f72eee00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f72e8113690 con 0x7f72e8103750 2024-05-08T06:05:22.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.396+0000 7f72eee00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f72e8113510 con 0x7f72e8108850 2024-05-08T06:05:22.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.396+0000 7f72eee00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f72e8113810 con 0x7f72e8104100 2024-05-08T06:05:22.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.397+0000 7f72e7400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72e8104100 0x7f72e819fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:22.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.397+0000 7f72e7400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72e8104100 0x7f72e819fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:41434/0 (socket says 172.21.15.123:41434) 2024-05-08T06:05:22.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.397+0000 7f72e7400700 1 -- 172.21.15.123:0/813870229 learned_addr learned my addr 172.21.15.123:0/813870229 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:05:22.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.397+0000 7f72ece00700 1 --2- 172.21.15.123:0/813870229 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72e8108850 0x7f72e8199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:22.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.397+0000 7f72e7e00700 1 --2- 172.21.15.123:0/813870229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72e8103750 0x7f72e819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:22.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.397+0000 7f72e7400700 1 -- 172.21.15.123:0/813870229 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72e8108850 msgr2=0x7f72e8199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:22.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.397+0000 7f72e7400700 1 --2- 172.21.15.123:0/813870229 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72e8108850 0x7f72e8199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:22.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.397+0000 7f72e7400700 1 -- 172.21.15.123:0/813870229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72e8103750 msgr2=0x7f72e819f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:22.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.397+0000 7f72e7400700 1 --2- 172.21.15.123:0/813870229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72e8103750 0x7f72e819f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:22.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.397+0000 7f72e7400700 1 -- 172.21.15.123:0/813870229 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72e819a090 con 0x7f72e8104100 2024-05-08T06:05:22.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.398+0000 7f72e7400700 1 --2- 172.21.15.123:0/813870229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72e8104100 0x7f72e819fcb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f72d400e9a0 tx=0x7f72d400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:05:22.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.398+0000 7f72e4c00700 1 -- 172.21.15.123:0/813870229 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d400ca10 con 0x7f72e8104100 2024-05-08T06:05:22.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.398+0000 7f72e4c00700 1 -- 172.21.15.123:0/813870229 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72d400cbb0 con 0x7f72e8104100 2024-05-08T06:05:22.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.398+0000 7f72eee00700 1 -- 172.21.15.123:0/813870229 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72e819a380 con 0x7f72e8104100 2024-05-08T06:05:22.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.398+0000 7f72e4c00700 1 -- 172.21.15.123:0/813870229 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d4018680 con 0x7f72e8104100 2024-05-08T06:05:22.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.398+0000 7f72eee00700 1 -- 172.21.15.123:0/813870229 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72e81a6640 con 0x7f72e8104100 2024-05-08T06:05:22.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.399+0000 7f72eee00700 1 -- 172.21.15.123:0/813870229 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72e810ba90 con 0x7f72e8104100 2024-05-08T06:05:22.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.402+0000 7f72e4c00700 1 -- 172.21.15.123:0/813870229 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f72d404aab0 con 0x7f72e8104100 2024-05-08T06:05:22.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.404+0000 7f72e4c00700 1 --2- 172.21.15.123:0/813870229 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f72cc0b2900 0x7f72cc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:22.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.404+0000 7f72e4c00700 1 -- 172.21.15.123:0/813870229 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f72d40e1fe0 con 0x7f72e8104100 2024-05-08T06:05:22.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.404+0000 7f72e7e00700 1 --2- 172.21.15.123:0/813870229 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f72cc0b2900 0x7f72cc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:22.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.405+0000 7f72e7e00700 1 --2- 172.21.15.123:0/813870229 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f72cc0b2900 0x7f72cc0b4dc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f72d800b790 tx=0x7f72d801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:05:22.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.405+0000 7f72e4c00700 1 -- 172.21.15.123:0/813870229 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f72d40aa100 con 0x7f72e8104100 2024-05-08T06:05:22.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:22 smithi163 ceph-mon[34520]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:22.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:22 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:22.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:22 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:22.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:22 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:22.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:22 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T06:05:22.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:22 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T06:05:22.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:22 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:05:22.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:22 smithi192 ceph-mon[34664]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:22.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:22 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:22.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:22 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:22.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:22 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:22.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:22 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T06:05:22.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:22 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T06:05:22.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:22 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:05:22.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.701+0000 7f72eee00700 1 -- 172.21.15.123:0/813870229 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72e806a190 con 0x7f72e8104100 2024-05-08T06:05:22.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.702+0000 7f72e4c00700 1 -- 172.21.15.123:0/813870229 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8839 (secure 0 0 0) 0x7f72d40b2180 con 0x7f72e8104100 2024-05-08T06:05:22.703 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:05:22.703 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3646476522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3646476522},{"type":"v1","addr":"172.21.15.123:6835","nonce":3646476522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2763706497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2763706497},{"type":"v1","addr":"172.21.15.123:6837","nonce":2763706497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/274635265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":274635265},{"type":"v1","addr":"172.21.15.123:6839","nonce":274635265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/175423115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":175423115},{"type":"v1","addr":"172.21.15.192:6833","nonce":175423115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/3263009472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3263009472},{"type":"v1","addr":"172.21.15.163:6833","nonce":3263009472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3308612826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3308612826},{"type":"v1","addr":"172.21.15.163:6835","nonce":3308612826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/202513680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":202513680},{"type":"v1","addr":"172.21.15.192:6835","nonce":202513680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/2483545969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2483545969},{"type":"v1","addr":"172.21.15.192:6837","nonce":2483545969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/660511838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":660511838},{"type":"v1","addr":"172.21.15.163:6837","nonce":660511838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/3598682360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":3598682360},{"type":"v1","addr":"172.21.15.192:6839","nonce":3598682360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-08T06:05:22.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.706+0000 7f72eee00700 1 -- 172.21.15.123:0/813870229 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f72cc0b2900 msgr2=0x7f72cc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:22.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.706+0000 7f72eee00700 1 --2- 172.21.15.123:0/813870229 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f72cc0b2900 0x7f72cc0b4dc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f72d800b790 tx=0x7f72d801a040 comp rx=0 tx=0).stop 2024-05-08T06:05:22.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.706+0000 7f72eee00700 1 -- 172.21.15.123:0/813870229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72e8104100 msgr2=0x7f72e819fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:22.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.706+0000 7f72eee00700 1 --2- 172.21.15.123:0/813870229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72e8104100 0x7f72e819fcb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f72d400e9a0 tx=0x7f72d400ecb0 comp rx=0 tx=0).stop 2024-05-08T06:05:22.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.706+0000 7f72eee00700 1 -- 172.21.15.123:0/813870229 shutdown_connections 2024-05-08T06:05:22.706 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.706+0000 7f72eee00700 1 --2- 172.21.15.123:0/813870229 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f72cc0b2900 0x7f72cc0b4dc0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:22.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.706+0000 7f72eee00700 1 --2- 172.21.15.123:0/813870229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72e8103750 0x7f72e819f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:22.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.706+0000 7f72eee00700 1 --2- 172.21.15.123:0/813870229 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72e8108850 0x7f72e8199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:22.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.706+0000 7f72eee00700 1 --2- 172.21.15.123:0/813870229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72e8104100 0x7f72e819fcb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:22.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.706+0000 7f72eee00700 1 -- 172.21.15.123:0/813870229 >> 172.21.15.123:0/813870229 conn(0x7f72e807a600 msgr2=0x7f72e81016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:05:22.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.706+0000 7f72eee00700 1 -- 172.21.15.123:0/813870229 shutdown_connections 2024-05-08T06:05:22.707 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:22.707+0000 7f72eee00700 1 -- 172.21.15.123:0/813870229 wait complete. 2024-05-08T06:05:22.708 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 11 2024-05-08T06:05:23.356 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:23 smithi163 ceph-mon[34520]: Deploying daemon mds.a on smithi123 2024-05-08T06:05:23.356 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:23 smithi163 ceph-mon[34520]: mds.? [v2:172.21.15.123:6838/274635265,v1:172.21.15.123:6839/274635265] up:boot 2024-05-08T06:05:23.356 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:23 smithi163 ceph-mon[34520]: fsmap 10 up:standby 2024-05-08T06:05:23.356 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:23 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T06:05:23.356 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:23 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/813870229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:05:23.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:23 smithi123 ceph-mon[29787]: Deploying daemon mds.a on smithi123 2024-05-08T06:05:23.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:23 smithi123 ceph-mon[29787]: mds.? [v2:172.21.15.123:6838/274635265,v1:172.21.15.123:6839/274635265] up:boot 2024-05-08T06:05:23.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:23 smithi123 ceph-mon[29787]: fsmap 10 up:standby 2024-05-08T06:05:23.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:23 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T06:05:23.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:23 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/813870229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:05:23.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:23 smithi192 ceph-mon[34664]: Deploying daemon mds.a on smithi123 2024-05-08T06:05:23.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:23 smithi192 ceph-mon[34664]: mds.? [v2:172.21.15.123:6838/274635265,v1:172.21.15.123:6839/274635265] up:boot 2024-05-08T06:05:23.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:23 smithi192 ceph-mon[34664]: fsmap 10 up:standby 2024-05-08T06:05:23.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:23 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T06:05:23.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:23 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/813870229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:05:24.411 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:24 smithi123 ceph-mon[29787]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:24.534 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:24 smithi163 ceph-mon[34520]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:24.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:24 smithi192 ceph-mon[34664]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:25.837 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T06:05:26.444 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:26 smithi123 ceph-mon[29787]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:26.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:26 smithi163 ceph-mon[34520]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:26.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:26 smithi192 ceph-mon[34664]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:26.705 INFO:journalctl@ceph.mds.a.smithi123.stdout:May 08 06:05:26 smithi123 podman[71240]: 2024-05-08 06:05:26.493698893 +0000 UTC m=+0.183267680 container create daf390136276f52772900b181b2ebb6da473dd321aa6044ae27bf3dfb450ca07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-a, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, ceph=True) 2024-05-08T06:05:27.052 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:05:27.204 INFO:journalctl@ceph.mds.a.smithi123.stdout:May 08 06:05:26 smithi123 podman[71240]: 2024-05-08 06:05:26.839350742 +0000 UTC m=+0.528919523 container init daf390136276f52772900b181b2ebb6da473dd321aa6044ae27bf3dfb450ca07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-08T06:05:27.205 INFO:journalctl@ceph.mds.a.smithi123.stdout:May 08 06:05:26 smithi123 podman[71240]: 2024-05-08 06:05:26.849640403 +0000 UTC m=+0.539209207 container start daf390136276f52772900b181b2ebb6da473dd321aa6044ae27bf3dfb450ca07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-08T06:05:27.205 INFO:journalctl@ceph.mds.a.smithi123.stdout:May 08 06:05:26 smithi123 ceph-mds[71277]: -- 172.21.15.123:0/3467606702 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d0eed05a40 con 0x55d0eecd8c00 2024-05-08T06:05:27.205 INFO:journalctl@ceph.mds.a.smithi123.stdout:May 08 06:05:26 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-a[71273]: starting mds.a at 2024-05-08T06:05:27.205 INFO:journalctl@ceph.mds.a.smithi123.stdout:May 08 06:05:26 smithi123 bash[71240]: daf390136276f52772900b181b2ebb6da473dd321aa6044ae27bf3dfb450ca07 2024-05-08T06:05:27.205 INFO:journalctl@ceph.mds.a.smithi123.stdout:May 08 06:05:26 smithi123 systemd[1]: Started Ceph mds.a for 85cfb2e6-0cff-11ef-bc97-c7b262605968. 2024-05-08T06:05:28.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:28 smithi123 ceph-mon[29787]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:28.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:28 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:28.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:28 smithi123 ceph-mon[29787]: mds.? [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] up:boot 2024-05-08T06:05:28.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:28 smithi123 ceph-mon[29787]: fsmap 11 up:standby 2024-05-08T06:05:28.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:28 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T06:05:28.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:28 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:28.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:28 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:28.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:28 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T06:05:28.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:28 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T06:05:28.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:28 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:05:28.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:28 smithi123 ceph-mon[29787]: Deploying daemon mds.b on smithi163 2024-05-08T06:05:28.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:28 smithi163 ceph-mon[34520]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:28.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:28 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:28.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:28 smithi163 ceph-mon[34520]: mds.? [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] up:boot 2024-05-08T06:05:28.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:28 smithi163 ceph-mon[34520]: fsmap 11 up:standby 2024-05-08T06:05:28.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:28 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T06:05:28.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:28 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:28.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:28 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:28.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:28 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T06:05:28.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:28 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T06:05:28.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:28 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:05:28.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:28 smithi163 ceph-mon[34520]: Deploying daemon mds.b on smithi163 2024-05-08T06:05:28.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:28 smithi192 ceph-mon[34664]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:28.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:28 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:28.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:28 smithi192 ceph-mon[34664]: mds.? [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] up:boot 2024-05-08T06:05:28.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:28 smithi192 ceph-mon[34664]: fsmap 11 up:standby 2024-05-08T06:05:28.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:28 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T06:05:28.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:28 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:28.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:28 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:28.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:28 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T06:05:28.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:28 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T06:05:28.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:28 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:05:28.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:28 smithi192 ceph-mon[34664]: Deploying daemon mds.b on smithi163 2024-05-08T06:05:29.527 INFO:journalctl@ceph.mds.b.smithi163.stdout:May 08 06:05:29 smithi163 systemd[1]: Starting Ceph mds.b for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T06:05:29.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.570+0000 7ffa5ca00700 1 -- 172.21.15.123:0/2618351012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa58104f50 msgr2=0x7ffa5810d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:29.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.570+0000 7ffa5ca00700 1 --2- 172.21.15.123:0/2618351012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa58104f50 0x7ffa5810d440 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ffa44009990 tx=0x7ffa44009ca0 comp rx=0 tx=0).stop 2024-05-08T06:05:29.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.570+0000 7ffa5ca00700 1 -- 172.21.15.123:0/2618351012 shutdown_connections 2024-05-08T06:05:29.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.570+0000 7ffa5ca00700 1 --2- 172.21.15.123:0/2618351012 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa5810da10 0x7ffa5810fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:29.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.570+0000 7ffa5ca00700 1 --2- 172.21.15.123:0/2618351012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa58104630 0x7ffa58104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:29.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.570+0000 7ffa5ca00700 1 --2- 172.21.15.123:0/2618351012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa58104f50 0x7ffa5810d440 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:29.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.570+0000 7ffa5ca00700 1 -- 172.21.15.123:0/2618351012 >> 172.21.15.123:0/2618351012 conn(0x7ffa580fddc0 msgr2=0x7ffa581001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:05:29.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.570+0000 7ffa5ca00700 1 -- 172.21.15.123:0/2618351012 shutdown_connections 2024-05-08T06:05:29.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.570+0000 7ffa5ca00700 1 -- 172.21.15.123:0/2618351012 wait complete. 2024-05-08T06:05:29.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.571+0000 7ffa5ca00700 1 Processor -- start 2024-05-08T06:05:29.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.572+0000 7ffa5ca00700 1 -- start start 2024-05-08T06:05:29.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.572+0000 7ffa5ca00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa58104630 0x7ffa58102290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:29.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.572+0000 7ffa5ca00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa58104f50 0x7ffa581027d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:29.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.572+0000 7ffa5ca00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa5810da10 0x7ffa58102d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:29.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.572+0000 7ffa5ca00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffa581135f0 con 0x7ffa5810da10 2024-05-08T06:05:29.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.572+0000 7ffa5ca00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ffa58113470 con 0x7ffa58104f50 2024-05-08T06:05:29.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.572+0000 7ffa5ca00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ffa58113770 con 0x7ffa58104630 2024-05-08T06:05:29.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.572+0000 7ffa57e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa5810da10 0x7ffa58102d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:29.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.572+0000 7ffa57e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa5810da10 0x7ffa58102d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52366/0 (socket says 172.21.15.123:52366) 2024-05-08T06:05:29.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.572+0000 7ffa57e00700 1 -- 172.21.15.123:0/2678103203 learned_addr learned my addr 172.21.15.123:0/2678103203 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:05:29.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.572+0000 7ffa56a00700 1 --2- 172.21.15.123:0/2678103203 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa58104f50 0x7ffa581027d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:29.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.572+0000 7ffa57400700 1 --2- 172.21.15.123:0/2678103203 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa58104630 0x7ffa58102290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:29.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.573+0000 7ffa56a00700 1 -- 172.21.15.123:0/2678103203 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa58104630 msgr2=0x7ffa58102290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:29.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.573+0000 7ffa56a00700 1 --2- 172.21.15.123:0/2678103203 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa58104630 0x7ffa58102290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:29.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.573+0000 7ffa56a00700 1 -- 172.21.15.123:0/2678103203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa5810da10 msgr2=0x7ffa58102d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:29.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.573+0000 7ffa56a00700 1 --2- 172.21.15.123:0/2678103203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa5810da10 0x7ffa58102d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:29.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.573+0000 7ffa56a00700 1 -- 172.21.15.123:0/2678103203 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa581a8040 con 0x7ffa58104f50 2024-05-08T06:05:29.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.573+0000 7ffa56a00700 1 --2- 172.21.15.123:0/2678103203 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa58104f50 0x7ffa581027d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ffa440098e0 tx=0x7ffa440057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:05:29.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.573+0000 7ffa4fe00700 1 -- 172.21.15.123:0/2678103203 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44027070 con 0x7ffa58104f50 2024-05-08T06:05:29.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.573+0000 7ffa4fe00700 1 -- 172.21.15.123:0/2678103203 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa44022470 con 0x7ffa58104f50 2024-05-08T06:05:29.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.573+0000 7ffa5ca00700 1 -- 172.21.15.123:0/2678103203 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa581a82d0 con 0x7ffa58104f50 2024-05-08T06:05:29.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.573+0000 7ffa4fe00700 1 -- 172.21.15.123:0/2678103203 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44017650 con 0x7ffa58104f50 2024-05-08T06:05:29.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.574+0000 7ffa5ca00700 1 -- 172.21.15.123:0/2678103203 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa581a87b0 con 0x7ffa58104f50 2024-05-08T06:05:29.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.575+0000 7ffa5ca00700 1 -- 172.21.15.123:0/2678103203 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa58103770 con 0x7ffa58104f50 2024-05-08T06:05:29.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.577+0000 7ffa4fe00700 1 -- 172.21.15.123:0/2678103203 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7ffa4402a070 con 0x7ffa58104f50 2024-05-08T06:05:29.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.578+0000 7ffa4fe00700 1 --2- 172.21.15.123:0/2678103203 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7ffa3c0b2880 0x7ffa3c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:29.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.578+0000 7ffa4fe00700 1 -- 172.21.15.123:0/2678103203 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7ffa440e2ac0 con 0x7ffa58104f50 2024-05-08T06:05:29.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.578+0000 7ffa57400700 1 --2- 172.21.15.123:0/2678103203 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7ffa3c0b2880 0x7ffa3c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:29.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.579+0000 7ffa57400700 1 --2- 172.21.15.123:0/2678103203 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7ffa3c0b2880 0x7ffa3c0b4d40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ffa48009c60 tx=0x7ffa48009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:05:29.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.581+0000 7ffa4fe00700 1 -- 172.21.15.123:0/2678103203 <== mon.2 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffa44013070 con 0x7ffa58104f50 2024-05-08T06:05:29.788 INFO:journalctl@ceph.mds.b.smithi163.stdout:May 08 06:05:29 smithi163 podman[65022]: 2024-05-08 06:05:29.431415208 +0000 UTC m=+0.018094542 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:05:29.788 INFO:journalctl@ceph.mds.b.smithi163.stdout:May 08 06:05:29 smithi163 podman[65022]: 2024-05-08 06:05:29.561229286 +0000 UTC m=+0.147908603 container create 222c13906a21dea588a675641b306a4ab79d1ea56abd064146c95193658bb259 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-08T06:05:29.789 INFO:journalctl@ceph.mds.b.smithi163.stdout:May 08 06:05:29 smithi163 podman[65022]: 2024-05-08 06:05:29.772131492 +0000 UTC m=+0.358810810 container init 222c13906a21dea588a675641b306a4ab79d1ea56abd064146c95193658bb259 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-08T06:05:29.789 INFO:journalctl@ceph.mds.b.smithi163.stdout:May 08 06:05:29 smithi163 podman[65022]: 2024-05-08 06:05:29.774862221 +0000 UTC m=+0.361541531 container start 222c13906a21dea588a675641b306a4ab79d1ea56abd064146c95193658bb259 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-b, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-08T06:05:29.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.861+0000 7ffa5ca00700 1 -- 172.21.15.123:0/2678103203 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa5802e510 con 0x7ffa58104f50 2024-05-08T06:05:29.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.862+0000 7ffa4fe00700 1 -- 172.21.15.123:0/2678103203 <== mon.2 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9668 (secure 0 0 0) 0x7ffa440aac70 con 0x7ffa58104f50 2024-05-08T06:05:29.862 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:05:29.863 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3646476522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3646476522},{"type":"v1","addr":"172.21.15.123:6835","nonce":3646476522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2763706497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2763706497},{"type":"v1","addr":"172.21.15.123:6837","nonce":2763706497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/274635265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":274635265},{"type":"v1","addr":"172.21.15.123:6839","nonce":274635265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/175423115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":175423115},{"type":"v1","addr":"172.21.15.192:6833","nonce":175423115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/3263009472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3263009472},{"type":"v1","addr":"172.21.15.163:6833","nonce":3263009472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3308612826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3308612826},{"type":"v1","addr":"172.21.15.163:6835","nonce":3308612826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/202513680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":202513680},{"type":"v1","addr":"172.21.15.192:6835","nonce":202513680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/2483545969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2483545969},{"type":"v1","addr":"172.21.15.192:6837","nonce":2483545969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/660511838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":660511838},{"type":"v1","addr":"172.21.15.163:6837","nonce":660511838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/3598682360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":3598682360},{"type":"v1","addr":"172.21.15.192:6839","nonce":3598682360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24512,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6841/1571726231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":1571726231},{"type":"v1","addr":"172.21.15.123:6841","nonce":1571726231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[]} 2024-05-08T06:05:29.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.866+0000 7ffa5ca00700 1 -- 172.21.15.123:0/2678103203 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7ffa3c0b2880 msgr2=0x7ffa3c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:29.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.866+0000 7ffa5ca00700 1 --2- 172.21.15.123:0/2678103203 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7ffa3c0b2880 0x7ffa3c0b4d40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ffa48009c60 tx=0x7ffa48009260 comp rx=0 tx=0).stop 2024-05-08T06:05:29.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.866+0000 7ffa5ca00700 1 -- 172.21.15.123:0/2678103203 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa58104f50 msgr2=0x7ffa581027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:29.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.866+0000 7ffa5ca00700 1 --2- 172.21.15.123:0/2678103203 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa58104f50 0x7ffa581027d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ffa440098e0 tx=0x7ffa440057c0 comp rx=0 tx=0).stop 2024-05-08T06:05:29.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.867+0000 7ffa5ca00700 1 -- 172.21.15.123:0/2678103203 shutdown_connections 2024-05-08T06:05:29.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.867+0000 7ffa5ca00700 1 --2- 172.21.15.123:0/2678103203 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7ffa3c0b2880 0x7ffa3c0b4d40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:29.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.867+0000 7ffa5ca00700 1 --2- 172.21.15.123:0/2678103203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa5810da10 0x7ffa58102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:29.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.867+0000 7ffa5ca00700 1 --2- 172.21.15.123:0/2678103203 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa58104630 0x7ffa58102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:29.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.867+0000 7ffa5ca00700 1 --2- 172.21.15.123:0/2678103203 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa58104f50 0x7ffa581027d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:29.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.867+0000 7ffa5ca00700 1 -- 172.21.15.123:0/2678103203 >> 172.21.15.123:0/2678103203 conn(0x7ffa580fddc0 msgr2=0x7ffa580feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:05:29.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.867+0000 7ffa5ca00700 1 -- 172.21.15.123:0/2678103203 shutdown_connections 2024-05-08T06:05:29.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:29.867+0000 7ffa5ca00700 1 -- 172.21.15.123:0/2678103203 wait complete. 2024-05-08T06:05:29.869 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 12 2024-05-08T06:05:30.133 INFO:journalctl@ceph.mds.b.smithi163.stdout:May 08 06:05:29 smithi163 ceph-mds[65041]: -- 172.21.15.163:0/2093543401 <== mon.2 v2:172.21.15.163:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563f06c5da40 con 0x563f06c30800 2024-05-08T06:05:30.133 INFO:journalctl@ceph.mds.b.smithi163.stdout:May 08 06:05:29 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-b[65037]: starting mds.b at 2024-05-08T06:05:30.134 INFO:journalctl@ceph.mds.b.smithi163.stdout:May 08 06:05:29 smithi163 bash[65022]: 222c13906a21dea588a675641b306a4ab79d1ea56abd064146c95193658bb259 2024-05-08T06:05:30.134 INFO:journalctl@ceph.mds.b.smithi163.stdout:May 08 06:05:29 smithi163 systemd[1]: Started Ceph mds.b for 85cfb2e6-0cff-11ef-bc97-c7b262605968. 2024-05-08T06:05:30.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:30 smithi123 ceph-mon[29787]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:30.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:30 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/2678103203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:05:30.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:30 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:30.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:30 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:30.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:30 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:30.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:30 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:30.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:30 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:30.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:30 smithi163 ceph-mon[34520]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:30.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:30 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/2678103203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:05:30.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:30 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:30.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:30 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:30.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:30 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:30.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:30 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:30.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:30 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:30.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:30 smithi192 ceph-mon[34664]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:30.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:30 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/2678103203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:05:30.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:30 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:30.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:30 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:30.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:30 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:30.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:30 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:30.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:30 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:31.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:31 smithi163 ceph-mon[34520]: mds.? [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] up:boot 2024-05-08T06:05:31.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:31 smithi163 ceph-mon[34520]: fsmap 12 up:standby 2024-05-08T06:05:31.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:31 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T06:05:31.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:31 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:05:31.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:31 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:31.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:31 smithi123 ceph-mon[29787]: mds.? [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] up:boot 2024-05-08T06:05:31.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:31 smithi123 ceph-mon[29787]: fsmap 12 up:standby 2024-05-08T06:05:31.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:31 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T06:05:31.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:31 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:05:31.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:31 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:31.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:31 smithi192 ceph-mon[34664]: mds.? [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] up:boot 2024-05-08T06:05:31.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:31 smithi192 ceph-mon[34664]: fsmap 12 up:standby 2024-05-08T06:05:31.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:31 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T06:05:31.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:31 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:05:31.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:31 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:32.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:32 smithi123 ceph-mon[29787]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:32.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:32 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:32.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:32 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:32.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:32 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:32.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:32 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:32.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:32 smithi163 ceph-mon[34520]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:32.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:32 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:32.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:32 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:32.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:32 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:32.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:32 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:32.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:32 smithi192 ceph-mon[34664]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:32.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:32 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:32.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:32 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:32.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:32 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:32.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:32 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:32.894 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T06:05:33.349 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:05:34.446 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:34 smithi192 ceph-mon[34664]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:34.446 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:34 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:34.446 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:34 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:34.447 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:34 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:05:34.447 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:34 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:05:34.447 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:34 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:34.447 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:34 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:05:34.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:34 smithi123 ceph-mon[29787]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:34.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:34 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:34.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:34 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:34.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:34 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:05:34.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:34 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:05:34.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:34 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:34.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:34 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:05:34.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:34 smithi163 ceph-mon[34520]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:34.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:34 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:34.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:34 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:34.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:34 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:05:34.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:34 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:05:34.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:34 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:34.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:34 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:05:35.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.894+0000 7f426a400700 1 -- 172.21.15.123:0/3575938362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4264103750 msgr2=0x7f4264103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:35.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.894+0000 7f426a400700 1 --2- 172.21.15.123:0/3575938362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4264103750 0x7f4264103b30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f42540099c0 tx=0x7f4254009cd0 comp rx=0 tx=0).stop 2024-05-08T06:05:35.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.894+0000 7f426a400700 1 -- 172.21.15.123:0/3575938362 shutdown_connections 2024-05-08T06:05:35.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.894+0000 7f426a400700 1 --2- 172.21.15.123:0/3575938362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4264103750 0x7f4264103b30 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:35.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.894+0000 7f426a400700 1 --2- 172.21.15.123:0/3575938362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4264108850 0x7f42641103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:35.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.894+0000 7f426a400700 1 --2- 172.21.15.123:0/3575938362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4264104100 0x7f4264108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:35.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.894+0000 7f426a400700 1 -- 172.21.15.123:0/3575938362 >> 172.21.15.123:0/3575938362 conn(0x7f426407a600 msgr2=0x7f426407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:05:35.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.894+0000 7f426a400700 1 -- 172.21.15.123:0/3575938362 shutdown_connections 2024-05-08T06:05:35.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.894+0000 7f426a400700 1 -- 172.21.15.123:0/3575938362 wait complete. 2024-05-08T06:05:35.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.895+0000 7f426a400700 1 Processor -- start 2024-05-08T06:05:35.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.896+0000 7f426a400700 1 -- start start 2024-05-08T06:05:35.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.896+0000 7f426a400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4264103750 0x7f426410a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:35.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.896+0000 7f426a400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4264104100 0x7f426410a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:35.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.896+0000 7f426a400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4264108850 0x7f426410ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:35.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.896+0000 7f426a400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4264113690 con 0x7f4264108850 2024-05-08T06:05:35.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.896+0000 7f426a400700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4264113510 con 0x7f4264104100 2024-05-08T06:05:35.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.896+0000 7f426a400700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4264113810 con 0x7f4264103750 2024-05-08T06:05:35.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.896+0000 7f4263e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4264108850 0x7f426410ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:35.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.896+0000 7f4262a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4264104100 0x7f426410a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:35.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.896+0000 7f4263e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4264108850 0x7f426410ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35694/0 (socket says 172.21.15.123:35694) 2024-05-08T06:05:35.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.896+0000 7f4263e00700 1 -- 172.21.15.123:0/313893551 learned_addr learned my addr 172.21.15.123:0/313893551 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:05:35.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.896+0000 7f4263400700 1 --2- 172.21.15.123:0/313893551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4264103750 0x7f426410a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:35.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.897+0000 7f4262a00700 1 -- 172.21.15.123:0/313893551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4264103750 msgr2=0x7f426410a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:35.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.897+0000 7f4262a00700 1 --2- 172.21.15.123:0/313893551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4264103750 0x7f426410a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:35.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.897+0000 7f4262a00700 1 -- 172.21.15.123:0/313893551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4264108850 msgr2=0x7f426410ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:35.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.897+0000 7f4262a00700 1 --2- 172.21.15.123:0/313893551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4264108850 0x7f426410ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:35.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.897+0000 7f4262a00700 1 -- 172.21.15.123:0/313893551 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42641ac3d0 con 0x7f4264104100 2024-05-08T06:05:35.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.897+0000 7f4263e00700 1 --2- 172.21.15.123:0/313893551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4264108850 0x7f426410ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:05:35.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.897+0000 7f4262a00700 1 --2- 172.21.15.123:0/313893551 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4264104100 0x7f426410a920 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f425000e9f0 tx=0x7f425000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:05:35.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.897+0000 7f425be00700 1 -- 172.21.15.123:0/313893551 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4250004220 con 0x7f4264104100 2024-05-08T06:05:35.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.897+0000 7f426a400700 1 -- 172.21.15.123:0/313893551 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42641ac6c0 con 0x7f4264104100 2024-05-08T06:05:35.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.897+0000 7f425be00700 1 -- 172.21.15.123:0/313893551 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42500043c0 con 0x7f4264104100 2024-05-08T06:05:35.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.897+0000 7f425be00700 1 -- 172.21.15.123:0/313893551 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4250005020 con 0x7f4264104100 2024-05-08T06:05:35.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.898+0000 7f426a400700 1 -- 172.21.15.123:0/313893551 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42641acbb0 con 0x7f4264104100 2024-05-08T06:05:35.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.899+0000 7f426a400700 1 -- 172.21.15.123:0/313893551 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4264104be0 con 0x7f4264104100 2024-05-08T06:05:35.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.902+0000 7f425be00700 1 -- 172.21.15.123:0/313893551 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f4250004750 con 0x7f4264104100 2024-05-08T06:05:35.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.903+0000 7f425be00700 1 --2- 172.21.15.123:0/313893551 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f42480b28b0 0x7f42480b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:35.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.903+0000 7f425be00700 1 -- 172.21.15.123:0/313893551 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f42500e1c30 con 0x7f4264104100 2024-05-08T06:05:35.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.903+0000 7f4263400700 1 --2- 172.21.15.123:0/313893551 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f42480b28b0 0x7f42480b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:35.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.904+0000 7f4263400700 1 --2- 172.21.15.123:0/313893551 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f42480b28b0 0x7f42480b4d70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4254000c00 tx=0x7f4254005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:05:35.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:35.904+0000 7f425be00700 1 -- 172.21.15.123:0/313893551 <== mon.2 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f42500a9dd0 con 0x7f4264104100 2024-05-08T06:05:36.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:36.182+0000 7f426a400700 1 -- 172.21.15.123:0/313893551 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f426406a190 con 0x7f4264104100 2024-05-08T06:05:36.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:36.183+0000 7f425be00700 1 -- 172.21.15.123:0/313893551 <== mon.2 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10497 (secure 0 0 0) 0x7f42500a96d0 con 0x7f4264104100 2024-05-08T06:05:36.183 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:05:36.183 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3646476522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3646476522},{"type":"v1","addr":"172.21.15.123:6835","nonce":3646476522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2763706497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2763706497},{"type":"v1","addr":"172.21.15.123:6837","nonce":2763706497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/274635265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":274635265},{"type":"v1","addr":"172.21.15.123:6839","nonce":274635265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/175423115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":175423115},{"type":"v1","addr":"172.21.15.192:6833","nonce":175423115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/3263009472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3263009472},{"type":"v1","addr":"172.21.15.163:6833","nonce":3263009472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3308612826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3308612826},{"type":"v1","addr":"172.21.15.163:6835","nonce":3308612826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/202513680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":202513680},{"type":"v1","addr":"172.21.15.192:6835","nonce":202513680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/2483545969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2483545969},{"type":"v1","addr":"172.21.15.192:6837","nonce":2483545969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/660511838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":660511838},{"type":"v1","addr":"172.21.15.163:6837","nonce":660511838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/3598682360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":3598682360},{"type":"v1","addr":"172.21.15.192:6839","nonce":3598682360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24512,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6841/1571726231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":1571726231},{"type":"v1","addr":"172.21.15.123:6841","nonce":1571726231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6839/1568161515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":1568161515},{"type":"v1","addr":"172.21.15.163:6839","nonce":1568161515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-05-08T06:05:36.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:36.186+0000 7f426a400700 1 -- 172.21.15.123:0/313893551 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f42480b28b0 msgr2=0x7f42480b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:36.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:36.186+0000 7f426a400700 1 --2- 172.21.15.123:0/313893551 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f42480b28b0 0x7f42480b4d70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4254000c00 tx=0x7f4254005fb0 comp rx=0 tx=0).stop 2024-05-08T06:05:36.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:36.186+0000 7f426a400700 1 -- 172.21.15.123:0/313893551 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4264104100 msgr2=0x7f426410a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:36.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:36.186+0000 7f426a400700 1 --2- 172.21.15.123:0/313893551 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4264104100 0x7f426410a920 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f425000e9f0 tx=0x7f425000ed00 comp rx=0 tx=0).stop 2024-05-08T06:05:36.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:36.187+0000 7f426a400700 1 -- 172.21.15.123:0/313893551 shutdown_connections 2024-05-08T06:05:36.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:36.187+0000 7f426a400700 1 --2- 172.21.15.123:0/313893551 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f42480b28b0 0x7f42480b4d70 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:36.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:36.187+0000 7f426a400700 1 --2- 172.21.15.123:0/313893551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4264108850 0x7f426410ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:36.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:36.187+0000 7f426a400700 1 --2- 172.21.15.123:0/313893551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4264103750 0x7f426410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:36.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:36.187+0000 7f426a400700 1 --2- 172.21.15.123:0/313893551 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4264104100 0x7f426410a920 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:36.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:36.187+0000 7f426a400700 1 -- 172.21.15.123:0/313893551 >> 172.21.15.123:0/313893551 conn(0x7f426407a600 msgr2=0x7f42641099a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:05:36.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:36.187+0000 7f426a400700 1 -- 172.21.15.123:0/313893551 shutdown_connections 2024-05-08T06:05:36.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:36.187+0000 7f426a400700 1 -- 172.21.15.123:0/313893551 wait complete. 2024-05-08T06:05:36.188 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 13 2024-05-08T06:05:36.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:36.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:36 smithi163 ceph-mon[34520]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:36.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:36.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:36.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:36 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:36.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:36 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:36.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:36 smithi192 ceph-mon[34664]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:36.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:36 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:36.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:36 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:36.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:36 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:36.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:36 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:36.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:36 smithi123 ceph-mon[29787]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:36.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:36 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:36.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:36 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:36.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:36 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:37.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:37 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/313893551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:05:37.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:37 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/313893551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:05:37.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:37 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/313893551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:05:37.607 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-08T06:05:38.059 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:05:38.439 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:38 smithi123 ceph-mon[29787]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:38.506 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:38 smithi192 ceph-mon[34664]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:38.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:38 smithi163 ceph-mon[34520]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:40.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.406+0000 7f595b200700 1 -- 172.21.15.123:0/3013851383 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5954105f90 msgr2=0x7f59541124a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:40.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.406+0000 7f595b200700 1 --2- 172.21.15.123:0/3013851383 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5954105f90 0x7f59541124a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f595000b0a0 tx=0x7f595000b3b0 comp rx=0 tx=0).stop 2024-05-08T06:05:40.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.406+0000 7f595b200700 1 -- 172.21.15.123:0/3013851383 shutdown_connections 2024-05-08T06:05:40.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.406+0000 7f595b200700 1 --2- 172.21.15.123:0/3013851383 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5954105f90 0x7f59541124a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:40.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.406+0000 7f595b200700 1 --2- 172.21.15.123:0/3013851383 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f595410b660 0x7f595410ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:40.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.406+0000 7f595b200700 1 --2- 172.21.15.123:0/3013851383 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59541055d0 0x7f5954105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:40.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.406+0000 7f595b200700 1 -- 172.21.15.123:0/3013851383 >> 172.21.15.123:0/3013851383 conn(0x7f59541010c0 msgr2=0x7f59541034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:05:40.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.406+0000 7f595b200700 1 -- 172.21.15.123:0/3013851383 shutdown_connections 2024-05-08T06:05:40.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.406+0000 7f595b200700 1 -- 172.21.15.123:0/3013851383 wait complete. 2024-05-08T06:05:40.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.408+0000 7f595b200700 1 Processor -- start 2024-05-08T06:05:40.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.408+0000 7f595b200700 1 -- start start 2024-05-08T06:05:40.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.408+0000 7f595b200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59541055d0 0x7f59541a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:40.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.408+0000 7f595b200700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5954105f90 0x7f59541a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:40.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.408+0000 7f595b200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f595410b660 0x7f595419b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:40.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.408+0000 7f595b200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f59541157d0 con 0x7f595410b660 2024-05-08T06:05:40.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.408+0000 7f595b200700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5954115650 con 0x7f59541055d0 2024-05-08T06:05:40.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.408+0000 7f595b200700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5954115950 con 0x7f5954105f90 2024-05-08T06:05:40.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.408+0000 7f5959e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59541055d0 0x7f59541a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:40.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.409+0000 7f5959e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59541055d0 0x7f59541a17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:50672/0 (socket says 172.21.15.123:50672) 2024-05-08T06:05:40.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.409+0000 7f5959e00700 1 -- 172.21.15.123:0/2121720167 learned_addr learned my addr 172.21.15.123:0/2121720167 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:05:40.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.409+0000 7f595a800700 1 --2- 172.21.15.123:0/2121720167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f595410b660 0x7f595419b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:40.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.409+0000 7f5959400700 1 --2- 172.21.15.123:0/2121720167 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5954105f90 0x7f59541a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:40.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.409+0000 7f5959e00700 1 -- 172.21.15.123:0/2121720167 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5954105f90 msgr2=0x7f59541a1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:40.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.409+0000 7f5959e00700 1 --2- 172.21.15.123:0/2121720167 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5954105f90 0x7f59541a1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:40.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.409+0000 7f5959e00700 1 -- 172.21.15.123:0/2121720167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f595410b660 msgr2=0x7f595419b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:40.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.409+0000 7f5959e00700 1 --2- 172.21.15.123:0/2121720167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f595410b660 0x7f595419b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:40.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.409+0000 7f5959e00700 1 -- 172.21.15.123:0/2121720167 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f595419c130 con 0x7f59541055d0 2024-05-08T06:05:40.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.409+0000 7f595a800700 1 --2- 172.21.15.123:0/2121720167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f595410b660 0x7f595419b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:05:40.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.410+0000 7f5959e00700 1 --2- 172.21.15.123:0/2121720167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59541055d0 0x7f59541a17c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f594800e950 tx=0x7f594800ec60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:05:40.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.410+0000 7f594ea00700 1 -- 172.21.15.123:0/2121720167 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5948004230 con 0x7f59541055d0 2024-05-08T06:05:40.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.410+0000 7f594ea00700 1 -- 172.21.15.123:0/2121720167 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59480043d0 con 0x7f59541055d0 2024-05-08T06:05:40.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.410+0000 7f595b200700 1 -- 172.21.15.123:0/2121720167 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f595419c420 con 0x7f59541055d0 2024-05-08T06:05:40.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.410+0000 7f594ea00700 1 -- 172.21.15.123:0/2121720167 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5948003d60 con 0x7f59541055d0 2024-05-08T06:05:40.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.410+0000 7f595b200700 1 -- 172.21.15.123:0/2121720167 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59541a8690 con 0x7f59541055d0 2024-05-08T06:05:40.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.411+0000 7f595b200700 1 -- 172.21.15.123:0/2121720167 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f595406a190 con 0x7f59541055d0 2024-05-08T06:05:40.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.413+0000 7f594ea00700 1 -- 172.21.15.123:0/2121720167 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f594800c7e0 con 0x7f59541055d0 2024-05-08T06:05:40.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.415+0000 7f594ea00700 1 --2- 172.21.15.123:0/2121720167 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f593c0b2900 0x7f593c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:40.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.415+0000 7f594ea00700 1 -- 172.21.15.123:0/2121720167 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f59480e1f30 con 0x7f59541055d0 2024-05-08T06:05:40.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.415+0000 7f5959400700 1 --2- 172.21.15.123:0/2121720167 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f593c0b2900 0x7f593c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:40.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.416+0000 7f5959400700 1 --2- 172.21.15.123:0/2121720167 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f593c0b2900 0x7f593c0b4dc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f595410ac90 tx=0x7f59440073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:05:40.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.419+0000 7f594ea00700 1 -- 172.21.15.123:0/2121720167 <== mon.2 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f59480aa050 con 0x7f59541055d0 2024-05-08T06:05:40.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:40 smithi163 ceph-mon[34520]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:40.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:40 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:40.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:40 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:40.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:40 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:05:40.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:40 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:05:40.539 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:40 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:40.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:40 smithi192 ceph-mon[34664]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:40.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:40 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:40.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:40 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:40.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:40 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:05:40.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:40 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:05:40.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:40 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:40.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.652+0000 7f595b200700 1 -- 172.21.15.123:0/2121720167 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f595419ceb0 con 0x7f59541055d0 2024-05-08T06:05:40.653 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:40 smithi123 ceph-mon[29787]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:40.653 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:40 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:40.653 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:40 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:40.653 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:40 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:05:40.653 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:40 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:05:40.653 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:40 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:05:40.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.653+0000 7f594ea00700 1 -- 172.21.15.123:0/2121720167 <== mon.2 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v73) ==== 84+0+65 (secure 0 0 0) 0x7f59480a9950 con 0x7f59541055d0 2024-05-08T06:05:40.654 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:05:40.654 INFO:teuthology.orchestra.run.smithi123.stdout:[ 2024-05-08T06:05:40.654 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:05:40.654 INFO:teuthology.orchestra.run.smithi123.stdout: "poolnum": 1, 2024-05-08T06:05:40.654 INFO:teuthology.orchestra.run.smithi123.stdout: "poolname": ".mgr" 2024-05-08T06:05:40.654 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:05:40.654 INFO:teuthology.orchestra.run.smithi123.stdout:] 2024-05-08T06:05:40.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.657+0000 7f595b200700 1 -- 172.21.15.123:0/2121720167 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f593c0b2900 msgr2=0x7f593c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:40.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.657+0000 7f595b200700 1 --2- 172.21.15.123:0/2121720167 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f593c0b2900 0x7f593c0b4dc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f595410ac90 tx=0x7f59440073d0 comp rx=0 tx=0).stop 2024-05-08T06:05:40.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.657+0000 7f595b200700 1 -- 172.21.15.123:0/2121720167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59541055d0 msgr2=0x7f59541a17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:40.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.657+0000 7f595b200700 1 --2- 172.21.15.123:0/2121720167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59541055d0 0x7f59541a17c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f594800e950 tx=0x7f594800ec60 comp rx=0 tx=0).stop 2024-05-08T06:05:40.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.658+0000 7f595b200700 1 -- 172.21.15.123:0/2121720167 shutdown_connections 2024-05-08T06:05:40.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.658+0000 7f595b200700 1 --2- 172.21.15.123:0/2121720167 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f593c0b2900 0x7f593c0b4dc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:40.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.658+0000 7f595b200700 1 --2- 172.21.15.123:0/2121720167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f595410b660 0x7f595419b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:40.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.658+0000 7f595b200700 1 --2- 172.21.15.123:0/2121720167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59541055d0 0x7f59541a17c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:40.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.658+0000 7f595b200700 1 --2- 172.21.15.123:0/2121720167 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5954105f90 0x7f59541a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:40.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.658+0000 7f595b200700 1 -- 172.21.15.123:0/2121720167 >> 172.21.15.123:0/2121720167 conn(0x7f59541010c0 msgr2=0x7f59541026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:05:40.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.658+0000 7f595b200700 1 -- 172.21.15.123:0/2121720167 shutdown_connections 2024-05-08T06:05:40.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:40.658+0000 7f595b200700 1 -- 172.21.15.123:0/2121720167 wait complete. 2024-05-08T06:05:41.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:41 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/2121720167' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-08T06:05:41.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:41 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/2121720167' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-08T06:05:41.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:41 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/2121720167' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-08T06:05:41.626 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T06:05:41.627 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T06:05:42.077 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:05:42.485 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:42 smithi123 ceph-mon[29787]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:42.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:42 smithi192 ceph-mon[34664]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:42.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:42 smithi163 ceph-mon[34520]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:44.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.552+0000 7f9fd1a00700 1 -- 172.21.15.123:0/2051078661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fcc109510 msgr2=0x7f9fcc1098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:44.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.552+0000 7f9fd1a00700 1 --2- 172.21.15.123:0/2051078661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fcc109510 0x7f9fcc1098f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc009990 tx=0x7f9fbc009ca0 comp rx=0 tx=0).stop 2024-05-08T06:05:44.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.552+0000 7f9fd1a00700 1 -- 172.21.15.123:0/2051078661 shutdown_connections 2024-05-08T06:05:44.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.552+0000 7f9fd1a00700 1 --2- 172.21.15.123:0/2051078661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fcc109510 0x7f9fcc1098f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:44.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.552+0000 7f9fd1a00700 1 --2- 172.21.15.123:0/2051078661 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fcc06bb40 0x7f9fcc110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:44.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.552+0000 7f9fd1a00700 1 --2- 172.21.15.123:0/2051078661 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fcc06b180 0x7f9fcc06b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:44.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.552+0000 7f9fd1a00700 1 -- 172.21.15.123:0/2051078661 >> 172.21.15.123:0/2051078661 conn(0x7f9fcc079350 msgr2=0x7f9fcc079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:05:44.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.552+0000 7f9fd1a00700 1 -- 172.21.15.123:0/2051078661 shutdown_connections 2024-05-08T06:05:44.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.552+0000 7f9fd1a00700 1 -- 172.21.15.123:0/2051078661 wait complete. 2024-05-08T06:05:44.553 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:44 smithi123 ceph-mon[29787]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:44.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.553+0000 7f9fd1a00700 1 Processor -- start 2024-05-08T06:05:44.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.553+0000 7f9fd1a00700 1 -- start start 2024-05-08T06:05:44.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.553+0000 7f9fd1a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fcc06b180 0x7f9fcc199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:44.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.553+0000 7f9fd1a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fcc06bb40 0x7f9fcc19a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:44.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.554+0000 7f9fd1a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fcc109510 0x7f9fcc19e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:44.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.554+0000 7f9fd1a00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc113690 con 0x7f9fcc06bb40 2024-05-08T06:05:44.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.554+0000 7f9fd1a00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc113510 con 0x7f9fcc06b180 2024-05-08T06:05:44.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.554+0000 7f9fd1a00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc113810 con 0x7f9fcc109510 2024-05-08T06:05:44.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.554+0000 7f9fc2000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fcc06bb40 0x7f9fcc19a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:44.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.554+0000 7f9fcaa00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fcc06b180 0x7f9fcc199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:44.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.554+0000 7f9fc2000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fcc06bb40 0x7f9fcc19a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35736/0 (socket says 172.21.15.123:35736) 2024-05-08T06:05:44.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.554+0000 7f9fc2000700 1 -- 172.21.15.123:0/4254276107 learned_addr learned my addr 172.21.15.123:0/4254276107 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:05:44.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.554+0000 7f9fcb400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fcc109510 0x7f9fcc19e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:44.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.554+0000 7f9fc2000700 1 -- 172.21.15.123:0/4254276107 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fcc109510 msgr2=0x7f9fcc19e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:44.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.554+0000 7f9fc2000700 1 --2- 172.21.15.123:0/4254276107 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fcc109510 0x7f9fcc19e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:44.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.554+0000 7f9fc2000700 1 -- 172.21.15.123:0/4254276107 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fcc06b180 msgr2=0x7f9fcc199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:44.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.554+0000 7f9fc2000700 1 --2- 172.21.15.123:0/4254276107 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fcc06b180 0x7f9fcc199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:44.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.554+0000 7f9fc2000700 1 -- 172.21.15.123:0/4254276107 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fcc19ed60 con 0x7f9fcc06bb40 2024-05-08T06:05:44.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.554+0000 7f9fcaa00700 1 --2- 172.21.15.123:0/4254276107 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fcc06b180 0x7f9fcc199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:05:44.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.555+0000 7f9fc2000700 1 --2- 172.21.15.123:0/4254276107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fcc06bb40 0x7f9fcc19a290 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9fb400e9c0 tx=0x7f9fb400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:05:44.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.555+0000 7f9fc3e00700 1 -- 172.21.15.123:0/4254276107 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb400cbd0 con 0x7f9fcc06bb40 2024-05-08T06:05:44.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.555+0000 7f9fd1a00700 1 -- 172.21.15.123:0/4254276107 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fcc19f050 con 0x7f9fcc06bb40 2024-05-08T06:05:44.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.555+0000 7f9fc3e00700 1 -- 172.21.15.123:0/4254276107 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fb400ee60 con 0x7f9fcc06bb40 2024-05-08T06:05:44.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.555+0000 7f9fc3e00700 1 -- 172.21.15.123:0/4254276107 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb40055c0 con 0x7f9fcc06bb40 2024-05-08T06:05:44.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.555+0000 7f9fd1a00700 1 -- 172.21.15.123:0/4254276107 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fcc1023a0 con 0x7f9fcc06bb40 2024-05-08T06:05:44.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.556+0000 7f9fd1a00700 1 -- 172.21.15.123:0/4254276107 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fcc10b9f0 con 0x7f9fcc06bb40 2024-05-08T06:05:44.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.558+0000 7f9fc3e00700 1 -- 172.21.15.123:0/4254276107 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f9fb4004750 con 0x7f9fcc06bb40 2024-05-08T06:05:44.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.560+0000 7f9fc3e00700 1 --2- 172.21.15.123:0/4254276107 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9fb00b2900 0x7f9fb00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:44.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.560+0000 7f9fc3e00700 1 -- 172.21.15.123:0/4254276107 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f9fb40e1ff0 con 0x7f9fcc06bb40 2024-05-08T06:05:44.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.560+0000 7f9fcaa00700 1 --2- 172.21.15.123:0/4254276107 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9fb00b2900 0x7f9fb00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:44.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.561+0000 7f9fcaa00700 1 --2- 172.21.15.123:0/4254276107 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9fb00b2900 0x7f9fb00b4dc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc00b2e0 tx=0x7f9fbc01e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:05:44.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.565+0000 7f9fc3e00700 1 -- 172.21.15.123:0/4254276107 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9fb40aa130 con 0x7f9fcc06bb40 2024-05-08T06:05:44.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:44 smithi163 ceph-mon[34520]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:44.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:44.796+0000 7f9fd1a00700 1 -- 172.21.15.123:0/4254276107 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f9fcc06a190 con 0x7f9fcc06bb40 2024-05-08T06:05:44.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:44 smithi192 ceph-mon[34664]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:45.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:45.315+0000 7f9fc3e00700 1 -- 172.21.15.123:0/4254276107 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v74) ==== 138+0+0 (secure 0 0 0) 0x7f9fb40a9a30 con 0x7f9fcc06bb40 2024-05-08T06:05:45.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:45.322+0000 7f9fd1a00700 1 -- 172.21.15.123:0/4254276107 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9fb00b2900 msgr2=0x7f9fb00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:45.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:45.322+0000 7f9fd1a00700 1 --2- 172.21.15.123:0/4254276107 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9fb00b2900 0x7f9fb00b4dc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc00b2e0 tx=0x7f9fbc01e040 comp rx=0 tx=0).stop 2024-05-08T06:05:45.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:45.322+0000 7f9fd1a00700 1 -- 172.21.15.123:0/4254276107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fcc06bb40 msgr2=0x7f9fcc19a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:45.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:45.322+0000 7f9fd1a00700 1 --2- 172.21.15.123:0/4254276107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fcc06bb40 0x7f9fcc19a290 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9fb400e9c0 tx=0x7f9fb400ecd0 comp rx=0 tx=0).stop 2024-05-08T06:05:45.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:45.323+0000 7f9fd1a00700 1 -- 172.21.15.123:0/4254276107 shutdown_connections 2024-05-08T06:05:45.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:45.324+0000 7f9fd1a00700 1 --2- 172.21.15.123:0/4254276107 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9fb00b2900 0x7f9fb00b4dc0 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc00b2e0 tx=0x7f9fbc01e040 comp rx=0 tx=0).stop 2024-05-08T06:05:45.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:45.324+0000 7f9fd1a00700 1 --2- 172.21.15.123:0/4254276107 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fcc109510 0x7f9fcc19e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:45.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:45.324+0000 7f9fd1a00700 1 --2- 172.21.15.123:0/4254276107 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fcc06b180 0x7f9fcc199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:45.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:45.324+0000 7f9fd1a00700 1 --2- 172.21.15.123:0/4254276107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fcc06bb40 0x7f9fcc19a290 secure :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9fb400e9c0 tx=0x7f9fb400ecd0 comp rx=0 tx=0).stop 2024-05-08T06:05:45.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:45.324+0000 7f9fd1a00700 1 -- 172.21.15.123:0/4254276107 >> 172.21.15.123:0/4254276107 conn(0x7f9fcc079350 msgr2=0x7f9fcc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:05:45.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:45.325+0000 7f9fd1a00700 1 -- 172.21.15.123:0/4254276107 shutdown_connections 2024-05-08T06:05:45.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:45.325+0000 7f9fd1a00700 1 -- 172.21.15.123:0/4254276107 wait complete. 2024-05-08T06:05:45.329 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_metadata' created 2024-05-08T06:05:45.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:45 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/4254276107' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T06:05:45.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:45 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/4254276107' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T06:05:45.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:45 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/4254276107' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T06:05:46.357 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T06:05:46.417 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:46 smithi123 ceph-mon[29787]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:46.417 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:46 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/4254276107' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-08T06:05:46.417 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:46 smithi123 ceph-mon[29787]: osdmap e74: 12 total, 12 up, 12 in 2024-05-08T06:05:46.785 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:05:46.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:46 smithi163 ceph-mon[34520]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:46.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:46 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/4254276107' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-08T06:05:46.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:46 smithi163 ceph-mon[34520]: osdmap e74: 12 total, 12 up, 12 in 2024-05-08T06:05:46.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:46 smithi192 ceph-mon[34664]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:46.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:46 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/4254276107' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-08T06:05:46.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:46 smithi192 ceph-mon[34664]: osdmap e74: 12 total, 12 up, 12 in 2024-05-08T06:05:47.447 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:47 smithi123 ceph-mon[29787]: osdmap e75: 12 total, 12 up, 12 in 2024-05-08T06:05:47.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:47 smithi163 ceph-mon[34520]: osdmap e75: 12 total, 12 up, 12 in 2024-05-08T06:05:47.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:47 smithi192 ceph-mon[34664]: osdmap e75: 12 total, 12 up, 12 in 2024-05-08T06:05:48.682 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:48 smithi123 ceph-mon[29787]: pgmap v313: 33 pgs: 15 creating+peering, 4 active+clean, 14 unknown; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:48.682 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:48 smithi123 ceph-mon[29787]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T06:05:48.682 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:48 smithi123 ceph-mon[29787]: osdmap e76: 12 total, 12 up, 12 in 2024-05-08T06:05:48.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:48 smithi163 ceph-mon[34520]: pgmap v313: 33 pgs: 15 creating+peering, 4 active+clean, 14 unknown; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:48.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:48 smithi163 ceph-mon[34520]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T06:05:48.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:48 smithi163 ceph-mon[34520]: osdmap e76: 12 total, 12 up, 12 in 2024-05-08T06:05:48.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:48 smithi192 ceph-mon[34664]: pgmap v313: 33 pgs: 15 creating+peering, 4 active+clean, 14 unknown; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:48.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:48 smithi192 ceph-mon[34664]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T06:05:48.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:48 smithi192 ceph-mon[34664]: osdmap e76: 12 total, 12 up, 12 in 2024-05-08T06:05:49.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.342+0000 7fa52ea00700 1 -- 172.21.15.123:0/570106999 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5281083c0 msgr2=0x7fa52806ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:49.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.342+0000 7fa52ea00700 1 --2- 172.21.15.123:0/570106999 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5281083c0 0x7fa52806ba40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa518009990 tx=0x7fa518009ca0 comp rx=0 tx=0).stop 2024-05-08T06:05:49.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.342+0000 7fa52ea00700 1 -- 172.21.15.123:0/570106999 shutdown_connections 2024-05-08T06:05:49.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.342+0000 7fa52ea00700 1 --2- 172.21.15.123:0/570106999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa528105ed0 0x7fa528106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:49.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.342+0000 7fa52ea00700 1 --2- 172.21.15.123:0/570106999 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5281083c0 0x7fa52806ba40 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:49.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.342+0000 7fa52ea00700 1 --2- 172.21.15.123:0/570106999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa52806bf80 0x7fa52806c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:49.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.342+0000 7fa52ea00700 1 -- 172.21.15.123:0/570106999 >> 172.21.15.123:0/570106999 conn(0x7fa528079310 msgr2=0x7fa528079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:05:49.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.342+0000 7fa52ea00700 1 -- 172.21.15.123:0/570106999 shutdown_connections 2024-05-08T06:05:49.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.342+0000 7fa52ea00700 1 -- 172.21.15.123:0/570106999 wait complete. 2024-05-08T06:05:49.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.343+0000 7fa52ea00700 1 Processor -- start 2024-05-08T06:05:49.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.343+0000 7fa52ea00700 1 -- start start 2024-05-08T06:05:49.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.344+0000 7fa52ea00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa52806bf80 0x7fa5281a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:49.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.344+0000 7fa52ea00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa528105ed0 0x7fa5281a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:49.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.344+0000 7fa52ea00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5281083c0 0x7fa5281a4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:49.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.344+0000 7fa52ea00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa528119c20 con 0x7fa52806bf80 2024-05-08T06:05:49.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.344+0000 7fa52ea00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa528119aa0 con 0x7fa528105ed0 2024-05-08T06:05:49.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.344+0000 7fa52ea00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa528119da0 con 0x7fa5281083c0 2024-05-08T06:05:49.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.344+0000 7fa527e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa52806bf80 0x7fa5281a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:49.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.344+0000 7fa527e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa52806bf80 0x7fa5281a0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33496/0 (socket says 172.21.15.123:33496) 2024-05-08T06:05:49.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.344+0000 7fa527e00700 1 -- 172.21.15.123:0/2954510651 learned_addr learned my addr 172.21.15.123:0/2954510651 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:05:49.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.344+0000 7fa527400700 1 --2- 172.21.15.123:0/2954510651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa528105ed0 0x7fa5281a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:49.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.344+0000 7fa52ca00700 1 --2- 172.21.15.123:0/2954510651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5281083c0 0x7fa5281a4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:49.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.345+0000 7fa527e00700 1 -- 172.21.15.123:0/2954510651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5281083c0 msgr2=0x7fa5281a4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:49.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.345+0000 7fa527e00700 1 --2- 172.21.15.123:0/2954510651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5281083c0 0x7fa5281a4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:49.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.345+0000 7fa527e00700 1 -- 172.21.15.123:0/2954510651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa528105ed0 msgr2=0x7fa5281a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:49.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.345+0000 7fa527e00700 1 --2- 172.21.15.123:0/2954510651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa528105ed0 0x7fa5281a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:49.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.345+0000 7fa527e00700 1 -- 172.21.15.123:0/2954510651 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5281a5140 con 0x7fa52806bf80 2024-05-08T06:05:49.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.345+0000 7fa527e00700 1 --2- 172.21.15.123:0/2954510651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa52806bf80 0x7fa5281a0130 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa518000c00 tx=0x7fa51800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:05:49.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.345+0000 7fa524c00700 1 -- 172.21.15.123:0/2954510651 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa518027070 con 0x7fa52806bf80 2024-05-08T06:05:49.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.345+0000 7fa524c00700 1 -- 172.21.15.123:0/2954510651 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa518022470 con 0x7fa52806bf80 2024-05-08T06:05:49.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.345+0000 7fa52ea00700 1 -- 172.21.15.123:0/2954510651 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5281a53d0 con 0x7fa52806bf80 2024-05-08T06:05:49.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.345+0000 7fa524c00700 1 -- 172.21.15.123:0/2954510651 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa518017650 con 0x7fa52806bf80 2024-05-08T06:05:49.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.346+0000 7fa52ea00700 1 -- 172.21.15.123:0/2954510651 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5281accb0 con 0x7fa52806bf80 2024-05-08T06:05:49.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.347+0000 7fa52ea00700 1 -- 172.21.15.123:0/2954510651 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa52806a190 con 0x7fa52806bf80 2024-05-08T06:05:49.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.349+0000 7fa524c00700 1 -- 172.21.15.123:0/2954510651 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fa51802a070 con 0x7fa52806bf80 2024-05-08T06:05:49.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.351+0000 7fa524c00700 1 --2- 172.21.15.123:0/2954510651 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa50c0b2900 0x7fa50c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:49.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.351+0000 7fa524c00700 1 -- 172.21.15.123:0/2954510651 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7497+0+0 (secure 0 0 0) 0x7fa5180e2dd0 con 0x7fa52806bf80 2024-05-08T06:05:49.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.351+0000 7fa527400700 1 --2- 172.21.15.123:0/2954510651 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa50c0b2900 0x7fa50c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:49.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.352+0000 7fa527400700 1 --2- 172.21.15.123:0/2954510651 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa50c0b2900 0x7fa50c0b4dc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa52806be40 tx=0x7fa514009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:05:49.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.354+0000 7fa524c00700 1 -- 172.21.15.123:0/2954510651 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa518013070 con 0x7fa52806bf80 2024-05-08T06:05:49.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:49.587+0000 7fa52ea00700 1 -- 172.21.15.123:0/2954510651 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fa5281a1490 con 0x7fa52806bf80 2024-05-08T06:05:50.372 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:50.372+0000 7fa524c00700 1 -- 172.21.15.123:0/2954510651 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v77) ==== 170+0+0 (secure 0 0 0) 0x7fa5281a1490 con 0x7fa52806bf80 2024-05-08T06:05:50.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:50.380+0000 7fa52ea00700 1 -- 172.21.15.123:0/2954510651 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa50c0b2900 msgr2=0x7fa50c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:50.381 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:50.380+0000 7fa52ea00700 1 --2- 172.21.15.123:0/2954510651 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa50c0b2900 0x7fa50c0b4dc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa52806be40 tx=0x7fa514009260 comp rx=0 tx=0).stop 2024-05-08T06:05:50.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:50.380+0000 7fa52ea00700 1 -- 172.21.15.123:0/2954510651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa52806bf80 msgr2=0x7fa5281a0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:50.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:50.380+0000 7fa52ea00700 1 --2- 172.21.15.123:0/2954510651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa52806bf80 0x7fa5281a0130 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa518000c00 tx=0x7fa51800bfd0 comp rx=0 tx=0).stop 2024-05-08T06:05:50.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:50.381+0000 7fa52ea00700 1 -- 172.21.15.123:0/2954510651 shutdown_connections 2024-05-08T06:05:50.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:50.381+0000 7fa52ea00700 1 --2- 172.21.15.123:0/2954510651 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa50c0b2900 0x7fa50c0b4dc0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:50.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:50.381+0000 7fa52ea00700 1 --2- 172.21.15.123:0/2954510651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa52806bf80 0x7fa5281a0130 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:50.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:50.381+0000 7fa52ea00700 1 --2- 172.21.15.123:0/2954510651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5281083c0 0x7fa5281a4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:50.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:50.381+0000 7fa52ea00700 1 --2- 172.21.15.123:0/2954510651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa528105ed0 0x7fa5281a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:50.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:50.381+0000 7fa52ea00700 1 -- 172.21.15.123:0/2954510651 >> 172.21.15.123:0/2954510651 conn(0x7fa528079310 msgr2=0x7fa528101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:05:50.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:50.381+0000 7fa52ea00700 1 -- 172.21.15.123:0/2954510651 shutdown_connections 2024-05-08T06:05:50.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:50.382+0000 7fa52ea00700 1 -- 172.21.15.123:0/2954510651 wait complete. 2024-05-08T06:05:50.386 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_data' created 2024-05-08T06:05:50.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:50 smithi123 ceph-mon[29787]: pgmap v315: 33 pgs: 15 creating+peering, 11 active+clean, 7 unknown; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:50.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:50 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/2954510651' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T06:05:50.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:50 smithi163 ceph-mon[34520]: pgmap v315: 33 pgs: 15 creating+peering, 11 active+clean, 7 unknown; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:50.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:50 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/2954510651' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T06:05:50.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:50 smithi192 ceph-mon[34664]: pgmap v315: 33 pgs: 15 creating+peering, 11 active+clean, 7 unknown; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:50.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:50 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/2954510651' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T06:05:51.423 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T06:05:51.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:51 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/2954510651' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-08T06:05:51.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:51 smithi123 ceph-mon[29787]: osdmap e77: 12 total, 12 up, 12 in 2024-05-08T06:05:51.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:51 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/2954510651' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-08T06:05:51.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:51 smithi163 ceph-mon[34520]: osdmap e77: 12 total, 12 up, 12 in 2024-05-08T06:05:51.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:51 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/2954510651' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-08T06:05:51.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:51 smithi192 ceph-mon[34664]: osdmap e77: 12 total, 12 up, 12 in 2024-05-08T06:05:51.853 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:05:52.566 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:52 smithi123 ceph-mon[29787]: pgmap v317: 97 pgs: 16 creating+peering, 48 unknown, 33 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:52.567 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:52 smithi123 ceph-mon[29787]: osdmap e78: 12 total, 12 up, 12 in 2024-05-08T06:05:52.689 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:52 smithi163 ceph-mon[34520]: pgmap v317: 97 pgs: 16 creating+peering, 48 unknown, 33 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:52.690 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:52 smithi163 ceph-mon[34520]: osdmap e78: 12 total, 12 up, 12 in 2024-05-08T06:05:52.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:52 smithi192 ceph-mon[34664]: pgmap v317: 97 pgs: 16 creating+peering, 48 unknown, 33 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:52.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:52 smithi192 ceph-mon[34664]: osdmap e78: 12 total, 12 up, 12 in 2024-05-08T06:05:53.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:53 smithi123 ceph-mon[29787]: osdmap e79: 12 total, 12 up, 12 in 2024-05-08T06:05:53.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:53 smithi123 ceph-mon[29787]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T06:05:53.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:53 smithi163 ceph-mon[34520]: osdmap e79: 12 total, 12 up, 12 in 2024-05-08T06:05:53.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:53 smithi163 ceph-mon[34520]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T06:05:53.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:53 smithi192 ceph-mon[34664]: osdmap e79: 12 total, 12 up, 12 in 2024-05-08T06:05:53.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:53 smithi192 ceph-mon[34664]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T06:05:54.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.458+0000 7fd1fa200700 1 -- 172.21.15.123:0/425125400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1f41056c0 msgr2=0x7fd1f4105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:54.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.458+0000 7fd1fa200700 1 --2- 172.21.15.123:0/425125400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1f41056c0 0x7fd1f4105b40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc009990 tx=0x7fd1dc009ca0 comp rx=0 tx=0).stop 2024-05-08T06:05:54.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.458+0000 7fd1fa200700 1 -- 172.21.15.123:0/425125400 shutdown_connections 2024-05-08T06:05:54.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.458+0000 7fd1fa200700 1 --2- 172.21.15.123:0/425125400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1f4106080 0x7fd1f4112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:54.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.458+0000 7fd1fa200700 1 --2- 172.21.15.123:0/425125400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1f410b6c0 0x7fd1f410baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:54.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.458+0000 7fd1fa200700 1 --2- 172.21.15.123:0/425125400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1f41056c0 0x7fd1f4105b40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:54.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.458+0000 7fd1fa200700 1 -- 172.21.15.123:0/425125400 >> 172.21.15.123:0/425125400 conn(0x7fd1f41011b0 msgr2=0x7fd1f41035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:05:54.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.458+0000 7fd1fa200700 1 -- 172.21.15.123:0/425125400 shutdown_connections 2024-05-08T06:05:54.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.458+0000 7fd1fa200700 1 -- 172.21.15.123:0/425125400 wait complete. 2024-05-08T06:05:54.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.459+0000 7fd1fa200700 1 Processor -- start 2024-05-08T06:05:54.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.459+0000 7fd1fa200700 1 -- start start 2024-05-08T06:05:54.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.460+0000 7fd1fa200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1f41056c0 0x7fd1f419bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:54.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.460+0000 7fd1fa200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1f4106080 0x7fd1f419c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:54.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.460+0000 7fd1fa200700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1f410b6c0 0x7fd1f41a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:54.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.460+0000 7fd1fa200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f4115900 con 0x7fd1f41056c0 2024-05-08T06:05:54.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.460+0000 7fd1fa200700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f4115780 con 0x7fd1f4106080 2024-05-08T06:05:54.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.460+0000 7fd1fa200700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f4115a80 con 0x7fd1f410b6c0 2024-05-08T06:05:54.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.460+0000 7fd1f3400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1f41056c0 0x7fd1f419bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:54.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.460+0000 7fd1f3400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1f41056c0 0x7fd1f419bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33510/0 (socket says 172.21.15.123:33510) 2024-05-08T06:05:54.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.460+0000 7fd1f3400700 1 -- 172.21.15.123:0/1087083174 learned_addr learned my addr 172.21.15.123:0/1087083174 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:05:54.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.460+0000 7fd1f3e00700 1 --2- 172.21.15.123:0/1087083174 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1f410b6c0 0x7fd1f41a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:54.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.460+0000 7fd1ebe00700 1 --2- 172.21.15.123:0/1087083174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1f4106080 0x7fd1f419c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:54.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.460+0000 7fd1f3400700 1 -- 172.21.15.123:0/1087083174 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1f410b6c0 msgr2=0x7fd1f41a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:54.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.460+0000 7fd1f3400700 1 --2- 172.21.15.123:0/1087083174 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1f410b6c0 0x7fd1f41a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:54.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.460+0000 7fd1f3400700 1 -- 172.21.15.123:0/1087083174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1f4106080 msgr2=0x7fd1f419c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:54.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.460+0000 7fd1f3400700 1 --2- 172.21.15.123:0/1087083174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1f4106080 0x7fd1f419c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:54.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.460+0000 7fd1f3400700 1 -- 172.21.15.123:0/1087083174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f41a0dc0 con 0x7fd1f41056c0 2024-05-08T06:05:54.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.460+0000 7fd1f3e00700 1 --2- 172.21.15.123:0/1087083174 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1f410b6c0 0x7fd1f41a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:05:54.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.461+0000 7fd1f3400700 1 --2- 172.21.15.123:0/1087083174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1f41056c0 0x7fd1f419bdb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd1e400e9a0 tx=0x7fd1e400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:05:54.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.461+0000 7fd1f0c00700 1 -- 172.21.15.123:0/1087083174 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e400ca10 con 0x7fd1f41056c0 2024-05-08T06:05:54.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.461+0000 7fd1f0c00700 1 -- 172.21.15.123:0/1087083174 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1e400cbb0 con 0x7fd1f41056c0 2024-05-08T06:05:54.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.461+0000 7fd1fa200700 1 -- 172.21.15.123:0/1087083174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1f41a10b0 con 0x7fd1f41056c0 2024-05-08T06:05:54.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.461+0000 7fd1f0c00700 1 -- 172.21.15.123:0/1087083174 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4018680 con 0x7fd1f41056c0 2024-05-08T06:05:54.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.461+0000 7fd1fa200700 1 -- 172.21.15.123:0/1087083174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1f41a8930 con 0x7fd1f41056c0 2024-05-08T06:05:54.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.462+0000 7fd1fa200700 1 -- 172.21.15.123:0/1087083174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1f406a190 con 0x7fd1f41056c0 2024-05-08T06:05:54.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.465+0000 7fd1f0c00700 1 -- 172.21.15.123:0/1087083174 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fd1e4010430 con 0x7fd1f41056c0 2024-05-08T06:05:54.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.466+0000 7fd1f0c00700 1 --2- 172.21.15.123:0/1087083174 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd1d80b2980 0x7fd1d80b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:54.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.467+0000 7fd1f0c00700 1 -- 172.21.15.123:0/1087083174 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7879+0+0 (secure 0 0 0) 0x7fd1e40e24f0 con 0x7fd1f41056c0 2024-05-08T06:05:54.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.467+0000 7fd1ebe00700 1 --2- 172.21.15.123:0/1087083174 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd1d80b2980 0x7fd1d80b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:54.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.467+0000 7fd1ebe00700 1 --2- 172.21.15.123:0/1087083174 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd1d80b2980 0x7fd1d80b4e40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4103800 tx=0x7fd1dc00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:05:54.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.470+0000 7fd1f0c00700 1 -- 172.21.15.123:0/1087083174 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd1e40aa320 con 0x7fd1f41056c0 2024-05-08T06:05:54.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:54 smithi123 ceph-mon[29787]: pgmap v320: 97 pgs: 16 creating+peering, 24 unknown, 57 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:54.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:54.753+0000 7fd1fa200700 1 -- 172.21.15.123:0/1087083174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd1f402e510 con 0x7fd1f41056c0 2024-05-08T06:05:54.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:54 smithi163 ceph-mon[34520]: pgmap v320: 97 pgs: 16 creating+peering, 24 unknown, 57 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:54.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:54 smithi192 ceph-mon[34664]: pgmap v320: 97 pgs: 16 creating+peering, 24 unknown, 57 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:55.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:55.430+0000 7fd1f0c00700 1 -- 172.21.15.123:0/1087083174 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fd1e40b23a0 con 0x7fd1f41056c0 2024-05-08T06:05:55.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:55.434+0000 7fd1fa200700 1 -- 172.21.15.123:0/1087083174 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd1d80b2980 msgr2=0x7fd1d80b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:55.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:55.434+0000 7fd1fa200700 1 --2- 172.21.15.123:0/1087083174 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd1d80b2980 0x7fd1d80b4e40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4103800 tx=0x7fd1dc00b260 comp rx=0 tx=0).stop 2024-05-08T06:05:55.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:55.434+0000 7fd1fa200700 1 -- 172.21.15.123:0/1087083174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1f41056c0 msgr2=0x7fd1f419bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:55.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:55.434+0000 7fd1fa200700 1 --2- 172.21.15.123:0/1087083174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1f41056c0 0x7fd1f419bdb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd1e400e9a0 tx=0x7fd1e400ecb0 comp rx=0 tx=0).stop 2024-05-08T06:05:55.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:55.435+0000 7fd1fa200700 1 -- 172.21.15.123:0/1087083174 shutdown_connections 2024-05-08T06:05:55.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:55.435+0000 7fd1fa200700 1 --2- 172.21.15.123:0/1087083174 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd1d80b2980 0x7fd1d80b4e40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:55.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:55.435+0000 7fd1fa200700 1 --2- 172.21.15.123:0/1087083174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1f41056c0 0x7fd1f419bdb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:55.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:55.435+0000 7fd1fa200700 1 --2- 172.21.15.123:0/1087083174 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1f410b6c0 0x7fd1f41a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:55.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:55.435+0000 7fd1fa200700 1 --2- 172.21.15.123:0/1087083174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1f4106080 0x7fd1f419c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:55.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:55.435+0000 7fd1fa200700 1 -- 172.21.15.123:0/1087083174 >> 172.21.15.123:0/1087083174 conn(0x7fd1f41011b0 msgr2=0x7fd1f41018e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:05:55.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:55.435+0000 7fd1fa200700 1 -- 172.21.15.123:0/1087083174 shutdown_connections 2024-05-08T06:05:55.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:55.436+0000 7fd1fa200700 1 -- 172.21.15.123:0/1087083174 wait complete. 2024-05-08T06:05:55.440 INFO:teuthology.orchestra.run.smithi123.stderr:new fs with metadata pool 2 and data pool 3 2024-05-08T06:05:55.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:55 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/1087083174' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-08T06:05:55.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:55 smithi123 ceph-mon[29787]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-08T06:05:55.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:55 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/1087083174' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-08T06:05:55.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:55 smithi163 ceph-mon[34520]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-08T06:05:55.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:55 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/1087083174' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-08T06:05:55.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:55 smithi192 ceph-mon[34664]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-08T06:05:56.318 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T06:05:56.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:56 smithi123 ceph-mon[29787]: pgmap v321: 97 pgs: 16 creating+peering, 81 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:56.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:56 smithi123 ceph-mon[29787]: osdmap e80: 12 total, 12 up, 12 in 2024-05-08T06:05:56.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:56 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/1087083174' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-08T06:05:56.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:56 smithi123 ceph-mon[29787]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-08T06:05:56.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:56 smithi123 ceph-mon[29787]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-08T06:05:56.758 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:05:56.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:56 smithi163 ceph-mon[34520]: pgmap v321: 97 pgs: 16 creating+peering, 81 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:56.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:56 smithi163 ceph-mon[34520]: osdmap e80: 12 total, 12 up, 12 in 2024-05-08T06:05:56.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:56 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/1087083174' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-08T06:05:56.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:56 smithi163 ceph-mon[34520]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-08T06:05:56.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:56 smithi163 ceph-mon[34520]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-08T06:05:56.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:56 smithi192 ceph-mon[34664]: pgmap v321: 97 pgs: 16 creating+peering, 81 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:05:56.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:56 smithi192 ceph-mon[34664]: osdmap e80: 12 total, 12 up, 12 in 2024-05-08T06:05:56.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:56 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/1087083174' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-08T06:05:56.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:56 smithi192 ceph-mon[34664]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-08T06:05:56.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:56 smithi192 ceph-mon[34664]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-08T06:05:57.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:57 smithi123 ceph-mon[29787]: mds.? [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] up:active 2024-05-08T06:05:57.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:57 smithi123 ceph-mon[29787]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T06:05:57.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:57 smithi123 ceph-mon[29787]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T06:05:57.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:57 smithi163 ceph-mon[34520]: mds.? [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] up:active 2024-05-08T06:05:57.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:57 smithi163 ceph-mon[34520]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T06:05:57.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:57 smithi163 ceph-mon[34520]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T06:05:57.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:57 smithi192 ceph-mon[34664]: mds.? [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] up:active 2024-05-08T06:05:57.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:57 smithi192 ceph-mon[34664]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T06:05:57.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:57 smithi192 ceph-mon[34664]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T06:05:58.561 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:58 smithi123 ceph-mon[29787]: pgmap v323: 97 pgs: 97 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-05-08T06:05:58.561 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:58 smithi123 ceph-mon[29787]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-08T06:05:58.561 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:05:58 smithi123 ceph-mon[29787]: Cluster is now healthy 2024-05-08T06:05:58.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:58 smithi163 ceph-mon[34520]: pgmap v323: 97 pgs: 97 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-05-08T06:05:58.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:58 smithi163 ceph-mon[34520]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-08T06:05:58.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:05:58 smithi163 ceph-mon[34520]: Cluster is now healthy 2024-05-08T06:05:58.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:58 smithi192 ceph-mon[34664]: pgmap v323: 97 pgs: 97 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-05-08T06:05:58.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:58 smithi192 ceph-mon[34664]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-08T06:05:58.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:05:58 smithi192 ceph-mon[34664]: Cluster is now healthy 2024-05-08T06:05:59.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.236+0000 7f9e7c800700 1 -- 172.21.15.123:0/3804001615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e7410b6c0 msgr2=0x7f9e7410baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:59.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.236+0000 7f9e7c800700 1 --2- 172.21.15.123:0/3804001615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e7410b6c0 0x7f9e7410baa0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9e68009990 tx=0x7f9e68009ca0 comp rx=0 tx=0).stop 2024-05-08T06:05:59.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.236+0000 7f9e7c800700 1 -- 172.21.15.123:0/3804001615 shutdown_connections 2024-05-08T06:05:59.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.236+0000 7f9e7c800700 1 --2- 172.21.15.123:0/3804001615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e7410b6c0 0x7f9e7410baa0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:59.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.236+0000 7f9e7c800700 1 --2- 172.21.15.123:0/3804001615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e74106080 0x7f9e74112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:59.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.236+0000 7f9e7c800700 1 --2- 172.21.15.123:0/3804001615 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e741056c0 0x7f9e74105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:59.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.236+0000 7f9e7c800700 1 -- 172.21.15.123:0/3804001615 >> 172.21.15.123:0/3804001615 conn(0x7f9e74101170 msgr2=0x7f9e74103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:05:59.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.236+0000 7f9e7c800700 1 -- 172.21.15.123:0/3804001615 shutdown_connections 2024-05-08T06:05:59.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.236+0000 7f9e7c800700 1 -- 172.21.15.123:0/3804001615 wait complete. 2024-05-08T06:05:59.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.238+0000 7f9e7c800700 1 Processor -- start 2024-05-08T06:05:59.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.238+0000 7f9e7c800700 1 -- start start 2024-05-08T06:05:59.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.238+0000 7f9e7c800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e741056c0 0x7f9e7419bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:59.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.238+0000 7f9e7c800700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e74106080 0x7f9e7419c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:59.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.238+0000 7f9e7c800700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e7410b6c0 0x7f9e741a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:59.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.238+0000 7f9e7c800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9e74115910 con 0x7f9e741056c0 2024-05-08T06:05:59.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.238+0000 7f9e7c800700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9e74115790 con 0x7f9e7410b6c0 2024-05-08T06:05:59.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.238+0000 7f9e7c800700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9e74115a90 con 0x7f9e74106080 2024-05-08T06:05:59.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.238+0000 7f9e79e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e741056c0 0x7f9e7419bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:59.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.239+0000 7f9e79e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e741056c0 0x7f9e7419bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45342/0 (socket says 172.21.15.123:45342) 2024-05-08T06:05:59.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.239+0000 7f9e79e00700 1 -- 172.21.15.123:0/585924242 learned_addr learned my addr 172.21.15.123:0/585924242 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:05:59.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.239+0000 7f9e79400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e74106080 0x7f9e7419c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:59.240 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.239+0000 7f9e7a800700 1 --2- 172.21.15.123:0/585924242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e7410b6c0 0x7f9e741a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:59.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.239+0000 7f9e79400700 1 -- 172.21.15.123:0/585924242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e7410b6c0 msgr2=0x7f9e741a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:59.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.239+0000 7f9e79400700 1 --2- 172.21.15.123:0/585924242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e7410b6c0 0x7f9e741a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:59.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.239+0000 7f9e79400700 1 -- 172.21.15.123:0/585924242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e741056c0 msgr2=0x7f9e7419bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:59.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.239+0000 7f9e79400700 1 --2- 172.21.15.123:0/585924242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e741056c0 0x7f9e7419bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:59.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.239+0000 7f9e79400700 1 -- 172.21.15.123:0/585924242 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e741a0db0 con 0x7f9e74106080 2024-05-08T06:05:59.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.239+0000 7f9e79400700 1 --2- 172.21.15.123:0/585924242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e74106080 0x7f9e7419c2e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9e6400e9f0 tx=0x7f9e6400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:05:59.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.240+0000 7f9e6ea00700 1 -- 172.21.15.123:0/585924242 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6400c9c0 con 0x7f9e74106080 2024-05-08T06:05:59.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.240+0000 7f9e7c800700 1 -- 172.21.15.123:0/585924242 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e741a10a0 con 0x7f9e74106080 2024-05-08T06:05:59.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.240+0000 7f9e6ea00700 1 -- 172.21.15.123:0/585924242 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e6400cb60 con 0x7f9e74106080 2024-05-08T06:05:59.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.240+0000 7f9e6ea00700 1 -- 172.21.15.123:0/585924242 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e64018680 con 0x7f9e74106080 2024-05-08T06:05:59.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.240+0000 7f9e7c800700 1 -- 172.21.15.123:0/585924242 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e741a8920 con 0x7f9e74106080 2024-05-08T06:05:59.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.241+0000 7f9e7c800700 1 -- 172.21.15.123:0/585924242 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e7406a190 con 0x7f9e74106080 2024-05-08T06:05:59.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.244+0000 7f9e6ea00700 1 -- 172.21.15.123:0/585924242 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f9e64010430 con 0x7f9e74106080 2024-05-08T06:05:59.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.245+0000 7f9e6ea00700 1 --2- 172.21.15.123:0/585924242 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9e5c0b2880 0x7f9e5c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:05:59.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.246+0000 7f9e6ea00700 1 -- 172.21.15.123:0/585924242 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f9e640e2630 con 0x7f9e74106080 2024-05-08T06:05:59.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.246+0000 7f9e79e00700 1 --2- 172.21.15.123:0/585924242 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9e5c0b2880 0x7f9e5c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:05:59.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.246+0000 7f9e79e00700 1 --2- 172.21.15.123:0/585924242 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9e5c0b2880 0x7f9e5c0b4d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9e68006010 tx=0x7f9e6800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:05:59.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.247+0000 7f9e6ea00700 1 -- 172.21.15.123:0/585924242 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9e640aa400 con 0x7f9e74106080 2024-05-08T06:05:59.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.488+0000 7f9e7c800700 1 -- 172.21.15.123:0/585924242 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9e7402e510 con 0x7f9e74106080 2024-05-08T06:05:59.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.490+0000 7f9e6ea00700 1 -- 172.21.15.123:0/585924242 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22288 (secure 0 0 0) 0x7f9e640a9d00 con 0x7f9e74106080 2024-05-08T06:05:59.492 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:05:59.492 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":80,"fsid":"85cfb2e6-0cff-11ef-bc97-c7b262605968","created":"2024-05-08T05:56:12.544750+0000","modified":"2024-05-08T06:05:54.755598+0000","last_up_change":"2024-05-08T06:04:33.654727+0000","last_in_change":"2024-05-08T06:04:13.809050+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T06:00:17.050518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T06:05:44.815334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T06:05:49.604457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a14fde9b-bfb1-40a7-a105-305e8981a778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6803","nonce":3608618058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6805","nonce":3608618058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6809","nonce":3608618058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6807","nonce":3608618058}]},"public_addr":"172.21.15.123:6803/3608618058","cluster_addr":"172.21.15.123:6805/3608618058","heartbeat_back_addr":"172.21.15.123:6809/3608618058","heartbeat_front_addr":"172.21.15.123:6807/3608618058","state":["exists","up"]},{"osd":1,"uuid":"e023ae3d-a82e-4bb6-8fef-889dd07fe183","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6801","nonce":1884753449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6803","nonce":1884753449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6807","nonce":1884753449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6805","nonce":1884753449}]},"public_addr":"172.21.15.163:6801/1884753449","cluster_addr":"172.21.15.163:6803/1884753449","heartbeat_back_addr":"172.21.15.163:6807/1884753449","heartbeat_front_addr":"172.21.15.163:6805/1884753449","state":["exists","up"]},{"osd":2,"uuid":"69f4cfca-63cb-4862-9480-d79891b0f338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6801","nonce":4161570404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6803","nonce":4161570404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6807","nonce":4161570404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6805","nonce":4161570404}]},"public_addr":"172.21.15.192:6801/4161570404","cluster_addr":"172.21.15.192:6803/4161570404","heartbeat_back_addr":"172.21.15.192:6807/4161570404","heartbeat_front_addr":"172.21.15.192:6805/4161570404","state":["exists","up"]},{"osd":3,"uuid":"71bb5ecd-c88c-45fb-8b51-0eefb53941c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6811","nonce":1243003258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6813","nonce":1243003258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6817","nonce":1243003258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6815","nonce":1243003258}]},"public_addr":"172.21.15.123:6811/1243003258","cluster_addr":"172.21.15.123:6813/1243003258","heartbeat_back_addr":"172.21.15.123:6817/1243003258","heartbeat_front_addr":"172.21.15.123:6815/1243003258","state":["exists","up"]},{"osd":4,"uuid":"5a7a3462-e74f-4d15-a85c-709b5d423745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6809","nonce":591680273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6811","nonce":591680273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6815","nonce":591680273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6813","nonce":591680273}]},"public_addr":"172.21.15.163:6809/591680273","cluster_addr":"172.21.15.163:6811/591680273","heartbeat_back_addr":"172.21.15.163:6815/591680273","heartbeat_front_addr":"172.21.15.163:6813/591680273","state":["exists","up"]},{"osd":5,"uuid":"85e3f1ab-f8c6-4025-bab9-9ddb6bd3e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6809","nonce":1734821180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6811","nonce":1734821180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6815","nonce":1734821180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6813","nonce":1734821180}]},"public_addr":"172.21.15.192:6809/1734821180","cluster_addr":"172.21.15.192:6811/1734821180","heartbeat_back_addr":"172.21.15.192:6815/1734821180","heartbeat_front_addr":"172.21.15.192:6813/1734821180","state":["exists","up"]},{"osd":6,"uuid":"f21a5624-6f8f-48be-bc6d-0a883d6b7b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6819","nonce":1002138878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6821","nonce":1002138878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6825","nonce":1002138878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6823","nonce":1002138878}]},"public_addr":"172.21.15.123:6819/1002138878","cluster_addr":"172.21.15.123:6821/1002138878","heartbeat_back_addr":"172.21.15.123:6825/1002138878","heartbeat_front_addr":"172.21.15.123:6823/1002138878","state":["exists","up"]},{"osd":7,"uuid":"6eb16729-d260-4041-9829-3e868795b111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6817","nonce":354411978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6819","nonce":354411978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6823","nonce":354411978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6821","nonce":354411978}]},"public_addr":"172.21.15.163:6817/354411978","cluster_addr":"172.21.15.163:6819/354411978","heartbeat_back_addr":"172.21.15.163:6823/354411978","heartbeat_front_addr":"172.21.15.163:6821/354411978","state":["exists","up"]},{"osd":8,"uuid":"122a281e-18b8-4728-97db-458e7432b2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6817","nonce":3669753689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6819","nonce":3669753689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6823","nonce":3669753689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6821","nonce":3669753689}]},"public_addr":"172.21.15.192:6817/3669753689","cluster_addr":"172.21.15.192:6819/3669753689","heartbeat_back_addr":"172.21.15.192:6823/3669753689","heartbeat_front_addr":"172.21.15.192:6821/3669753689","state":["exists","up"]},{"osd":9,"uuid":"9fa022a9-b873-4e07-806c-641a706a150f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6827","nonce":3104436473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6829","nonce":3104436473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6833","nonce":3104436473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6831","nonce":3104436473}]},"public_addr":"172.21.15.123:6827/3104436473","cluster_addr":"172.21.15.123:6829/3104436473","heartbeat_back_addr":"172.21.15.123:6833/3104436473","heartbeat_front_addr":"172.21.15.123:6831/3104436473","state":["exists","up"]},{"osd":10,"uuid":"fb59339b-e11b-4ffc-b202-7f67cb21da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6825","nonce":2628487883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6827","nonce":2628487883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6831","nonce":2628487883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6829","nonce":2628487883}]},"public_addr":"172.21.15.163:6825/2628487883","cluster_addr":"172.21.15.163:6827/2628487883","heartbeat_back_addr":"172.21.15.163:6831/2628487883","heartbeat_front_addr":"172.21.15.163:6829/2628487883","state":["exists","up"]},{"osd":11,"uuid":"be7a5f36-c1be-41d7-80ee-004d0546eb9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6825","nonce":1339159347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6827","nonce":1339159347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6831","nonce":1339159347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6829","nonce":1339159347}]},"public_addr":"172.21.15.192:6825/1339159347","cluster_addr":"172.21.15.192:6827/1339159347","heartbeat_back_addr":"172.21.15.192:6831/1339159347","heartbeat_front_addr":"172.21.15.192:6829/1339159347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:59:24.923234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:59:49.867071+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:00:13.454062+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:00:46.983594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:01:12.820452+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:01:37.451382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:02:13.325181+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:02:39.453436+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:03:03.837509+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:03:39.460579+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:04:06.314082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:04:31.723154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6801/3057688649":"2024-05-09T05:57:30.887936+0000","172.21.15.123:6800/3057688649":"2024-05-09T05:57:30.887936+0000","172.21.15.123:6800/2769724312":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/399884043":"2024-05-09T05:57:30.887936+0000","172.21.15.123:0/3406061897":"2024-05-09T05:57:30.887936+0000","172.21.15.123:0/644457268":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/3055092197":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/343390357":"2024-05-09T05:57:30.887936+0000","172.21.15.123:6801/2769724312":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/2263171562":"2024-05-09T05:56:55.849036+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T06:05:59.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.494+0000 7f9e7c800700 1 -- 172.21.15.123:0/585924242 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9e5c0b2880 msgr2=0x7f9e5c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:59.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.494+0000 7f9e7c800700 1 --2- 172.21.15.123:0/585924242 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9e5c0b2880 0x7f9e5c0b4d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9e68006010 tx=0x7f9e6800b260 comp rx=0 tx=0).stop 2024-05-08T06:05:59.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.494+0000 7f9e7c800700 1 -- 172.21.15.123:0/585924242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e74106080 msgr2=0x7f9e7419c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:05:59.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.494+0000 7f9e7c800700 1 --2- 172.21.15.123:0/585924242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e74106080 0x7f9e7419c2e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9e6400e9f0 tx=0x7f9e6400ed00 comp rx=0 tx=0).stop 2024-05-08T06:05:59.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.494+0000 7f9e7c800700 1 -- 172.21.15.123:0/585924242 shutdown_connections 2024-05-08T06:05:59.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.494+0000 7f9e7c800700 1 --2- 172.21.15.123:0/585924242 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9e5c0b2880 0x7f9e5c0b4d40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:59.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.494+0000 7f9e7c800700 1 --2- 172.21.15.123:0/585924242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e741056c0 0x7f9e7419bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:59.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.494+0000 7f9e7c800700 1 --2- 172.21.15.123:0/585924242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e7410b6c0 0x7f9e741a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:59.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.494+0000 7f9e7c800700 1 --2- 172.21.15.123:0/585924242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e74106080 0x7f9e7419c2e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:05:59.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.494+0000 7f9e7c800700 1 -- 172.21.15.123:0/585924242 >> 172.21.15.123:0/585924242 conn(0x7f9e74101170 msgr2=0x7f9e741018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:05:59.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.494+0000 7f9e7c800700 1 -- 172.21.15.123:0/585924242 shutdown_connections 2024-05-08T06:05:59.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:05:59.494+0000 7f9e7c800700 1 -- 172.21.15.123:0/585924242 wait complete. 2024-05-08T06:06:00.404 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T06:06:00.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:00 smithi123 ceph-mon[29787]: pgmap v324: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-05-08T06:06:00.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:00 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/585924242' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:06:00.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:00 smithi163 ceph-mon[34520]: pgmap v324: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-05-08T06:06:00.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:00 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/585924242' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:06:00.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:00 smithi192 ceph-mon[34664]: pgmap v324: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-05-08T06:06:00.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:00 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/585924242' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:06:00.884 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:06:01.538 INFO:journalctl@ceph.mds.b.smithi163.stdout:May 08 06:06:01 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-b[65037]: 2024-05-08T06:06:01.131+0000 7fcfb960d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T06:06:02.692 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:02 smithi123 ceph-mon[29787]: pgmap v325: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-05-08T06:06:02.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:02 smithi163 ceph-mon[34520]: pgmap v325: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-05-08T06:06:02.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:02 smithi192 ceph-mon[34664]: pgmap v325: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-05-08T06:06:03.286 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.284+0000 7fcb36e00700 1 -- 172.21.15.123:0/1867954046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb301060a0 msgr2=0x7fcb301125b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:03.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.284+0000 7fcb36e00700 1 --2- 172.21.15.123:0/1867954046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb301060a0 0x7fcb301125b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcb2800b0a0 tx=0x7fcb2800b3b0 comp rx=0 tx=0).stop 2024-05-08T06:06:03.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.285+0000 7fcb36e00700 1 -- 172.21.15.123:0/1867954046 shutdown_connections 2024-05-08T06:06:03.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.285+0000 7fcb36e00700 1 --2- 172.21.15.123:0/1867954046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb301060a0 0x7fcb301125b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:03.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.285+0000 7fcb36e00700 1 --2- 172.21.15.123:0/1867954046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb3010b6e0 0x7fcb3010bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:03.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.285+0000 7fcb36e00700 1 --2- 172.21.15.123:0/1867954046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb301056e0 0x7fcb30105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:03.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.285+0000 7fcb36e00700 1 -- 172.21.15.123:0/1867954046 >> 172.21.15.123:0/1867954046 conn(0x7fcb301011d0 msgr2=0x7fcb301035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:06:03.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.285+0000 7fcb36e00700 1 -- 172.21.15.123:0/1867954046 shutdown_connections 2024-05-08T06:06:03.287 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.285+0000 7fcb36e00700 1 -- 172.21.15.123:0/1867954046 wait complete. 2024-05-08T06:06:03.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.286+0000 7fcb36e00700 1 Processor -- start 2024-05-08T06:06:03.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.286+0000 7fcb36e00700 1 -- start start 2024-05-08T06:06:03.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.287+0000 7fcb36e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb301056e0 0x7fcb301a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:03.289 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.287+0000 7fcb36e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb301060a0 0x7fcb301a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:03.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.287+0000 7fcb36e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb3010b6e0 0x7fcb3019bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:03.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.287+0000 7fcb36e00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcb30115920 con 0x7fcb301060a0 2024-05-08T06:06:03.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.287+0000 7fcb36e00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fcb301157a0 con 0x7fcb3010b6e0 2024-05-08T06:06:03.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.287+0000 7fcb36e00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcb30115aa0 con 0x7fcb301056e0 2024-05-08T06:06:03.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.287+0000 7fcb2f400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb301060a0 0x7fcb301a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:03.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.287+0000 7fcb2fe00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb301056e0 0x7fcb301a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:03.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.287+0000 7fcb2fe00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb301056e0 0x7fcb301a1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:41834/0 (socket says 172.21.15.123:41834) 2024-05-08T06:06:03.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.287+0000 7fcb2fe00700 1 -- 172.21.15.123:0/2710414870 learned_addr learned my addr 172.21.15.123:0/2710414870 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:06:03.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.287+0000 7fcb34e00700 1 --2- 172.21.15.123:0/2710414870 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb3010b6e0 0x7fcb3019bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:03.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.288+0000 7fcb2f400700 1 -- 172.21.15.123:0/2710414870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb301056e0 msgr2=0x7fcb301a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:03.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.288+0000 7fcb2f400700 1 --2- 172.21.15.123:0/2710414870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb301056e0 0x7fcb301a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:03.290 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.288+0000 7fcb2f400700 1 -- 172.21.15.123:0/2710414870 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb3010b6e0 msgr2=0x7fcb3019bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:03.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.288+0000 7fcb2f400700 1 --2- 172.21.15.123:0/2710414870 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb3010b6e0 0x7fcb3019bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:03.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.288+0000 7fcb2f400700 1 -- 172.21.15.123:0/2710414870 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb3019c460 con 0x7fcb301060a0 2024-05-08T06:06:03.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.288+0000 7fcb2f400700 1 --2- 172.21.15.123:0/2710414870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb301060a0 0x7fcb301a1fa0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c00e9f0 tx=0x7fcb1c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:06:03.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.288+0000 7fcb2cc00700 1 -- 172.21.15.123:0/2710414870 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c00c9c0 con 0x7fcb301060a0 2024-05-08T06:06:03.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.288+0000 7fcb2cc00700 1 -- 172.21.15.123:0/2710414870 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb1c00cb60 con 0x7fcb301060a0 2024-05-08T06:06:03.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.288+0000 7fcb36e00700 1 -- 172.21.15.123:0/2710414870 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb3019c750 con 0x7fcb301060a0 2024-05-08T06:06:03.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.288+0000 7fcb2cc00700 1 -- 172.21.15.123:0/2710414870 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c018680 con 0x7fcb301060a0 2024-05-08T06:06:03.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.289+0000 7fcb36e00700 1 -- 172.21.15.123:0/2710414870 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb301a89e0 con 0x7fcb301060a0 2024-05-08T06:06:03.291 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.290+0000 7fcb36e00700 1 -- 172.21.15.123:0/2710414870 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb30106e20 con 0x7fcb301060a0 2024-05-08T06:06:03.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.293+0000 7fcb2cc00700 1 -- 172.21.15.123:0/2710414870 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fcb1c010430 con 0x7fcb301060a0 2024-05-08T06:06:03.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.294+0000 7fcb2cc00700 1 --2- 172.21.15.123:0/2710414870 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fcb140b2900 0x7fcb140b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:03.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.295+0000 7fcb2cc00700 1 -- 172.21.15.123:0/2710414870 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fcb1c0e25b0 con 0x7fcb301060a0 2024-05-08T06:06:03.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.295+0000 7fcb2fe00700 1 --2- 172.21.15.123:0/2710414870 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fcb140b2900 0x7fcb140b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:03.296 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.295+0000 7fcb2fe00700 1 --2- 172.21.15.123:0/2710414870 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fcb140b2900 0x7fcb140b4dc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcb20009ca0 tx=0x7fcb20009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:06:03.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.296+0000 7fcb2cc00700 1 -- 172.21.15.123:0/2710414870 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcb1c0aa380 con 0x7fcb301060a0 2024-05-08T06:06:03.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.528+0000 7fcb36e00700 1 -- 172.21.15.123:0/2710414870 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcb3019cff0 con 0x7fcb301060a0 2024-05-08T06:06:03.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.531+0000 7fcb2cc00700 1 -- 172.21.15.123:0/2710414870 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22288 (secure 0 0 0) 0x7fcb1c0b2400 con 0x7fcb301060a0 2024-05-08T06:06:03.532 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:06:03.532 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":80,"fsid":"85cfb2e6-0cff-11ef-bc97-c7b262605968","created":"2024-05-08T05:56:12.544750+0000","modified":"2024-05-08T06:05:54.755598+0000","last_up_change":"2024-05-08T06:04:33.654727+0000","last_in_change":"2024-05-08T06:04:13.809050+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T06:00:17.050518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T06:05:44.815334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T06:05:49.604457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a14fde9b-bfb1-40a7-a105-305e8981a778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6803","nonce":3608618058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6805","nonce":3608618058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6809","nonce":3608618058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6807","nonce":3608618058}]},"public_addr":"172.21.15.123:6803/3608618058","cluster_addr":"172.21.15.123:6805/3608618058","heartbeat_back_addr":"172.21.15.123:6809/3608618058","heartbeat_front_addr":"172.21.15.123:6807/3608618058","state":["exists","up"]},{"osd":1,"uuid":"e023ae3d-a82e-4bb6-8fef-889dd07fe183","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6801","nonce":1884753449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6803","nonce":1884753449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6807","nonce":1884753449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6805","nonce":1884753449}]},"public_addr":"172.21.15.163:6801/1884753449","cluster_addr":"172.21.15.163:6803/1884753449","heartbeat_back_addr":"172.21.15.163:6807/1884753449","heartbeat_front_addr":"172.21.15.163:6805/1884753449","state":["exists","up"]},{"osd":2,"uuid":"69f4cfca-63cb-4862-9480-d79891b0f338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6801","nonce":4161570404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6803","nonce":4161570404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6807","nonce":4161570404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6805","nonce":4161570404}]},"public_addr":"172.21.15.192:6801/4161570404","cluster_addr":"172.21.15.192:6803/4161570404","heartbeat_back_addr":"172.21.15.192:6807/4161570404","heartbeat_front_addr":"172.21.15.192:6805/4161570404","state":["exists","up"]},{"osd":3,"uuid":"71bb5ecd-c88c-45fb-8b51-0eefb53941c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6811","nonce":1243003258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6813","nonce":1243003258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6817","nonce":1243003258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6815","nonce":1243003258}]},"public_addr":"172.21.15.123:6811/1243003258","cluster_addr":"172.21.15.123:6813/1243003258","heartbeat_back_addr":"172.21.15.123:6817/1243003258","heartbeat_front_addr":"172.21.15.123:6815/1243003258","state":["exists","up"]},{"osd":4,"uuid":"5a7a3462-e74f-4d15-a85c-709b5d423745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6809","nonce":591680273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6811","nonce":591680273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6815","nonce":591680273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6813","nonce":591680273}]},"public_addr":"172.21.15.163:6809/591680273","cluster_addr":"172.21.15.163:6811/591680273","heartbeat_back_addr":"172.21.15.163:6815/591680273","heartbeat_front_addr":"172.21.15.163:6813/591680273","state":["exists","up"]},{"osd":5,"uuid":"85e3f1ab-f8c6-4025-bab9-9ddb6bd3e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6809","nonce":1734821180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6811","nonce":1734821180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6815","nonce":1734821180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6813","nonce":1734821180}]},"public_addr":"172.21.15.192:6809/1734821180","cluster_addr":"172.21.15.192:6811/1734821180","heartbeat_back_addr":"172.21.15.192:6815/1734821180","heartbeat_front_addr":"172.21.15.192:6813/1734821180","state":["exists","up"]},{"osd":6,"uuid":"f21a5624-6f8f-48be-bc6d-0a883d6b7b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6819","nonce":1002138878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6821","nonce":1002138878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6825","nonce":1002138878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6823","nonce":1002138878}]},"public_addr":"172.21.15.123:6819/1002138878","cluster_addr":"172.21.15.123:6821/1002138878","heartbeat_back_addr":"172.21.15.123:6825/1002138878","heartbeat_front_addr":"172.21.15.123:6823/1002138878","state":["exists","up"]},{"osd":7,"uuid":"6eb16729-d260-4041-9829-3e868795b111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6817","nonce":354411978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6819","nonce":354411978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6823","nonce":354411978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6821","nonce":354411978}]},"public_addr":"172.21.15.163:6817/354411978","cluster_addr":"172.21.15.163:6819/354411978","heartbeat_back_addr":"172.21.15.163:6823/354411978","heartbeat_front_addr":"172.21.15.163:6821/354411978","state":["exists","up"]},{"osd":8,"uuid":"122a281e-18b8-4728-97db-458e7432b2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6817","nonce":3669753689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6819","nonce":3669753689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6823","nonce":3669753689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6821","nonce":3669753689}]},"public_addr":"172.21.15.192:6817/3669753689","cluster_addr":"172.21.15.192:6819/3669753689","heartbeat_back_addr":"172.21.15.192:6823/3669753689","heartbeat_front_addr":"172.21.15.192:6821/3669753689","state":["exists","up"]},{"osd":9,"uuid":"9fa022a9-b873-4e07-806c-641a706a150f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6827","nonce":3104436473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6829","nonce":3104436473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6833","nonce":3104436473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6831","nonce":3104436473}]},"public_addr":"172.21.15.123:6827/3104436473","cluster_addr":"172.21.15.123:6829/3104436473","heartbeat_back_addr":"172.21.15.123:6833/3104436473","heartbeat_front_addr":"172.21.15.123:6831/3104436473","state":["exists","up"]},{"osd":10,"uuid":"fb59339b-e11b-4ffc-b202-7f67cb21da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6825","nonce":2628487883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6827","nonce":2628487883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6831","nonce":2628487883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6829","nonce":2628487883}]},"public_addr":"172.21.15.163:6825/2628487883","cluster_addr":"172.21.15.163:6827/2628487883","heartbeat_back_addr":"172.21.15.163:6831/2628487883","heartbeat_front_addr":"172.21.15.163:6829/2628487883","state":["exists","up"]},{"osd":11,"uuid":"be7a5f36-c1be-41d7-80ee-004d0546eb9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6825","nonce":1339159347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6827","nonce":1339159347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6831","nonce":1339159347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6829","nonce":1339159347}]},"public_addr":"172.21.15.192:6825/1339159347","cluster_addr":"172.21.15.192:6827/1339159347","heartbeat_back_addr":"172.21.15.192:6831/1339159347","heartbeat_front_addr":"172.21.15.192:6829/1339159347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:59:24.923234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:59:49.867071+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:00:13.454062+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:00:46.983594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:01:12.820452+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:01:37.451382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:02:13.325181+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:02:39.453436+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:03:03.837509+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:03:39.460579+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:04:06.314082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:04:31.723154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6801/3057688649":"2024-05-09T05:57:30.887936+0000","172.21.15.123:6800/3057688649":"2024-05-09T05:57:30.887936+0000","172.21.15.123:6800/2769724312":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/399884043":"2024-05-09T05:57:30.887936+0000","172.21.15.123:0/3406061897":"2024-05-09T05:57:30.887936+0000","172.21.15.123:0/644457268":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/3055092197":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/343390357":"2024-05-09T05:57:30.887936+0000","172.21.15.123:6801/2769724312":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/2263171562":"2024-05-09T05:56:55.849036+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T06:06:03.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.534+0000 7fcb36e00700 1 -- 172.21.15.123:0/2710414870 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fcb140b2900 msgr2=0x7fcb140b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:03.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.534+0000 7fcb36e00700 1 --2- 172.21.15.123:0/2710414870 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fcb140b2900 0x7fcb140b4dc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcb20009ca0 tx=0x7fcb20009260 comp rx=0 tx=0).stop 2024-05-08T06:06:03.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.534+0000 7fcb36e00700 1 -- 172.21.15.123:0/2710414870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb301060a0 msgr2=0x7fcb301a1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:03.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.534+0000 7fcb36e00700 1 --2- 172.21.15.123:0/2710414870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb301060a0 0x7fcb301a1fa0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c00e9f0 tx=0x7fcb1c00ed00 comp rx=0 tx=0).stop 2024-05-08T06:06:03.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.534+0000 7fcb36e00700 1 -- 172.21.15.123:0/2710414870 shutdown_connections 2024-05-08T06:06:03.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.534+0000 7fcb36e00700 1 --2- 172.21.15.123:0/2710414870 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fcb140b2900 0x7fcb140b4dc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:03.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.534+0000 7fcb36e00700 1 --2- 172.21.15.123:0/2710414870 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb3010b6e0 0x7fcb3019bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:03.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.534+0000 7fcb36e00700 1 --2- 172.21.15.123:0/2710414870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb301056e0 0x7fcb301a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:03.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.534+0000 7fcb36e00700 1 --2- 172.21.15.123:0/2710414870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb301060a0 0x7fcb301a1fa0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:03.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.534+0000 7fcb36e00700 1 -- 172.21.15.123:0/2710414870 >> 172.21.15.123:0/2710414870 conn(0x7fcb301011d0 msgr2=0x7fcb301018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:06:03.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.535+0000 7fcb36e00700 1 -- 172.21.15.123:0/2710414870 shutdown_connections 2024-05-08T06:06:03.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:03.535+0000 7fcb36e00700 1 -- 172.21.15.123:0/2710414870 wait complete. 2024-05-08T06:06:04.512 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-08T06:06:04.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:04 smithi123 ceph-mon[29787]: pgmap v326: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-08T06:06:04.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:04 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/2710414870' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:06:04.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:04 smithi163 ceph-mon[34520]: pgmap v326: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-08T06:06:04.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:04 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/2710414870' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:06:04.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:04 smithi192 ceph-mon[34664]: pgmap v326: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-08T06:06:04.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:04 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/2710414870' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:06:04.952 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:06:06.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:06 smithi163 ceph-mon[34520]: pgmap v327: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-08T06:06:06.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:06 smithi192 ceph-mon[34664]: pgmap v327: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-08T06:06:06.924 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:06 smithi123 ceph-mon[29787]: pgmap v327: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-08T06:06:07.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.559+0000 7ff406c00700 1 -- 172.21.15.123:0/3984233110 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff400103750 msgr2=0x7ff400103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:07.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.559+0000 7ff406c00700 1 --2- 172.21.15.123:0/3984233110 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff400103750 0x7ff400103b30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000b0a0 tx=0x7ff3f000b3b0 comp rx=0 tx=0).stop 2024-05-08T06:06:07.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.559+0000 7ff406c00700 1 -- 172.21.15.123:0/3984233110 shutdown_connections 2024-05-08T06:06:07.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.559+0000 7ff406c00700 1 --2- 172.21.15.123:0/3984233110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff400108850 0x7ff4001103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:07.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.559+0000 7ff406c00700 1 --2- 172.21.15.123:0/3984233110 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff400103750 0x7ff400103b30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:07.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.559+0000 7ff406c00700 1 --2- 172.21.15.123:0/3984233110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff400104100 0x7ff400108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:07.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.559+0000 7ff406c00700 1 -- 172.21.15.123:0/3984233110 >> 172.21.15.123:0/3984233110 conn(0x7ff40007a600 msgr2=0x7ff40007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:06:07.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.560+0000 7ff406c00700 1 -- 172.21.15.123:0/3984233110 shutdown_connections 2024-05-08T06:06:07.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.560+0000 7ff406c00700 1 -- 172.21.15.123:0/3984233110 wait complete. 2024-05-08T06:06:07.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.561+0000 7ff406c00700 1 Processor -- start 2024-05-08T06:06:07.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.561+0000 7ff406c00700 1 -- start start 2024-05-08T06:06:07.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.561+0000 7ff406c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff400103750 0x7ff40019bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:07.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.561+0000 7ff406c00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff400104100 0x7ff40019c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:07.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.561+0000 7ff406c00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff400108850 0x7ff4001a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:07.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.561+0000 7ff406c00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff400113690 con 0x7ff400103750 2024-05-08T06:06:07.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.561+0000 7ff406c00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff400113510 con 0x7ff400108850 2024-05-08T06:06:07.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.561+0000 7ff406c00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff400113810 con 0x7ff400104100 2024-05-08T06:06:07.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.562+0000 7ff3ffe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff400103750 0x7ff40019bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:07.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.562+0000 7ff3ffe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff400103750 0x7ff40019bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48458/0 (socket says 172.21.15.123:48458) 2024-05-08T06:06:07.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.562+0000 7ff3ffe00700 1 -- 172.21.15.123:0/1009340338 learned_addr learned my addr 172.21.15.123:0/1009340338 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:06:07.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.562+0000 7ff404c00700 1 --2- 172.21.15.123:0/1009340338 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff400108850 0x7ff4001a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:07.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.562+0000 7ff3f7e00700 1 --2- 172.21.15.123:0/1009340338 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff400104100 0x7ff40019c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:07.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.562+0000 7ff404c00700 1 -- 172.21.15.123:0/1009340338 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff400104100 msgr2=0x7ff40019c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:07.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.562+0000 7ff404c00700 1 --2- 172.21.15.123:0/1009340338 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff400104100 0x7ff40019c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:07.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.562+0000 7ff404c00700 1 -- 172.21.15.123:0/1009340338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff400103750 msgr2=0x7ff40019bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:07.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.562+0000 7ff404c00700 1 --2- 172.21.15.123:0/1009340338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff400103750 0x7ff40019bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:07.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.562+0000 7ff404c00700 1 -- 172.21.15.123:0/1009340338 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4001a0db0 con 0x7ff400108850 2024-05-08T06:06:07.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.562+0000 7ff3ffe00700 1 --2- 172.21.15.123:0/1009340338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff400103750 0x7ff40019bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:06:07.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.562+0000 7ff404c00700 1 --2- 172.21.15.123:0/1009340338 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff400108850 0x7ff4001a0670 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800d750 tx=0x7ff3f800da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:06:07.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.563+0000 7ff3fd600700 1 -- 172.21.15.123:0/1009340338 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8014030 con 0x7ff400108850 2024-05-08T06:06:07.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.563+0000 7ff3fd600700 1 -- 172.21.15.123:0/1009340338 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3f80044d0 con 0x7ff400108850 2024-05-08T06:06:07.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.563+0000 7ff406c00700 1 -- 172.21.15.123:0/1009340338 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4001a10a0 con 0x7ff400108850 2024-05-08T06:06:07.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.563+0000 7ff3fd600700 1 -- 172.21.15.123:0/1009340338 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8005420 con 0x7ff400108850 2024-05-08T06:06:07.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.563+0000 7ff406c00700 1 -- 172.21.15.123:0/1009340338 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4001a8920 con 0x7ff400108850 2024-05-08T06:06:07.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.564+0000 7ff406c00700 1 -- 172.21.15.123:0/1009340338 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff40006a190 con 0x7ff400108850 2024-05-08T06:06:07.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.566+0000 7ff3fd600700 1 -- 172.21.15.123:0/1009340338 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7ff3f8004b50 con 0x7ff400108850 2024-05-08T06:06:07.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.567+0000 7ff3fd600700 1 --2- 172.21.15.123:0/1009340338 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7ff3e40b2900 0x7ff3e40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:07.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.568+0000 7ff3fd600700 1 -- 172.21.15.123:0/1009340338 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7ff3f80e1940 con 0x7ff400108850 2024-05-08T06:06:07.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.568+0000 7ff3ffe00700 1 --2- 172.21.15.123:0/1009340338 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7ff3e40b2900 0x7ff3e40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:07.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.568+0000 7ff3ffe00700 1 --2- 172.21.15.123:0/1009340338 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7ff3e40b2900 0x7ff3e40b4dc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000b780 tx=0x7ff3f0009c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:06:07.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.572+0000 7ff3fd600700 1 -- 172.21.15.123:0/1009340338 <== mon.2 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff3f80a9710 con 0x7ff400108850 2024-05-08T06:06:07.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:07.833+0000 7ff406c00700 1 -- 172.21.15.123:0/1009340338 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff4001a8af0 con 0x7ff400108850 2024-05-08T06:06:08.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:08.464+0000 7ff3fd600700 1 -- 172.21.15.123:0/1009340338 <== mon.2 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7ff3f80a9010 con 0x7ff400108850 2024-05-08T06:06:08.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:08.469+0000 7ff406c00700 1 -- 172.21.15.123:0/1009340338 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7ff3e40b2900 msgr2=0x7ff3e40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:08.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:08.469+0000 7ff406c00700 1 --2- 172.21.15.123:0/1009340338 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7ff3e40b2900 0x7ff3e40b4dc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000b780 tx=0x7ff3f0009c40 comp rx=0 tx=0).stop 2024-05-08T06:06:08.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:08.469+0000 7ff406c00700 1 -- 172.21.15.123:0/1009340338 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff400108850 msgr2=0x7ff4001a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:08.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:08.469+0000 7ff406c00700 1 --2- 172.21.15.123:0/1009340338 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff400108850 0x7ff4001a0670 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800d750 tx=0x7ff3f800da60 comp rx=0 tx=0).stop 2024-05-08T06:06:08.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:08.469+0000 7ff406c00700 1 -- 172.21.15.123:0/1009340338 shutdown_connections 2024-05-08T06:06:08.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:08.469+0000 7ff406c00700 1 --2- 172.21.15.123:0/1009340338 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7ff3e40b2900 0x7ff3e40b4dc0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:08.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:08.469+0000 7ff406c00700 1 --2- 172.21.15.123:0/1009340338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff400103750 0x7ff40019bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:08.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:08.469+0000 7ff406c00700 1 --2- 172.21.15.123:0/1009340338 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff400108850 0x7ff4001a0670 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:08.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:08.469+0000 7ff406c00700 1 --2- 172.21.15.123:0/1009340338 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff400104100 0x7ff40019c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:08.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:08.469+0000 7ff406c00700 1 -- 172.21.15.123:0/1009340338 >> 172.21.15.123:0/1009340338 conn(0x7ff40007a600 msgr2=0x7ff4001014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:06:08.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:08.469+0000 7ff406c00700 1 -- 172.21.15.123:0/1009340338 shutdown_connections 2024-05-08T06:06:08.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:08.469+0000 7ff406c00700 1 -- 172.21.15.123:0/1009340338 wait complete. 2024-05-08T06:06:08.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:08 smithi163 ceph-mon[34520]: pgmap v328: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-08T06:06:08.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:08 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/1009340338' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T06:06:08.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:08 smithi163 ceph-mon[34520]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T06:06:08.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:08 smithi192 ceph-mon[34664]: pgmap v328: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-08T06:06:08.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:08 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/1009340338' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T06:06:08.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:08 smithi192 ceph-mon[34664]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T06:06:08.891 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:08 smithi123 ceph-mon[29787]: pgmap v328: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-08T06:06:08.891 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:08 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/1009340338' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T06:06:08.891 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:08 smithi123 ceph-mon[29787]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T06:06:09.389 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-05-08T06:06:09.390 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-05-08T06:06:09.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:09 smithi123 ceph-mon[29787]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-08T06:06:09.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:09 smithi123 ceph-mon[29787]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T06:06:09.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:09 smithi163 ceph-mon[34520]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-08T06:06:09.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:09 smithi163 ceph-mon[34520]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T06:06:09.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:09 smithi192 ceph-mon[34664]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-08T06:06:09.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:09 smithi192 ceph-mon[34664]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T06:06:09.838 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:06:10.513 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:10 smithi123 ceph-mon[29787]: pgmap v329: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-08T06:06:10.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:10 smithi163 ceph-mon[34520]: pgmap v329: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-08T06:06:10.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:10 smithi192 ceph-mon[34664]: pgmap v329: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-08T06:06:12.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.310+0000 7fae91400700 1 -- 172.21.15.123:0/2722324850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae8c106080 msgr2=0x7fae8c112590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:12.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.310+0000 7fae91400700 1 --2- 172.21.15.123:0/2722324850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae8c106080 0x7fae8c112590 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fae8400b0a0 tx=0x7fae8400b3b0 comp rx=0 tx=0).stop 2024-05-08T06:06:12.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.310+0000 7fae91400700 1 -- 172.21.15.123:0/2722324850 shutdown_connections 2024-05-08T06:06:12.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.310+0000 7fae91400700 1 --2- 172.21.15.123:0/2722324850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae8c10b6c0 0x7fae8c10baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:12.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.310+0000 7fae91400700 1 --2- 172.21.15.123:0/2722324850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae8c106080 0x7fae8c112590 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:12.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.310+0000 7fae91400700 1 --2- 172.21.15.123:0/2722324850 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae8c1056c0 0x7fae8c105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:12.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.310+0000 7fae91400700 1 -- 172.21.15.123:0/2722324850 >> 172.21.15.123:0/2722324850 conn(0x7fae8c1011b0 msgr2=0x7fae8c1035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:06:12.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.310+0000 7fae91400700 1 -- 172.21.15.123:0/2722324850 shutdown_connections 2024-05-08T06:06:12.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.310+0000 7fae91400700 1 -- 172.21.15.123:0/2722324850 wait complete. 2024-05-08T06:06:12.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.311+0000 7fae91400700 1 Processor -- start 2024-05-08T06:06:12.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.311+0000 7fae91400700 1 -- start start 2024-05-08T06:06:12.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.312+0000 7fae91400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae8c1056c0 0x7fae8c19bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:12.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.312+0000 7fae91400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae8c106080 0x7fae8c19c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:12.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.312+0000 7fae91400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae8c10b6c0 0x7fae8c1a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:12.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.312+0000 7fae91400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c115900 con 0x7fae8c106080 2024-05-08T06:06:12.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.312+0000 7fae91400700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c115780 con 0x7fae8c1056c0 2024-05-08T06:06:12.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.312+0000 7fae91400700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c115a80 con 0x7fae8c10b6c0 2024-05-08T06:06:12.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.312+0000 7fae89e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae8c106080 0x7fae8c19c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:12.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.312+0000 7fae89e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae8c106080 0x7fae8c19c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48478/0 (socket says 172.21.15.123:48478) 2024-05-08T06:06:12.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.312+0000 7fae8a800700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae8c1056c0 0x7fae8c19bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:12.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.312+0000 7fae89e00700 1 -- 172.21.15.123:0/3919846993 learned_addr learned my addr 172.21.15.123:0/3919846993 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:06:12.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.312+0000 7fae8b200700 1 --2- 172.21.15.123:0/3919846993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae8c10b6c0 0x7fae8c1a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:12.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.313+0000 7fae89e00700 1 -- 172.21.15.123:0/3919846993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae8c10b6c0 msgr2=0x7fae8c1a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:12.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.313+0000 7fae89e00700 1 --2- 172.21.15.123:0/3919846993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae8c10b6c0 0x7fae8c1a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:12.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.313+0000 7fae89e00700 1 -- 172.21.15.123:0/3919846993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae8c1056c0 msgr2=0x7fae8c19bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:12.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.313+0000 7fae89e00700 1 --2- 172.21.15.123:0/3919846993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae8c1056c0 0x7fae8c19bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:12.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.313+0000 7fae89e00700 1 -- 172.21.15.123:0/3919846993 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae8c1a0d20 con 0x7fae8c106080 2024-05-08T06:06:12.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.313+0000 7fae8b200700 1 --2- 172.21.15.123:0/3919846993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae8c10b6c0 0x7fae8c1a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:06:12.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.313+0000 7fae8a800700 1 --2- 172.21.15.123:0/3919846993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae8c1056c0 0x7fae8c19bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:06:12.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.313+0000 7fae89e00700 1 --2- 172.21.15.123:0/3919846993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae8c106080 0x7fae8c19c250 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fae7800e9a0 tx=0x7fae7800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:06:12.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.313+0000 7fae83400700 1 -- 172.21.15.123:0/3919846993 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7800ca10 con 0x7fae8c106080 2024-05-08T06:06:12.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.313+0000 7fae83400700 1 -- 172.21.15.123:0/3919846993 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae7800cbb0 con 0x7fae8c106080 2024-05-08T06:06:12.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.313+0000 7fae91400700 1 -- 172.21.15.123:0/3919846993 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae8c1a1010 con 0x7fae8c106080 2024-05-08T06:06:12.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.313+0000 7fae83400700 1 -- 172.21.15.123:0/3919846993 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae78018680 con 0x7fae8c106080 2024-05-08T06:06:12.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.313+0000 7fae91400700 1 -- 172.21.15.123:0/3919846993 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae8c1a8890 con 0x7fae8c106080 2024-05-08T06:06:12.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.314+0000 7fae91400700 1 -- 172.21.15.123:0/3919846993 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae8c06a190 con 0x7fae8c106080 2024-05-08T06:06:12.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.318+0000 7fae83400700 1 -- 172.21.15.123:0/3919846993 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fae78010430 con 0x7fae8c106080 2024-05-08T06:06:12.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.319+0000 7fae83400700 1 --2- 172.21.15.123:0/3919846993 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fae700b2880 0x7fae700b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:12.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.320+0000 7fae83400700 1 -- 172.21.15.123:0/3919846993 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fae780e2610 con 0x7fae8c106080 2024-05-08T06:06:12.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.320+0000 7fae8a800700 1 --2- 172.21.15.123:0/3919846993 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fae700b2880 0x7fae700b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:12.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.320+0000 7fae8a800700 1 --2- 172.21.15.123:0/3919846993 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fae700b2880 0x7fae700b4d40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fae7c009fd0 tx=0x7fae7c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:06:12.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.321+0000 7fae83400700 1 -- 172.21.15.123:0/3919846993 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fae780aa3e0 con 0x7fae8c106080 2024-05-08T06:06:12.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:12.606+0000 7fae91400700 1 -- 172.21.15.123:0/3919846993 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fae8c02e510 con 0x7fae8c106080 2024-05-08T06:06:12.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:12 smithi163 ceph-mon[34520]: pgmap v330: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-08T06:06:12.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:12 smithi192 ceph-mon[34664]: pgmap v330: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-08T06:06:12.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:12 smithi123 ceph-mon[29787]: pgmap v330: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-08T06:06:13.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:13.382+0000 7fae83400700 1 -- 172.21.15.123:0/3919846993 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7fae780a9ce0 con 0x7fae8c106080 2024-05-08T06:06:13.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:13.388+0000 7fae91400700 1 -- 172.21.15.123:0/3919846993 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fae700b2880 msgr2=0x7fae700b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:13.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:13.388+0000 7fae91400700 1 --2- 172.21.15.123:0/3919846993 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fae700b2880 0x7fae700b4d40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fae7c009fd0 tx=0x7fae7c009260 comp rx=0 tx=0).stop 2024-05-08T06:06:13.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:13.388+0000 7fae91400700 1 -- 172.21.15.123:0/3919846993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae8c106080 msgr2=0x7fae8c19c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:13.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:13.388+0000 7fae91400700 1 --2- 172.21.15.123:0/3919846993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae8c106080 0x7fae8c19c250 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fae7800e9a0 tx=0x7fae7800ecb0 comp rx=0 tx=0).stop 2024-05-08T06:06:13.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:13.388+0000 7fae91400700 1 -- 172.21.15.123:0/3919846993 shutdown_connections 2024-05-08T06:06:13.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:13.388+0000 7fae91400700 1 --2- 172.21.15.123:0/3919846993 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fae700b2880 0x7fae700b4d40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:13.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:13.388+0000 7fae91400700 1 --2- 172.21.15.123:0/3919846993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae8c10b6c0 0x7fae8c1a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:13.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:13.388+0000 7fae91400700 1 --2- 172.21.15.123:0/3919846993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae8c1056c0 0x7fae8c19bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:13.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:13.388+0000 7fae91400700 1 --2- 172.21.15.123:0/3919846993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae8c106080 0x7fae8c19c250 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:13.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:13.388+0000 7fae91400700 1 -- 172.21.15.123:0/3919846993 >> 172.21.15.123:0/3919846993 conn(0x7fae8c1011b0 msgr2=0x7fae8c101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:06:13.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:13.388+0000 7fae91400700 1 -- 172.21.15.123:0/3919846993 shutdown_connections 2024-05-08T06:06:13.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:13.388+0000 7fae91400700 1 -- 172.21.15.123:0/3919846993 wait complete. 2024-05-08T06:06:13.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:13 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/3919846993' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-08T06:06:13.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:13 smithi123 ceph-mon[29787]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-08T06:06:13.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:13 smithi123 ceph-mon[29787]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-08T06:06:13.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:13 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/3919846993' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-08T06:06:13.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:13 smithi123 ceph-mon[29787]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-05-08T06:06:13.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:13 smithi123 ceph-mon[29787]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-05-08T06:06:13.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:13 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/3919846993' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-08T06:06:13.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:13 smithi163 ceph-mon[34520]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-08T06:06:13.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:13 smithi163 ceph-mon[34520]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-08T06:06:13.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:13 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/3919846993' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-08T06:06:13.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:13 smithi163 ceph-mon[34520]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-05-08T06:06:13.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:13 smithi163 ceph-mon[34520]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-05-08T06:06:13.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:13 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/3919846993' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-08T06:06:13.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:13 smithi192 ceph-mon[34664]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-08T06:06:13.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:13 smithi192 ceph-mon[34664]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-08T06:06:13.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:13 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/3919846993' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-08T06:06:13.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:13 smithi192 ceph-mon[34664]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-05-08T06:06:13.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:13 smithi192 ceph-mon[34664]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-05-08T06:06:14.328 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-08T06:06:14.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:14 smithi123 ceph-mon[29787]: pgmap v331: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:06:14.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:14 smithi123 ceph-mon[29787]: mds.? [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] up:active 2024-05-08T06:06:14.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:14 smithi123 ceph-mon[29787]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-08T06:06:14.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:14 smithi123 ceph-mon[29787]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-08T06:06:14.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:14 smithi123 ceph-mon[29787]: Cluster is now healthy 2024-05-08T06:06:14.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:14 smithi123 ceph-mon[29787]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-05-08T06:06:14.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:14 smithi123 ceph-mon[29787]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-05-08T06:06:14.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:14 smithi123 ceph-mon[29787]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-08T06:06:14.769 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:06:14.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:14 smithi163 ceph-mon[34520]: pgmap v331: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:06:14.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:14 smithi163 ceph-mon[34520]: mds.? [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] up:active 2024-05-08T06:06:14.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:14 smithi163 ceph-mon[34520]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-08T06:06:14.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:14 smithi163 ceph-mon[34520]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-08T06:06:14.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:14 smithi163 ceph-mon[34520]: Cluster is now healthy 2024-05-08T06:06:14.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:14 smithi163 ceph-mon[34520]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-05-08T06:06:14.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:14 smithi163 ceph-mon[34520]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-05-08T06:06:14.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:14 smithi163 ceph-mon[34520]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-08T06:06:14.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:14 smithi192 ceph-mon[34664]: pgmap v331: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:06:14.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:14 smithi192 ceph-mon[34664]: mds.? [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] up:active 2024-05-08T06:06:14.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:14 smithi192 ceph-mon[34664]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-08T06:06:14.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:14 smithi192 ceph-mon[34664]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-08T06:06:14.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:14 smithi192 ceph-mon[34664]: Cluster is now healthy 2024-05-08T06:06:14.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:14 smithi192 ceph-mon[34664]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-05-08T06:06:14.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:14 smithi192 ceph-mon[34664]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-05-08T06:06:14.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:14 smithi192 ceph-mon[34664]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-08T06:06:16.397 INFO:journalctl@ceph.mds.b.smithi163.stdout:May 08 06:06:16 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-b[65037]: 2024-05-08T06:06:16.133+0000 7fcfb960d700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-08T06:06:16.398 INFO:journalctl@ceph.mds.b.smithi163.stdout:May 08 06:06:16 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-b[65037]: 2024-05-08T06:06:16.133+0000 7fcfb960d700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-08T06:06:16.528 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:16 smithi123 ceph-mon[29787]: pgmap v332: 97 pgs: 97 active+clean; 580 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-08T06:06:16.528 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:16 smithi123 ceph-mon[29787]: mds.? [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] up:active 2024-05-08T06:06:16.528 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:16 smithi123 ceph-mon[29787]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-05-08T06:06:16.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:16 smithi163 ceph-mon[34520]: pgmap v332: 97 pgs: 97 active+clean; 580 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-08T06:06:16.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:16 smithi163 ceph-mon[34520]: mds.? [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] up:active 2024-05-08T06:06:16.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:16 smithi163 ceph-mon[34520]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-05-08T06:06:16.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:16 smithi192 ceph-mon[34664]: pgmap v332: 97 pgs: 97 active+clean; 580 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-08T06:06:16.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:16 smithi192 ceph-mon[34664]: mds.? [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] up:active 2024-05-08T06:06:16.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:16 smithi192 ceph-mon[34664]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-05-08T06:06:17.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.150+0000 7f67e1200700 1 -- 172.21.15.123:0/2271302776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67dc10b6c0 msgr2=0x7f67dc10baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:17.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.150+0000 7f67e1200700 1 --2- 172.21.15.123:0/2271302776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67dc10b6c0 0x7f67dc10baa0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f67cc009990 tx=0x7f67cc009ca0 comp rx=0 tx=0).stop 2024-05-08T06:06:17.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.150+0000 7f67e1200700 1 -- 172.21.15.123:0/2271302776 shutdown_connections 2024-05-08T06:06:17.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.150+0000 7f67e1200700 1 --2- 172.21.15.123:0/2271302776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67dc10b6c0 0x7f67dc10baa0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:17.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.150+0000 7f67e1200700 1 --2- 172.21.15.123:0/2271302776 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67dc106080 0x7f67dc112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:17.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.150+0000 7f67e1200700 1 --2- 172.21.15.123:0/2271302776 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67dc1056c0 0x7f67dc105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:17.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.150+0000 7f67e1200700 1 -- 172.21.15.123:0/2271302776 >> 172.21.15.123:0/2271302776 conn(0x7f67dc1011b0 msgr2=0x7f67dc1035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:06:17.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.150+0000 7f67e1200700 1 -- 172.21.15.123:0/2271302776 shutdown_connections 2024-05-08T06:06:17.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.151+0000 7f67e1200700 1 -- 172.21.15.123:0/2271302776 wait complete. 2024-05-08T06:06:17.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.152+0000 7f67e1200700 1 Processor -- start 2024-05-08T06:06:17.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.152+0000 7f67e1200700 1 -- start start 2024-05-08T06:06:17.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.152+0000 7f67e1200700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67dc1056c0 0x7f67dc19be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:17.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.152+0000 7f67e1200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67dc106080 0x7f67dc19c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:17.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.152+0000 7f67e1200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67dc10b6c0 0x7f67dc1a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:17.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.152+0000 7f67e1200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc115900 con 0x7f67dc10b6c0 2024-05-08T06:06:17.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.152+0000 7f67e1200700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc115780 con 0x7f67dc106080 2024-05-08T06:06:17.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.152+0000 7f67e1200700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc115a80 con 0x7f67dc1056c0 2024-05-08T06:06:17.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.153+0000 7f67d9400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67dc106080 0x7f67dc19c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:17.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.153+0000 7f67d9400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67dc106080 0x7f67dc19c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:59632/0 (socket says 172.21.15.123:59632) 2024-05-08T06:06:17.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.153+0000 7f67d9400700 1 -- 172.21.15.123:0/705064958 learned_addr learned my addr 172.21.15.123:0/705064958 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:06:17.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.153+0000 7f67da800700 1 --2- 172.21.15.123:0/705064958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67dc10b6c0 0x7f67dc1a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:17.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.153+0000 7f67d9e00700 1 --2- 172.21.15.123:0/705064958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67dc1056c0 0x7f67dc19be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:17.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.153+0000 7f67d9400700 1 -- 172.21.15.123:0/705064958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67dc1056c0 msgr2=0x7f67dc19be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:17.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.153+0000 7f67d9400700 1 --2- 172.21.15.123:0/705064958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67dc1056c0 0x7f67dc19be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:17.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.153+0000 7f67d9400700 1 -- 172.21.15.123:0/705064958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67dc10b6c0 msgr2=0x7f67dc1a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:17.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.153+0000 7f67d9400700 1 --2- 172.21.15.123:0/705064958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67dc10b6c0 0x7f67dc1a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:17.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.153+0000 7f67d9400700 1 -- 172.21.15.123:0/705064958 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67dc1a0ea0 con 0x7f67dc106080 2024-05-08T06:06:17.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.153+0000 7f67d9400700 1 --2- 172.21.15.123:0/705064958 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67dc106080 0x7f67dc19c3d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f67c800d980 tx=0x7f67c800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:06:17.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.154+0000 7f67d2a00700 1 -- 172.21.15.123:0/705064958 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c8016070 con 0x7f67dc106080 2024-05-08T06:06:17.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.154+0000 7f67d2a00700 1 -- 172.21.15.123:0/705064958 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67c8004830 con 0x7f67dc106080 2024-05-08T06:06:17.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.154+0000 7f67d2a00700 1 -- 172.21.15.123:0/705064958 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c8005540 con 0x7f67dc106080 2024-05-08T06:06:17.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.154+0000 7f67e1200700 1 -- 172.21.15.123:0/705064958 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67dc1a1190 con 0x7f67dc106080 2024-05-08T06:06:17.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.154+0000 7f67e1200700 1 -- 172.21.15.123:0/705064958 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67dc1a8a10 con 0x7f67dc106080 2024-05-08T06:06:17.156 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.155+0000 7f67e1200700 1 -- 172.21.15.123:0/705064958 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67dc06a190 con 0x7f67dc106080 2024-05-08T06:06:17.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.158+0000 7f67d2a00700 1 -- 172.21.15.123:0/705064958 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f67c8004220 con 0x7f67dc106080 2024-05-08T06:06:17.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.159+0000 7f67d2a00700 1 --2- 172.21.15.123:0/705064958 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f67c00b2900 0x7f67c00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:17.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.160+0000 7f67d2a00700 1 -- 172.21.15.123:0/705064958 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f67c80e24a0 con 0x7f67dc106080 2024-05-08T06:06:17.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.160+0000 7f67d9e00700 1 --2- 172.21.15.123:0/705064958 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f67c00b2900 0x7f67c00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:17.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.160+0000 7f67d9e00700 1 --2- 172.21.15.123:0/705064958 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f67c00b2900 0x7f67c00b4dc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f67cc006010 tx=0x7f67cc00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:06:17.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.161+0000 7f67d2a00700 1 -- 172.21.15.123:0/705064958 <== mon.2 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f67c801f020 con 0x7f67dc106080 2024-05-08T06:06:17.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:17.442+0000 7f67e1200700 1 -- 172.21.15.123:0/705064958 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f67dc02e510 con 0x7f67dc106080 2024-05-08T06:06:18.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:18.390+0000 7f67d2a00700 1 -- 172.21.15.123:0/705064958 <== mon.2 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f67c80cae20 con 0x7f67dc106080 2024-05-08T06:06:18.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:18.394+0000 7f67e1200700 1 -- 172.21.15.123:0/705064958 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f67c00b2900 msgr2=0x7f67c00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:18.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:18.394+0000 7f67e1200700 1 --2- 172.21.15.123:0/705064958 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f67c00b2900 0x7f67c00b4dc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f67cc006010 tx=0x7f67cc00b260 comp rx=0 tx=0).stop 2024-05-08T06:06:18.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:18.394+0000 7f67e1200700 1 -- 172.21.15.123:0/705064958 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67dc106080 msgr2=0x7f67dc19c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:18.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:18.394+0000 7f67e1200700 1 --2- 172.21.15.123:0/705064958 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67dc106080 0x7f67dc19c3d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f67c800d980 tx=0x7f67c800dc90 comp rx=0 tx=0).stop 2024-05-08T06:06:18.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:18.394+0000 7f67e1200700 1 -- 172.21.15.123:0/705064958 shutdown_connections 2024-05-08T06:06:18.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:18.394+0000 7f67e1200700 1 --2- 172.21.15.123:0/705064958 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f67c00b2900 0x7f67c00b4dc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:18.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:18.394+0000 7f67e1200700 1 --2- 172.21.15.123:0/705064958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67dc10b6c0 0x7f67dc1a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:18.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:18.394+0000 7f67e1200700 1 --2- 172.21.15.123:0/705064958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67dc1056c0 0x7f67dc19be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:18.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:18.394+0000 7f67e1200700 1 --2- 172.21.15.123:0/705064958 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67dc106080 0x7f67dc19c3d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:18.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:18.394+0000 7f67e1200700 1 -- 172.21.15.123:0/705064958 >> 172.21.15.123:0/705064958 conn(0x7f67dc1011b0 msgr2=0x7f67dc101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:06:18.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:18.395+0000 7f67e1200700 1 -- 172.21.15.123:0/705064958 shutdown_connections 2024-05-08T06:06:18.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:18.395+0000 7f67e1200700 1 -- 172.21.15.123:0/705064958 wait complete. 2024-05-08T06:06:18.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:18 smithi123 ceph-mon[29787]: pgmap v333: 97 pgs: 97 active+clean; 582 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-05-08T06:06:18.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:18 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/705064958' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T06:06:18.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:18 smithi123 ceph-mon[29787]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T06:06:18.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:18 smithi123 ceph-mon[29787]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-08T06:06:18.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:18 smithi123 ceph-mon[29787]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T06:06:18.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:18 smithi163 ceph-mon[34520]: pgmap v333: 97 pgs: 97 active+clean; 582 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-05-08T06:06:18.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:18 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/705064958' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T06:06:18.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:18 smithi163 ceph-mon[34520]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T06:06:18.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:18 smithi163 ceph-mon[34520]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-08T06:06:18.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:18 smithi163 ceph-mon[34520]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T06:06:18.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:18 smithi192 ceph-mon[34664]: pgmap v333: 97 pgs: 97 active+clean; 582 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-05-08T06:06:18.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:18 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/705064958' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T06:06:18.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:18 smithi192 ceph-mon[34664]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T06:06:18.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:18 smithi192 ceph-mon[34664]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-08T06:06:18.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:18 smithi192 ceph-mon[34664]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T06:06:19.282 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-08T06:06:19.739 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:06:20.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:20 smithi123 ceph-mon[29787]: pgmap v334: 97 pgs: 97 active+clean; 582 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-05-08T06:06:20.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:20 smithi163 ceph-mon[34520]: pgmap v334: 97 pgs: 97 active+clean; 582 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-05-08T06:06:20.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:20 smithi192 ceph-mon[34664]: pgmap v334: 97 pgs: 97 active+clean; 582 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-05-08T06:06:22.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.298+0000 7fabe9c00700 1 -- 172.21.15.123:0/410992047 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabe410a490 msgr2=0x7fabe41169a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:22.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.298+0000 7fabe9c00700 1 --2- 172.21.15.123:0/410992047 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabe410a490 0x7fabe41169a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fabdc00b0a0 tx=0x7fabdc00b3b0 comp rx=0 tx=0).stop 2024-05-08T06:06:22.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.298+0000 7fabe9c00700 1 -- 172.21.15.123:0/410992047 shutdown_connections 2024-05-08T06:06:22.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.298+0000 7fabe9c00700 1 --2- 172.21.15.123:0/410992047 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabe410a490 0x7fabe41169a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:22.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.298+0000 7fabe9c00700 1 --2- 172.21.15.123:0/410992047 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabe410fb60 0x7fabe410ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:22.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.298+0000 7fabe9c00700 1 --2- 172.21.15.123:0/410992047 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabe4109ad0 0x7fabe4109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:22.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.298+0000 7fabe9c00700 1 -- 172.21.15.123:0/410992047 >> 172.21.15.123:0/410992047 conn(0x7fabe4076c70 msgr2=0x7fabe41082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:06:22.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.298+0000 7fabe9c00700 1 -- 172.21.15.123:0/410992047 shutdown_connections 2024-05-08T06:06:22.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.298+0000 7fabe9c00700 1 -- 172.21.15.123:0/410992047 wait complete. 2024-05-08T06:06:22.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.299+0000 7fabe9c00700 1 Processor -- start 2024-05-08T06:06:22.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.299+0000 7fabe9c00700 1 -- start start 2024-05-08T06:06:22.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.299+0000 7fabe9c00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabe4109ad0 0x7fabe41a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:22.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.300+0000 7fabe9c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabe410a490 0x7fabe41a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:22.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.300+0000 7fabe9c00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabe410fb60 0x7fabe41a4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:22.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.300+0000 7fabe9c00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fabe4119d20 con 0x7fabe410a490 2024-05-08T06:06:22.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.300+0000 7fabe9c00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fabe4119ba0 con 0x7fabe410fb60 2024-05-08T06:06:22.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.300+0000 7fabe9c00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fabe4119ea0 con 0x7fabe4109ad0 2024-05-08T06:06:22.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.300+0000 7fabe2000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabe410a490 0x7fabe41a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:22.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.300+0000 7fabe2a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabe4109ad0 0x7fabe41a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:22.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.300+0000 7fabe2a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabe4109ad0 0x7fabe41a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:54936/0 (socket says 172.21.15.123:54936) 2024-05-08T06:06:22.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.300+0000 7fabe2a00700 1 -- 172.21.15.123:0/1327408610 learned_addr learned my addr 172.21.15.123:0/1327408610 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:06:22.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.300+0000 7fabe3400700 1 --2- 172.21.15.123:0/1327408610 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabe410fb60 0x7fabe41a4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:22.302 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.300+0000 7fabe2a00700 1 -- 172.21.15.123:0/1327408610 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabe410fb60 msgr2=0x7fabe41a4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:22.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.300+0000 7fabe2a00700 1 --2- 172.21.15.123:0/1327408610 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabe410fb60 0x7fabe41a4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:22.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.300+0000 7fabe2a00700 1 -- 172.21.15.123:0/1327408610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabe410a490 msgr2=0x7fabe41a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:22.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.300+0000 7fabe2a00700 1 --2- 172.21.15.123:0/1327408610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabe410a490 0x7fabe41a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:22.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.300+0000 7fabe2a00700 1 -- 172.21.15.123:0/1327408610 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe41a5150 con 0x7fabe4109ad0 2024-05-08T06:06:22.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.301+0000 7fabe2a00700 1 --2- 172.21.15.123:0/1327408610 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabe4109ad0 0x7fabe41a0190 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fabd400d980 tx=0x7fabd400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:06:22.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.301+0000 7fabdb400700 1 -- 172.21.15.123:0/1327408610 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd4016070 con 0x7fabe4109ad0 2024-05-08T06:06:22.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.301+0000 7fabe9c00700 1 -- 172.21.15.123:0/1327408610 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabe41a5440 con 0x7fabe4109ad0 2024-05-08T06:06:22.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.301+0000 7fabe9c00700 1 -- 172.21.15.123:0/1327408610 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabe41accc0 con 0x7fabe4109ad0 2024-05-08T06:06:22.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.301+0000 7fabdb400700 1 -- 172.21.15.123:0/1327408610 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabd4004830 con 0x7fabe4109ad0 2024-05-08T06:06:22.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.301+0000 7fabdb400700 1 -- 172.21.15.123:0/1327408610 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd4005420 con 0x7fabe4109ad0 2024-05-08T06:06:22.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.302+0000 7fabe9c00700 1 -- 172.21.15.123:0/1327408610 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabc4005330 con 0x7fabe4109ad0 2024-05-08T06:06:22.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.305+0000 7fabdb400700 1 -- 172.21.15.123:0/1327408610 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fabd40055c0 con 0x7fabe4109ad0 2024-05-08T06:06:22.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.306+0000 7fabdb400700 1 --2- 172.21.15.123:0/1327408610 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fabc80b2900 0x7fabc80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:22.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.306+0000 7fabdb400700 1 -- 172.21.15.123:0/1327408610 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fabd40e22b0 con 0x7fabe4109ad0 2024-05-08T06:06:22.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.306+0000 7fabe2000700 1 --2- 172.21.15.123:0/1327408610 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fabc80b2900 0x7fabc80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:22.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.307+0000 7fabe2000700 1 --2- 172.21.15.123:0/1327408610 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fabc80b2900 0x7fabc80b4dc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fabe4078b80 tx=0x7fabd0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:06:22.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.309+0000 7fabdb400700 1 -- 172.21.15.123:0/1327408610 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fabd401f020 con 0x7fabe4109ad0 2024-05-08T06:06:22.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:22.591+0000 7fabe9c00700 1 -- 172.21.15.123:0/1327408610 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fabc4005620 con 0x7fabe4109ad0 2024-05-08T06:06:22.591 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:22 smithi123 ceph-mon[29787]: pgmap v335: 97 pgs: 97 active+clean; 582 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-08T06:06:22.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:22 smithi163 ceph-mon[34520]: pgmap v335: 97 pgs: 97 active+clean; 582 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-08T06:06:22.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:22 smithi192 ceph-mon[34664]: pgmap v335: 97 pgs: 97 active+clean; 582 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-08T06:06:23.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:23.421+0000 7fabdb400700 1 -- 172.21.15.123:0/1327408610 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) ==== 115+0+0 (secure 0 0 0) 0x7fabd40aa080 con 0x7fabe4109ad0 2024-05-08T06:06:23.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:23.428+0000 7fabe9c00700 1 -- 172.21.15.123:0/1327408610 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fabc80b2900 msgr2=0x7fabc80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:23.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:23.428+0000 7fabe9c00700 1 --2- 172.21.15.123:0/1327408610 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fabc80b2900 0x7fabc80b4dc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fabe4078b80 tx=0x7fabd0009260 comp rx=0 tx=0).stop 2024-05-08T06:06:23.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:23.428+0000 7fabe9c00700 1 -- 172.21.15.123:0/1327408610 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabe4109ad0 msgr2=0x7fabe41a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:23.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:23.428+0000 7fabe9c00700 1 --2- 172.21.15.123:0/1327408610 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabe4109ad0 0x7fabe41a0190 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fabd400d980 tx=0x7fabd400dc90 comp rx=0 tx=0).stop 2024-05-08T06:06:23.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:23.428+0000 7fabe9c00700 1 -- 172.21.15.123:0/1327408610 shutdown_connections 2024-05-08T06:06:23.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:23.428+0000 7fabe9c00700 1 --2- 172.21.15.123:0/1327408610 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fabc80b2900 0x7fabc80b4dc0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:23.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:23.428+0000 7fabe9c00700 1 --2- 172.21.15.123:0/1327408610 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabe410fb60 0x7fabe41a4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:23.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:23.428+0000 7fabe9c00700 1 --2- 172.21.15.123:0/1327408610 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabe4109ad0 0x7fabe41a0190 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:23.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:23.428+0000 7fabe9c00700 1 --2- 172.21.15.123:0/1327408610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabe410a490 0x7fabe41a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:23.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:23.428+0000 7fabe9c00700 1 -- 172.21.15.123:0/1327408610 >> 172.21.15.123:0/1327408610 conn(0x7fabe4076c70 msgr2=0x7fabe4077230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:06:23.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:23.429+0000 7fabe9c00700 1 -- 172.21.15.123:0/1327408610 shutdown_connections 2024-05-08T06:06:23.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:23.429+0000 7fabe9c00700 1 -- 172.21.15.123:0/1327408610 wait complete. 2024-05-08T06:06:23.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:23 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/1327408610' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T06:06:23.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:23 smithi123 ceph-mon[29787]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T06:06:23.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:23 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/1327408610' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T06:06:23.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:23 smithi163 ceph-mon[34520]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T06:06:23.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:23 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/1327408610' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T06:06:23.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:23 smithi192 ceph-mon[34664]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T06:06:24.401 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-05-08T06:06:24.402 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-08T06:06:24.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:24 smithi123 ceph-mon[29787]: pgmap v336: 97 pgs: 97 active+clean; 582 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-08T06:06:24.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:24 smithi123 ceph-mon[29787]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-08T06:06:24.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:24 smithi123 ceph-mon[29787]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T06:06:24.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:24 smithi163 ceph-mon[34520]: pgmap v336: 97 pgs: 97 active+clean; 582 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-08T06:06:24.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:24 smithi163 ceph-mon[34520]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-08T06:06:24.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:24 smithi163 ceph-mon[34520]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T06:06:24.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:24 smithi192 ceph-mon[34664]: pgmap v336: 97 pgs: 97 active+clean; 582 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-08T06:06:24.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:24 smithi192 ceph-mon[34664]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-08T06:06:24.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:24 smithi192 ceph-mon[34664]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T06:06:24.845 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:06:26.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:26 smithi123 ceph-mon[29787]: pgmap v337: 97 pgs: 97 active+clean; 582 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-08T06:06:26.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:26 smithi163 ceph-mon[34520]: pgmap v337: 97 pgs: 97 active+clean; 582 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-08T06:06:26.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:26 smithi192 ceph-mon[34664]: pgmap v337: 97 pgs: 97 active+clean; 582 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-08T06:06:27.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.382+0000 7fa5abc00700 1 -- 172.21.15.123:0/989670173 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5a4106070 msgr2=0x7fa5a4112580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:27.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.382+0000 7fa5abc00700 1 --2- 172.21.15.123:0/989670173 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5a4106070 0x7fa5a4112580 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa59c00b0a0 tx=0x7fa59c00b3b0 comp rx=0 tx=0).stop 2024-05-08T06:06:27.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.382+0000 7fa5abc00700 1 -- 172.21.15.123:0/989670173 shutdown_connections 2024-05-08T06:06:27.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.382+0000 7fa5abc00700 1 --2- 172.21.15.123:0/989670173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5a410b6b0 0x7fa5a410ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:27.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.382+0000 7fa5abc00700 1 --2- 172.21.15.123:0/989670173 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5a4106070 0x7fa5a4112580 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:27.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.382+0000 7fa5abc00700 1 --2- 172.21.15.123:0/989670173 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5a41056b0 0x7fa5a4105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:27.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.382+0000 7fa5abc00700 1 -- 172.21.15.123:0/989670173 >> 172.21.15.123:0/989670173 conn(0x7fa5a41011a0 msgr2=0x7fa5a41035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:06:27.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.382+0000 7fa5abc00700 1 -- 172.21.15.123:0/989670173 shutdown_connections 2024-05-08T06:06:27.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.382+0000 7fa5abc00700 1 -- 172.21.15.123:0/989670173 wait complete. 2024-05-08T06:06:27.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.383+0000 7fa5abc00700 1 Processor -- start 2024-05-08T06:06:27.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.383+0000 7fa5abc00700 1 -- start start 2024-05-08T06:06:27.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.384+0000 7fa5abc00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5a41056b0 0x7fa5a41a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:27.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.384+0000 7fa5abc00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5a4106070 0x7fa5a41a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:27.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.384+0000 7fa5abc00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5a410b6b0 0x7fa5a419bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:27.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.384+0000 7fa5abc00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a41158f0 con 0x7fa5a4106070 2024-05-08T06:06:27.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.384+0000 7fa5abc00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a4115770 con 0x7fa5a41056b0 2024-05-08T06:06:27.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.384+0000 7fa5abc00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a4115a70 con 0x7fa5a410b6b0 2024-05-08T06:06:27.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.384+0000 7fa5a9c00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5a410b6b0 0x7fa5a419bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:27.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.384+0000 7fa5a3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5a4106070 0x7fa5a41a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:27.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.384+0000 7fa5a9c00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5a410b6b0 0x7fa5a419bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:41756/0 (socket says 172.21.15.123:41756) 2024-05-08T06:06:27.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.384+0000 7fa5a9c00700 1 -- 172.21.15.123:0/2949826183 learned_addr learned my addr 172.21.15.123:0/2949826183 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:06:27.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.384+0000 7fa5a9200700 1 --2- 172.21.15.123:0/2949826183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5a41056b0 0x7fa5a41a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:27.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.384+0000 7fa5a9c00700 1 -- 172.21.15.123:0/2949826183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5a41056b0 msgr2=0x7fa5a41a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:27.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.384+0000 7fa5a9c00700 1 --2- 172.21.15.123:0/2949826183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5a41056b0 0x7fa5a41a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:27.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.384+0000 7fa5a9c00700 1 -- 172.21.15.123:0/2949826183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5a4106070 msgr2=0x7fa5a41a1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:27.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.384+0000 7fa5a9c00700 1 --2- 172.21.15.123:0/2949826183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5a4106070 0x7fa5a41a1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:27.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.384+0000 7fa5a9c00700 1 -- 172.21.15.123:0/2949826183 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a419c3f0 con 0x7fa5a410b6b0 2024-05-08T06:06:27.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.385+0000 7fa5a9c00700 1 --2- 172.21.15.123:0/2949826183 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5a410b6b0 0x7fa5a419bb10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa59c00b070 tx=0x7fa59c009c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:06:27.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.385+0000 7fa5a1600700 1 -- 172.21.15.123:0/2949826183 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c021070 con 0x7fa5a410b6b0 2024-05-08T06:06:27.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.385+0000 7fa5abc00700 1 -- 172.21.15.123:0/2949826183 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a419c680 con 0x7fa5a410b6b0 2024-05-08T06:06:27.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.385+0000 7fa5a1600700 1 -- 172.21.15.123:0/2949826183 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa59c004e40 con 0x7fa5a410b6b0 2024-05-08T06:06:27.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.385+0000 7fa5a1600700 1 -- 172.21.15.123:0/2949826183 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c004630 con 0x7fa5a410b6b0 2024-05-08T06:06:27.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.385+0000 7fa5abc00700 1 -- 172.21.15.123:0/2949826183 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a41a89c0 con 0x7fa5a410b6b0 2024-05-08T06:06:27.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.386+0000 7fa5abc00700 1 -- 172.21.15.123:0/2949826183 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5a4106df0 con 0x7fa5a410b6b0 2024-05-08T06:06:27.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.389+0000 7fa5a1600700 1 -- 172.21.15.123:0/2949826183 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fa59c007980 con 0x7fa5a410b6b0 2024-05-08T06:06:27.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.390+0000 7fa5a1600700 1 --2- 172.21.15.123:0/2949826183 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa58c0b2900 0x7fa58c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:27.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.391+0000 7fa5a1600700 1 -- 172.21.15.123:0/2949826183 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fa59c00f020 con 0x7fa5a410b6b0 2024-05-08T06:06:27.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.391+0000 7fa5a9200700 1 --2- 172.21.15.123:0/2949826183 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa58c0b2900 0x7fa58c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:27.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.391+0000 7fa5a9200700 1 --2- 172.21.15.123:0/2949826183 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa58c0b2900 0x7fa58c0b4dc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa598009be0 tx=0x7fa598009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:06:27.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.392+0000 7fa5a1600700 1 -- 172.21.15.123:0/2949826183 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa59c02a020 con 0x7fa5a410b6b0 2024-05-08T06:06:27.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.658+0000 7fa5abc00700 1 -- 172.21.15.123:0/2949826183 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fa5a419cfd0 con 0x7fa58c0b2900 2024-05-08T06:06:27.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.695+0000 7fa5a1600700 1 -- 172.21.15.123:0/2949826183 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa5a419cfd0 con 0x7fa58c0b2900 2024-05-08T06:06:27.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.698+0000 7fa5abc00700 1 -- 172.21.15.123:0/2949826183 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa58c0b2900 msgr2=0x7fa58c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:27.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.698+0000 7fa5abc00700 1 --2- 172.21.15.123:0/2949826183 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa58c0b2900 0x7fa58c0b4dc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa598009be0 tx=0x7fa598009260 comp rx=0 tx=0).stop 2024-05-08T06:06:27.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.698+0000 7fa5abc00700 1 -- 172.21.15.123:0/2949826183 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5a410b6b0 msgr2=0x7fa5a419bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:27.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.698+0000 7fa5abc00700 1 --2- 172.21.15.123:0/2949826183 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5a410b6b0 0x7fa5a419bb10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa59c00b070 tx=0x7fa59c009c90 comp rx=0 tx=0).stop 2024-05-08T06:06:27.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.699+0000 7fa5abc00700 1 -- 172.21.15.123:0/2949826183 shutdown_connections 2024-05-08T06:06:27.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.699+0000 7fa5abc00700 1 --2- 172.21.15.123:0/2949826183 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa58c0b2900 0x7fa58c0b4dc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:27.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.699+0000 7fa5abc00700 1 --2- 172.21.15.123:0/2949826183 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5a410b6b0 0x7fa5a419bb10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:27.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.699+0000 7fa5abc00700 1 --2- 172.21.15.123:0/2949826183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5a41056b0 0x7fa5a41a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:27.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.699+0000 7fa5abc00700 1 --2- 172.21.15.123:0/2949826183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5a4106070 0x7fa5a41a1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:27.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.699+0000 7fa5abc00700 1 -- 172.21.15.123:0/2949826183 >> 172.21.15.123:0/2949826183 conn(0x7fa5a41011a0 msgr2=0x7fa5a41017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:06:27.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.699+0000 7fa5abc00700 1 -- 172.21.15.123:0/2949826183 shutdown_connections 2024-05-08T06:06:27.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:27.699+0000 7fa5abc00700 1 -- 172.21.15.123:0/2949826183 wait complete. 2024-05-08T06:06:27.955 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 06:06:27 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T06:06:27.667+0000 7f780ca00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T06:06:27.956 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 06:06:27 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T06:06:27.667+0000 7f780ca00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T06:06:27.956 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 06:06:27 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T06:06:27.667+0000 7f780ca00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T06:06:27.956 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 06:06:27 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T06:06:27.667+0000 7f780ca00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T06:06:27.956 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 06:06:27 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T06:06:27.667+0000 7f780ca00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T06:06:28.610 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-08T06:06:28.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:28 smithi123 ceph-mon[29787]: pgmap v338: 97 pgs: 97 active+clean; 582 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-05-08T06:06:28.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:28 smithi123 ceph-mon[29787]: from='client.24592 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:06:28.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:28 smithi163 ceph-mon[34520]: pgmap v338: 97 pgs: 97 active+clean; 582 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-05-08T06:06:28.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:28 smithi163 ceph-mon[34520]: from='client.24592 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:06:28.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:28 smithi192 ceph-mon[34664]: pgmap v338: 97 pgs: 97 active+clean; 582 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-05-08T06:06:28.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:28 smithi192 ceph-mon[34664]: from='client.24592 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:06:29.065 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:06:30.615 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:30 smithi123 ceph-mon[29787]: pgmap v339: 97 pgs: 97 active+clean; 588 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 852 B/s wr, 7 op/s 2024-05-08T06:06:30.615 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:30 smithi123 ceph-mon[29787]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-08T06:06:30.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:30 smithi163 ceph-mon[34520]: pgmap v339: 97 pgs: 97 active+clean; 588 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 852 B/s wr, 7 op/s 2024-05-08T06:06:30.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:30 smithi163 ceph-mon[34520]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-08T06:06:30.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:30 smithi192 ceph-mon[34664]: pgmap v339: 97 pgs: 97 active+clean; 588 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 852 B/s wr, 7 op/s 2024-05-08T06:06:30.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:30 smithi192 ceph-mon[34664]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-08T06:06:31.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.526+0000 7fe45ac00700 1 -- 172.21.15.123:0/1085463316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4541099e0 msgr2=0x7fe454109e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:31.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.526+0000 7fe45ac00700 1 --2- 172.21.15.123:0/1085463316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4541099e0 0x7fe454109e60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe440009990 tx=0x7fe440009ca0 comp rx=0 tx=0).stop 2024-05-08T06:06:31.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.526+0000 7fe45ac00700 1 -- 172.21.15.123:0/1085463316 shutdown_connections 2024-05-08T06:06:31.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.526+0000 7fe45ac00700 1 --2- 172.21.15.123:0/1085463316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe45410a3a0 0x7fe454116840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:31.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.526+0000 7fe45ac00700 1 --2- 172.21.15.123:0/1085463316 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe45410e8b0 0x7fe45410ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:31.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.526+0000 7fe45ac00700 1 --2- 172.21.15.123:0/1085463316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4541099e0 0x7fe454109e60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:31.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.526+0000 7fe45ac00700 1 -- 172.21.15.123:0/1085463316 >> 172.21.15.123:0/1085463316 conn(0x7fe454079230 msgr2=0x7fe4541082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:06:31.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.526+0000 7fe45ac00700 1 -- 172.21.15.123:0/1085463316 shutdown_connections 2024-05-08T06:06:31.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.526+0000 7fe45ac00700 1 -- 172.21.15.123:0/1085463316 wait complete. 2024-05-08T06:06:31.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.527+0000 7fe45ac00700 1 Processor -- start 2024-05-08T06:06:31.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.527+0000 7fe45ac00700 1 -- start start 2024-05-08T06:06:31.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.528+0000 7fe45ac00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4541099e0 0x7fe4541a0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:31.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.528+0000 7fe45ac00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe45410a3a0 0x7fe4541a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:31.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.528+0000 7fe45ac00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe45410e8b0 0x7fe4541a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:31.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.528+0000 7fe45ac00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe454119b70 con 0x7fe45410a3a0 2024-05-08T06:06:31.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.528+0000 7fe45ac00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe4541199f0 con 0x7fe4541099e0 2024-05-08T06:06:31.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.528+0000 7fe45ac00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe454119cf0 con 0x7fe45410e8b0 2024-05-08T06:06:31.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.528+0000 7fe458e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe45410a3a0 0x7fe4541a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:31.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.528+0000 7fe458e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe45410a3a0 0x7fe4541a05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40432/0 (socket says 172.21.15.123:40432) 2024-05-08T06:06:31.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.528+0000 7fe458e00700 1 -- 172.21.15.123:0/3028927413 learned_addr learned my addr 172.21.15.123:0/3028927413 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:06:31.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.528+0000 7fe459800700 1 --2- 172.21.15.123:0/3028927413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4541099e0 0x7fe4541a0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:31.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.528+0000 7fe45a200700 1 --2- 172.21.15.123:0/3028927413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe45410e8b0 0x7fe4541a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:31.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.529+0000 7fe458e00700 1 -- 172.21.15.123:0/3028927413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe45410e8b0 msgr2=0x7fe4541a4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:31.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.529+0000 7fe458e00700 1 --2- 172.21.15.123:0/3028927413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe45410e8b0 0x7fe4541a4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:31.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.529+0000 7fe458e00700 1 -- 172.21.15.123:0/3028927413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4541099e0 msgr2=0x7fe4541a0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:31.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.529+0000 7fe458e00700 1 --2- 172.21.15.123:0/3028927413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4541099e0 0x7fe4541a0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:31.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.529+0000 7fe458e00700 1 -- 172.21.15.123:0/3028927413 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4541a5070 con 0x7fe45410a3a0 2024-05-08T06:06:31.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.529+0000 7fe458e00700 1 --2- 172.21.15.123:0/3028927413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe45410a3a0 0x7fe4541a05a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe440000c00 tx=0x7fe4400047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:06:31.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.529+0000 7fe452000700 1 -- 172.21.15.123:0/3028927413 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe440027070 con 0x7fe45410a3a0 2024-05-08T06:06:31.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.529+0000 7fe452000700 1 -- 172.21.15.123:0/3028927413 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe440003f20 con 0x7fe45410a3a0 2024-05-08T06:06:31.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.529+0000 7fe45ac00700 1 -- 172.21.15.123:0/3028927413 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4541a5300 con 0x7fe45410a3a0 2024-05-08T06:06:31.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.529+0000 7fe452000700 1 -- 172.21.15.123:0/3028927413 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe440022ca0 con 0x7fe45410a3a0 2024-05-08T06:06:31.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.529+0000 7fe45ac00700 1 -- 172.21.15.123:0/3028927413 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4541acbe0 con 0x7fe45410a3a0 2024-05-08T06:06:31.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.531+0000 7fe45ac00700 1 -- 172.21.15.123:0/3028927413 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe45406a190 con 0x7fe45410a3a0 2024-05-08T06:06:31.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.533+0000 7fe452000700 1 -- 172.21.15.123:0/3028927413 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fe44002a070 con 0x7fe45410a3a0 2024-05-08T06:06:31.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.534+0000 7fe452000700 1 --2- 172.21.15.123:0/3028927413 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe43c0b2990 0x7fe43c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:31.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.534+0000 7fe452000700 1 -- 172.21.15.123:0/3028927413 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8211+0+0 (secure 0 0 0) 0x7fe4400e31d0 con 0x7fe45410a3a0 2024-05-08T06:06:31.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.534+0000 7fe459800700 1 --2- 172.21.15.123:0/3028927413 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe43c0b2990 0x7fe43c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:31.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.535+0000 7fe459800700 1 --2- 172.21.15.123:0/3028927413 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe43c0b2990 0x7fe43c0b4e50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe448007b60 tx=0x7fe4480073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:06:31.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.539+0000 7fe452000700 1 -- 172.21.15.123:0/3028927413 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe440013070 con 0x7fe45410a3a0 2024-05-08T06:06:31.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:31 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-05-08T06:06:31.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:31 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-05-08T06:06:31.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:31 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-05-08T06:06:31.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:31 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-05-08T06:06:31.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:31 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]: dispatch 2024-05-08T06:06:31.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:31 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-05-08T06:06:31.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:31 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-05-08T06:06:31.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:31 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-05-08T06:06:31.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:31 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-05-08T06:06:31.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:31 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-05-08T06:06:31.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:31 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-05-08T06:06:31.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:31 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-05-08T06:06:31.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:31 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]: dispatch 2024-05-08T06:06:31.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:31 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-05-08T06:06:31.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:31 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-05-08T06:06:31.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:31 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-05-08T06:06:31.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.805+0000 7fe45ac00700 1 -- 172.21.15.123:0/3028927413 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fe4541a13c0 con 0x7fe43c0b2990 2024-05-08T06:06:31.805 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:31 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-05-08T06:06:31.805 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:31 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-05-08T06:06:31.806 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:31 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-05-08T06:06:31.806 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:31 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-05-08T06:06:31.806 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:31 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]: dispatch 2024-05-08T06:06:31.806 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:31 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-05-08T06:06:31.806 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:31 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-05-08T06:06:31.806 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:31 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-05-08T06:06:31.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.848+0000 7fe452000700 1 -- 172.21.15.123:0/3028927413 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe4541a13c0 con 0x7fe43c0b2990 2024-05-08T06:06:31.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.852+0000 7fe45ac00700 1 -- 172.21.15.123:0/3028927413 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe43c0b2990 msgr2=0x7fe43c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:31.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.852+0000 7fe45ac00700 1 --2- 172.21.15.123:0/3028927413 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe43c0b2990 0x7fe43c0b4e50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe448007b60 tx=0x7fe4480073d0 comp rx=0 tx=0).stop 2024-05-08T06:06:31.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.852+0000 7fe45ac00700 1 -- 172.21.15.123:0/3028927413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe45410a3a0 msgr2=0x7fe4541a05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:31.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.852+0000 7fe45ac00700 1 --2- 172.21.15.123:0/3028927413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe45410a3a0 0x7fe4541a05a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe440000c00 tx=0x7fe4400047b0 comp rx=0 tx=0).stop 2024-05-08T06:06:31.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.852+0000 7fe45ac00700 1 -- 172.21.15.123:0/3028927413 shutdown_connections 2024-05-08T06:06:31.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.852+0000 7fe45ac00700 1 --2- 172.21.15.123:0/3028927413 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe43c0b2990 0x7fe43c0b4e50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:31.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.852+0000 7fe45ac00700 1 --2- 172.21.15.123:0/3028927413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe45410e8b0 0x7fe4541a4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:31.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.852+0000 7fe45ac00700 1 --2- 172.21.15.123:0/3028927413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4541099e0 0x7fe4541a0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:31.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.852+0000 7fe45ac00700 1 --2- 172.21.15.123:0/3028927413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe45410a3a0 0x7fe4541a05a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:31.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.852+0000 7fe45ac00700 1 -- 172.21.15.123:0/3028927413 >> 172.21.15.123:0/3028927413 conn(0x7fe454079230 msgr2=0x7fe454079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:06:31.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.853+0000 7fe45ac00700 1 -- 172.21.15.123:0/3028927413 shutdown_connections 2024-05-08T06:06:31.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:31.853+0000 7fe45ac00700 1 -- 172.21.15.123:0/3028927413 wait complete. 2024-05-08T06:06:32.699 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:32 smithi123 ceph-mon[29787]: pgmap v340: 97 pgs: 97 active+clean; 588 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 852 B/s wr, 7 op/s 2024-05-08T06:06:32.700 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:32 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-05-08T06:06:32.700 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:32 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-05-08T06:06:32.700 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:32 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-05-08T06:06:32.700 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:32 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-05-08T06:06:32.700 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:32 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]': finished 2024-05-08T06:06:32.700 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:32 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-05-08T06:06:32.700 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:32 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-05-08T06:06:32.700 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:32 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-05-08T06:06:32.700 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:32 smithi123 ceph-mon[29787]: osdmap e81: 12 total, 12 up, 12 in 2024-05-08T06:06:32.700 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:32 smithi123 ceph-mon[29787]: from='client.14490 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:06:32.759 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-08T06:06:32.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:32 smithi163 ceph-mon[34520]: pgmap v340: 97 pgs: 97 active+clean; 588 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 852 B/s wr, 7 op/s 2024-05-08T06:06:32.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:32 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-05-08T06:06:32.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:32 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-05-08T06:06:32.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:32 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-05-08T06:06:32.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:32 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-05-08T06:06:32.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:32 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]': finished 2024-05-08T06:06:32.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:32 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-05-08T06:06:32.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:32 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-05-08T06:06:32.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:32 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-05-08T06:06:32.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:32 smithi163 ceph-mon[34520]: osdmap e81: 12 total, 12 up, 12 in 2024-05-08T06:06:32.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:32 smithi163 ceph-mon[34520]: from='client.14490 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:06:32.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:32 smithi192 ceph-mon[34664]: pgmap v340: 97 pgs: 97 active+clean; 588 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 852 B/s wr, 7 op/s 2024-05-08T06:06:32.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:32 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-05-08T06:06:32.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:32 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-05-08T06:06:32.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:32 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-05-08T06:06:32.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:32 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-05-08T06:06:32.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:32 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]': finished 2024-05-08T06:06:32.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:32 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-05-08T06:06:32.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:32 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-05-08T06:06:32.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:32 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-05-08T06:06:32.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:32 smithi192 ceph-mon[34664]: osdmap e81: 12 total, 12 up, 12 in 2024-05-08T06:06:32.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:32 smithi192 ceph-mon[34664]: from='client.14490 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:06:33.217 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:06:33.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:33 smithi123 ceph-mon[29787]: osdmap e82: 12 total, 12 up, 12 in 2024-05-08T06:06:33.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:33 smithi163 ceph-mon[34520]: osdmap e82: 12 total, 12 up, 12 in 2024-05-08T06:06:33.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:33 smithi192 ceph-mon[34664]: osdmap e82: 12 total, 12 up, 12 in 2024-05-08T06:06:34.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:34 smithi163 ceph-mon[34520]: pgmap v343: 97 pgs: 2 peering, 95 active+clean; 588 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-05-08T06:06:34.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:34 smithi192 ceph-mon[34664]: pgmap v343: 97 pgs: 2 peering, 95 active+clean; 588 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-05-08T06:06:34.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:34 smithi123 ceph-mon[29787]: pgmap v343: 97 pgs: 2 peering, 95 active+clean; 588 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-05-08T06:06:35.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.510+0000 7f71fdc00700 1 -- 172.21.15.123:0/3535118863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71f8103eb0 msgr2=0x7f71f8110390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:35.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.510+0000 7f71fdc00700 1 --2- 172.21.15.123:0/3535118863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71f8103eb0 0x7f71f8110390 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f71f000b0a0 tx=0x7f71f000b3b0 comp rx=0 tx=0).stop 2024-05-08T06:06:35.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.511+0000 7f71fdc00700 1 -- 172.21.15.123:0/3535118863 shutdown_connections 2024-05-08T06:06:35.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.511+0000 7f71fdc00700 1 --2- 172.21.15.123:0/3535118863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71f8103eb0 0x7f71f8110390 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:35.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.511+0000 7f71fdc00700 1 --2- 172.21.15.123:0/3535118863 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71f8109600 0x7f71f81099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:35.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.511+0000 7f71fdc00700 1 --2- 172.21.15.123:0/3535118863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71f81034f0 0x7f71f8103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:35.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.511+0000 7f71fdc00700 1 -- 172.21.15.123:0/3535118863 >> 172.21.15.123:0/3535118863 conn(0x7f71f807a650 msgr2=0x7f71f807aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:06:35.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.511+0000 7f71fdc00700 1 -- 172.21.15.123:0/3535118863 shutdown_connections 2024-05-08T06:06:35.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.511+0000 7f71fdc00700 1 -- 172.21.15.123:0/3535118863 wait complete. 2024-05-08T06:06:35.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.512+0000 7f71fdc00700 1 Processor -- start 2024-05-08T06:06:35.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.512+0000 7f71fdc00700 1 -- start start 2024-05-08T06:06:35.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.513+0000 7f71fdc00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71f81034f0 0x7f71f819f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:35.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.513+0000 7f71fdc00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71f8103eb0 0x7f71f819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:35.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.513+0000 7f71fdc00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71f8109600 0x7f71f8199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:35.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.513+0000 7f71fdc00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f71f81136a0 con 0x7f71f8109600 2024-05-08T06:06:35.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.513+0000 7f71fdc00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f71f8113520 con 0x7f71f8103eb0 2024-05-08T06:06:35.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.513+0000 7f71fdc00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f71f8113820 con 0x7f71f81034f0 2024-05-08T06:06:35.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.513+0000 7f71f6a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71f81034f0 0x7f71f819f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:35.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.513+0000 7f71f6000700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71f8103eb0 0x7f71f819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:35.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.513+0000 7f71f6a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71f81034f0 0x7f71f819f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:52130/0 (socket says 172.21.15.123:52130) 2024-05-08T06:06:35.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.513+0000 7f71f6a00700 1 -- 172.21.15.123:0/302311362 learned_addr learned my addr 172.21.15.123:0/302311362 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:06:35.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.513+0000 7f71f7400700 1 --2- 172.21.15.123:0/302311362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71f8109600 0x7f71f8199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:35.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.514+0000 7f71f6a00700 1 -- 172.21.15.123:0/302311362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71f8103eb0 msgr2=0x7f71f819fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:35.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.514+0000 7f71f6a00700 1 --2- 172.21.15.123:0/302311362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71f8103eb0 0x7f71f819fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:35.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.514+0000 7f71f6a00700 1 -- 172.21.15.123:0/302311362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71f8109600 msgr2=0x7f71f8199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:35.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.514+0000 7f71f6a00700 1 --2- 172.21.15.123:0/302311362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71f8109600 0x7f71f8199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:35.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.514+0000 7f71f6a00700 1 -- 172.21.15.123:0/302311362 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f819a060 con 0x7f71f81034f0 2024-05-08T06:06:35.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.514+0000 7f71f7400700 1 --2- 172.21.15.123:0/302311362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71f8109600 0x7f71f8199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:06:35.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.514+0000 7f71f6a00700 1 --2- 172.21.15.123:0/302311362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71f81034f0 0x7f71f819f740 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f71e800e9a0 tx=0x7f71e800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:06:35.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.514+0000 7f71ef400700 1 -- 172.21.15.123:0/302311362 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e800cbd0 con 0x7f71f81034f0 2024-05-08T06:06:35.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.514+0000 7f71fdc00700 1 -- 172.21.15.123:0/302311362 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71f819a350 con 0x7f71f81034f0 2024-05-08T06:06:35.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.514+0000 7f71fdc00700 1 -- 172.21.15.123:0/302311362 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71f81a6610 con 0x7f71f81034f0 2024-05-08T06:06:35.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.514+0000 7f71ef400700 1 -- 172.21.15.123:0/302311362 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71e800cd70 con 0x7f71f81034f0 2024-05-08T06:06:35.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.514+0000 7f71ef400700 1 -- 172.21.15.123:0/302311362 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e8018680 con 0x7f71f81034f0 2024-05-08T06:06:35.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.516+0000 7f71fdc00700 1 -- 172.21.15.123:0/302311362 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71f806a190 con 0x7f71f81034f0 2024-05-08T06:06:35.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.518+0000 7f71ef400700 1 -- 172.21.15.123:0/302311362 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f71e8010430 con 0x7f71f81034f0 2024-05-08T06:06:35.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.519+0000 7f71ef400700 1 --2- 172.21.15.123:0/302311362 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f71dc0b2990 0x7f71dc0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:35.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.519+0000 7f71ef400700 1 -- 172.21.15.123:0/302311362 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f71e80e25e0 con 0x7f71f81034f0 2024-05-08T06:06:35.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.520+0000 7f71f6000700 1 --2- 172.21.15.123:0/302311362 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f71dc0b2990 0x7f71dc0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:35.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.520+0000 7f71f6000700 1 --2- 172.21.15.123:0/302311362 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f71dc0b2990 0x7f71dc0b4e50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f71f807ab70 tx=0x7f71e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:06:35.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.523+0000 7f71ef400700 1 -- 172.21.15.123:0/302311362 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f71e80aa150 con 0x7f71f81034f0 2024-05-08T06:06:35.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.797+0000 7f71fdc00700 1 -- 172.21.15.123:0/302311362 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f71f80638e0 con 0x7f71dc0b2990 2024-05-08T06:06:35.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.810+0000 7f71ef400700 1 -- 172.21.15.123:0/302311362 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f71f80638e0 con 0x7f71dc0b2990 2024-05-08T06:06:35.810 INFO:teuthology.orchestra.run.smithi123.stdout:/volumes/qa/sv_0/0f197f29-b450-4957-b81b-d073c4904a56 2024-05-08T06:06:35.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.813+0000 7f71fdc00700 1 -- 172.21.15.123:0/302311362 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f71dc0b2990 msgr2=0x7f71dc0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:35.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.813+0000 7f71fdc00700 1 --2- 172.21.15.123:0/302311362 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f71dc0b2990 0x7f71dc0b4e50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f71f807ab70 tx=0x7f71e4009260 comp rx=0 tx=0).stop 2024-05-08T06:06:35.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.813+0000 7f71fdc00700 1 -- 172.21.15.123:0/302311362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71f81034f0 msgr2=0x7f71f819f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:35.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.813+0000 7f71fdc00700 1 --2- 172.21.15.123:0/302311362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71f81034f0 0x7f71f819f740 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f71e800e9a0 tx=0x7f71e800ecb0 comp rx=0 tx=0).stop 2024-05-08T06:06:35.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.813+0000 7f71fdc00700 1 -- 172.21.15.123:0/302311362 shutdown_connections 2024-05-08T06:06:35.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.814+0000 7f71fdc00700 1 --2- 172.21.15.123:0/302311362 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f71dc0b2990 0x7f71dc0b4e50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:35.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.814+0000 7f71fdc00700 1 --2- 172.21.15.123:0/302311362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71f8109600 0x7f71f8199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:35.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.814+0000 7f71fdc00700 1 --2- 172.21.15.123:0/302311362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71f81034f0 0x7f71f819f740 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:35.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.814+0000 7f71fdc00700 1 --2- 172.21.15.123:0/302311362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71f8103eb0 0x7f71f819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:35.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.814+0000 7f71fdc00700 1 -- 172.21.15.123:0/302311362 >> 172.21.15.123:0/302311362 conn(0x7f71f807a650 msgr2=0x7f71f8101fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:06:35.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.814+0000 7f71fdc00700 1 -- 172.21.15.123:0/302311362 shutdown_connections 2024-05-08T06:06:35.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:35.814+0000 7f71fdc00700 1 -- 172.21.15.123:0/302311362 wait complete. 2024-05-08T06:06:36.612 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-08T06:06:36.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:36 smithi163 ceph-mon[34520]: pgmap v344: 97 pgs: 2 peering, 95 active+clean; 602 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.2 KiB/s wr, 7 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-05-08T06:06:36.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:36 smithi163 ceph-mon[34520]: from='client.24607 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:06:36.796 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:36 smithi123 ceph-mon[29787]: pgmap v344: 97 pgs: 2 peering, 95 active+clean; 602 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.2 KiB/s wr, 7 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-05-08T06:06:36.796 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:36 smithi123 ceph-mon[29787]: from='client.24607 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:06:36.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:36 smithi192 ceph-mon[34664]: pgmap v344: 97 pgs: 2 peering, 95 active+clean; 602 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.2 KiB/s wr, 7 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-05-08T06:06:36.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:36 smithi192 ceph-mon[34664]: from='client.24607 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:06:37.028 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:06:38.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:38 smithi163 ceph-mon[34520]: pgmap v345: 97 pgs: 97 active+clean; 602 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.4 KiB/s wr, 7 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-05-08T06:06:38.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:38 smithi192 ceph-mon[34664]: pgmap v345: 97 pgs: 97 active+clean; 602 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.4 KiB/s wr, 7 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-05-08T06:06:38.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:38 smithi123 ceph-mon[29787]: pgmap v345: 97 pgs: 97 active+clean; 602 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.4 KiB/s wr, 7 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-05-08T06:06:39.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.582+0000 7fb427200700 1 -- 172.21.15.123:0/2683855298 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4281055d0 msgr2=0x7fb428105a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:39.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.582+0000 7fb427200700 1 --2- 172.21.15.123:0/2683855298 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4281055d0 0x7fb428105a50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb410009990 tx=0x7fb410009ca0 comp rx=0 tx=0).stop 2024-05-08T06:06:39.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.582+0000 7fb427200700 1 -- 172.21.15.123:0/2683855298 shutdown_connections 2024-05-08T06:06:39.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.582+0000 7fb427200700 1 --2- 172.21.15.123:0/2683855298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb42810b660 0x7fb42810ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:39.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.582+0000 7fb427200700 1 --2- 172.21.15.123:0/2683855298 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb428105f90 0x7fb4281124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:39.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.582+0000 7fb427200700 1 --2- 172.21.15.123:0/2683855298 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4281055d0 0x7fb428105a50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:39.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.582+0000 7fb427200700 1 -- 172.21.15.123:0/2683855298 >> 172.21.15.123:0/2683855298 conn(0x7fb4281010c0 msgr2=0x7fb4281034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:06:39.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.582+0000 7fb427200700 1 -- 172.21.15.123:0/2683855298 shutdown_connections 2024-05-08T06:06:39.583 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.582+0000 7fb427200700 1 -- 172.21.15.123:0/2683855298 wait complete. 2024-05-08T06:06:39.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.583+0000 7fb427200700 1 Processor -- start 2024-05-08T06:06:39.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.583+0000 7fb427200700 1 -- start start 2024-05-08T06:06:39.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.584+0000 7fb427200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4281055d0 0x7fb4281a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:39.584 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.584+0000 7fb427200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb428105f90 0x7fb4281a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:39.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.584+0000 7fb427200700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb42810b660 0x7fb42819b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:39.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.584+0000 7fb427200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb4281157d0 con 0x7fb428105f90 2024-05-08T06:06:39.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.584+0000 7fb427200700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb428115650 con 0x7fb4281055d0 2024-05-08T06:06:39.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.584+0000 7fb427200700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb428115950 con 0x7fb42810b660 2024-05-08T06:06:39.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.584+0000 7fb41fe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb428105f90 0x7fb4281a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:39.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.584+0000 7fb41fe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb428105f90 0x7fb4281a1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39296/0 (socket says 172.21.15.123:39296) 2024-05-08T06:06:39.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.584+0000 7fb41fe00700 1 -- 172.21.15.123:0/3942902572 learned_addr learned my addr 172.21.15.123:0/3942902572 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:06:39.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.584+0000 7fb425e00700 1 --2- 172.21.15.123:0/3942902572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4281055d0 0x7fb4281a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:39.585 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.584+0000 7fb426800700 1 --2- 172.21.15.123:0/3942902572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb42810b660 0x7fb42819b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:39.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.585+0000 7fb41fe00700 1 -- 172.21.15.123:0/3942902572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb42810b660 msgr2=0x7fb42819b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:39.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.585+0000 7fb41fe00700 1 --2- 172.21.15.123:0/3942902572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb42810b660 0x7fb42819b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:39.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.585+0000 7fb41fe00700 1 -- 172.21.15.123:0/3942902572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4281055d0 msgr2=0x7fb4281a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:39.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.585+0000 7fb41fe00700 1 --2- 172.21.15.123:0/3942902572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4281055d0 0x7fb4281a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:39.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.585+0000 7fb41fe00700 1 -- 172.21.15.123:0/3942902572 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb42819c150 con 0x7fb428105f90 2024-05-08T06:06:39.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.585+0000 7fb41fe00700 1 --2- 172.21.15.123:0/3942902572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb428105f90 0x7fb4281a1d70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb410000c00 tx=0x7fb41000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:06:39.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.586+0000 7fb41ea00700 1 -- 172.21.15.123:0/3942902572 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb410027070 con 0x7fb428105f90 2024-05-08T06:06:39.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.586+0000 7fb41ea00700 1 -- 172.21.15.123:0/3942902572 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb410022470 con 0x7fb428105f90 2024-05-08T06:06:39.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.586+0000 7fb427200700 1 -- 172.21.15.123:0/3942902572 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb42819c3e0 con 0x7fb428105f90 2024-05-08T06:06:39.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.586+0000 7fb41ea00700 1 -- 172.21.15.123:0/3942902572 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb410017650 con 0x7fb428105f90 2024-05-08T06:06:39.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.586+0000 7fb427200700 1 -- 172.21.15.123:0/3942902572 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4281a8700 con 0x7fb428105f90 2024-05-08T06:06:39.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.587+0000 7fb427200700 1 -- 172.21.15.123:0/3942902572 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb428106d10 con 0x7fb428105f90 2024-05-08T06:06:39.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.591+0000 7fb41ea00700 1 -- 172.21.15.123:0/3942902572 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fb41002a070 con 0x7fb428105f90 2024-05-08T06:06:39.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.592+0000 7fb41ea00700 1 --2- 172.21.15.123:0/3942902572 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fb40c0b2910 0x7fb40c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:39.592 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.593+0000 7fb41ea00700 1 -- 172.21.15.123:0/3942902572 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fb4100e32a0 con 0x7fb428105f90 2024-05-08T06:06:39.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.593+0000 7fb425e00700 1 --2- 172.21.15.123:0/3942902572 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fb40c0b2910 0x7fb40c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:39.593 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.593+0000 7fb425e00700 1 --2- 172.21.15.123:0/3942902572 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fb40c0b2910 0x7fb40c0b4dd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb42819cb30 tx=0x7fb418009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:06:39.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.595+0000 7fb41ea00700 1 -- 172.21.15.123:0/3942902572 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb410013070 con 0x7fb428105f90 2024-05-08T06:06:39.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.889+0000 7fb427200700 1 -- 172.21.15.123:0/3942902572 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fb4280638e0 con 0x7fb40c0b2910 2024-05-08T06:06:39.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.922+0000 7fb41ea00700 1 -- 172.21.15.123:0/3942902572 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb4280638e0 con 0x7fb40c0b2910 2024-05-08T06:06:39.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.925+0000 7fb427200700 1 -- 172.21.15.123:0/3942902572 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fb40c0b2910 msgr2=0x7fb40c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:39.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.925+0000 7fb427200700 1 --2- 172.21.15.123:0/3942902572 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fb40c0b2910 0x7fb40c0b4dd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb42819cb30 tx=0x7fb418009260 comp rx=0 tx=0).stop 2024-05-08T06:06:39.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.925+0000 7fb427200700 1 -- 172.21.15.123:0/3942902572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb428105f90 msgr2=0x7fb4281a1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:39.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.925+0000 7fb427200700 1 --2- 172.21.15.123:0/3942902572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb428105f90 0x7fb4281a1d70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb410000c00 tx=0x7fb41000bfd0 comp rx=0 tx=0).stop 2024-05-08T06:06:39.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.926+0000 7fb427200700 1 -- 172.21.15.123:0/3942902572 shutdown_connections 2024-05-08T06:06:39.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.926+0000 7fb427200700 1 --2- 172.21.15.123:0/3942902572 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fb40c0b2910 0x7fb40c0b4dd0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:39.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.926+0000 7fb427200700 1 --2- 172.21.15.123:0/3942902572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb42810b660 0x7fb42819b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:39.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.926+0000 7fb427200700 1 --2- 172.21.15.123:0/3942902572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4281055d0 0x7fb4281a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:39.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.926+0000 7fb427200700 1 --2- 172.21.15.123:0/3942902572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb428105f90 0x7fb4281a1d70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:39.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.926+0000 7fb427200700 1 -- 172.21.15.123:0/3942902572 >> 172.21.15.123:0/3942902572 conn(0x7fb4281010c0 msgr2=0x7fb428102730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:06:39.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.926+0000 7fb427200700 1 -- 172.21.15.123:0/3942902572 shutdown_connections 2024-05-08T06:06:39.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:39.926+0000 7fb427200700 1 -- 172.21.15.123:0/3942902572 wait complete. 2024-05-08T06:06:40.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:40 smithi123 ceph-mon[29787]: pgmap v346: 97 pgs: 97 active+clean; 608 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.1 KiB/s wr, 7 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-05-08T06:06:40.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:40 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:06:40.759 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:40 smithi163 ceph-mon[34520]: pgmap v346: 97 pgs: 97 active+clean; 608 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.1 KiB/s wr, 7 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-05-08T06:06:40.759 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:40 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:06:40.766 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:40 smithi192 ceph-mon[34664]: pgmap v346: 97 pgs: 97 active+clean; 608 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.1 KiB/s wr, 7 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-05-08T06:06:40.766 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:40 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:06:40.993 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-08T06:06:41.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:41 smithi163 ceph-mon[34520]: from='client.14502 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:06:41.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:41 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:06:41.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:41 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:06:41.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:41 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:06:41.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:41 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:06:41.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:41 smithi192 ceph-mon[34664]: from='client.14502 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:06:41.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:41 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:06:41.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:41 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:06:41.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:41 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:06:41.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:41 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:06:41.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:41 smithi123 ceph-mon[29787]: from='client.14502 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:06:41.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:41 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:06:41.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:41 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:06:41.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:41 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:06:41.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:41 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:06:42.199 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:06:42.605 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:42 smithi123 ceph-mon[29787]: pgmap v347: 97 pgs: 97 active+clean; 608 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.6 KiB/s wr, 7 op/s; 5 B/s, 1 keys/s, 0 objects/s recovering 2024-05-08T06:06:42.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:42 smithi163 ceph-mon[34520]: pgmap v347: 97 pgs: 97 active+clean; 608 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.6 KiB/s wr, 7 op/s; 5 B/s, 1 keys/s, 0 objects/s recovering 2024-05-08T06:06:42.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:42 smithi192 ceph-mon[34664]: pgmap v347: 97 pgs: 97 active+clean; 608 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.6 KiB/s wr, 7 op/s; 5 B/s, 1 keys/s, 0 objects/s recovering 2024-05-08T06:06:44.622 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:44 smithi123 ceph-mon[29787]: pgmap v348: 97 pgs: 97 active+clean; 608 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.5 KiB/s wr, 7 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T06:06:44.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.624+0000 7fa209e00700 1 -- 172.21.15.123:0/1819790838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa204103d90 msgr2=0x7fa2041102a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:44.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.624+0000 7fa209e00700 1 --2- 172.21.15.123:0/1819790838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa204103d90 0x7fa2041102a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc00b0a0 tx=0x7fa1fc00b3b0 comp rx=0 tx=0).stop 2024-05-08T06:06:44.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.624+0000 7fa209e00700 1 -- 172.21.15.123:0/1819790838 shutdown_connections 2024-05-08T06:06:44.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.624+0000 7fa209e00700 1 --2- 172.21.15.123:0/1819790838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa204103d90 0x7fa2041102a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:44.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.624+0000 7fa209e00700 1 --2- 172.21.15.123:0/1819790838 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa204109460 0x7fa204109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:44.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.624+0000 7fa209e00700 1 --2- 172.21.15.123:0/1819790838 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2041033d0 0x7fa204103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:44.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.624+0000 7fa209e00700 1 -- 172.21.15.123:0/1819790838 >> 172.21.15.123:0/1819790838 conn(0x7fa2040fef20 msgr2=0x7fa204101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:06:44.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.624+0000 7fa209e00700 1 -- 172.21.15.123:0/1819790838 shutdown_connections 2024-05-08T06:06:44.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.624+0000 7fa209e00700 1 -- 172.21.15.123:0/1819790838 wait complete. 2024-05-08T06:06:44.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.625+0000 7fa209e00700 1 Processor -- start 2024-05-08T06:06:44.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.625+0000 7fa209e00700 1 -- start start 2024-05-08T06:06:44.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.626+0000 7fa209e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2041033d0 0x7fa2041a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:44.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.626+0000 7fa209e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa204103d90 0x7fa2041a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:44.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.626+0000 7fa209e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa204109460 0x7fa20419ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:44.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.626+0000 7fa209e00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa2041135f0 con 0x7fa204103d90 2024-05-08T06:06:44.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.626+0000 7fa209e00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa204113470 con 0x7fa204109460 2024-05-08T06:06:44.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.626+0000 7fa209e00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa204113770 con 0x7fa2041033d0 2024-05-08T06:06:44.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.626+0000 7fa1fbe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa204103d90 0x7fa2041a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:44.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.626+0000 7fa1fbe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa204103d90 0x7fa2041a1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39308/0 (socket says 172.21.15.123:39308) 2024-05-08T06:06:44.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.626+0000 7fa1fbe00700 1 -- 172.21.15.123:0/446125881 learned_addr learned my addr 172.21.15.123:0/446125881 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:06:44.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.626+0000 7fa208a00700 1 --2- 172.21.15.123:0/446125881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2041033d0 0x7fa2041a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:44.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.626+0000 7fa209400700 1 --2- 172.21.15.123:0/446125881 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa204109460 0x7fa20419ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:44.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.627+0000 7fa1fbe00700 1 -- 172.21.15.123:0/446125881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2041033d0 msgr2=0x7fa2041a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:44.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.627+0000 7fa1fbe00700 1 --2- 172.21.15.123:0/446125881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2041033d0 0x7fa2041a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:44.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.627+0000 7fa1fbe00700 1 -- 172.21.15.123:0/446125881 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa204109460 msgr2=0x7fa20419ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:44.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.627+0000 7fa1fbe00700 1 --2- 172.21.15.123:0/446125881 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa204109460 0x7fa20419ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:44.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.627+0000 7fa1fbe00700 1 -- 172.21.15.123:0/446125881 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20419c1d0 con 0x7fa204103d90 2024-05-08T06:06:44.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.627+0000 7fa1fbe00700 1 --2- 172.21.15.123:0/446125881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa204103d90 0x7fa2041a1df0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec00e9c0 tx=0x7fa1ec00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:06:44.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.627+0000 7fa202000700 1 -- 172.21.15.123:0/446125881 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec00cbd0 con 0x7fa204103d90 2024-05-08T06:06:44.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.627+0000 7fa202000700 1 -- 172.21.15.123:0/446125881 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1ec00ee60 con 0x7fa204103d90 2024-05-08T06:06:44.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.627+0000 7fa202000700 1 -- 172.21.15.123:0/446125881 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec0055c0 con 0x7fa204103d90 2024-05-08T06:06:44.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.628+0000 7fa209e00700 1 -- 172.21.15.123:0/446125881 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa20419c4c0 con 0x7fa204103d90 2024-05-08T06:06:44.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.628+0000 7fa209e00700 1 -- 172.21.15.123:0/446125881 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2041a8780 con 0x7fa204103d90 2024-05-08T06:06:44.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.629+0000 7fa209e00700 1 -- 172.21.15.123:0/446125881 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa20406a190 con 0x7fa204103d90 2024-05-08T06:06:44.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.631+0000 7fa202000700 1 -- 172.21.15.123:0/446125881 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fa1ec004750 con 0x7fa204103d90 2024-05-08T06:06:44.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.633+0000 7fa202000700 1 --2- 172.21.15.123:0/446125881 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa1e80b2910 0x7fa1e80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:44.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.633+0000 7fa202000700 1 -- 172.21.15.123:0/446125881 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fa1ec0e2960 con 0x7fa204103d90 2024-05-08T06:06:44.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.633+0000 7fa208a00700 1 --2- 172.21.15.123:0/446125881 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa1e80b2910 0x7fa1e80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:44.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.634+0000 7fa208a00700 1 --2- 172.21.15.123:0/446125881 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa1e80b2910 0x7fa1e80b4dd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa20419cbb0 tx=0x7fa1f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:06:44.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.638+0000 7fa202000700 1 -- 172.21.15.123:0/446125881 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa1ec0aa560 con 0x7fa204103d90 2024-05-08T06:06:44.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:44 smithi163 ceph-mon[34520]: pgmap v348: 97 pgs: 97 active+clean; 608 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.5 KiB/s wr, 7 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T06:06:44.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:44 smithi192 ceph-mon[34664]: pgmap v348: 97 pgs: 97 active+clean; 608 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.5 KiB/s wr, 7 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T06:06:44.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.915+0000 7fa209e00700 1 -- 172.21.15.123:0/446125881 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fa204100310 con 0x7fa1e80b2910 2024-05-08T06:06:44.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.937+0000 7fa202000700 1 -- 172.21.15.123:0/446125881 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fa204100310 con 0x7fa1e80b2910 2024-05-08T06:06:44.939 INFO:teuthology.orchestra.run.smithi123.stdout:/volumes/qa/sv_1/2af9b403-9f5f-4bf9-875d-231916530d9c 2024-05-08T06:06:44.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.941+0000 7fa209e00700 1 -- 172.21.15.123:0/446125881 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa1e80b2910 msgr2=0x7fa1e80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:44.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.941+0000 7fa209e00700 1 --2- 172.21.15.123:0/446125881 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa1e80b2910 0x7fa1e80b4dd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa20419cbb0 tx=0x7fa1f4009260 comp rx=0 tx=0).stop 2024-05-08T06:06:44.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.941+0000 7fa209e00700 1 -- 172.21.15.123:0/446125881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa204103d90 msgr2=0x7fa2041a1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:44.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.941+0000 7fa209e00700 1 --2- 172.21.15.123:0/446125881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa204103d90 0x7fa2041a1df0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec00e9c0 tx=0x7fa1ec00ecd0 comp rx=0 tx=0).stop 2024-05-08T06:06:44.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.941+0000 7fa209e00700 1 -- 172.21.15.123:0/446125881 shutdown_connections 2024-05-08T06:06:44.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.941+0000 7fa209e00700 1 --2- 172.21.15.123:0/446125881 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa1e80b2910 0x7fa1e80b4dd0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:44.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.941+0000 7fa209e00700 1 --2- 172.21.15.123:0/446125881 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa204109460 0x7fa20419ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:44.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.941+0000 7fa209e00700 1 --2- 172.21.15.123:0/446125881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2041033d0 0x7fa2041a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:44.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.941+0000 7fa209e00700 1 --2- 172.21.15.123:0/446125881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa204103d90 0x7fa2041a1df0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:44.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.941+0000 7fa209e00700 1 -- 172.21.15.123:0/446125881 >> 172.21.15.123:0/446125881 conn(0x7fa2040fef20 msgr2=0x7fa20410eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:06:44.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.942+0000 7fa209e00700 1 -- 172.21.15.123:0/446125881 shutdown_connections 2024-05-08T06:06:44.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:44.942+0000 7fa209e00700 1 -- 172.21.15.123:0/446125881 wait complete. 2024-05-08T06:06:45.918 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T06:06:46.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:45 smithi123 ceph-mon[29787]: from='client.14508 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:06:46.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:45 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:06:46.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:45 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:06:46.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:45 smithi123 ceph-mon[29787]: pgmap v349: 97 pgs: 97 active+clean; 625 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.7 KiB/s wr, 8 op/s; 4 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T06:06:46.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:45 smithi163 ceph-mon[34520]: from='client.14508 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:06:46.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:45 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:06:46.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:45 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:06:46.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:45 smithi163 ceph-mon[34520]: pgmap v349: 97 pgs: 97 active+clean; 625 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.7 KiB/s wr, 8 op/s; 4 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T06:06:46.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:45 smithi192 ceph-mon[34664]: from='client.14508 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:06:46.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:45 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:06:46.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:45 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:06:46.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:45 smithi192 ceph-mon[34664]: pgmap v349: 97 pgs: 97 active+clean; 625 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.7 KiB/s wr, 8 op/s; 4 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T06:06:46.351 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:06:47.058 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:46 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:06:47.058 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:46 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:06:47.058 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:46 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:06:47.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:46 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:06:47.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:46 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:06:47.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:46 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:06:47.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:46 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:06:47.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:46 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:06:47.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:46 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:06:48.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:47 smithi163 ceph-mon[34520]: pgmap v350: 97 pgs: 97 active+clean; 625 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.6 KiB/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-08T06:06:48.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:47 smithi192 ceph-mon[34664]: pgmap v350: 97 pgs: 97 active+clean; 625 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.6 KiB/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-08T06:06:48.310 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:47 smithi123 ceph-mon[29787]: pgmap v350: 97 pgs: 97 active+clean; 625 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.6 KiB/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-08T06:06:48.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.897+0000 7f39d5400700 1 -- 172.21.15.123:0/2821372240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39d006c3d0 msgr2=0x7f39d0110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:48.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.897+0000 7f39d5400700 1 --2- 172.21.15.123:0/2821372240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39d006c3d0 0x7f39d0110320 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f39c8009f90 tx=0x7f39c800b330 comp rx=0 tx=0).stop 2024-05-08T06:06:48.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.898+0000 7f39d5400700 1 -- 172.21.15.123:0/2821372240 shutdown_connections 2024-05-08T06:06:48.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.898+0000 7f39d5400700 1 --2- 172.21.15.123:0/2821372240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39d0109500 0x7f39d01098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:48.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.898+0000 7f39d5400700 1 --2- 172.21.15.123:0/2821372240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39d006c3d0 0x7f39d0110320 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:48.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.898+0000 7f39d5400700 1 --2- 172.21.15.123:0/2821372240 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39d006ba10 0x7f39d006be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:48.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.898+0000 7f39d5400700 1 -- 172.21.15.123:0/2821372240 >> 172.21.15.123:0/2821372240 conn(0x7f39d0079310 msgr2=0x7f39d0079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:06:48.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.898+0000 7f39d5400700 1 -- 172.21.15.123:0/2821372240 shutdown_connections 2024-05-08T06:06:48.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.898+0000 7f39d5400700 1 -- 172.21.15.123:0/2821372240 wait complete. 2024-05-08T06:06:48.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.899+0000 7f39d5400700 1 Processor -- start 2024-05-08T06:06:48.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.899+0000 7f39d5400700 1 -- start start 2024-05-08T06:06:48.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.899+0000 7f39d5400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39d006ba10 0x7f39d0102460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:48.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.899+0000 7f39d5400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39d006c3d0 0x7f39d01029a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:48.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.899+0000 7f39d5400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39d0109500 0x7f39d0102ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:48.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.900+0000 7f39d5400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f39d0113610 con 0x7f39d006ba10 2024-05-08T06:06:48.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.900+0000 7f39d5400700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f39d0113490 con 0x7f39d0109500 2024-05-08T06:06:48.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.900+0000 7f39d5400700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f39d0113790 con 0x7f39d006c3d0 2024-05-08T06:06:48.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.900+0000 7f39ce800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39d006ba10 0x7f39d0102460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:48.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.900+0000 7f39ce800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39d006ba10 0x7f39d0102460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43300/0 (socket says 172.21.15.123:43300) 2024-05-08T06:06:48.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.900+0000 7f39ce800700 1 -- 172.21.15.123:0/2369195839 learned_addr learned my addr 172.21.15.123:0/2369195839 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:06:48.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.900+0000 7f39cde00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39d006c3d0 0x7f39d01029a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:48.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.900+0000 7f39cf200700 1 --2- 172.21.15.123:0/2369195839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39d0109500 0x7f39d0102ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:48.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.900+0000 7f39ce800700 1 -- 172.21.15.123:0/2369195839 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39d006c3d0 msgr2=0x7f39d01029a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:48.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.900+0000 7f39ce800700 1 --2- 172.21.15.123:0/2369195839 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39d006c3d0 0x7f39d01029a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:48.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.900+0000 7f39ce800700 1 -- 172.21.15.123:0/2369195839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39d0109500 msgr2=0x7f39d0102ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:48.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.900+0000 7f39ce800700 1 --2- 172.21.15.123:0/2369195839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39d0109500 0x7f39d0102ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:48.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.900+0000 7f39ce800700 1 -- 172.21.15.123:0/2369195839 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d01a7ff0 con 0x7f39d006ba10 2024-05-08T06:06:48.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.900+0000 7f39cf200700 1 --2- 172.21.15.123:0/2369195839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39d0109500 0x7f39d0102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:06:48.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.900+0000 7f39cde00700 1 --2- 172.21.15.123:0/2369195839 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39d006c3d0 0x7f39d01029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:06:48.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.901+0000 7f39ce800700 1 --2- 172.21.15.123:0/2369195839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39d006ba10 0x7f39d0102460 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f39c000e9c0 tx=0x7f39c000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:06:48.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.901+0000 7f39c7400700 1 -- 172.21.15.123:0/2369195839 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c000cbd0 con 0x7f39d006ba10 2024-05-08T06:06:48.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.901+0000 7f39d5400700 1 -- 172.21.15.123:0/2369195839 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d01a82e0 con 0x7f39d006ba10 2024-05-08T06:06:48.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.901+0000 7f39c7400700 1 -- 172.21.15.123:0/2369195839 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39c000cd70 con 0x7f39d006ba10 2024-05-08T06:06:48.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.901+0000 7f39c7400700 1 -- 172.21.15.123:0/2369195839 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c0005550 con 0x7f39d006ba10 2024-05-08T06:06:48.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.901+0000 7f39d5400700 1 -- 172.21.15.123:0/2369195839 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d01a8870 con 0x7f39d006ba10 2024-05-08T06:06:48.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.902+0000 7f39d5400700 1 -- 172.21.15.123:0/2369195839 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39d006a190 con 0x7f39d006ba10 2024-05-08T06:06:48.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.905+0000 7f39c7400700 1 -- 172.21.15.123:0/2369195839 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f39c0004750 con 0x7f39d006ba10 2024-05-08T06:06:48.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.906+0000 7f39c7400700 1 --2- 172.21.15.123:0/2369195839 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f39b40b2910 0x7f39b40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:48.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.906+0000 7f39c7400700 1 -- 172.21.15.123:0/2369195839 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f39c00e27c0 con 0x7f39d006ba10 2024-05-08T06:06:48.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.907+0000 7f39cde00700 1 --2- 172.21.15.123:0/2369195839 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f39b40b2910 0x7f39b40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:48.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.907+0000 7f39cde00700 1 --2- 172.21.15.123:0/2369195839 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f39b40b2910 0x7f39b40b4dd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f39d00799f0 tx=0x7f39bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:06:48.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:48.909+0000 7f39c7400700 1 -- 172.21.15.123:0/2369195839 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f39c0054df0 con 0x7f39d006ba10 2024-05-08T06:06:49.187 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:49.187+0000 7f39d5400700 1 -- 172.21.15.123:0/2369195839 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39d002e510 con 0x7f39d006ba10 2024-05-08T06:06:49.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:49.189+0000 7f39c7400700 1 -- 172.21.15.123:0/2369195839 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11885 (secure 0 0 0) 0x7f39c00aa330 con 0x7f39d006ba10 2024-05-08T06:06:49.189 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:06:49.189 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3646476522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3646476522},{"type":"v1","addr":"172.21.15.123:6835","nonce":3646476522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2763706497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2763706497},{"type":"v1","addr":"172.21.15.123:6837","nonce":2763706497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/274635265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":274635265},{"type":"v1","addr":"172.21.15.123:6839","nonce":274635265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/175423115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":175423115},{"type":"v1","addr":"172.21.15.192:6833","nonce":175423115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/3263009472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3263009472},{"type":"v1","addr":"172.21.15.163:6833","nonce":3263009472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3308612826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3308612826},{"type":"v1","addr":"172.21.15.163:6835","nonce":3308612826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T06:05:54.755550+0000","modified":"2024-05-08T06:06:22.594178+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24512,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/202513680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":202513680},{"type":"v1","addr":"172.21.15.192:6835","nonce":202513680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/2483545969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2483545969},{"type":"v1","addr":"172.21.15.192:6837","nonce":2483545969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/660511838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":660511838},{"type":"v1","addr":"172.21.15.163:6837","nonce":660511838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.192:6839/3598682360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":3598682360},{"type":"v1","addr":"172.21.15.192:6839","nonce":3598682360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6841/1571726231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":1571726231},{"type":"v1","addr":"172.21.15.123:6841","nonce":1571726231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.163:6839/1568161515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":1568161515},{"type":"v1","addr":"172.21.15.163:6839","nonce":1568161515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24512,24524,24496]},"id":1}]} 2024-05-08T06:06:49.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:49.193+0000 7f39d5400700 1 -- 172.21.15.123:0/2369195839 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f39b40b2910 msgr2=0x7f39b40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:49.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:49.193+0000 7f39d5400700 1 --2- 172.21.15.123:0/2369195839 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f39b40b2910 0x7f39b40b4dd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f39d00799f0 tx=0x7f39bc009260 comp rx=0 tx=0).stop 2024-05-08T06:06:49.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:49.193+0000 7f39d5400700 1 -- 172.21.15.123:0/2369195839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39d006ba10 msgr2=0x7f39d0102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:49.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:49.193+0000 7f39d5400700 1 --2- 172.21.15.123:0/2369195839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39d006ba10 0x7f39d0102460 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f39c000e9c0 tx=0x7f39c000ecd0 comp rx=0 tx=0).stop 2024-05-08T06:06:49.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:49.193+0000 7f39d5400700 1 -- 172.21.15.123:0/2369195839 shutdown_connections 2024-05-08T06:06:49.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:49.193+0000 7f39d5400700 1 --2- 172.21.15.123:0/2369195839 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f39b40b2910 0x7f39b40b4dd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:49.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:49.193+0000 7f39d5400700 1 --2- 172.21.15.123:0/2369195839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39d006ba10 0x7f39d0102460 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:49.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:49.193+0000 7f39d5400700 1 --2- 172.21.15.123:0/2369195839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39d0109500 0x7f39d0102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:49.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:49.193+0000 7f39d5400700 1 --2- 172.21.15.123:0/2369195839 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39d006c3d0 0x7f39d01029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:49.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:49.193+0000 7f39d5400700 1 -- 172.21.15.123:0/2369195839 >> 172.21.15.123:0/2369195839 conn(0x7f39d0079310 msgr2=0x7f39d0101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:06:49.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:49.193+0000 7f39d5400700 1 -- 172.21.15.123:0/2369195839 shutdown_connections 2024-05-08T06:06:49.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:49.193+0000 7f39d5400700 1 -- 172.21.15.123:0/2369195839 wait complete. 2024-05-08T06:06:49.195 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 23 2024-05-08T06:06:50.124 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T06:06:50.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:50 smithi123 ceph-mon[29787]: pgmap v351: 97 pgs: 97 active+clean; 634 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-05-08T06:06:50.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:50 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/2369195839' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:06:50.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:50 smithi163 ceph-mon[34520]: pgmap v351: 97 pgs: 97 active+clean; 634 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-05-08T06:06:50.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:50 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/2369195839' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:06:50.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:50 smithi192 ceph-mon[34664]: pgmap v351: 97 pgs: 97 active+clean; 634 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-05-08T06:06:50.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:50 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/2369195839' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:06:50.555 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:06:52.256 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:52 smithi123 ceph-mon[29787]: pgmap v352: 97 pgs: 97 active+clean; 634 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-05-08T06:06:52.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:52 smithi163 ceph-mon[34520]: pgmap v352: 97 pgs: 97 active+clean; 634 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-05-08T06:06:52.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:52 smithi192 ceph-mon[34664]: pgmap v352: 97 pgs: 97 active+clean; 634 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-05-08T06:06:52.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.854+0000 7feef3000700 1 -- 172.21.15.123:0/4106601671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feeec10fea0 msgr2=0x7feeec112290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:52.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.854+0000 7feef3000700 1 --2- 172.21.15.123:0/4106601671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feeec10fea0 0x7feeec112290 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7feee400b0a0 tx=0x7feee400b3b0 comp rx=0 tx=0).stop 2024-05-08T06:06:52.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.854+0000 7feef3000700 1 -- 172.21.15.123:0/4106601671 shutdown_connections 2024-05-08T06:06:52.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.854+0000 7feef3000700 1 --2- 172.21.15.123:0/4106601671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feeec10fea0 0x7feeec112290 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:52.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.854+0000 7feef3000700 1 --2- 172.21.15.123:0/4106601671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feeec0756d0 0x7feeec075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:52.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.854+0000 7feef3000700 1 --2- 172.21.15.123:0/4106601671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feeec075ff0 0x7feeec10f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:52.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.854+0000 7feef3000700 1 -- 172.21.15.123:0/4106601671 >> 172.21.15.123:0/4106601671 conn(0x7feeec07a650 msgr2=0x7feeec07aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:06:52.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.855+0000 7feef3000700 1 -- 172.21.15.123:0/4106601671 shutdown_connections 2024-05-08T06:06:52.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.855+0000 7feef3000700 1 -- 172.21.15.123:0/4106601671 wait complete. 2024-05-08T06:06:52.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.856+0000 7feef3000700 1 Processor -- start 2024-05-08T06:06:52.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.856+0000 7feef3000700 1 -- start start 2024-05-08T06:06:52.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.856+0000 7feef3000700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feeec0756d0 0x7feeec19bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:52.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.856+0000 7feef3000700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feeec075ff0 0x7feeec19c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:52.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.856+0000 7feef3000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feeec10fea0 0x7feeec1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:52.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.856+0000 7feef3000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feeec115930 con 0x7feeec10fea0 2024-05-08T06:06:52.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.856+0000 7feef3000700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7feeec1157b0 con 0x7feeec0756d0 2024-05-08T06:06:52.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.856+0000 7feef3000700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7feeec115ab0 con 0x7feeec075ff0 2024-05-08T06:06:52.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.856+0000 7feeebe00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feeec0756d0 0x7feeec19bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:52.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.856+0000 7feef1000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feeec10fea0 0x7feeec1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:52.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.856+0000 7feeeb400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feeec075ff0 0x7feeec19c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:52.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.856+0000 7feeebe00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feeec0756d0 0x7feeec19bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:45274/0 (socket says 172.21.15.123:45274) 2024-05-08T06:06:52.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.857+0000 7feeebe00700 1 -- 172.21.15.123:0/3694091816 learned_addr learned my addr 172.21.15.123:0/3694091816 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:06:52.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.857+0000 7feeebe00700 1 -- 172.21.15.123:0/3694091816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feeec075ff0 msgr2=0x7feeec19c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:52.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.857+0000 7feeebe00700 1 --2- 172.21.15.123:0/3694091816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feeec075ff0 0x7feeec19c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:52.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.857+0000 7feeebe00700 1 -- 172.21.15.123:0/3694091816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feeec10fea0 msgr2=0x7feeec1a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:52.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.857+0000 7feeebe00700 1 --2- 172.21.15.123:0/3694091816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feeec10fea0 0x7feeec1a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:52.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.857+0000 7feeebe00700 1 -- 172.21.15.123:0/3694091816 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeec1a0d80 con 0x7feeec0756d0 2024-05-08T06:06:52.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.857+0000 7feef1000700 1 --2- 172.21.15.123:0/3694091816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feeec10fea0 0x7feeec1a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:06:52.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.857+0000 7feeeb400700 1 --2- 172.21.15.123:0/3694091816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feeec075ff0 0x7feeec19c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:06:52.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.857+0000 7feeebe00700 1 --2- 172.21.15.123:0/3694091816 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feeec0756d0 0x7feeec19bdc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7feedc00e9a0 tx=0x7feedc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:06:52.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.857+0000 7feee8c00700 1 -- 172.21.15.123:0/3694091816 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feedc00ca10 con 0x7feeec0756d0 2024-05-08T06:06:52.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.858+0000 7feef3000700 1 -- 172.21.15.123:0/3694091816 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeec1a1070 con 0x7feeec0756d0 2024-05-08T06:06:52.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.858+0000 7feee8c00700 1 -- 172.21.15.123:0/3694091816 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feedc00cbb0 con 0x7feeec0756d0 2024-05-08T06:06:52.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.858+0000 7feee8c00700 1 -- 172.21.15.123:0/3694091816 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feedc018680 con 0x7feeec0756d0 2024-05-08T06:06:52.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.858+0000 7feef3000700 1 -- 172.21.15.123:0/3694091816 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeec1a88f0 con 0x7feeec0756d0 2024-05-08T06:06:52.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.859+0000 7feef3000700 1 -- 172.21.15.123:0/3694091816 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feeec06a190 con 0x7feeec0756d0 2024-05-08T06:06:52.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.861+0000 7feee8c00700 1 -- 172.21.15.123:0/3694091816 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7feedc04aae0 con 0x7feeec0756d0 2024-05-08T06:06:52.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.862+0000 7feee8c00700 1 --2- 172.21.15.123:0/3694091816 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7feed00b2910 0x7feed00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:52.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.863+0000 7feee8c00700 1 -- 172.21.15.123:0/3694091816 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7feedc0e2650 con 0x7feeec0756d0 2024-05-08T06:06:52.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.863+0000 7feeeb400700 1 --2- 172.21.15.123:0/3694091816 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7feed00b2910 0x7feed00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:52.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.863+0000 7feeeb400700 1 --2- 172.21.15.123:0/3694091816 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7feed00b2910 0x7feed00b4dd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7feeec111bc0 tx=0x7feed8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:06:52.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:52.865+0000 7feee8c00700 1 -- 172.21.15.123:0/3694091816 <== mon.2 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7feedc0aa1c0 con 0x7feeec0756d0 2024-05-08T06:06:53.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:53.097+0000 7feef3000700 1 -- 172.21.15.123:0/3694091816 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feeec02e510 con 0x7feeec0756d0 2024-05-08T06:06:53.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:53.099+0000 7feee8c00700 1 -- 172.21.15.123:0/3694091816 <== mon.2 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22719 (secure 0 0 0) 0x7feedc0b2240 con 0x7feeec0756d0 2024-05-08T06:06:53.100 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:06:53.100 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":82,"fsid":"85cfb2e6-0cff-11ef-bc97-c7b262605968","created":"2024-05-08T05:56:12.544750+0000","modified":"2024-05-08T06:06:32.471993+0000","last_up_change":"2024-05-08T06:04:33.654727+0000","last_in_change":"2024-05-08T06:04:13.809050+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T06:00:17.050518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T06:05:44.815334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T06:05:49.604457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a14fde9b-bfb1-40a7-a105-305e8981a778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6803","nonce":3608618058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6805","nonce":3608618058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6809","nonce":3608618058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6807","nonce":3608618058}]},"public_addr":"172.21.15.123:6803/3608618058","cluster_addr":"172.21.15.123:6805/3608618058","heartbeat_back_addr":"172.21.15.123:6809/3608618058","heartbeat_front_addr":"172.21.15.123:6807/3608618058","state":["exists","up"]},{"osd":1,"uuid":"e023ae3d-a82e-4bb6-8fef-889dd07fe183","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6801","nonce":1884753449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6803","nonce":1884753449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6807","nonce":1884753449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6805","nonce":1884753449}]},"public_addr":"172.21.15.163:6801/1884753449","cluster_addr":"172.21.15.163:6803/1884753449","heartbeat_back_addr":"172.21.15.163:6807/1884753449","heartbeat_front_addr":"172.21.15.163:6805/1884753449","state":["exists","up"]},{"osd":2,"uuid":"69f4cfca-63cb-4862-9480-d79891b0f338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6801","nonce":4161570404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6803","nonce":4161570404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6807","nonce":4161570404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6805","nonce":4161570404}]},"public_addr":"172.21.15.192:6801/4161570404","cluster_addr":"172.21.15.192:6803/4161570404","heartbeat_back_addr":"172.21.15.192:6807/4161570404","heartbeat_front_addr":"172.21.15.192:6805/4161570404","state":["exists","up"]},{"osd":3,"uuid":"71bb5ecd-c88c-45fb-8b51-0eefb53941c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6811","nonce":1243003258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6813","nonce":1243003258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6817","nonce":1243003258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6815","nonce":1243003258}]},"public_addr":"172.21.15.123:6811/1243003258","cluster_addr":"172.21.15.123:6813/1243003258","heartbeat_back_addr":"172.21.15.123:6817/1243003258","heartbeat_front_addr":"172.21.15.123:6815/1243003258","state":["exists","up"]},{"osd":4,"uuid":"5a7a3462-e74f-4d15-a85c-709b5d423745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6809","nonce":591680273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6811","nonce":591680273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6815","nonce":591680273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6813","nonce":591680273}]},"public_addr":"172.21.15.163:6809/591680273","cluster_addr":"172.21.15.163:6811/591680273","heartbeat_back_addr":"172.21.15.163:6815/591680273","heartbeat_front_addr":"172.21.15.163:6813/591680273","state":["exists","up"]},{"osd":5,"uuid":"85e3f1ab-f8c6-4025-bab9-9ddb6bd3e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6809","nonce":1734821180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6811","nonce":1734821180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6815","nonce":1734821180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6813","nonce":1734821180}]},"public_addr":"172.21.15.192:6809/1734821180","cluster_addr":"172.21.15.192:6811/1734821180","heartbeat_back_addr":"172.21.15.192:6815/1734821180","heartbeat_front_addr":"172.21.15.192:6813/1734821180","state":["exists","up"]},{"osd":6,"uuid":"f21a5624-6f8f-48be-bc6d-0a883d6b7b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6819","nonce":1002138878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6821","nonce":1002138878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6825","nonce":1002138878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6823","nonce":1002138878}]},"public_addr":"172.21.15.123:6819/1002138878","cluster_addr":"172.21.15.123:6821/1002138878","heartbeat_back_addr":"172.21.15.123:6825/1002138878","heartbeat_front_addr":"172.21.15.123:6823/1002138878","state":["exists","up"]},{"osd":7,"uuid":"6eb16729-d260-4041-9829-3e868795b111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6817","nonce":354411978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6819","nonce":354411978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6823","nonce":354411978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6821","nonce":354411978}]},"public_addr":"172.21.15.163:6817/354411978","cluster_addr":"172.21.15.163:6819/354411978","heartbeat_back_addr":"172.21.15.163:6823/354411978","heartbeat_front_addr":"172.21.15.163:6821/354411978","state":["exists","up"]},{"osd":8,"uuid":"122a281e-18b8-4728-97db-458e7432b2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6817","nonce":3669753689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6819","nonce":3669753689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6823","nonce":3669753689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6821","nonce":3669753689}]},"public_addr":"172.21.15.192:6817/3669753689","cluster_addr":"172.21.15.192:6819/3669753689","heartbeat_back_addr":"172.21.15.192:6823/3669753689","heartbeat_front_addr":"172.21.15.192:6821/3669753689","state":["exists","up"]},{"osd":9,"uuid":"9fa022a9-b873-4e07-806c-641a706a150f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6827","nonce":3104436473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6829","nonce":3104436473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6833","nonce":3104436473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6831","nonce":3104436473}]},"public_addr":"172.21.15.123:6827/3104436473","cluster_addr":"172.21.15.123:6829/3104436473","heartbeat_back_addr":"172.21.15.123:6833/3104436473","heartbeat_front_addr":"172.21.15.123:6831/3104436473","state":["exists","up"]},{"osd":10,"uuid":"fb59339b-e11b-4ffc-b202-7f67cb21da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6825","nonce":2628487883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6827","nonce":2628487883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6831","nonce":2628487883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6829","nonce":2628487883}]},"public_addr":"172.21.15.163:6825/2628487883","cluster_addr":"172.21.15.163:6827/2628487883","heartbeat_back_addr":"172.21.15.163:6831/2628487883","heartbeat_front_addr":"172.21.15.163:6829/2628487883","state":["exists","up"]},{"osd":11,"uuid":"be7a5f36-c1be-41d7-80ee-004d0546eb9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6825","nonce":1339159347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6827","nonce":1339159347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6831","nonce":1339159347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6829","nonce":1339159347}]},"public_addr":"172.21.15.192:6825/1339159347","cluster_addr":"172.21.15.192:6827/1339159347","heartbeat_back_addr":"172.21.15.192:6831/1339159347","heartbeat_front_addr":"172.21.15.192:6829/1339159347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:59:24.923234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:59:49.867071+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:00:13.454062+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:00:46.983594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:01:12.820452+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:01:37.451382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:02:13.325181+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:02:39.453436+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:03:03.837509+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:03:39.460579+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:04:06.314082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:04:31.723154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6801/3057688649":"2024-05-09T05:57:30.887936+0000","172.21.15.123:6800/3057688649":"2024-05-09T05:57:30.887936+0000","172.21.15.123:6800/2769724312":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/399884043":"2024-05-09T05:57:30.887936+0000","172.21.15.123:0/3406061897":"2024-05-09T05:57:30.887936+0000","172.21.15.123:0/644457268":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/3055092197":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/343390357":"2024-05-09T05:57:30.887936+0000","172.21.15.123:6801/2769724312":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/2263171562":"2024-05-09T05:56:55.849036+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T06:06:53.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:53.102+0000 7feef3000700 1 -- 172.21.15.123:0/3694091816 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7feed00b2910 msgr2=0x7feed00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:53.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:53.102+0000 7feef3000700 1 --2- 172.21.15.123:0/3694091816 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7feed00b2910 0x7feed00b4dd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7feeec111bc0 tx=0x7feed8009260 comp rx=0 tx=0).stop 2024-05-08T06:06:53.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:53.102+0000 7feef3000700 1 -- 172.21.15.123:0/3694091816 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feeec0756d0 msgr2=0x7feeec19bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:53.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:53.102+0000 7feef3000700 1 --2- 172.21.15.123:0/3694091816 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feeec0756d0 0x7feeec19bdc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7feedc00e9a0 tx=0x7feedc00ecb0 comp rx=0 tx=0).stop 2024-05-08T06:06:53.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:53.102+0000 7feef3000700 1 -- 172.21.15.123:0/3694091816 shutdown_connections 2024-05-08T06:06:53.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:53.102+0000 7feef3000700 1 --2- 172.21.15.123:0/3694091816 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7feed00b2910 0x7feed00b4dd0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:53.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:53.102+0000 7feef3000700 1 --2- 172.21.15.123:0/3694091816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feeec10fea0 0x7feeec1a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:53.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:53.102+0000 7feef3000700 1 --2- 172.21.15.123:0/3694091816 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feeec0756d0 0x7feeec19bdc0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:53.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:53.102+0000 7feef3000700 1 --2- 172.21.15.123:0/3694091816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feeec075ff0 0x7feeec19c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:53.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:53.102+0000 7feef3000700 1 -- 172.21.15.123:0/3694091816 >> 172.21.15.123:0/3694091816 conn(0x7feeec07a650 msgr2=0x7feeec1006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:06:53.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:53.102+0000 7feef3000700 1 -- 172.21.15.123:0/3694091816 shutdown_connections 2024-05-08T06:06:53.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:53.102+0000 7feef3000700 1 -- 172.21.15.123:0/3694091816 wait complete. 2024-05-08T06:06:53.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:53 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/3694091816' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:06:53.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:53 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/3694091816' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:06:53.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:53 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/3694091816' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:06:54.207 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-08T06:06:54.207 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T06:06:54.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:54 smithi123 ceph-mon[29787]: pgmap v353: 97 pgs: 97 active+clean; 634 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-05-08T06:06:54.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:54 smithi163 ceph-mon[34520]: pgmap v353: 97 pgs: 97 active+clean; 634 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-05-08T06:06:54.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:54 smithi192 ceph-mon[34664]: pgmap v353: 97 pgs: 97 active+clean; 634 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-05-08T06:06:54.659 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:06:56.408 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:56 smithi123 ceph-mon[29787]: pgmap v354: 97 pgs: 97 active+clean; 638 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-05-08T06:06:56.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:56 smithi163 ceph-mon[34520]: pgmap v354: 97 pgs: 97 active+clean; 638 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-05-08T06:06:56.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:56 smithi192 ceph-mon[34664]: pgmap v354: 97 pgs: 97 active+clean; 638 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-05-08T06:06:57.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.048+0000 7f2380200700 1 -- 172.21.15.123:0/1720489575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2378108410 msgr2=0x7f23781087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:57.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.048+0000 7f2380200700 1 --2- 172.21.15.123:0/1720489575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2378108410 0x7f23781087f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f236c009990 tx=0x7f236c009ca0 comp rx=0 tx=0).stop 2024-05-08T06:06:57.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.048+0000 7f2380200700 1 -- 172.21.15.123:0/1720489575 shutdown_connections 2024-05-08T06:06:57.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.048+0000 7f2380200700 1 --2- 172.21.15.123:0/1720489575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2378108410 0x7f23781087f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:57.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.048+0000 7f2380200700 1 --2- 172.21.15.123:0/1720489575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2378102d60 0x7f237810f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:57.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.048+0000 7f2380200700 1 --2- 172.21.15.123:0/1720489575 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23781023a0 0x7f2378102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:57.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.048+0000 7f2380200700 1 -- 172.21.15.123:0/1720489575 >> 172.21.15.123:0/1720489575 conn(0x7f2378079350 msgr2=0x7f2378079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:06:57.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.048+0000 7f2380200700 1 -- 172.21.15.123:0/1720489575 shutdown_connections 2024-05-08T06:06:57.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.048+0000 7f2380200700 1 -- 172.21.15.123:0/1720489575 wait complete. 2024-05-08T06:06:57.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.050+0000 7f2380200700 1 Processor -- start 2024-05-08T06:06:57.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.050+0000 7f2380200700 1 -- start start 2024-05-08T06:06:57.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.050+0000 7f2380200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23781023a0 0x7f237819bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:57.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.050+0000 7f2380200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2378102d60 0x7f237819c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:57.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.050+0000 7f2380200700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2378108410 0x7f23781a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:57.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.050+0000 7f2380200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2378112580 con 0x7f23781023a0 2024-05-08T06:06:57.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.050+0000 7f2380200700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2378112400 con 0x7f2378102d60 2024-05-08T06:06:57.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.050+0000 7f2380200700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2378112700 con 0x7f2378108410 2024-05-08T06:06:57.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.051+0000 7f237d800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23781023a0 0x7f237819bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:57.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.051+0000 7f237d800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23781023a0 0x7f237819bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56402/0 (socket says 172.21.15.123:56402) 2024-05-08T06:06:57.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.051+0000 7f237d800700 1 -- 172.21.15.123:0/4022331736 learned_addr learned my addr 172.21.15.123:0/4022331736 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:06:57.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.051+0000 7f237ce00700 1 --2- 172.21.15.123:0/4022331736 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2378102d60 0x7f237819c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:57.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.051+0000 7f237e200700 1 --2- 172.21.15.123:0/4022331736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2378108410 0x7f23781a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:57.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.051+0000 7f237d800700 1 -- 172.21.15.123:0/4022331736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2378108410 msgr2=0x7f23781a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:57.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.051+0000 7f237d800700 1 --2- 172.21.15.123:0/4022331736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2378108410 0x7f23781a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:57.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.051+0000 7f237d800700 1 -- 172.21.15.123:0/4022331736 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2378102d60 msgr2=0x7f237819c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:57.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.051+0000 7f237d800700 1 --2- 172.21.15.123:0/4022331736 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2378102d60 0x7f237819c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:57.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.051+0000 7f237d800700 1 -- 172.21.15.123:0/4022331736 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23781a0d40 con 0x7f23781023a0 2024-05-08T06:06:57.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.051+0000 7f237e200700 1 --2- 172.21.15.123:0/4022331736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2378108410 0x7f23781a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:06:57.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.052+0000 7f237d800700 1 --2- 172.21.15.123:0/4022331736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23781023a0 0x7f237819bd30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f236c000c00 tx=0x7f236c0057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:06:57.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.052+0000 7f2376000700 1 -- 172.21.15.123:0/4022331736 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236c027070 con 0x7f23781023a0 2024-05-08T06:06:57.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.052+0000 7f2376000700 1 -- 172.21.15.123:0/4022331736 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f236c003f20 con 0x7f23781023a0 2024-05-08T06:06:57.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.052+0000 7f2380200700 1 -- 172.21.15.123:0/4022331736 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23781a0fd0 con 0x7f23781023a0 2024-05-08T06:06:57.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.052+0000 7f2376000700 1 -- 172.21.15.123:0/4022331736 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236c022bd0 con 0x7f23781023a0 2024-05-08T06:06:57.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.052+0000 7f2380200700 1 -- 172.21.15.123:0/4022331736 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23781a88b0 con 0x7f23781023a0 2024-05-08T06:06:57.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.053+0000 7f2380200700 1 -- 172.21.15.123:0/4022331736 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f237806a190 con 0x7f23781023a0 2024-05-08T06:06:57.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.055+0000 7f2376000700 1 -- 172.21.15.123:0/4022331736 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f236c02a070 con 0x7f23781023a0 2024-05-08T06:06:57.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.057+0000 7f2376000700 1 --2- 172.21.15.123:0/4022331736 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f23600b2990 0x7f23600b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:06:57.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.057+0000 7f2376000700 1 -- 172.21.15.123:0/4022331736 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f236c0e3ab0 con 0x7f23781023a0 2024-05-08T06:06:57.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.057+0000 7f237ce00700 1 --2- 172.21.15.123:0/4022331736 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f23600b2990 0x7f23600b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:06:57.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.058+0000 7f237ce00700 1 --2- 172.21.15.123:0/4022331736 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f23600b2990 0x7f23600b4e50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2378102330 tx=0x7f23640073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:06:57.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.060+0000 7f2376000700 1 -- 172.21.15.123:0/4022331736 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f236c031020 con 0x7f23781023a0 2024-05-08T06:06:57.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.284+0000 7f2380200700 1 -- 172.21.15.123:0/4022331736 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f237806b890 con 0x7f23600b2990 2024-05-08T06:06:57.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.310+0000 7f2376000700 1 -- 172.21.15.123:0/4022331736 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258934 (secure 0 0 0) 0x7f237806b890 con 0x7f23600b2990 2024-05-08T06:06:57.315 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:06:57.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.316+0000 7f2380200700 1 -- 172.21.15.123:0/4022331736 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f23600b2990 msgr2=0x7f23600b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:57.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.316+0000 7f2380200700 1 --2- 172.21.15.123:0/4022331736 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f23600b2990 0x7f23600b4e50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2378102330 tx=0x7f23640073d0 comp rx=0 tx=0).stop 2024-05-08T06:06:57.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.316+0000 7f2380200700 1 -- 172.21.15.123:0/4022331736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23781023a0 msgr2=0x7f237819bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:06:57.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.316+0000 7f2380200700 1 --2- 172.21.15.123:0/4022331736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23781023a0 0x7f237819bd30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f236c000c00 tx=0x7f236c0057a0 comp rx=0 tx=0).stop 2024-05-08T06:06:57.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.316+0000 7f2380200700 1 -- 172.21.15.123:0/4022331736 shutdown_connections 2024-05-08T06:06:57.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.316+0000 7f2380200700 1 --2- 172.21.15.123:0/4022331736 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f23600b2990 0x7f23600b4e50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:57.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.316+0000 7f2380200700 1 --2- 172.21.15.123:0/4022331736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23781023a0 0x7f237819bd30 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:57.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.316+0000 7f2380200700 1 --2- 172.21.15.123:0/4022331736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2378108410 0x7f23781a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:57.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.316+0000 7f2380200700 1 --2- 172.21.15.123:0/4022331736 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2378102d60 0x7f237819c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:06:57.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.316+0000 7f2380200700 1 -- 172.21.15.123:0/4022331736 >> 172.21.15.123:0/4022331736 conn(0x7f2378079350 msgr2=0x7f237806b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:06:57.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.316+0000 7f2380200700 1 -- 172.21.15.123:0/4022331736 shutdown_connections 2024-05-08T06:06:57.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:06:57.316+0000 7f2380200700 1 -- 172.21.15.123:0/4022331736 wait complete. 2024-05-08T06:06:57.319 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-05-08T06:06:58.409 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:58 smithi123 ceph-mon[29787]: pgmap v355: 97 pgs: 97 active+clean; 638 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-05-08T06:06:58.410 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:06:58 smithi123 ceph-mon[29787]: from='client.14526 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T06:06:58.413 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-05-08T06:06:57.067080+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653418,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":320,"num_read_kb":251,"num_write":290,"num_write_kb":2261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":2951896,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":234,"ondisk_log_size":234,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368344,"kb_used_data":10840,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573480,"statfs":{"total":1151940427776,"available":1151563243520,"internally_reserved":0,"allocated":11100160,"data_stored":7139163,"data_compressed":113232,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":13304,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":72,"num_read_kb":49,"num_write":13,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005412"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.477766+0000","last_change":"2024-05-08T06:05:51.431744+0000","last_active":"2024-05-08T06:06:32.477766+0000","last_peered":"2024-05-08T06:06:32.477766+0000","last_clean":"2024-05-08T06:06:32.477766+0000","last_became_active":"2024-05-08T06:05:51.431405+0000","last_became_peered":"2024-05-08T06:05:51.431405+0000","last_unstale":"2024-05-08T06:06:32.477766+0000","last_undegraded":"2024-05-08T06:06:32.477766+0000","last_fullsized":"2024-05-08T06:06:32.477766+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:56:40.503607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.488683+0000","last_change":"2024-05-08T06:05:51.426473+0000","last_active":"2024-05-08T06:06:32.488683+0000","last_peered":"2024-05-08T06:06:32.488683+0000","last_clean":"2024-05-08T06:06:32.488683+0000","last_became_active":"2024-05-08T06:05:51.424477+0000","last_became_peered":"2024-05-08T06:05:51.424477+0000","last_unstale":"2024-05-08T06:06:32.488683+0000","last_undegraded":"2024-05-08T06:06:32.488683+0000","last_fullsized":"2024-05-08T06:06:32.488683+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:26:55.747808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.489112+0000","last_change":"2024-05-08T06:05:51.423767+0000","last_active":"2024-05-08T06:06:32.489112+0000","last_peered":"2024-05-08T06:06:32.489112+0000","last_clean":"2024-05-08T06:06:32.489112+0000","last_became_active":"2024-05-08T06:05:51.423273+0000","last_became_peered":"2024-05-08T06:05:51.423273+0000","last_unstale":"2024-05-08T06:06:32.489112+0000","last_undegraded":"2024-05-08T06:06:32.489112+0000","last_fullsized":"2024-05-08T06:06:32.489112+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:24:44.099743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.597288+0000","last_change":"2024-05-08T06:05:51.416604+0000","last_active":"2024-05-08T06:06:32.597288+0000","last_peered":"2024-05-08T06:06:32.597288+0000","last_clean":"2024-05-08T06:06:32.597288+0000","last_became_active":"2024-05-08T06:05:51.416284+0000","last_became_peered":"2024-05-08T06:05:51.416284+0000","last_unstale":"2024-05-08T06:06:32.597288+0000","last_undegraded":"2024-05-08T06:06:32.597288+0000","last_fullsized":"2024-05-08T06:06:32.597288+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:05:26.014323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.479834+0000","last_change":"2024-05-08T06:05:51.423915+0000","last_active":"2024-05-08T06:06:32.479834+0000","last_peered":"2024-05-08T06:06:32.479834+0000","last_clean":"2024-05-08T06:06:32.479834+0000","last_became_active":"2024-05-08T06:05:51.423534+0000","last_became_peered":"2024-05-08T06:05:51.423534+0000","last_unstale":"2024-05-08T06:06:32.479834+0000","last_undegraded":"2024-05-08T06:06:32.479834+0000","last_fullsized":"2024-05-08T06:06:32.479834+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:06:46.197071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.477712+0000","last_change":"2024-05-08T06:05:51.411102+0000","last_active":"2024-05-08T06:06:32.477712+0000","last_peered":"2024-05-08T06:06:32.477712+0000","last_clean":"2024-05-08T06:06:32.477712+0000","last_became_active":"2024-05-08T06:05:51.410791+0000","last_became_peered":"2024-05-08T06:05:51.410791+0000","last_unstale":"2024-05-08T06:06:32.477712+0000","last_undegraded":"2024-05-08T06:06:32.477712+0000","last_fullsized":"2024-05-08T06:06:32.477712+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:31:47.861663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482197+0000","last_change":"2024-05-08T06:05:51.417756+0000","last_active":"2024-05-08T06:06:32.482197+0000","last_peered":"2024-05-08T06:06:32.482197+0000","last_clean":"2024-05-08T06:06:32.482197+0000","last_became_active":"2024-05-08T06:05:51.417373+0000","last_became_peered":"2024-05-08T06:05:51.417373+0000","last_unstale":"2024-05-08T06:06:32.482197+0000","last_undegraded":"2024-05-08T06:06:32.482197+0000","last_fullsized":"2024-05-08T06:06:32.482197+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:07:48.330401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.489273+0000","last_change":"2024-05-08T06:05:51.413883+0000","last_active":"2024-05-08T06:06:32.489273+0000","last_peered":"2024-05-08T06:06:32.489273+0000","last_clean":"2024-05-08T06:06:32.489273+0000","last_became_active":"2024-05-08T06:05:51.413565+0000","last_became_peered":"2024-05-08T06:05:51.413565+0000","last_unstale":"2024-05-08T06:06:32.489273+0000","last_undegraded":"2024-05-08T06:06:32.489273+0000","last_fullsized":"2024-05-08T06:06:32.489273+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:06:38.186225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.596722+0000","last_change":"2024-05-08T06:05:51.416300+0000","last_active":"2024-05-08T06:06:32.596722+0000","last_peered":"2024-05-08T06:06:32.596722+0000","last_clean":"2024-05-08T06:06:32.596722+0000","last_became_active":"2024-05-08T06:05:51.415970+0000","last_became_peered":"2024-05-08T06:05:51.415970+0000","last_unstale":"2024-05-08T06:06:32.596722+0000","last_undegraded":"2024-05-08T06:06:32.596722+0000","last_fullsized":"2024-05-08T06:06:32.596722+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:44:51.853339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.477997+0000","last_change":"2024-05-08T06:05:51.410396+0000","last_active":"2024-05-08T06:06:32.477997+0000","last_peered":"2024-05-08T06:06:32.477997+0000","last_clean":"2024-05-08T06:06:32.477997+0000","last_became_active":"2024-05-08T06:05:51.410067+0000","last_became_peered":"2024-05-08T06:05:51.410067+0000","last_unstale":"2024-05-08T06:06:32.477997+0000","last_undegraded":"2024-05-08T06:06:32.477997+0000","last_fullsized":"2024-05-08T06:06:32.477997+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:32:00.217217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482522+0000","last_change":"2024-05-08T06:05:51.409276+0000","last_active":"2024-05-08T06:06:32.482522+0000","last_peered":"2024-05-08T06:06:32.482522+0000","last_clean":"2024-05-08T06:06:32.482522+0000","last_became_active":"2024-05-08T06:05:51.408768+0000","last_became_peered":"2024-05-08T06:05:51.408768+0000","last_unstale":"2024-05-08T06:06:32.482522+0000","last_undegraded":"2024-05-08T06:06:32.482522+0000","last_fullsized":"2024-05-08T06:06:32.482522+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:22:57.483944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.597286+0000","last_change":"2024-05-08T06:05:51.419083+0000","last_active":"2024-05-08T06:06:32.597286+0000","last_peered":"2024-05-08T06:06:32.597286+0000","last_clean":"2024-05-08T06:06:32.597286+0000","last_became_active":"2024-05-08T06:05:51.418685+0000","last_became_peered":"2024-05-08T06:05:51.418685+0000","last_unstale":"2024-05-08T06:06:32.597286+0000","last_undegraded":"2024-05-08T06:06:32.597286+0000","last_fullsized":"2024-05-08T06:06:32.597286+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:25:49.256242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.491600+0000","last_change":"2024-05-08T06:05:51.420399+0000","last_active":"2024-05-08T06:06:32.491600+0000","last_peered":"2024-05-08T06:06:32.491600+0000","last_clean":"2024-05-08T06:06:32.491600+0000","last_became_active":"2024-05-08T06:05:51.420076+0000","last_became_peered":"2024-05-08T06:05:51.420076+0000","last_unstale":"2024-05-08T06:06:32.491600+0000","last_undegraded":"2024-05-08T06:06:32.491600+0000","last_fullsized":"2024-05-08T06:06:32.491600+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:51:38.579280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.484011+0000","last_change":"2024-05-08T06:05:51.431518+0000","last_active":"2024-05-08T06:06:32.484011+0000","last_peered":"2024-05-08T06:06:32.484011+0000","last_clean":"2024-05-08T06:06:32.484011+0000","last_became_active":"2024-05-08T06:05:51.431029+0000","last_became_peered":"2024-05-08T06:05:51.431029+0000","last_unstale":"2024-05-08T06:06:32.484011+0000","last_undegraded":"2024-05-08T06:06:32.484011+0000","last_fullsized":"2024-05-08T06:06:32.484011+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:11:09.596221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.498002+0000","last_change":"2024-05-08T06:06:32.498002+0000","last_active":"2024-05-08T06:06:32.498002+0000","last_peered":"2024-05-08T06:06:32.498002+0000","last_clean":"2024-05-08T06:06:32.498002+0000","last_became_active":"2024-05-08T06:06:32.497367+0000","last_became_peered":"2024-05-08T06:06:32.497367+0000","last_unstale":"2024-05-08T06:06:32.498002+0000","last_undegraded":"2024-05-08T06:06:32.498002+0000","last_fullsized":"2024-05-08T06:06:32.498002+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:32:36.461777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.595891+0000","last_change":"2024-05-08T06:05:51.423857+0000","last_active":"2024-05-08T06:06:32.595891+0000","last_peered":"2024-05-08T06:06:32.595891+0000","last_clean":"2024-05-08T06:06:32.595891+0000","last_became_active":"2024-05-08T06:05:51.423487+0000","last_became_peered":"2024-05-08T06:05:51.423487+0000","last_unstale":"2024-05-08T06:06:32.595891+0000","last_undegraded":"2024-05-08T06:06:32.595891+0000","last_fullsized":"2024-05-08T06:06:32.595891+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:17:33.269354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.477873+0000","last_change":"2024-05-08T06:05:51.431795+0000","last_active":"2024-05-08T06:06:32.477873+0000","last_peered":"2024-05-08T06:06:32.477873+0000","last_clean":"2024-05-08T06:06:32.477873+0000","last_became_active":"2024-05-08T06:05:51.431432+0000","last_became_peered":"2024-05-08T06:05:51.431432+0000","last_unstale":"2024-05-08T06:06:32.477873+0000","last_undegraded":"2024-05-08T06:06:32.477873+0000","last_fullsized":"2024-05-08T06:06:32.477873+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:11:16.561478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.484127+0000","last_change":"2024-05-08T06:05:51.414805+0000","last_active":"2024-05-08T06:06:32.484127+0000","last_peered":"2024-05-08T06:06:32.484127+0000","last_clean":"2024-05-08T06:06:32.484127+0000","last_became_active":"2024-05-08T06:05:51.414482+0000","last_became_peered":"2024-05-08T06:05:51.414482+0000","last_unstale":"2024-05-08T06:06:32.484127+0000","last_undegraded":"2024-05-08T06:06:32.484127+0000","last_fullsized":"2024-05-08T06:06:32.484127+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:08:13.444591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480668+0000","last_change":"2024-05-08T06:05:51.418516+0000","last_active":"2024-05-08T06:06:32.480668+0000","last_peered":"2024-05-08T06:06:32.480668+0000","last_clean":"2024-05-08T06:06:32.480668+0000","last_became_active":"2024-05-08T06:05:51.418182+0000","last_became_peered":"2024-05-08T06:05:51.418182+0000","last_unstale":"2024-05-08T06:06:32.480668+0000","last_undegraded":"2024-05-08T06:06:32.480668+0000","last_fullsized":"2024-05-08T06:06:32.480668+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:32:33.338311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482309+0000","last_change":"2024-05-08T06:05:46.350509+0000","last_active":"2024-05-08T06:06:32.482309+0000","last_peered":"2024-05-08T06:06:32.482309+0000","last_clean":"2024-05-08T06:06:32.482309+0000","last_became_active":"2024-05-08T06:05:46.349986+0000","last_became_peered":"2024-05-08T06:05:46.349986+0000","last_unstale":"2024-05-08T06:06:32.482309+0000","last_undegraded":"2024-05-08T06:06:32.482309+0000","last_fullsized":"2024-05-08T06:06:32.482309+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:53:00.928971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.492031+0000","last_change":"2024-05-08T06:05:51.413621+0000","last_active":"2024-05-08T06:06:32.492031+0000","last_peered":"2024-05-08T06:06:32.492031+0000","last_clean":"2024-05-08T06:06:32.492031+0000","last_became_active":"2024-05-08T06:05:51.413188+0000","last_became_peered":"2024-05-08T06:05:51.413188+0000","last_unstale":"2024-05-08T06:06:32.492031+0000","last_undegraded":"2024-05-08T06:06:32.492031+0000","last_fullsized":"2024-05-08T06:06:32.492031+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:13:21.413696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.492646+0000","last_change":"2024-05-08T06:05:46.552444+0000","last_active":"2024-05-08T06:06:32.492646+0000","last_peered":"2024-05-08T06:06:32.492646+0000","last_clean":"2024-05-08T06:06:32.492646+0000","last_became_active":"2024-05-08T06:05:46.551310+0000","last_became_peered":"2024-05-08T06:05:46.551310+0000","last_unstale":"2024-05-08T06:06:32.492646+0000","last_undegraded":"2024-05-08T06:06:32.492646+0000","last_fullsized":"2024-05-08T06:06:32.492646+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:58:17.632832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.487033+0000","last_change":"2024-05-08T06:05:51.425649+0000","last_active":"2024-05-08T06:06:32.487033+0000","last_peered":"2024-05-08T06:06:32.487033+0000","last_clean":"2024-05-08T06:06:32.487033+0000","last_became_active":"2024-05-08T06:05:51.425288+0000","last_became_peered":"2024-05-08T06:05:51.425288+0000","last_unstale":"2024-05-08T06:06:32.487033+0000","last_undegraded":"2024-05-08T06:06:32.487033+0000","last_fullsized":"2024-05-08T06:06:32.487033+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:42:40.580641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480600+0000","last_change":"2024-05-08T06:05:46.350300+0000","last_active":"2024-05-08T06:06:32.480600+0000","last_peered":"2024-05-08T06:06:32.480600+0000","last_clean":"2024-05-08T06:06:32.480600+0000","last_became_active":"2024-05-08T06:05:46.349918+0000","last_became_peered":"2024-05-08T06:05:46.349918+0000","last_unstale":"2024-05-08T06:06:32.480600+0000","last_undegraded":"2024-05-08T06:06:32.480600+0000","last_fullsized":"2024-05-08T06:06:32.480600+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:32:41.952772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.596199+0000","last_change":"2024-05-08T06:05:51.430130+0000","last_active":"2024-05-08T06:06:32.596199+0000","last_peered":"2024-05-08T06:06:32.596199+0000","last_clean":"2024-05-08T06:06:32.596199+0000","last_became_active":"2024-05-08T06:05:51.429794+0000","last_became_peered":"2024-05-08T06:05:51.429794+0000","last_unstale":"2024-05-08T06:06:32.596199+0000","last_undegraded":"2024-05-08T06:06:32.596199+0000","last_fullsized":"2024-05-08T06:06:32.596199+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:21:55.773309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480488+0000","last_change":"2024-05-08T06:05:46.351664+0000","last_active":"2024-05-08T06:06:32.480488+0000","last_peered":"2024-05-08T06:06:32.480488+0000","last_clean":"2024-05-08T06:06:32.480488+0000","last_became_active":"2024-05-08T06:05:46.351305+0000","last_became_peered":"2024-05-08T06:05:46.351305+0000","last_unstale":"2024-05-08T06:06:32.480488+0000","last_undegraded":"2024-05-08T06:06:32.480488+0000","last_fullsized":"2024-05-08T06:06:32.480488+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:24:36.164315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.596189+0000","last_change":"2024-05-08T06:05:51.414793+0000","last_active":"2024-05-08T06:06:32.596189+0000","last_peered":"2024-05-08T06:06:32.596189+0000","last_clean":"2024-05-08T06:06:32.596189+0000","last_became_active":"2024-05-08T06:05:51.414317+0000","last_became_peered":"2024-05-08T06:05:51.414317+0000","last_unstale":"2024-05-08T06:06:32.596189+0000","last_undegraded":"2024-05-08T06:06:32.596189+0000","last_fullsized":"2024-05-08T06:06:32.596189+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:57:56.152836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":70,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:53.445586+0000","last_change":"2024-05-08T06:05:46.352683+0000","last_active":"2024-05-08T06:06:53.445586+0000","last_peered":"2024-05-08T06:06:53.445586+0000","last_clean":"2024-05-08T06:06:53.445586+0000","last_became_active":"2024-05-08T06:05:46.352278+0000","last_became_peered":"2024-05-08T06:05:46.352278+0000","last_unstale":"2024-05-08T06:06:53.445586+0000","last_undegraded":"2024-05-08T06:06:53.445586+0000","last_fullsized":"2024-05-08T06:06:53.445586+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:39:22.655516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480098+0000","last_change":"2024-05-08T06:05:51.415822+0000","last_active":"2024-05-08T06:06:32.480098+0000","last_peered":"2024-05-08T06:06:32.480098+0000","last_clean":"2024-05-08T06:06:32.480098+0000","last_became_active":"2024-05-08T06:05:51.415449+0000","last_became_peered":"2024-05-08T06:05:51.415449+0000","last_unstale":"2024-05-08T06:06:32.480098+0000","last_undegraded":"2024-05-08T06:06:32.480098+0000","last_fullsized":"2024-05-08T06:06:32.480098+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:44:49.127335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.477620+0000","last_change":"2024-05-08T06:05:46.361288+0000","last_active":"2024-05-08T06:06:32.477620+0000","last_peered":"2024-05-08T06:06:32.477620+0000","last_clean":"2024-05-08T06:06:32.477620+0000","last_became_active":"2024-05-08T06:05:46.360761+0000","last_became_peered":"2024-05-08T06:05:46.360761+0000","last_unstale":"2024-05-08T06:06:32.477620+0000","last_undegraded":"2024-05-08T06:06:32.477620+0000","last_fullsized":"2024-05-08T06:06:32.477620+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:14:59.858935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.596064+0000","last_change":"2024-05-08T06:05:51.418547+0000","last_active":"2024-05-08T06:06:32.596064+0000","last_peered":"2024-05-08T06:06:32.596064+0000","last_clean":"2024-05-08T06:06:32.596064+0000","last_became_active":"2024-05-08T06:05:51.417588+0000","last_became_peered":"2024-05-08T06:05:51.417588+0000","last_unstale":"2024-05-08T06:06:32.596064+0000","last_undegraded":"2024-05-08T06:06:32.596064+0000","last_fullsized":"2024-05-08T06:06:32.596064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:16:41.451351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.491600+0000","last_change":"2024-05-08T06:05:46.362077+0000","last_active":"2024-05-08T06:06:32.491600+0000","last_peered":"2024-05-08T06:06:32.491600+0000","last_clean":"2024-05-08T06:06:32.491600+0000","last_became_active":"2024-05-08T06:05:46.360823+0000","last_became_peered":"2024-05-08T06:05:46.360823+0000","last_unstale":"2024-05-08T06:06:32.491600+0000","last_undegraded":"2024-05-08T06:06:32.491600+0000","last_fullsized":"2024-05-08T06:06:32.491600+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:14.169822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.488967+0000","last_change":"2024-05-08T06:05:51.423687+0000","last_active":"2024-05-08T06:06:32.488967+0000","last_peered":"2024-05-08T06:06:32.488967+0000","last_clean":"2024-05-08T06:06:32.488967+0000","last_became_active":"2024-05-08T06:05:51.417656+0000","last_became_peered":"2024-05-08T06:05:51.417656+0000","last_unstale":"2024-05-08T06:06:32.488967+0000","last_undegraded":"2024-05-08T06:06:32.488967+0000","last_fullsized":"2024-05-08T06:06:32.488967+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:59:31.319915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.595724+0000","last_change":"2024-05-08T06:05:46.351892+0000","last_active":"2024-05-08T06:06:32.595724+0000","last_peered":"2024-05-08T06:06:32.595724+0000","last_clean":"2024-05-08T06:06:32.595724+0000","last_became_active":"2024-05-08T06:05:46.351523+0000","last_became_peered":"2024-05-08T06:05:46.351523+0000","last_unstale":"2024-05-08T06:06:32.595724+0000","last_undegraded":"2024-05-08T06:06:32.595724+0000","last_fullsized":"2024-05-08T06:06:32.595724+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:11:23.794313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.483084+0000","last_change":"2024-05-08T06:05:51.407681+0000","last_active":"2024-05-08T06:06:32.483084+0000","last_peered":"2024-05-08T06:06:32.483084+0000","last_clean":"2024-05-08T06:06:32.483084+0000","last_became_active":"2024-05-08T06:05:51.407243+0000","last_became_peered":"2024-05-08T06:05:51.407243+0000","last_unstale":"2024-05-08T06:06:32.483084+0000","last_undegraded":"2024-05-08T06:06:32.483084+0000","last_fullsized":"2024-05-08T06:06:32.483084+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:06:30.490268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'3","reported_seq":64,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:51.441926+0000","last_change":"2024-05-08T06:05:46.356675+0000","last_active":"2024-05-08T06:06:51.441926+0000","last_peered":"2024-05-08T06:06:51.441926+0000","last_clean":"2024-05-08T06:06:51.441926+0000","last_became_active":"2024-05-08T06:05:46.356360+0000","last_became_peered":"2024-05-08T06:05:46.356360+0000","last_unstale":"2024-05-08T06:06:51.441926+0000","last_undegraded":"2024-05-08T06:06:51.441926+0000","last_fullsized":"2024-05-08T06:06:51.441926+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:46:48.131551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.596564+0000","last_change":"2024-05-08T06:05:51.421325+0000","last_active":"2024-05-08T06:06:32.596564+0000","last_peered":"2024-05-08T06:06:32.596564+0000","last_clean":"2024-05-08T06:06:32.596564+0000","last_became_active":"2024-05-08T06:05:51.420905+0000","last_became_peered":"2024-05-08T06:05:51.420905+0000","last_unstale":"2024-05-08T06:06:32.596564+0000","last_undegraded":"2024-05-08T06:06:32.596564+0000","last_fullsized":"2024-05-08T06:06:32.596564+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:37:27.324920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482674+0000","last_change":"2024-05-08T06:05:46.351958+0000","last_active":"2024-05-08T06:06:32.482674+0000","last_peered":"2024-05-08T06:06:32.482674+0000","last_clean":"2024-05-08T06:06:32.482674+0000","last_became_active":"2024-05-08T06:05:46.351561+0000","last_became_peered":"2024-05-08T06:05:46.351561+0000","last_unstale":"2024-05-08T06:06:32.482674+0000","last_undegraded":"2024-05-08T06:06:32.482674+0000","last_fullsized":"2024-05-08T06:06:32.482674+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:11:27.727029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.486739+0000","last_change":"2024-05-08T06:05:51.431809+0000","last_active":"2024-05-08T06:06:32.486739+0000","last_peered":"2024-05-08T06:06:32.486739+0000","last_clean":"2024-05-08T06:06:32.486739+0000","last_became_active":"2024-05-08T06:05:51.431416+0000","last_became_peered":"2024-05-08T06:05:51.431416+0000","last_unstale":"2024-05-08T06:06:32.486739+0000","last_undegraded":"2024-05-08T06:06:32.486739+0000","last_fullsized":"2024-05-08T06:06:32.486739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:48:16.704134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.596297+0000","last_change":"2024-05-08T06:05:46.550763+0000","last_active":"2024-05-08T06:06:32.596297+0000","last_peered":"2024-05-08T06:06:32.596297+0000","last_clean":"2024-05-08T06:06:32.596297+0000","last_became_active":"2024-05-08T06:05:46.550297+0000","last_became_peered":"2024-05-08T06:05:46.550297+0000","last_unstale":"2024-05-08T06:06:32.596297+0000","last_undegraded":"2024-05-08T06:06:32.596297+0000","last_fullsized":"2024-05-08T06:06:32.596297+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:32:19.907220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.596339+0000","last_change":"2024-05-08T06:05:51.414738+0000","last_active":"2024-05-08T06:06:32.596339+0000","last_peered":"2024-05-08T06:06:32.596339+0000","last_clean":"2024-05-08T06:06:32.596339+0000","last_became_active":"2024-05-08T06:05:51.414369+0000","last_became_peered":"2024-05-08T06:05:51.414369+0000","last_unstale":"2024-05-08T06:06:32.596339+0000","last_undegraded":"2024-05-08T06:06:32.596339+0000","last_fullsized":"2024-05-08T06:06:32.596339+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:57:44.215343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'29","reported_seq":133,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:52.443563+0000","last_change":"2024-05-08T06:05:46.349825+0000","last_active":"2024-05-08T06:06:52.443563+0000","last_peered":"2024-05-08T06:06:52.443563+0000","last_clean":"2024-05-08T06:06:52.443563+0000","last_became_active":"2024-05-08T06:05:46.349425+0000","last_became_peered":"2024-05-08T06:05:46.349425+0000","last_unstale":"2024-05-08T06:06:52.443563+0000","last_undegraded":"2024-05-08T06:06:52.443563+0000","last_fullsized":"2024-05-08T06:06:52.443563+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:29:39.123817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58815,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":78,"num_read_kb":96,"num_write":30,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480389+0000","last_change":"2024-05-08T06:05:51.411341+0000","last_active":"2024-05-08T06:06:32.480389+0000","last_peered":"2024-05-08T06:06:32.480389+0000","last_clean":"2024-05-08T06:06:32.480389+0000","last_became_active":"2024-05-08T06:05:51.411002+0000","last_became_peered":"2024-05-08T06:05:51.411002+0000","last_unstale":"2024-05-08T06:06:32.480389+0000","last_undegraded":"2024-05-08T06:06:32.480389+0000","last_fullsized":"2024-05-08T06:06:32.480389+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:27:07.013809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.509330+0000","last_change":"2024-05-08T06:06:32.508904+0000","last_active":"2024-05-08T06:06:32.509330+0000","last_peered":"2024-05-08T06:06:32.509330+0000","last_clean":"2024-05-08T06:06:32.509330+0000","last_became_active":"2024-05-08T06:06:32.496001+0000","last_became_peered":"2024-05-08T06:06:32.496001+0000","last_unstale":"2024-05-08T06:06:32.509330+0000","last_undegraded":"2024-05-08T06:06:32.509330+0000","last_fullsized":"2024-05-08T06:06:32.509330+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:34:32.417105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.491251+0000","last_change":"2024-05-08T06:05:51.418395+0000","last_active":"2024-05-08T06:06:32.491251+0000","last_peered":"2024-05-08T06:06:32.491251+0000","last_clean":"2024-05-08T06:06:32.491251+0000","last_became_active":"2024-05-08T06:05:51.418035+0000","last_became_peered":"2024-05-08T06:05:51.418035+0000","last_unstale":"2024-05-08T06:06:32.491251+0000","last_undegraded":"2024-05-08T06:06:32.491251+0000","last_fullsized":"2024-05-08T06:06:32.491251+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:58:16.529144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.595936+0000","last_change":"2024-05-08T06:05:46.363088+0000","last_active":"2024-05-08T06:06:32.595936+0000","last_peered":"2024-05-08T06:06:32.595936+0000","last_clean":"2024-05-08T06:06:32.595936+0000","last_became_active":"2024-05-08T06:05:46.362488+0000","last_became_peered":"2024-05-08T06:05:46.362488+0000","last_unstale":"2024-05-08T06:06:32.595936+0000","last_undegraded":"2024-05-08T06:06:32.595936+0000","last_fullsized":"2024-05-08T06:06:32.595936+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:28:10.039647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482481+0000","last_change":"2024-05-08T06:05:51.415567+0000","last_active":"2024-05-08T06:06:32.482481+0000","last_peered":"2024-05-08T06:06:32.482481+0000","last_clean":"2024-05-08T06:06:32.482481+0000","last_became_active":"2024-05-08T06:05:51.415148+0000","last_became_peered":"2024-05-08T06:05:51.415148+0000","last_unstale":"2024-05-08T06:06:32.482481+0000","last_undegraded":"2024-05-08T06:06:32.482481+0000","last_fullsized":"2024-05-08T06:06:32.482481+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:51:48.871149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'4","reported_seq":71,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:54.472375+0000","last_change":"2024-05-08T06:05:46.348769+0000","last_active":"2024-05-08T06:06:54.472375+0000","last_peered":"2024-05-08T06:06:54.472375+0000","last_clean":"2024-05-08T06:06:54.472375+0000","last_became_active":"2024-05-08T06:05:46.348072+0000","last_became_peered":"2024-05-08T06:05:46.348072+0000","last_unstale":"2024-05-08T06:06:54.472375+0000","last_undegraded":"2024-05-08T06:06:54.472375+0000","last_fullsized":"2024-05-08T06:06:54.472375+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:45.292436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482108+0000","last_change":"2024-05-08T06:05:51.414623+0000","last_active":"2024-05-08T06:06:32.482108+0000","last_peered":"2024-05-08T06:06:32.482108+0000","last_clean":"2024-05-08T06:06:32.482108+0000","last_became_active":"2024-05-08T06:05:51.414258+0000","last_became_peered":"2024-05-08T06:05:51.414258+0000","last_unstale":"2024-05-08T06:06:32.482108+0000","last_undegraded":"2024-05-08T06:06:32.482108+0000","last_fullsized":"2024-05-08T06:06:32.482108+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:38:48.297900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.479966+0000","last_change":"2024-05-08T06:05:46.546936+0000","last_active":"2024-05-08T06:06:32.479966+0000","last_peered":"2024-05-08T06:06:32.479966+0000","last_clean":"2024-05-08T06:06:32.479966+0000","last_became_active":"2024-05-08T06:05:46.546574+0000","last_became_peered":"2024-05-08T06:05:46.546574+0000","last_unstale":"2024-05-08T06:06:32.479966+0000","last_undegraded":"2024-05-08T06:06:32.479966+0000","last_fullsized":"2024-05-08T06:06:32.479966+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:16:29.720304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.595824+0000","last_change":"2024-05-08T06:05:51.418012+0000","last_active":"2024-05-08T06:06:32.595824+0000","last_peered":"2024-05-08T06:06:32.595824+0000","last_clean":"2024-05-08T06:06:32.595824+0000","last_became_active":"2024-05-08T06:05:51.417646+0000","last_became_peered":"2024-05-08T06:05:51.417646+0000","last_unstale":"2024-05-08T06:06:32.595824+0000","last_undegraded":"2024-05-08T06:06:32.595824+0000","last_fullsized":"2024-05-08T06:06:32.595824+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:43:02.733967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480019+0000","last_change":"2024-05-08T06:05:46.347748+0000","last_active":"2024-05-08T06:06:32.480019+0000","last_peered":"2024-05-08T06:06:32.480019+0000","last_clean":"2024-05-08T06:06:32.480019+0000","last_became_active":"2024-05-08T06:05:46.347367+0000","last_became_peered":"2024-05-08T06:05:46.347367+0000","last_unstale":"2024-05-08T06:06:32.480019+0000","last_undegraded":"2024-05-08T06:06:32.480019+0000","last_fullsized":"2024-05-08T06:06:32.480019+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:51:50.973855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.597018+0000","last_change":"2024-05-08T06:06:32.597018+0000","last_active":"2024-05-08T06:06:32.597018+0000","last_peered":"2024-05-08T06:06:32.597018+0000","last_clean":"2024-05-08T06:06:32.597018+0000","last_became_active":"2024-05-08T06:06:32.596677+0000","last_became_peered":"2024-05-08T06:06:32.596677+0000","last_unstale":"2024-05-08T06:06:32.597018+0000","last_undegraded":"2024-05-08T06:06:32.597018+0000","last_fullsized":"2024-05-08T06:06:32.597018+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:02:46.223949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.481565+0000","last_change":"2024-05-08T06:05:46.554443+0000","last_active":"2024-05-08T06:06:32.481565+0000","last_peered":"2024-05-08T06:06:32.481565+0000","last_clean":"2024-05-08T06:06:32.481565+0000","last_became_active":"2024-05-08T06:05:46.553226+0000","last_became_peered":"2024-05-08T06:05:46.553226+0000","last_unstale":"2024-05-08T06:06:32.481565+0000","last_undegraded":"2024-05-08T06:06:32.481565+0000","last_fullsized":"2024-05-08T06:06:32.481565+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:56:39.784652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.477829+0000","last_change":"2024-05-08T06:05:51.413121+0000","last_active":"2024-05-08T06:06:32.477829+0000","last_peered":"2024-05-08T06:06:32.477829+0000","last_clean":"2024-05-08T06:06:32.477829+0000","last_became_active":"2024-05-08T06:05:51.412767+0000","last_became_peered":"2024-05-08T06:05:51.412767+0000","last_unstale":"2024-05-08T06:06:32.477829+0000","last_undegraded":"2024-05-08T06:06:32.477829+0000","last_fullsized":"2024-05-08T06:06:32.477829+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:59:13.416981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482351+0000","last_change":"2024-05-08T06:05:46.361349+0000","last_active":"2024-05-08T06:06:32.482351+0000","last_peered":"2024-05-08T06:06:32.482351+0000","last_clean":"2024-05-08T06:06:32.482351+0000","last_became_active":"2024-05-08T06:05:46.360654+0000","last_became_peered":"2024-05-08T06:05:46.360654+0000","last_unstale":"2024-05-08T06:06:32.482351+0000","last_undegraded":"2024-05-08T06:06:32.482351+0000","last_fullsized":"2024-05-08T06:06:32.482351+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:19:22.144874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.477758+0000","last_change":"2024-05-08T06:05:51.423836+0000","last_active":"2024-05-08T06:06:32.477758+0000","last_peered":"2024-05-08T06:06:32.477758+0000","last_clean":"2024-05-08T06:06:32.477758+0000","last_became_active":"2024-05-08T06:05:51.423424+0000","last_became_peered":"2024-05-08T06:05:51.423424+0000","last_unstale":"2024-05-08T06:06:32.477758+0000","last_undegraded":"2024-05-08T06:06:32.477758+0000","last_fullsized":"2024-05-08T06:06:32.477758+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:56:14.779765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.598978+0000","last_change":"2024-05-08T06:06:32.598978+0000","last_active":"2024-05-08T06:06:32.598978+0000","last_peered":"2024-05-08T06:06:32.598978+0000","last_clean":"2024-05-08T06:06:32.598978+0000","last_became_active":"2024-05-08T06:06:32.598158+0000","last_became_peered":"2024-05-08T06:06:32.598158+0000","last_unstale":"2024-05-08T06:06:32.598978+0000","last_undegraded":"2024-05-08T06:06:32.598978+0000","last_fullsized":"2024-05-08T06:06:32.598978+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:51:09.270066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":68,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:54.471627+0000","last_change":"2024-05-08T06:05:46.547450+0000","last_active":"2024-05-08T06:06:54.471627+0000","last_peered":"2024-05-08T06:06:54.471627+0000","last_clean":"2024-05-08T06:06:54.471627+0000","last_became_active":"2024-05-08T06:05:46.546444+0000","last_became_peered":"2024-05-08T06:05:46.546444+0000","last_unstale":"2024-05-08T06:06:54.471627+0000","last_undegraded":"2024-05-08T06:06:54.471627+0000","last_fullsized":"2024-05-08T06:06:54.471627+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:17:14.124224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":38,"num_read_kb":38,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.489451+0000","last_change":"2024-05-08T06:05:51.423191+0000","last_active":"2024-05-08T06:06:32.489451+0000","last_peered":"2024-05-08T06:06:32.489451+0000","last_clean":"2024-05-08T06:06:32.489451+0000","last_became_active":"2024-05-08T06:05:51.420912+0000","last_became_peered":"2024-05-08T06:05:51.420912+0000","last_unstale":"2024-05-08T06:06:32.489451+0000","last_undegraded":"2024-05-08T06:06:32.489451+0000","last_fullsized":"2024-05-08T06:06:32.489451+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:27:57.678527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.597268+0000","last_change":"2024-05-08T06:05:46.348650+0000","last_active":"2024-05-08T06:06:32.597268+0000","last_peered":"2024-05-08T06:06:32.597268+0000","last_clean":"2024-05-08T06:06:32.597268+0000","last_became_active":"2024-05-08T06:05:46.347266+0000","last_became_peered":"2024-05-08T06:05:46.347266+0000","last_unstale":"2024-05-08T06:06:32.597268+0000","last_undegraded":"2024-05-08T06:06:32.597268+0000","last_fullsized":"2024-05-08T06:06:32.597268+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:37:30.623580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":100,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482311+0000","last_change":"2024-05-08T06:04:09.404663+0000","last_active":"2024-05-08T06:06:32.482311+0000","last_peered":"2024-05-08T06:06:32.482311+0000","last_clean":"2024-05-08T06:06:32.482311+0000","last_became_active":"2024-05-08T06:04:09.400115+0000","last_became_peered":"2024-05-08T06:04:09.400115+0000","last_unstale":"2024-05-08T06:06:32.482311+0000","last_undegraded":"2024-05-08T06:06:32.482311+0000","last_fullsized":"2024-05-08T06:06:32.482311+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:00:17.419712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:00:17.419712+0000","last_clean_scrub_stamp":"2024-05-08T06:00:17.419712+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:27:39.154694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.490968+0000","last_change":"2024-05-08T06:05:51.421826+0000","last_active":"2024-05-08T06:06:32.490968+0000","last_peered":"2024-05-08T06:06:32.490968+0000","last_clean":"2024-05-08T06:06:32.490968+0000","last_became_active":"2024-05-08T06:05:51.420706+0000","last_became_peered":"2024-05-08T06:05:51.420706+0000","last_unstale":"2024-05-08T06:06:32.490968+0000","last_undegraded":"2024-05-08T06:06:32.490968+0000","last_fullsized":"2024-05-08T06:06:32.490968+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:06:16.242332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482336+0000","last_change":"2024-05-08T06:05:46.351365+0000","last_active":"2024-05-08T06:06:32.482336+0000","last_peered":"2024-05-08T06:06:32.482336+0000","last_clean":"2024-05-08T06:06:32.482336+0000","last_became_active":"2024-05-08T06:05:46.351022+0000","last_became_peered":"2024-05-08T06:05:46.351022+0000","last_unstale":"2024-05-08T06:06:32.482336+0000","last_undegraded":"2024-05-08T06:06:32.482336+0000","last_fullsized":"2024-05-08T06:06:32.482336+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:32:44.134994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.501668+0000","last_change":"2024-05-08T06:06:32.501263+0000","last_active":"2024-05-08T06:06:32.501668+0000","last_peered":"2024-05-08T06:06:32.501668+0000","last_clean":"2024-05-08T06:06:32.501668+0000","last_became_active":"2024-05-08T06:06:32.493795+0000","last_became_peered":"2024-05-08T06:06:32.493795+0000","last_unstale":"2024-05-08T06:06:32.501668+0000","last_undegraded":"2024-05-08T06:06:32.501668+0000","last_fullsized":"2024-05-08T06:06:32.501668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:49:03.898162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.492437+0000","last_change":"2024-05-08T06:05:51.428382+0000","last_active":"2024-05-08T06:06:32.492437+0000","last_peered":"2024-05-08T06:06:32.492437+0000","last_clean":"2024-05-08T06:06:32.492437+0000","last_became_active":"2024-05-08T06:05:51.421443+0000","last_became_peered":"2024-05-08T06:05:51.421443+0000","last_unstale":"2024-05-08T06:06:32.492437+0000","last_undegraded":"2024-05-08T06:06:32.492437+0000","last_fullsized":"2024-05-08T06:06:32.492437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:26:10.215020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'12","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:46.444694+0000","last_change":"2024-05-08T06:05:46.364306+0000","last_active":"2024-05-08T06:06:46.444694+0000","last_peered":"2024-05-08T06:06:46.444694+0000","last_clean":"2024-05-08T06:06:46.444694+0000","last_became_active":"2024-05-08T06:05:46.363871+0000","last_became_peered":"2024-05-08T06:05:46.363871+0000","last_unstale":"2024-05-08T06:06:46.444694+0000","last_undegraded":"2024-05-08T06:06:46.444694+0000","last_fullsized":"2024-05-08T06:06:46.444694+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:38:34.531162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.495732+0000","last_change":"2024-05-08T06:05:51.415020+0000","last_active":"2024-05-08T06:06:32.495732+0000","last_peered":"2024-05-08T06:06:32.495732+0000","last_clean":"2024-05-08T06:06:32.495732+0000","last_became_active":"2024-05-08T06:05:51.413690+0000","last_became_peered":"2024-05-08T06:05:51.413690+0000","last_unstale":"2024-05-08T06:06:32.495732+0000","last_undegraded":"2024-05-08T06:06:32.495732+0000","last_fullsized":"2024-05-08T06:06:32.495732+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:55:38.121492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.597212+0000","last_change":"2024-05-08T06:05:46.359100+0000","last_active":"2024-05-08T06:06:32.597212+0000","last_peered":"2024-05-08T06:06:32.597212+0000","last_clean":"2024-05-08T06:06:32.597212+0000","last_became_active":"2024-05-08T06:05:46.358815+0000","last_became_peered":"2024-05-08T06:05:46.358815+0000","last_unstale":"2024-05-08T06:06:32.597212+0000","last_undegraded":"2024-05-08T06:06:32.597212+0000","last_fullsized":"2024-05-08T06:06:32.597212+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:22:58.532358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.488780+0000","last_change":"2024-05-08T06:05:51.429222+0000","last_active":"2024-05-08T06:06:32.488780+0000","last_peered":"2024-05-08T06:06:32.488780+0000","last_clean":"2024-05-08T06:06:32.488780+0000","last_became_active":"2024-05-08T06:05:51.428845+0000","last_became_peered":"2024-05-08T06:05:51.428845+0000","last_unstale":"2024-05-08T06:06:32.488780+0000","last_undegraded":"2024-05-08T06:06:32.488780+0000","last_fullsized":"2024-05-08T06:06:32.488780+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:36:41.435666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.596030+0000","last_change":"2024-05-08T06:05:46.547931+0000","last_active":"2024-05-08T06:06:32.596030+0000","last_peered":"2024-05-08T06:06:32.596030+0000","last_clean":"2024-05-08T06:06:32.596030+0000","last_became_active":"2024-05-08T06:05:46.547547+0000","last_became_peered":"2024-05-08T06:05:46.547547+0000","last_unstale":"2024-05-08T06:06:32.596030+0000","last_undegraded":"2024-05-08T06:06:32.596030+0000","last_fullsized":"2024-05-08T06:06:32.596030+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:27:03.502153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.479822+0000","last_change":"2024-05-08T06:05:51.413259+0000","last_active":"2024-05-08T06:06:32.479822+0000","last_peered":"2024-05-08T06:06:32.479822+0000","last_clean":"2024-05-08T06:06:32.479822+0000","last_became_active":"2024-05-08T06:05:51.412890+0000","last_became_peered":"2024-05-08T06:05:51.412890+0000","last_unstale":"2024-05-08T06:06:32.479822+0000","last_undegraded":"2024-05-08T06:06:32.479822+0000","last_fullsized":"2024-05-08T06:06:32.479822+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:04:56.690098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.489674+0000","last_change":"2024-05-08T06:05:46.365409+0000","last_active":"2024-05-08T06:06:32.489674+0000","last_peered":"2024-05-08T06:06:32.489674+0000","last_clean":"2024-05-08T06:06:32.489674+0000","last_became_active":"2024-05-08T06:05:46.361277+0000","last_became_peered":"2024-05-08T06:05:46.361277+0000","last_unstale":"2024-05-08T06:06:32.489674+0000","last_undegraded":"2024-05-08T06:06:32.489674+0000","last_fullsized":"2024-05-08T06:06:32.489674+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:06.927385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.477363+0000","last_change":"2024-05-08T06:05:51.423749+0000","last_active":"2024-05-08T06:06:32.477363+0000","last_peered":"2024-05-08T06:06:32.477363+0000","last_clean":"2024-05-08T06:06:32.477363+0000","last_became_active":"2024-05-08T06:05:51.423386+0000","last_became_peered":"2024-05-08T06:05:51.423386+0000","last_unstale":"2024-05-08T06:06:32.477363+0000","last_undegraded":"2024-05-08T06:06:32.477363+0000","last_fullsized":"2024-05-08T06:06:32.477363+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:41:26.261683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480381+0000","last_change":"2024-05-08T06:05:46.362497+0000","last_active":"2024-05-08T06:06:32.480381+0000","last_peered":"2024-05-08T06:06:32.480381+0000","last_clean":"2024-05-08T06:06:32.480381+0000","last_became_active":"2024-05-08T06:05:46.362110+0000","last_became_peered":"2024-05-08T06:05:46.362110+0000","last_unstale":"2024-05-08T06:06:32.480381+0000","last_undegraded":"2024-05-08T06:06:32.480381+0000","last_fullsized":"2024-05-08T06:06:32.480381+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:25:06.205380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.495078+0000","last_change":"2024-05-08T06:06:32.495078+0000","last_active":"2024-05-08T06:06:32.495078+0000","last_peered":"2024-05-08T06:06:32.495078+0000","last_clean":"2024-05-08T06:06:32.495078+0000","last_became_active":"2024-05-08T06:06:32.493855+0000","last_became_peered":"2024-05-08T06:06:32.493855+0000","last_unstale":"2024-05-08T06:06:32.495078+0000","last_undegraded":"2024-05-08T06:06:32.495078+0000","last_fullsized":"2024-05-08T06:06:32.495078+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:56:54.143080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":43,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.601826+0000","last_change":"2024-05-08T06:06:32.601826+0000","last_active":"2024-05-08T06:06:32.601826+0000","last_peered":"2024-05-08T06:06:32.601826+0000","last_clean":"2024-05-08T06:06:32.601826+0000","last_became_active":"2024-05-08T06:06:32.597823+0000","last_became_peered":"2024-05-08T06:06:32.597823+0000","last_unstale":"2024-05-08T06:06:32.601826+0000","last_undegraded":"2024-05-08T06:06:32.601826+0000","last_fullsized":"2024-05-08T06:06:32.601826+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:59:09.741096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482383+0000","last_change":"2024-05-08T06:05:51.413744+0000","last_active":"2024-05-08T06:06:32.482383+0000","last_peered":"2024-05-08T06:06:32.482383+0000","last_clean":"2024-05-08T06:06:32.482383+0000","last_became_active":"2024-05-08T06:05:51.413346+0000","last_became_peered":"2024-05-08T06:05:51.413346+0000","last_unstale":"2024-05-08T06:06:32.482383+0000","last_undegraded":"2024-05-08T06:06:32.482383+0000","last_fullsized":"2024-05-08T06:06:32.482383+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:46:56.109755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480874+0000","last_change":"2024-05-08T06:05:46.552604+0000","last_active":"2024-05-08T06:06:32.480874+0000","last_peered":"2024-05-08T06:06:32.480874+0000","last_clean":"2024-05-08T06:06:32.480874+0000","last_became_active":"2024-05-08T06:05:46.552248+0000","last_became_peered":"2024-05-08T06:05:46.552248+0000","last_unstale":"2024-05-08T06:06:32.480874+0000","last_undegraded":"2024-05-08T06:06:32.480874+0000","last_fullsized":"2024-05-08T06:06:32.480874+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:39:10.306692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482636+0000","last_change":"2024-05-08T06:05:51.404988+0000","last_active":"2024-05-08T06:06:32.482636+0000","last_peered":"2024-05-08T06:06:32.482636+0000","last_clean":"2024-05-08T06:06:32.482636+0000","last_became_active":"2024-05-08T06:05:51.404625+0000","last_became_peered":"2024-05-08T06:05:51.404625+0000","last_unstale":"2024-05-08T06:06:32.482636+0000","last_undegraded":"2024-05-08T06:06:32.482636+0000","last_fullsized":"2024-05-08T06:06:32.482636+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:33:27.919929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.596104+0000","last_change":"2024-05-08T06:05:46.361466+0000","last_active":"2024-05-08T06:06:32.596104+0000","last_peered":"2024-05-08T06:06:32.596104+0000","last_clean":"2024-05-08T06:06:32.596104+0000","last_became_active":"2024-05-08T06:05:46.361107+0000","last_became_peered":"2024-05-08T06:05:46.361107+0000","last_unstale":"2024-05-08T06:06:32.596104+0000","last_undegraded":"2024-05-08T06:06:32.596104+0000","last_fullsized":"2024-05-08T06:06:32.596104+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:15:31.314226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482603+0000","last_change":"2024-05-08T06:05:51.416437+0000","last_active":"2024-05-08T06:06:32.482603+0000","last_peered":"2024-05-08T06:06:32.482603+0000","last_clean":"2024-05-08T06:06:32.482603+0000","last_became_active":"2024-05-08T06:05:51.416070+0000","last_became_peered":"2024-05-08T06:05:51.416070+0000","last_unstale":"2024-05-08T06:06:32.482603+0000","last_undegraded":"2024-05-08T06:06:32.482603+0000","last_fullsized":"2024-05-08T06:06:32.482603+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:35:23.652373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.492808+0000","last_change":"2024-05-08T06:05:46.362475+0000","last_active":"2024-05-08T06:06:32.492808+0000","last_peered":"2024-05-08T06:06:32.492808+0000","last_clean":"2024-05-08T06:06:32.492808+0000","last_became_active":"2024-05-08T06:05:46.361852+0000","last_became_peered":"2024-05-08T06:05:46.361852+0000","last_unstale":"2024-05-08T06:06:32.492808+0000","last_undegraded":"2024-05-08T06:06:32.492808+0000","last_fullsized":"2024-05-08T06:06:32.492808+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:16:54.536330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482691+0000","last_change":"2024-05-08T06:05:51.410718+0000","last_active":"2024-05-08T06:06:32.482691+0000","last_peered":"2024-05-08T06:06:32.482691+0000","last_clean":"2024-05-08T06:06:32.482691+0000","last_became_active":"2024-05-08T06:05:51.408902+0000","last_became_peered":"2024-05-08T06:05:51.408902+0000","last_unstale":"2024-05-08T06:06:32.482691+0000","last_undegraded":"2024-05-08T06:06:32.482691+0000","last_fullsized":"2024-05-08T06:06:32.482691+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:23:33.857677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.477466+0000","last_change":"2024-05-08T06:05:51.414362+0000","last_active":"2024-05-08T06:06:32.477466+0000","last_peered":"2024-05-08T06:06:32.477466+0000","last_clean":"2024-05-08T06:06:32.477466+0000","last_became_active":"2024-05-08T06:05:51.413998+0000","last_became_peered":"2024-05-08T06:05:51.413998+0000","last_unstale":"2024-05-08T06:06:32.477466+0000","last_undegraded":"2024-05-08T06:06:32.477466+0000","last_fullsized":"2024-05-08T06:06:32.477466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:38:22.198219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480218+0000","last_change":"2024-05-08T06:05:51.415508+0000","last_active":"2024-05-08T06:06:32.480218+0000","last_peered":"2024-05-08T06:06:32.480218+0000","last_clean":"2024-05-08T06:06:32.480218+0000","last_became_active":"2024-05-08T06:05:51.415137+0000","last_became_peered":"2024-05-08T06:05:51.415137+0000","last_unstale":"2024-05-08T06:06:32.480218+0000","last_undegraded":"2024-05-08T06:06:32.480218+0000","last_fullsized":"2024-05-08T06:06:32.480218+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:15.007996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480258+0000","last_change":"2024-05-08T06:05:51.422093+0000","last_active":"2024-05-08T06:06:32.480258+0000","last_peered":"2024-05-08T06:06:32.480258+0000","last_clean":"2024-05-08T06:06:32.480258+0000","last_became_active":"2024-05-08T06:05:51.421733+0000","last_became_peered":"2024-05-08T06:05:51.421733+0000","last_unstale":"2024-05-08T06:06:32.480258+0000","last_undegraded":"2024-05-08T06:06:32.480258+0000","last_fullsized":"2024-05-08T06:06:32.480258+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:35:55.950229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.485656+0000","last_change":"2024-05-08T06:05:51.434705+0000","last_active":"2024-05-08T06:06:32.485656+0000","last_peered":"2024-05-08T06:06:32.485656+0000","last_clean":"2024-05-08T06:06:32.485656+0000","last_became_active":"2024-05-08T06:05:51.434324+0000","last_became_peered":"2024-05-08T06:05:51.434324+0000","last_unstale":"2024-05-08T06:06:32.485656+0000","last_undegraded":"2024-05-08T06:06:32.485656+0000","last_fullsized":"2024-05-08T06:06:32.485656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:38:23.101247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.499837+0000","last_change":"2024-05-08T06:06:32.499837+0000","last_active":"2024-05-08T06:06:32.499837+0000","last_peered":"2024-05-08T06:06:32.499837+0000","last_clean":"2024-05-08T06:06:32.499837+0000","last_became_active":"2024-05-08T06:06:32.499026+0000","last_became_peered":"2024-05-08T06:06:32.499026+0000","last_unstale":"2024-05-08T06:06:32.499837+0000","last_undegraded":"2024-05-08T06:06:32.499837+0000","last_fullsized":"2024-05-08T06:06:32.499837+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:48:51.910384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.477627+0000","last_change":"2024-05-08T06:05:51.412370+0000","last_active":"2024-05-08T06:06:32.477627+0000","last_peered":"2024-05-08T06:06:32.477627+0000","last_clean":"2024-05-08T06:06:32.477627+0000","last_became_active":"2024-05-08T06:05:51.412057+0000","last_became_peered":"2024-05-08T06:05:51.412057+0000","last_unstale":"2024-05-08T06:06:32.477627+0000","last_undegraded":"2024-05-08T06:06:32.477627+0000","last_fullsized":"2024-05-08T06:06:32.477627+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:20:51.549926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.481074+0000","last_change":"2024-05-08T06:05:51.408393+0000","last_active":"2024-05-08T06:06:32.481074+0000","last_peered":"2024-05-08T06:06:32.481074+0000","last_clean":"2024-05-08T06:06:32.481074+0000","last_became_active":"2024-05-08T06:05:51.408023+0000","last_became_peered":"2024-05-08T06:05:51.408023+0000","last_unstale":"2024-05-08T06:06:32.481074+0000","last_undegraded":"2024-05-08T06:06:32.481074+0000","last_fullsized":"2024-05-08T06:06:32.481074+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:53.898929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.489822+0000","last_change":"2024-05-08T06:05:51.418055+0000","last_active":"2024-05-08T06:06:32.489822+0000","last_peered":"2024-05-08T06:06:32.489822+0000","last_clean":"2024-05-08T06:06:32.489822+0000","last_became_active":"2024-05-08T06:05:51.417067+0000","last_became_peered":"2024-05-08T06:05:51.417067+0000","last_unstale":"2024-05-08T06:06:32.489822+0000","last_undegraded":"2024-05-08T06:06:32.489822+0000","last_fullsized":"2024-05-08T06:06:32.489822+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:07:35.413672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.595724+0000","last_change":"2024-05-08T06:05:51.421025+0000","last_active":"2024-05-08T06:06:32.595724+0000","last_peered":"2024-05-08T06:06:32.595724+0000","last_clean":"2024-05-08T06:06:32.595724+0000","last_became_active":"2024-05-08T06:05:51.420794+0000","last_became_peered":"2024-05-08T06:05:51.420794+0000","last_unstale":"2024-05-08T06:06:32.595724+0000","last_undegraded":"2024-05-08T06:06:32.595724+0000","last_fullsized":"2024-05-08T06:06:32.595724+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:45:01.852405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.489148+0000","last_change":"2024-05-08T06:05:51.417709+0000","last_active":"2024-05-08T06:06:32.489148+0000","last_peered":"2024-05-08T06:06:32.489148+0000","last_clean":"2024-05-08T06:06:32.489148+0000","last_became_active":"2024-05-08T06:05:51.417337+0000","last_became_peered":"2024-05-08T06:05:51.417337+0000","last_unstale":"2024-05-08T06:06:32.489148+0000","last_undegraded":"2024-05-08T06:06:32.489148+0000","last_fullsized":"2024-05-08T06:06:32.489148+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:32:24.551768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.596197+0000","last_change":"2024-05-08T06:05:51.425318+0000","last_active":"2024-05-08T06:06:32.596197+0000","last_peered":"2024-05-08T06:06:32.596197+0000","last_clean":"2024-05-08T06:06:32.596197+0000","last_became_active":"2024-05-08T06:05:51.424931+0000","last_became_peered":"2024-05-08T06:05:51.424931+0000","last_unstale":"2024-05-08T06:06:32.596197+0000","last_undegraded":"2024-05-08T06:06:32.596197+0000","last_fullsized":"2024-05-08T06:06:32.596197+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:59:45.110201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.492024+0000","last_change":"2024-05-08T06:05:51.433529+0000","last_active":"2024-05-08T06:06:32.492024+0000","last_peered":"2024-05-08T06:06:32.492024+0000","last_clean":"2024-05-08T06:06:32.492024+0000","last_became_active":"2024-05-08T06:05:51.433173+0000","last_became_peered":"2024-05-08T06:05:51.433173+0000","last_unstale":"2024-05-08T06:06:32.492024+0000","last_undegraded":"2024-05-08T06:06:32.492024+0000","last_fullsized":"2024-05-08T06:06:32.492024+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:00:29.042188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.477396+0000","last_change":"2024-05-08T06:05:51.411418+0000","last_active":"2024-05-08T06:06:32.477396+0000","last_peered":"2024-05-08T06:06:32.477396+0000","last_clean":"2024-05-08T06:06:32.477396+0000","last_became_active":"2024-05-08T06:05:51.411084+0000","last_became_peered":"2024-05-08T06:05:51.411084+0000","last_unstale":"2024-05-08T06:06:32.477396+0000","last_undegraded":"2024-05-08T06:06:32.477396+0000","last_fullsized":"2024-05-08T06:06:32.477396+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:08:11.522282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282383,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62816,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":244,"num_read_kb":187,"num_write":165,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":192705,"data_compressed":5508,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645342,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":431859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841571,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":1082490,"data_compressed":37744,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005098,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":897024,"data_stored":432039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201328,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":892928,"data_stored":431937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397556,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":1023889,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561082,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":431428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":490036,"data_compressed":1836,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":432699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":495609,"data_compressed":1836,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117068,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":431739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920838,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":432256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757312,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":1023182,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34281,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34281,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34281,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":58815,"data_compressed":1836,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":64388,"data_compressed":1836,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":60784,"data_compressed":1836,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T06:06:58.418 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T06:06:58.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:58 smithi163 ceph-mon[34520]: pgmap v355: 97 pgs: 97 active+clean; 638 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-05-08T06:06:58.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:06:58 smithi163 ceph-mon[34520]: from='client.14526 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T06:06:58.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:58 smithi192 ceph-mon[34664]: pgmap v355: 97 pgs: 97 active+clean; 638 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-05-08T06:06:58.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:06:58 smithi192 ceph-mon[34664]: from='client.14526 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T06:06:58.901 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:07:00.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:00 smithi123 ceph-mon[29787]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-08T06:07:00.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:00 smithi163 ceph-mon[34520]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-08T06:07:00.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:00 smithi192 ceph-mon[34664]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-05-08T06:07:01.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.349+0000 7f70e8800700 1 -- 172.21.15.123:0/4272363287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70e00756d0 msgr2=0x7f70e0075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:01.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.349+0000 7f70e8800700 1 --2- 172.21.15.123:0/4272363287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70e00756d0 0x7f70e0075ab0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f70d4009990 tx=0x7f70d4009ca0 comp rx=0 tx=0).stop 2024-05-08T06:07:01.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.349+0000 7f70e8800700 1 -- 172.21.15.123:0/4272363287 shutdown_connections 2024-05-08T06:07:01.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.349+0000 7f70e8800700 1 --2- 172.21.15.123:0/4272363287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70e00756d0 0x7f70e0075ab0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:01.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.349+0000 7f70e8800700 1 --2- 172.21.15.123:0/4272363287 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70e010fea0 0x7f70e0112290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:01.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.349+0000 7f70e8800700 1 --2- 172.21.15.123:0/4272363287 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70e0075ff0 0x7f70e010f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:01.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.349+0000 7f70e8800700 1 -- 172.21.15.123:0/4272363287 >> 172.21.15.123:0/4272363287 conn(0x7f70e007a600 msgr2=0x7f70e007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:07:01.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.349+0000 7f70e8800700 1 -- 172.21.15.123:0/4272363287 shutdown_connections 2024-05-08T06:07:01.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.349+0000 7f70e8800700 1 -- 172.21.15.123:0/4272363287 wait complete. 2024-05-08T06:07:01.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.350+0000 7f70e8800700 1 Processor -- start 2024-05-08T06:07:01.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.351+0000 7f70e8800700 1 -- start start 2024-05-08T06:07:01.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.351+0000 7f70e8800700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70e00756d0 0x7f70e019bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:01.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.351+0000 7f70e8800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70e0075ff0 0x7f70e019c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:01.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.351+0000 7f70e8800700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70e010fea0 0x7f70e01a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:01.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.351+0000 7f70e8800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f70e01158c0 con 0x7f70e0075ff0 2024-05-08T06:07:01.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.351+0000 7f70e8800700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f70e0115740 con 0x7f70e010fea0 2024-05-08T06:07:01.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.351+0000 7f70e8800700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f70e0115a40 con 0x7f70e00756d0 2024-05-08T06:07:01.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.351+0000 7f70e5400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70e0075ff0 0x7f70e019c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:01.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.351+0000 7f70e5e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70e00756d0 0x7f70e019bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:01.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.351+0000 7f70e5400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70e0075ff0 0x7f70e019c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56420/0 (socket says 172.21.15.123:56420) 2024-05-08T06:07:01.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.351+0000 7f70e5e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70e00756d0 0x7f70e019bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:57704/0 (socket says 172.21.15.123:57704) 2024-05-08T06:07:01.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.351+0000 7f70e5400700 1 -- 172.21.15.123:0/1446945248 learned_addr learned my addr 172.21.15.123:0/1446945248 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:07:01.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.351+0000 7f70e6800700 1 --2- 172.21.15.123:0/1446945248 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70e010fea0 0x7f70e01a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:01.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.352+0000 7f70e5e00700 1 -- 172.21.15.123:0/1446945248 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70e010fea0 msgr2=0x7f70e01a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:01.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.352+0000 7f70e5e00700 1 --2- 172.21.15.123:0/1446945248 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70e010fea0 0x7f70e01a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:01.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.352+0000 7f70e5e00700 1 -- 172.21.15.123:0/1446945248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70e0075ff0 msgr2=0x7f70e019c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:01.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.352+0000 7f70e5e00700 1 --2- 172.21.15.123:0/1446945248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70e0075ff0 0x7f70e019c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:01.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.352+0000 7f70e5e00700 1 -- 172.21.15.123:0/1446945248 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e01a0d60 con 0x7f70e00756d0 2024-05-08T06:07:01.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.352+0000 7f70e5400700 1 --2- 172.21.15.123:0/1446945248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70e0075ff0 0x7f70e019c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:07:01.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.352+0000 7f70e5e00700 1 --2- 172.21.15.123:0/1446945248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70e00756d0 0x7f70e019bd50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f70d40098e0 tx=0x7f70d40057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:01.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.352+0000 7f70daa00700 1 -- 172.21.15.123:0/1446945248 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d4027070 con 0x7f70e00756d0 2024-05-08T06:07:01.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.353+0000 7f70e8800700 1 -- 172.21.15.123:0/1446945248 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70e01a0ff0 con 0x7f70e00756d0 2024-05-08T06:07:01.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.353+0000 7f70daa00700 1 -- 172.21.15.123:0/1446945248 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70d4022470 con 0x7f70e00756d0 2024-05-08T06:07:01.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.353+0000 7f70daa00700 1 -- 172.21.15.123:0/1446945248 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d4017650 con 0x7f70e00756d0 2024-05-08T06:07:01.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.353+0000 7f70e8800700 1 -- 172.21.15.123:0/1446945248 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70e01a88d0 con 0x7f70e00756d0 2024-05-08T06:07:01.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.354+0000 7f70e8800700 1 -- 172.21.15.123:0/1446945248 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70e006a190 con 0x7f70e00756d0 2024-05-08T06:07:01.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.356+0000 7f70daa00700 1 -- 172.21.15.123:0/1446945248 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f70d402a070 con 0x7f70e00756d0 2024-05-08T06:07:01.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.357+0000 7f70daa00700 1 --2- 172.21.15.123:0/1446945248 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f70c80b2910 0x7f70c80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:01.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.358+0000 7f70daa00700 1 -- 172.21.15.123:0/1446945248 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f70d40e30b0 con 0x7f70e00756d0 2024-05-08T06:07:01.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.358+0000 7f70e5400700 1 --2- 172.21.15.123:0/1446945248 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f70c80b2910 0x7f70c80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:01.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.358+0000 7f70e5400700 1 --2- 172.21.15.123:0/1446945248 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f70c80b2910 0x7f70c80b4dd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f70e0111af0 tx=0x7f70d0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:01.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.360+0000 7f70daa00700 1 -- 172.21.15.123:0/1446945248 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f70d4013070 con 0x7f70e00756d0 2024-05-08T06:07:01.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.588+0000 7f70e8800700 1 -- 172.21.15.123:0/1446945248 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f70e019d0b0 con 0x7f70c80b2910 2024-05-08T06:07:01.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.601+0000 7f70daa00700 1 -- 172.21.15.123:0/1446945248 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258934 (secure 0 0 0) 0x7f70e019d0b0 con 0x7f70c80b2910 2024-05-08T06:07:01.606 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:07:01.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.607+0000 7f70e8800700 1 -- 172.21.15.123:0/1446945248 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f70c80b2910 msgr2=0x7f70c80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:01.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.607+0000 7f70e8800700 1 --2- 172.21.15.123:0/1446945248 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f70c80b2910 0x7f70c80b4dd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f70e0111af0 tx=0x7f70d0009260 comp rx=0 tx=0).stop 2024-05-08T06:07:01.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.607+0000 7f70e8800700 1 -- 172.21.15.123:0/1446945248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70e00756d0 msgr2=0x7f70e019bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:01.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.607+0000 7f70e8800700 1 --2- 172.21.15.123:0/1446945248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70e00756d0 0x7f70e019bd50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f70d40098e0 tx=0x7f70d40057c0 comp rx=0 tx=0).stop 2024-05-08T06:07:01.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.607+0000 7f70e8800700 1 -- 172.21.15.123:0/1446945248 shutdown_connections 2024-05-08T06:07:01.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.607+0000 7f70e8800700 1 --2- 172.21.15.123:0/1446945248 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f70c80b2910 0x7f70c80b4dd0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:01.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.607+0000 7f70e8800700 1 --2- 172.21.15.123:0/1446945248 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70e010fea0 0x7f70e01a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:01.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.607+0000 7f70e8800700 1 --2- 172.21.15.123:0/1446945248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70e00756d0 0x7f70e019bd50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:01.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.607+0000 7f70e8800700 1 --2- 172.21.15.123:0/1446945248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70e0075ff0 0x7f70e019c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:01.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.607+0000 7f70e8800700 1 -- 172.21.15.123:0/1446945248 >> 172.21.15.123:0/1446945248 conn(0x7f70e007a600 msgr2=0x7f70e0100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:07:01.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.607+0000 7f70e8800700 1 -- 172.21.15.123:0/1446945248 shutdown_connections 2024-05-08T06:07:01.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:01.607+0000 7f70e8800700 1 -- 172.21.15.123:0/1446945248 wait complete. 2024-05-08T06:07:01.609 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-05-08T06:07:02.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:02 smithi123 ceph-mon[29787]: pgmap v357: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-08T06:07:02.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:02 smithi123 ceph-mon[29787]: from='client.24643 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T06:07:02.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:02 smithi163 ceph-mon[34520]: pgmap v357: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-08T06:07:02.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:02 smithi163 ceph-mon[34520]: from='client.24643 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T06:07:02.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:02 smithi192 ceph-mon[34664]: pgmap v357: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-05-08T06:07:02.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:02 smithi192 ceph-mon[34664]: from='client.24643 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T06:07:02.750 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2024-05-08T06:07:01.068512+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657675,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":351,"num_read_kb":271,"num_write":291,"num_write_kb":2266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":2951896,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":235,"ondisk_log_size":235,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368352,"kb_used_data":10848,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573472,"statfs":{"total":1151940427776,"available":1151563235328,"internally_reserved":0,"allocated":11108352,"data_stored":7148815,"data_compressed":113232,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8705,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":77,"num_read_kb":45,"num_write":6,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005364"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.477766+0000","last_change":"2024-05-08T06:05:51.431744+0000","last_active":"2024-05-08T06:06:32.477766+0000","last_peered":"2024-05-08T06:06:32.477766+0000","last_clean":"2024-05-08T06:06:32.477766+0000","last_became_active":"2024-05-08T06:05:51.431405+0000","last_became_peered":"2024-05-08T06:05:51.431405+0000","last_unstale":"2024-05-08T06:06:32.477766+0000","last_undegraded":"2024-05-08T06:06:32.477766+0000","last_fullsized":"2024-05-08T06:06:32.477766+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:56:40.503607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.488683+0000","last_change":"2024-05-08T06:05:51.426473+0000","last_active":"2024-05-08T06:06:32.488683+0000","last_peered":"2024-05-08T06:06:32.488683+0000","last_clean":"2024-05-08T06:06:32.488683+0000","last_became_active":"2024-05-08T06:05:51.424477+0000","last_became_peered":"2024-05-08T06:05:51.424477+0000","last_unstale":"2024-05-08T06:06:32.488683+0000","last_undegraded":"2024-05-08T06:06:32.488683+0000","last_fullsized":"2024-05-08T06:06:32.488683+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:26:55.747808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.489112+0000","last_change":"2024-05-08T06:05:51.423767+0000","last_active":"2024-05-08T06:06:32.489112+0000","last_peered":"2024-05-08T06:06:32.489112+0000","last_clean":"2024-05-08T06:06:32.489112+0000","last_became_active":"2024-05-08T06:05:51.423273+0000","last_became_peered":"2024-05-08T06:05:51.423273+0000","last_unstale":"2024-05-08T06:06:32.489112+0000","last_undegraded":"2024-05-08T06:06:32.489112+0000","last_fullsized":"2024-05-08T06:06:32.489112+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:24:44.099743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.597288+0000","last_change":"2024-05-08T06:05:51.416604+0000","last_active":"2024-05-08T06:06:32.597288+0000","last_peered":"2024-05-08T06:06:32.597288+0000","last_clean":"2024-05-08T06:06:32.597288+0000","last_became_active":"2024-05-08T06:05:51.416284+0000","last_became_peered":"2024-05-08T06:05:51.416284+0000","last_unstale":"2024-05-08T06:06:32.597288+0000","last_undegraded":"2024-05-08T06:06:32.597288+0000","last_fullsized":"2024-05-08T06:06:32.597288+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:05:26.014323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.479834+0000","last_change":"2024-05-08T06:05:51.423915+0000","last_active":"2024-05-08T06:06:32.479834+0000","last_peered":"2024-05-08T06:06:32.479834+0000","last_clean":"2024-05-08T06:06:32.479834+0000","last_became_active":"2024-05-08T06:05:51.423534+0000","last_became_peered":"2024-05-08T06:05:51.423534+0000","last_unstale":"2024-05-08T06:06:32.479834+0000","last_undegraded":"2024-05-08T06:06:32.479834+0000","last_fullsized":"2024-05-08T06:06:32.479834+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:06:46.197071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.477712+0000","last_change":"2024-05-08T06:05:51.411102+0000","last_active":"2024-05-08T06:06:32.477712+0000","last_peered":"2024-05-08T06:06:32.477712+0000","last_clean":"2024-05-08T06:06:32.477712+0000","last_became_active":"2024-05-08T06:05:51.410791+0000","last_became_peered":"2024-05-08T06:05:51.410791+0000","last_unstale":"2024-05-08T06:06:32.477712+0000","last_undegraded":"2024-05-08T06:06:32.477712+0000","last_fullsized":"2024-05-08T06:06:32.477712+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:31:47.861663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482197+0000","last_change":"2024-05-08T06:05:51.417756+0000","last_active":"2024-05-08T06:06:32.482197+0000","last_peered":"2024-05-08T06:06:32.482197+0000","last_clean":"2024-05-08T06:06:32.482197+0000","last_became_active":"2024-05-08T06:05:51.417373+0000","last_became_peered":"2024-05-08T06:05:51.417373+0000","last_unstale":"2024-05-08T06:06:32.482197+0000","last_undegraded":"2024-05-08T06:06:32.482197+0000","last_fullsized":"2024-05-08T06:06:32.482197+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:07:48.330401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.489273+0000","last_change":"2024-05-08T06:05:51.413883+0000","last_active":"2024-05-08T06:06:32.489273+0000","last_peered":"2024-05-08T06:06:32.489273+0000","last_clean":"2024-05-08T06:06:32.489273+0000","last_became_active":"2024-05-08T06:05:51.413565+0000","last_became_peered":"2024-05-08T06:05:51.413565+0000","last_unstale":"2024-05-08T06:06:32.489273+0000","last_undegraded":"2024-05-08T06:06:32.489273+0000","last_fullsized":"2024-05-08T06:06:32.489273+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:06:38.186225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.596722+0000","last_change":"2024-05-08T06:05:51.416300+0000","last_active":"2024-05-08T06:06:32.596722+0000","last_peered":"2024-05-08T06:06:32.596722+0000","last_clean":"2024-05-08T06:06:32.596722+0000","last_became_active":"2024-05-08T06:05:51.415970+0000","last_became_peered":"2024-05-08T06:05:51.415970+0000","last_unstale":"2024-05-08T06:06:32.596722+0000","last_undegraded":"2024-05-08T06:06:32.596722+0000","last_fullsized":"2024-05-08T06:06:32.596722+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:44:51.853339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.477997+0000","last_change":"2024-05-08T06:05:51.410396+0000","last_active":"2024-05-08T06:06:32.477997+0000","last_peered":"2024-05-08T06:06:32.477997+0000","last_clean":"2024-05-08T06:06:32.477997+0000","last_became_active":"2024-05-08T06:05:51.410067+0000","last_became_peered":"2024-05-08T06:05:51.410067+0000","last_unstale":"2024-05-08T06:06:32.477997+0000","last_undegraded":"2024-05-08T06:06:32.477997+0000","last_fullsized":"2024-05-08T06:06:32.477997+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:32:00.217217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482522+0000","last_change":"2024-05-08T06:05:51.409276+0000","last_active":"2024-05-08T06:06:32.482522+0000","last_peered":"2024-05-08T06:06:32.482522+0000","last_clean":"2024-05-08T06:06:32.482522+0000","last_became_active":"2024-05-08T06:05:51.408768+0000","last_became_peered":"2024-05-08T06:05:51.408768+0000","last_unstale":"2024-05-08T06:06:32.482522+0000","last_undegraded":"2024-05-08T06:06:32.482522+0000","last_fullsized":"2024-05-08T06:06:32.482522+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:22:57.483944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.597286+0000","last_change":"2024-05-08T06:05:51.419083+0000","last_active":"2024-05-08T06:06:32.597286+0000","last_peered":"2024-05-08T06:06:32.597286+0000","last_clean":"2024-05-08T06:06:32.597286+0000","last_became_active":"2024-05-08T06:05:51.418685+0000","last_became_peered":"2024-05-08T06:05:51.418685+0000","last_unstale":"2024-05-08T06:06:32.597286+0000","last_undegraded":"2024-05-08T06:06:32.597286+0000","last_fullsized":"2024-05-08T06:06:32.597286+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:25:49.256242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.491600+0000","last_change":"2024-05-08T06:05:51.420399+0000","last_active":"2024-05-08T06:06:32.491600+0000","last_peered":"2024-05-08T06:06:32.491600+0000","last_clean":"2024-05-08T06:06:32.491600+0000","last_became_active":"2024-05-08T06:05:51.420076+0000","last_became_peered":"2024-05-08T06:05:51.420076+0000","last_unstale":"2024-05-08T06:06:32.491600+0000","last_undegraded":"2024-05-08T06:06:32.491600+0000","last_fullsized":"2024-05-08T06:06:32.491600+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:51:38.579280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.484011+0000","last_change":"2024-05-08T06:05:51.431518+0000","last_active":"2024-05-08T06:06:32.484011+0000","last_peered":"2024-05-08T06:06:32.484011+0000","last_clean":"2024-05-08T06:06:32.484011+0000","last_became_active":"2024-05-08T06:05:51.431029+0000","last_became_peered":"2024-05-08T06:05:51.431029+0000","last_unstale":"2024-05-08T06:06:32.484011+0000","last_undegraded":"2024-05-08T06:06:32.484011+0000","last_fullsized":"2024-05-08T06:06:32.484011+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:11:09.596221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.498002+0000","last_change":"2024-05-08T06:06:32.498002+0000","last_active":"2024-05-08T06:06:32.498002+0000","last_peered":"2024-05-08T06:06:32.498002+0000","last_clean":"2024-05-08T06:06:32.498002+0000","last_became_active":"2024-05-08T06:06:32.497367+0000","last_became_peered":"2024-05-08T06:06:32.497367+0000","last_unstale":"2024-05-08T06:06:32.498002+0000","last_undegraded":"2024-05-08T06:06:32.498002+0000","last_fullsized":"2024-05-08T06:06:32.498002+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:32:36.461777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.595891+0000","last_change":"2024-05-08T06:05:51.423857+0000","last_active":"2024-05-08T06:06:32.595891+0000","last_peered":"2024-05-08T06:06:32.595891+0000","last_clean":"2024-05-08T06:06:32.595891+0000","last_became_active":"2024-05-08T06:05:51.423487+0000","last_became_peered":"2024-05-08T06:05:51.423487+0000","last_unstale":"2024-05-08T06:06:32.595891+0000","last_undegraded":"2024-05-08T06:06:32.595891+0000","last_fullsized":"2024-05-08T06:06:32.595891+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:17:33.269354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.477873+0000","last_change":"2024-05-08T06:05:51.431795+0000","last_active":"2024-05-08T06:06:32.477873+0000","last_peered":"2024-05-08T06:06:32.477873+0000","last_clean":"2024-05-08T06:06:32.477873+0000","last_became_active":"2024-05-08T06:05:51.431432+0000","last_became_peered":"2024-05-08T06:05:51.431432+0000","last_unstale":"2024-05-08T06:06:32.477873+0000","last_undegraded":"2024-05-08T06:06:32.477873+0000","last_fullsized":"2024-05-08T06:06:32.477873+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:11:16.561478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.484127+0000","last_change":"2024-05-08T06:05:51.414805+0000","last_active":"2024-05-08T06:06:32.484127+0000","last_peered":"2024-05-08T06:06:32.484127+0000","last_clean":"2024-05-08T06:06:32.484127+0000","last_became_active":"2024-05-08T06:05:51.414482+0000","last_became_peered":"2024-05-08T06:05:51.414482+0000","last_unstale":"2024-05-08T06:06:32.484127+0000","last_undegraded":"2024-05-08T06:06:32.484127+0000","last_fullsized":"2024-05-08T06:06:32.484127+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:08:13.444591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480668+0000","last_change":"2024-05-08T06:05:51.418516+0000","last_active":"2024-05-08T06:06:32.480668+0000","last_peered":"2024-05-08T06:06:32.480668+0000","last_clean":"2024-05-08T06:06:32.480668+0000","last_became_active":"2024-05-08T06:05:51.418182+0000","last_became_peered":"2024-05-08T06:05:51.418182+0000","last_unstale":"2024-05-08T06:06:32.480668+0000","last_undegraded":"2024-05-08T06:06:32.480668+0000","last_fullsized":"2024-05-08T06:06:32.480668+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:32:33.338311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482309+0000","last_change":"2024-05-08T06:05:46.350509+0000","last_active":"2024-05-08T06:06:32.482309+0000","last_peered":"2024-05-08T06:06:32.482309+0000","last_clean":"2024-05-08T06:06:32.482309+0000","last_became_active":"2024-05-08T06:05:46.349986+0000","last_became_peered":"2024-05-08T06:05:46.349986+0000","last_unstale":"2024-05-08T06:06:32.482309+0000","last_undegraded":"2024-05-08T06:06:32.482309+0000","last_fullsized":"2024-05-08T06:06:32.482309+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:53:00.928971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.492031+0000","last_change":"2024-05-08T06:05:51.413621+0000","last_active":"2024-05-08T06:06:32.492031+0000","last_peered":"2024-05-08T06:06:32.492031+0000","last_clean":"2024-05-08T06:06:32.492031+0000","last_became_active":"2024-05-08T06:05:51.413188+0000","last_became_peered":"2024-05-08T06:05:51.413188+0000","last_unstale":"2024-05-08T06:06:32.492031+0000","last_undegraded":"2024-05-08T06:06:32.492031+0000","last_fullsized":"2024-05-08T06:06:32.492031+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:13:21.413696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.492646+0000","last_change":"2024-05-08T06:05:46.552444+0000","last_active":"2024-05-08T06:06:32.492646+0000","last_peered":"2024-05-08T06:06:32.492646+0000","last_clean":"2024-05-08T06:06:32.492646+0000","last_became_active":"2024-05-08T06:05:46.551310+0000","last_became_peered":"2024-05-08T06:05:46.551310+0000","last_unstale":"2024-05-08T06:06:32.492646+0000","last_undegraded":"2024-05-08T06:06:32.492646+0000","last_fullsized":"2024-05-08T06:06:32.492646+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:58:17.632832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.487033+0000","last_change":"2024-05-08T06:05:51.425649+0000","last_active":"2024-05-08T06:06:32.487033+0000","last_peered":"2024-05-08T06:06:32.487033+0000","last_clean":"2024-05-08T06:06:32.487033+0000","last_became_active":"2024-05-08T06:05:51.425288+0000","last_became_peered":"2024-05-08T06:05:51.425288+0000","last_unstale":"2024-05-08T06:06:32.487033+0000","last_undegraded":"2024-05-08T06:06:32.487033+0000","last_fullsized":"2024-05-08T06:06:32.487033+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:42:40.580641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480600+0000","last_change":"2024-05-08T06:05:46.350300+0000","last_active":"2024-05-08T06:06:32.480600+0000","last_peered":"2024-05-08T06:06:32.480600+0000","last_clean":"2024-05-08T06:06:32.480600+0000","last_became_active":"2024-05-08T06:05:46.349918+0000","last_became_peered":"2024-05-08T06:05:46.349918+0000","last_unstale":"2024-05-08T06:06:32.480600+0000","last_undegraded":"2024-05-08T06:06:32.480600+0000","last_fullsized":"2024-05-08T06:06:32.480600+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:32:41.952772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.596199+0000","last_change":"2024-05-08T06:05:51.430130+0000","last_active":"2024-05-08T06:06:32.596199+0000","last_peered":"2024-05-08T06:06:32.596199+0000","last_clean":"2024-05-08T06:06:32.596199+0000","last_became_active":"2024-05-08T06:05:51.429794+0000","last_became_peered":"2024-05-08T06:05:51.429794+0000","last_unstale":"2024-05-08T06:06:32.596199+0000","last_undegraded":"2024-05-08T06:06:32.596199+0000","last_fullsized":"2024-05-08T06:06:32.596199+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:21:55.773309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480488+0000","last_change":"2024-05-08T06:05:46.351664+0000","last_active":"2024-05-08T06:06:32.480488+0000","last_peered":"2024-05-08T06:06:32.480488+0000","last_clean":"2024-05-08T06:06:32.480488+0000","last_became_active":"2024-05-08T06:05:46.351305+0000","last_became_peered":"2024-05-08T06:05:46.351305+0000","last_unstale":"2024-05-08T06:06:32.480488+0000","last_undegraded":"2024-05-08T06:06:32.480488+0000","last_fullsized":"2024-05-08T06:06:32.480488+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:24:36.164315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.596189+0000","last_change":"2024-05-08T06:05:51.414793+0000","last_active":"2024-05-08T06:06:32.596189+0000","last_peered":"2024-05-08T06:06:32.596189+0000","last_clean":"2024-05-08T06:06:32.596189+0000","last_became_active":"2024-05-08T06:05:51.414317+0000","last_became_peered":"2024-05-08T06:05:51.414317+0000","last_unstale":"2024-05-08T06:06:32.596189+0000","last_undegraded":"2024-05-08T06:06:32.596189+0000","last_fullsized":"2024-05-08T06:06:32.596189+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:57:56.152836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":75,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:58.453156+0000","last_change":"2024-05-08T06:05:46.352683+0000","last_active":"2024-05-08T06:06:58.453156+0000","last_peered":"2024-05-08T06:06:58.453156+0000","last_clean":"2024-05-08T06:06:58.453156+0000","last_became_active":"2024-05-08T06:05:46.352278+0000","last_became_peered":"2024-05-08T06:05:46.352278+0000","last_unstale":"2024-05-08T06:06:58.453156+0000","last_undegraded":"2024-05-08T06:06:58.453156+0000","last_fullsized":"2024-05-08T06:06:58.453156+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:39:22.655516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":44,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480098+0000","last_change":"2024-05-08T06:05:51.415822+0000","last_active":"2024-05-08T06:06:32.480098+0000","last_peered":"2024-05-08T06:06:32.480098+0000","last_clean":"2024-05-08T06:06:32.480098+0000","last_became_active":"2024-05-08T06:05:51.415449+0000","last_became_peered":"2024-05-08T06:05:51.415449+0000","last_unstale":"2024-05-08T06:06:32.480098+0000","last_undegraded":"2024-05-08T06:06:32.480098+0000","last_fullsized":"2024-05-08T06:06:32.480098+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:44:49.127335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.477620+0000","last_change":"2024-05-08T06:05:46.361288+0000","last_active":"2024-05-08T06:06:32.477620+0000","last_peered":"2024-05-08T06:06:32.477620+0000","last_clean":"2024-05-08T06:06:32.477620+0000","last_became_active":"2024-05-08T06:05:46.360761+0000","last_became_peered":"2024-05-08T06:05:46.360761+0000","last_unstale":"2024-05-08T06:06:32.477620+0000","last_undegraded":"2024-05-08T06:06:32.477620+0000","last_fullsized":"2024-05-08T06:06:32.477620+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:14:59.858935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.596064+0000","last_change":"2024-05-08T06:05:51.418547+0000","last_active":"2024-05-08T06:06:32.596064+0000","last_peered":"2024-05-08T06:06:32.596064+0000","last_clean":"2024-05-08T06:06:32.596064+0000","last_became_active":"2024-05-08T06:05:51.417588+0000","last_became_peered":"2024-05-08T06:05:51.417588+0000","last_unstale":"2024-05-08T06:06:32.596064+0000","last_undegraded":"2024-05-08T06:06:32.596064+0000","last_fullsized":"2024-05-08T06:06:32.596064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:16:41.451351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.491600+0000","last_change":"2024-05-08T06:05:46.362077+0000","last_active":"2024-05-08T06:06:32.491600+0000","last_peered":"2024-05-08T06:06:32.491600+0000","last_clean":"2024-05-08T06:06:32.491600+0000","last_became_active":"2024-05-08T06:05:46.360823+0000","last_became_peered":"2024-05-08T06:05:46.360823+0000","last_unstale":"2024-05-08T06:06:32.491600+0000","last_undegraded":"2024-05-08T06:06:32.491600+0000","last_fullsized":"2024-05-08T06:06:32.491600+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:14.169822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.488967+0000","last_change":"2024-05-08T06:05:51.423687+0000","last_active":"2024-05-08T06:06:32.488967+0000","last_peered":"2024-05-08T06:06:32.488967+0000","last_clean":"2024-05-08T06:06:32.488967+0000","last_became_active":"2024-05-08T06:05:51.417656+0000","last_became_peered":"2024-05-08T06:05:51.417656+0000","last_unstale":"2024-05-08T06:06:32.488967+0000","last_undegraded":"2024-05-08T06:06:32.488967+0000","last_fullsized":"2024-05-08T06:06:32.488967+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:59:31.319915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.595724+0000","last_change":"2024-05-08T06:05:46.351892+0000","last_active":"2024-05-08T06:06:32.595724+0000","last_peered":"2024-05-08T06:06:32.595724+0000","last_clean":"2024-05-08T06:06:32.595724+0000","last_became_active":"2024-05-08T06:05:46.351523+0000","last_became_peered":"2024-05-08T06:05:46.351523+0000","last_unstale":"2024-05-08T06:06:32.595724+0000","last_undegraded":"2024-05-08T06:06:32.595724+0000","last_fullsized":"2024-05-08T06:06:32.595724+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:11:23.794313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.483084+0000","last_change":"2024-05-08T06:05:51.407681+0000","last_active":"2024-05-08T06:06:32.483084+0000","last_peered":"2024-05-08T06:06:32.483084+0000","last_clean":"2024-05-08T06:06:32.483084+0000","last_became_active":"2024-05-08T06:05:51.407243+0000","last_became_peered":"2024-05-08T06:05:51.407243+0000","last_unstale":"2024-05-08T06:06:32.483084+0000","last_undegraded":"2024-05-08T06:06:32.483084+0000","last_fullsized":"2024-05-08T06:06:32.483084+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:06:30.490268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'3","reported_seq":69,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:56.449405+0000","last_change":"2024-05-08T06:05:46.356675+0000","last_active":"2024-05-08T06:06:56.449405+0000","last_peered":"2024-05-08T06:06:56.449405+0000","last_clean":"2024-05-08T06:06:56.449405+0000","last_became_active":"2024-05-08T06:05:46.356360+0000","last_became_peered":"2024-05-08T06:05:46.356360+0000","last_unstale":"2024-05-08T06:06:56.449405+0000","last_undegraded":"2024-05-08T06:06:56.449405+0000","last_fullsized":"2024-05-08T06:06:56.449405+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:46:48.131551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.596564+0000","last_change":"2024-05-08T06:05:51.421325+0000","last_active":"2024-05-08T06:06:32.596564+0000","last_peered":"2024-05-08T06:06:32.596564+0000","last_clean":"2024-05-08T06:06:32.596564+0000","last_became_active":"2024-05-08T06:05:51.420905+0000","last_became_peered":"2024-05-08T06:05:51.420905+0000","last_unstale":"2024-05-08T06:06:32.596564+0000","last_undegraded":"2024-05-08T06:06:32.596564+0000","last_fullsized":"2024-05-08T06:06:32.596564+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:37:27.324920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482674+0000","last_change":"2024-05-08T06:05:46.351958+0000","last_active":"2024-05-08T06:06:32.482674+0000","last_peered":"2024-05-08T06:06:32.482674+0000","last_clean":"2024-05-08T06:06:32.482674+0000","last_became_active":"2024-05-08T06:05:46.351561+0000","last_became_peered":"2024-05-08T06:05:46.351561+0000","last_unstale":"2024-05-08T06:06:32.482674+0000","last_undegraded":"2024-05-08T06:06:32.482674+0000","last_fullsized":"2024-05-08T06:06:32.482674+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:11:27.727029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.486739+0000","last_change":"2024-05-08T06:05:51.431809+0000","last_active":"2024-05-08T06:06:32.486739+0000","last_peered":"2024-05-08T06:06:32.486739+0000","last_clean":"2024-05-08T06:06:32.486739+0000","last_became_active":"2024-05-08T06:05:51.431416+0000","last_became_peered":"2024-05-08T06:05:51.431416+0000","last_unstale":"2024-05-08T06:06:32.486739+0000","last_undegraded":"2024-05-08T06:06:32.486739+0000","last_fullsized":"2024-05-08T06:06:32.486739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:48:16.704134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.596297+0000","last_change":"2024-05-08T06:05:46.550763+0000","last_active":"2024-05-08T06:06:32.596297+0000","last_peered":"2024-05-08T06:06:32.596297+0000","last_clean":"2024-05-08T06:06:32.596297+0000","last_became_active":"2024-05-08T06:05:46.550297+0000","last_became_peered":"2024-05-08T06:05:46.550297+0000","last_unstale":"2024-05-08T06:06:32.596297+0000","last_undegraded":"2024-05-08T06:06:32.596297+0000","last_fullsized":"2024-05-08T06:06:32.596297+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:32:19.907220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.596339+0000","last_change":"2024-05-08T06:05:51.414738+0000","last_active":"2024-05-08T06:06:32.596339+0000","last_peered":"2024-05-08T06:06:32.596339+0000","last_clean":"2024-05-08T06:06:32.596339+0000","last_became_active":"2024-05-08T06:05:51.414369+0000","last_became_peered":"2024-05-08T06:05:51.414369+0000","last_unstale":"2024-05-08T06:06:32.596339+0000","last_undegraded":"2024-05-08T06:06:32.596339+0000","last_fullsized":"2024-05-08T06:06:32.596339+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:57:44.215343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'30","reported_seq":145,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:57.451011+0000","last_change":"2024-05-08T06:05:46.349825+0000","last_active":"2024-05-08T06:06:57.451011+0000","last_peered":"2024-05-08T06:06:57.451011+0000","last_clean":"2024-05-08T06:06:57.451011+0000","last_became_active":"2024-05-08T06:05:46.349425+0000","last_became_peered":"2024-05-08T06:05:46.349425+0000","last_unstale":"2024-05-08T06:06:57.451011+0000","last_undegraded":"2024-05-08T06:06:57.451011+0000","last_fullsized":"2024-05-08T06:06:57.451011+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:29:39.123817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63072,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":89,"num_read_kb":106,"num_write":31,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480389+0000","last_change":"2024-05-08T06:05:51.411341+0000","last_active":"2024-05-08T06:06:32.480389+0000","last_peered":"2024-05-08T06:06:32.480389+0000","last_clean":"2024-05-08T06:06:32.480389+0000","last_became_active":"2024-05-08T06:05:51.411002+0000","last_became_peered":"2024-05-08T06:05:51.411002+0000","last_unstale":"2024-05-08T06:06:32.480389+0000","last_undegraded":"2024-05-08T06:06:32.480389+0000","last_fullsized":"2024-05-08T06:06:32.480389+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:27:07.013809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.509330+0000","last_change":"2024-05-08T06:06:32.508904+0000","last_active":"2024-05-08T06:06:32.509330+0000","last_peered":"2024-05-08T06:06:32.509330+0000","last_clean":"2024-05-08T06:06:32.509330+0000","last_became_active":"2024-05-08T06:06:32.496001+0000","last_became_peered":"2024-05-08T06:06:32.496001+0000","last_unstale":"2024-05-08T06:06:32.509330+0000","last_undegraded":"2024-05-08T06:06:32.509330+0000","last_fullsized":"2024-05-08T06:06:32.509330+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:34:32.417105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.491251+0000","last_change":"2024-05-08T06:05:51.418395+0000","last_active":"2024-05-08T06:06:32.491251+0000","last_peered":"2024-05-08T06:06:32.491251+0000","last_clean":"2024-05-08T06:06:32.491251+0000","last_became_active":"2024-05-08T06:05:51.418035+0000","last_became_peered":"2024-05-08T06:05:51.418035+0000","last_unstale":"2024-05-08T06:06:32.491251+0000","last_undegraded":"2024-05-08T06:06:32.491251+0000","last_fullsized":"2024-05-08T06:06:32.491251+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:58:16.529144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.595936+0000","last_change":"2024-05-08T06:05:46.363088+0000","last_active":"2024-05-08T06:06:32.595936+0000","last_peered":"2024-05-08T06:06:32.595936+0000","last_clean":"2024-05-08T06:06:32.595936+0000","last_became_active":"2024-05-08T06:05:46.362488+0000","last_became_peered":"2024-05-08T06:05:46.362488+0000","last_unstale":"2024-05-08T06:06:32.595936+0000","last_undegraded":"2024-05-08T06:06:32.595936+0000","last_fullsized":"2024-05-08T06:06:32.595936+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:28:10.039647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482481+0000","last_change":"2024-05-08T06:05:51.415567+0000","last_active":"2024-05-08T06:06:32.482481+0000","last_peered":"2024-05-08T06:06:32.482481+0000","last_clean":"2024-05-08T06:06:32.482481+0000","last_became_active":"2024-05-08T06:05:51.415148+0000","last_became_peered":"2024-05-08T06:05:51.415148+0000","last_unstale":"2024-05-08T06:06:32.482481+0000","last_undegraded":"2024-05-08T06:06:32.482481+0000","last_fullsized":"2024-05-08T06:06:32.482481+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:51:48.871149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'4","reported_seq":76,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:59.482839+0000","last_change":"2024-05-08T06:05:46.348769+0000","last_active":"2024-05-08T06:06:59.482839+0000","last_peered":"2024-05-08T06:06:59.482839+0000","last_clean":"2024-05-08T06:06:59.482839+0000","last_became_active":"2024-05-08T06:05:46.348072+0000","last_became_peered":"2024-05-08T06:05:46.348072+0000","last_unstale":"2024-05-08T06:06:59.482839+0000","last_undegraded":"2024-05-08T06:06:59.482839+0000","last_fullsized":"2024-05-08T06:06:59.482839+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:45.292436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":47,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482108+0000","last_change":"2024-05-08T06:05:51.414623+0000","last_active":"2024-05-08T06:06:32.482108+0000","last_peered":"2024-05-08T06:06:32.482108+0000","last_clean":"2024-05-08T06:06:32.482108+0000","last_became_active":"2024-05-08T06:05:51.414258+0000","last_became_peered":"2024-05-08T06:05:51.414258+0000","last_unstale":"2024-05-08T06:06:32.482108+0000","last_undegraded":"2024-05-08T06:06:32.482108+0000","last_fullsized":"2024-05-08T06:06:32.482108+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:38:48.297900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.479966+0000","last_change":"2024-05-08T06:05:46.546936+0000","last_active":"2024-05-08T06:06:32.479966+0000","last_peered":"2024-05-08T06:06:32.479966+0000","last_clean":"2024-05-08T06:06:32.479966+0000","last_became_active":"2024-05-08T06:05:46.546574+0000","last_became_peered":"2024-05-08T06:05:46.546574+0000","last_unstale":"2024-05-08T06:06:32.479966+0000","last_undegraded":"2024-05-08T06:06:32.479966+0000","last_fullsized":"2024-05-08T06:06:32.479966+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:16:29.720304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.595824+0000","last_change":"2024-05-08T06:05:51.418012+0000","last_active":"2024-05-08T06:06:32.595824+0000","last_peered":"2024-05-08T06:06:32.595824+0000","last_clean":"2024-05-08T06:06:32.595824+0000","last_became_active":"2024-05-08T06:05:51.417646+0000","last_became_peered":"2024-05-08T06:05:51.417646+0000","last_unstale":"2024-05-08T06:06:32.595824+0000","last_undegraded":"2024-05-08T06:06:32.595824+0000","last_fullsized":"2024-05-08T06:06:32.595824+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:43:02.733967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480019+0000","last_change":"2024-05-08T06:05:46.347748+0000","last_active":"2024-05-08T06:06:32.480019+0000","last_peered":"2024-05-08T06:06:32.480019+0000","last_clean":"2024-05-08T06:06:32.480019+0000","last_became_active":"2024-05-08T06:05:46.347367+0000","last_became_peered":"2024-05-08T06:05:46.347367+0000","last_unstale":"2024-05-08T06:06:32.480019+0000","last_undegraded":"2024-05-08T06:06:32.480019+0000","last_fullsized":"2024-05-08T06:06:32.480019+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:51:50.973855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.597018+0000","last_change":"2024-05-08T06:06:32.597018+0000","last_active":"2024-05-08T06:06:32.597018+0000","last_peered":"2024-05-08T06:06:32.597018+0000","last_clean":"2024-05-08T06:06:32.597018+0000","last_became_active":"2024-05-08T06:06:32.596677+0000","last_became_peered":"2024-05-08T06:06:32.596677+0000","last_unstale":"2024-05-08T06:06:32.597018+0000","last_undegraded":"2024-05-08T06:06:32.597018+0000","last_fullsized":"2024-05-08T06:06:32.597018+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:02:46.223949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.481565+0000","last_change":"2024-05-08T06:05:46.554443+0000","last_active":"2024-05-08T06:06:32.481565+0000","last_peered":"2024-05-08T06:06:32.481565+0000","last_clean":"2024-05-08T06:06:32.481565+0000","last_became_active":"2024-05-08T06:05:46.553226+0000","last_became_peered":"2024-05-08T06:05:46.553226+0000","last_unstale":"2024-05-08T06:06:32.481565+0000","last_undegraded":"2024-05-08T06:06:32.481565+0000","last_fullsized":"2024-05-08T06:06:32.481565+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:56:39.784652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.477829+0000","last_change":"2024-05-08T06:05:51.413121+0000","last_active":"2024-05-08T06:06:32.477829+0000","last_peered":"2024-05-08T06:06:32.477829+0000","last_clean":"2024-05-08T06:06:32.477829+0000","last_became_active":"2024-05-08T06:05:51.412767+0000","last_became_peered":"2024-05-08T06:05:51.412767+0000","last_unstale":"2024-05-08T06:06:32.477829+0000","last_undegraded":"2024-05-08T06:06:32.477829+0000","last_fullsized":"2024-05-08T06:06:32.477829+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:59:13.416981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482351+0000","last_change":"2024-05-08T06:05:46.361349+0000","last_active":"2024-05-08T06:06:32.482351+0000","last_peered":"2024-05-08T06:06:32.482351+0000","last_clean":"2024-05-08T06:06:32.482351+0000","last_became_active":"2024-05-08T06:05:46.360654+0000","last_became_peered":"2024-05-08T06:05:46.360654+0000","last_unstale":"2024-05-08T06:06:32.482351+0000","last_undegraded":"2024-05-08T06:06:32.482351+0000","last_fullsized":"2024-05-08T06:06:32.482351+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:19:22.144874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.477758+0000","last_change":"2024-05-08T06:05:51.423836+0000","last_active":"2024-05-08T06:06:32.477758+0000","last_peered":"2024-05-08T06:06:32.477758+0000","last_clean":"2024-05-08T06:06:32.477758+0000","last_became_active":"2024-05-08T06:05:51.423424+0000","last_became_peered":"2024-05-08T06:05:51.423424+0000","last_unstale":"2024-05-08T06:06:32.477758+0000","last_undegraded":"2024-05-08T06:06:32.477758+0000","last_fullsized":"2024-05-08T06:06:32.477758+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:56:14.779765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.598978+0000","last_change":"2024-05-08T06:06:32.598978+0000","last_active":"2024-05-08T06:06:32.598978+0000","last_peered":"2024-05-08T06:06:32.598978+0000","last_clean":"2024-05-08T06:06:32.598978+0000","last_became_active":"2024-05-08T06:06:32.598158+0000","last_became_peered":"2024-05-08T06:06:32.598158+0000","last_unstale":"2024-05-08T06:06:32.598978+0000","last_undegraded":"2024-05-08T06:06:32.598978+0000","last_fullsized":"2024-05-08T06:06:32.598978+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:51:09.270066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":73,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:59.482080+0000","last_change":"2024-05-08T06:05:46.547450+0000","last_active":"2024-05-08T06:06:59.482080+0000","last_peered":"2024-05-08T06:06:59.482080+0000","last_clean":"2024-05-08T06:06:59.482080+0000","last_became_active":"2024-05-08T06:05:46.546444+0000","last_became_peered":"2024-05-08T06:05:46.546444+0000","last_unstale":"2024-05-08T06:06:59.482080+0000","last_undegraded":"2024-05-08T06:06:59.482080+0000","last_fullsized":"2024-05-08T06:06:59.482080+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:17:14.124224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":43,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.489451+0000","last_change":"2024-05-08T06:05:51.423191+0000","last_active":"2024-05-08T06:06:32.489451+0000","last_peered":"2024-05-08T06:06:32.489451+0000","last_clean":"2024-05-08T06:06:32.489451+0000","last_became_active":"2024-05-08T06:05:51.420912+0000","last_became_peered":"2024-05-08T06:05:51.420912+0000","last_unstale":"2024-05-08T06:06:32.489451+0000","last_undegraded":"2024-05-08T06:06:32.489451+0000","last_fullsized":"2024-05-08T06:06:32.489451+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:27:57.678527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.597268+0000","last_change":"2024-05-08T06:05:46.348650+0000","last_active":"2024-05-08T06:06:32.597268+0000","last_peered":"2024-05-08T06:06:32.597268+0000","last_clean":"2024-05-08T06:06:32.597268+0000","last_became_active":"2024-05-08T06:05:46.347266+0000","last_became_peered":"2024-05-08T06:05:46.347266+0000","last_unstale":"2024-05-08T06:06:32.597268+0000","last_undegraded":"2024-05-08T06:06:32.597268+0000","last_fullsized":"2024-05-08T06:06:32.597268+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:37:30.623580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":100,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482311+0000","last_change":"2024-05-08T06:04:09.404663+0000","last_active":"2024-05-08T06:06:32.482311+0000","last_peered":"2024-05-08T06:06:32.482311+0000","last_clean":"2024-05-08T06:06:32.482311+0000","last_became_active":"2024-05-08T06:04:09.400115+0000","last_became_peered":"2024-05-08T06:04:09.400115+0000","last_unstale":"2024-05-08T06:06:32.482311+0000","last_undegraded":"2024-05-08T06:06:32.482311+0000","last_fullsized":"2024-05-08T06:06:32.482311+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:00:17.419712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:00:17.419712+0000","last_clean_scrub_stamp":"2024-05-08T06:00:17.419712+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:27:39.154694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.490968+0000","last_change":"2024-05-08T06:05:51.421826+0000","last_active":"2024-05-08T06:06:32.490968+0000","last_peered":"2024-05-08T06:06:32.490968+0000","last_clean":"2024-05-08T06:06:32.490968+0000","last_became_active":"2024-05-08T06:05:51.420706+0000","last_became_peered":"2024-05-08T06:05:51.420706+0000","last_unstale":"2024-05-08T06:06:32.490968+0000","last_undegraded":"2024-05-08T06:06:32.490968+0000","last_fullsized":"2024-05-08T06:06:32.490968+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:06:16.242332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482336+0000","last_change":"2024-05-08T06:05:46.351365+0000","last_active":"2024-05-08T06:06:32.482336+0000","last_peered":"2024-05-08T06:06:32.482336+0000","last_clean":"2024-05-08T06:06:32.482336+0000","last_became_active":"2024-05-08T06:05:46.351022+0000","last_became_peered":"2024-05-08T06:05:46.351022+0000","last_unstale":"2024-05-08T06:06:32.482336+0000","last_undegraded":"2024-05-08T06:06:32.482336+0000","last_fullsized":"2024-05-08T06:06:32.482336+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:32:44.134994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.501668+0000","last_change":"2024-05-08T06:06:32.501263+0000","last_active":"2024-05-08T06:06:32.501668+0000","last_peered":"2024-05-08T06:06:32.501668+0000","last_clean":"2024-05-08T06:06:32.501668+0000","last_became_active":"2024-05-08T06:06:32.493795+0000","last_became_peered":"2024-05-08T06:06:32.493795+0000","last_unstale":"2024-05-08T06:06:32.501668+0000","last_undegraded":"2024-05-08T06:06:32.501668+0000","last_fullsized":"2024-05-08T06:06:32.501668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:49:03.898162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.492437+0000","last_change":"2024-05-08T06:05:51.428382+0000","last_active":"2024-05-08T06:06:32.492437+0000","last_peered":"2024-05-08T06:06:32.492437+0000","last_clean":"2024-05-08T06:06:32.492437+0000","last_became_active":"2024-05-08T06:05:51.421443+0000","last_became_peered":"2024-05-08T06:05:51.421443+0000","last_unstale":"2024-05-08T06:06:32.492437+0000","last_undegraded":"2024-05-08T06:06:32.492437+0000","last_fullsized":"2024-05-08T06:06:32.492437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:26:10.215020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'12","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:46.444694+0000","last_change":"2024-05-08T06:05:46.364306+0000","last_active":"2024-05-08T06:06:46.444694+0000","last_peered":"2024-05-08T06:06:46.444694+0000","last_clean":"2024-05-08T06:06:46.444694+0000","last_became_active":"2024-05-08T06:05:46.363871+0000","last_became_peered":"2024-05-08T06:05:46.363871+0000","last_unstale":"2024-05-08T06:06:46.444694+0000","last_undegraded":"2024-05-08T06:06:46.444694+0000","last_fullsized":"2024-05-08T06:06:46.444694+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:38:34.531162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.495732+0000","last_change":"2024-05-08T06:05:51.415020+0000","last_active":"2024-05-08T06:06:32.495732+0000","last_peered":"2024-05-08T06:06:32.495732+0000","last_clean":"2024-05-08T06:06:32.495732+0000","last_became_active":"2024-05-08T06:05:51.413690+0000","last_became_peered":"2024-05-08T06:05:51.413690+0000","last_unstale":"2024-05-08T06:06:32.495732+0000","last_undegraded":"2024-05-08T06:06:32.495732+0000","last_fullsized":"2024-05-08T06:06:32.495732+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:55:38.121492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.597212+0000","last_change":"2024-05-08T06:05:46.359100+0000","last_active":"2024-05-08T06:06:32.597212+0000","last_peered":"2024-05-08T06:06:32.597212+0000","last_clean":"2024-05-08T06:06:32.597212+0000","last_became_active":"2024-05-08T06:05:46.358815+0000","last_became_peered":"2024-05-08T06:05:46.358815+0000","last_unstale":"2024-05-08T06:06:32.597212+0000","last_undegraded":"2024-05-08T06:06:32.597212+0000","last_fullsized":"2024-05-08T06:06:32.597212+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:22:58.532358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.488780+0000","last_change":"2024-05-08T06:05:51.429222+0000","last_active":"2024-05-08T06:06:32.488780+0000","last_peered":"2024-05-08T06:06:32.488780+0000","last_clean":"2024-05-08T06:06:32.488780+0000","last_became_active":"2024-05-08T06:05:51.428845+0000","last_became_peered":"2024-05-08T06:05:51.428845+0000","last_unstale":"2024-05-08T06:06:32.488780+0000","last_undegraded":"2024-05-08T06:06:32.488780+0000","last_fullsized":"2024-05-08T06:06:32.488780+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:36:41.435666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.596030+0000","last_change":"2024-05-08T06:05:46.547931+0000","last_active":"2024-05-08T06:06:32.596030+0000","last_peered":"2024-05-08T06:06:32.596030+0000","last_clean":"2024-05-08T06:06:32.596030+0000","last_became_active":"2024-05-08T06:05:46.547547+0000","last_became_peered":"2024-05-08T06:05:46.547547+0000","last_unstale":"2024-05-08T06:06:32.596030+0000","last_undegraded":"2024-05-08T06:06:32.596030+0000","last_fullsized":"2024-05-08T06:06:32.596030+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:27:03.502153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.479822+0000","last_change":"2024-05-08T06:05:51.413259+0000","last_active":"2024-05-08T06:06:32.479822+0000","last_peered":"2024-05-08T06:06:32.479822+0000","last_clean":"2024-05-08T06:06:32.479822+0000","last_became_active":"2024-05-08T06:05:51.412890+0000","last_became_peered":"2024-05-08T06:05:51.412890+0000","last_unstale":"2024-05-08T06:06:32.479822+0000","last_undegraded":"2024-05-08T06:06:32.479822+0000","last_fullsized":"2024-05-08T06:06:32.479822+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:04:56.690098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.489674+0000","last_change":"2024-05-08T06:05:46.365409+0000","last_active":"2024-05-08T06:06:32.489674+0000","last_peered":"2024-05-08T06:06:32.489674+0000","last_clean":"2024-05-08T06:06:32.489674+0000","last_became_active":"2024-05-08T06:05:46.361277+0000","last_became_peered":"2024-05-08T06:05:46.361277+0000","last_unstale":"2024-05-08T06:06:32.489674+0000","last_undegraded":"2024-05-08T06:06:32.489674+0000","last_fullsized":"2024-05-08T06:06:32.489674+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:06.927385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.477363+0000","last_change":"2024-05-08T06:05:51.423749+0000","last_active":"2024-05-08T06:06:32.477363+0000","last_peered":"2024-05-08T06:06:32.477363+0000","last_clean":"2024-05-08T06:06:32.477363+0000","last_became_active":"2024-05-08T06:05:51.423386+0000","last_became_peered":"2024-05-08T06:05:51.423386+0000","last_unstale":"2024-05-08T06:06:32.477363+0000","last_undegraded":"2024-05-08T06:06:32.477363+0000","last_fullsized":"2024-05-08T06:06:32.477363+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:41:26.261683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480381+0000","last_change":"2024-05-08T06:05:46.362497+0000","last_active":"2024-05-08T06:06:32.480381+0000","last_peered":"2024-05-08T06:06:32.480381+0000","last_clean":"2024-05-08T06:06:32.480381+0000","last_became_active":"2024-05-08T06:05:46.362110+0000","last_became_peered":"2024-05-08T06:05:46.362110+0000","last_unstale":"2024-05-08T06:06:32.480381+0000","last_undegraded":"2024-05-08T06:06:32.480381+0000","last_fullsized":"2024-05-08T06:06:32.480381+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:25:06.205380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.495078+0000","last_change":"2024-05-08T06:06:32.495078+0000","last_active":"2024-05-08T06:06:32.495078+0000","last_peered":"2024-05-08T06:06:32.495078+0000","last_clean":"2024-05-08T06:06:32.495078+0000","last_became_active":"2024-05-08T06:06:32.493855+0000","last_became_peered":"2024-05-08T06:06:32.493855+0000","last_unstale":"2024-05-08T06:06:32.495078+0000","last_undegraded":"2024-05-08T06:06:32.495078+0000","last_fullsized":"2024-05-08T06:06:32.495078+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:56:54.143080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":43,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.601826+0000","last_change":"2024-05-08T06:06:32.601826+0000","last_active":"2024-05-08T06:06:32.601826+0000","last_peered":"2024-05-08T06:06:32.601826+0000","last_clean":"2024-05-08T06:06:32.601826+0000","last_became_active":"2024-05-08T06:06:32.597823+0000","last_became_peered":"2024-05-08T06:06:32.597823+0000","last_unstale":"2024-05-08T06:06:32.601826+0000","last_undegraded":"2024-05-08T06:06:32.601826+0000","last_fullsized":"2024-05-08T06:06:32.601826+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:59:09.741096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482383+0000","last_change":"2024-05-08T06:05:51.413744+0000","last_active":"2024-05-08T06:06:32.482383+0000","last_peered":"2024-05-08T06:06:32.482383+0000","last_clean":"2024-05-08T06:06:32.482383+0000","last_became_active":"2024-05-08T06:05:51.413346+0000","last_became_peered":"2024-05-08T06:05:51.413346+0000","last_unstale":"2024-05-08T06:06:32.482383+0000","last_undegraded":"2024-05-08T06:06:32.482383+0000","last_fullsized":"2024-05-08T06:06:32.482383+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:46:56.109755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480874+0000","last_change":"2024-05-08T06:05:46.552604+0000","last_active":"2024-05-08T06:06:32.480874+0000","last_peered":"2024-05-08T06:06:32.480874+0000","last_clean":"2024-05-08T06:06:32.480874+0000","last_became_active":"2024-05-08T06:05:46.552248+0000","last_became_peered":"2024-05-08T06:05:46.552248+0000","last_unstale":"2024-05-08T06:06:32.480874+0000","last_undegraded":"2024-05-08T06:06:32.480874+0000","last_fullsized":"2024-05-08T06:06:32.480874+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:39:10.306692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482636+0000","last_change":"2024-05-08T06:05:51.404988+0000","last_active":"2024-05-08T06:06:32.482636+0000","last_peered":"2024-05-08T06:06:32.482636+0000","last_clean":"2024-05-08T06:06:32.482636+0000","last_became_active":"2024-05-08T06:05:51.404625+0000","last_became_peered":"2024-05-08T06:05:51.404625+0000","last_unstale":"2024-05-08T06:06:32.482636+0000","last_undegraded":"2024-05-08T06:06:32.482636+0000","last_fullsized":"2024-05-08T06:06:32.482636+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:33:27.919929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.596104+0000","last_change":"2024-05-08T06:05:46.361466+0000","last_active":"2024-05-08T06:06:32.596104+0000","last_peered":"2024-05-08T06:06:32.596104+0000","last_clean":"2024-05-08T06:06:32.596104+0000","last_became_active":"2024-05-08T06:05:46.361107+0000","last_became_peered":"2024-05-08T06:05:46.361107+0000","last_unstale":"2024-05-08T06:06:32.596104+0000","last_undegraded":"2024-05-08T06:06:32.596104+0000","last_fullsized":"2024-05-08T06:06:32.596104+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:15:31.314226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482603+0000","last_change":"2024-05-08T06:05:51.416437+0000","last_active":"2024-05-08T06:06:32.482603+0000","last_peered":"2024-05-08T06:06:32.482603+0000","last_clean":"2024-05-08T06:06:32.482603+0000","last_became_active":"2024-05-08T06:05:51.416070+0000","last_became_peered":"2024-05-08T06:05:51.416070+0000","last_unstale":"2024-05-08T06:06:32.482603+0000","last_undegraded":"2024-05-08T06:06:32.482603+0000","last_fullsized":"2024-05-08T06:06:32.482603+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:35:23.652373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.492808+0000","last_change":"2024-05-08T06:05:46.362475+0000","last_active":"2024-05-08T06:06:32.492808+0000","last_peered":"2024-05-08T06:06:32.492808+0000","last_clean":"2024-05-08T06:06:32.492808+0000","last_became_active":"2024-05-08T06:05:46.361852+0000","last_became_peered":"2024-05-08T06:05:46.361852+0000","last_unstale":"2024-05-08T06:06:32.492808+0000","last_undegraded":"2024-05-08T06:06:32.492808+0000","last_fullsized":"2024-05-08T06:06:32.492808+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:16:54.536330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482691+0000","last_change":"2024-05-08T06:05:51.410718+0000","last_active":"2024-05-08T06:06:32.482691+0000","last_peered":"2024-05-08T06:06:32.482691+0000","last_clean":"2024-05-08T06:06:32.482691+0000","last_became_active":"2024-05-08T06:05:51.408902+0000","last_became_peered":"2024-05-08T06:05:51.408902+0000","last_unstale":"2024-05-08T06:06:32.482691+0000","last_undegraded":"2024-05-08T06:06:32.482691+0000","last_fullsized":"2024-05-08T06:06:32.482691+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:23:33.857677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.477466+0000","last_change":"2024-05-08T06:05:51.414362+0000","last_active":"2024-05-08T06:06:32.477466+0000","last_peered":"2024-05-08T06:06:32.477466+0000","last_clean":"2024-05-08T06:06:32.477466+0000","last_became_active":"2024-05-08T06:05:51.413998+0000","last_became_peered":"2024-05-08T06:05:51.413998+0000","last_unstale":"2024-05-08T06:06:32.477466+0000","last_undegraded":"2024-05-08T06:06:32.477466+0000","last_fullsized":"2024-05-08T06:06:32.477466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:38:22.198219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480218+0000","last_change":"2024-05-08T06:05:51.415508+0000","last_active":"2024-05-08T06:06:32.480218+0000","last_peered":"2024-05-08T06:06:32.480218+0000","last_clean":"2024-05-08T06:06:32.480218+0000","last_became_active":"2024-05-08T06:05:51.415137+0000","last_became_peered":"2024-05-08T06:05:51.415137+0000","last_unstale":"2024-05-08T06:06:32.480218+0000","last_undegraded":"2024-05-08T06:06:32.480218+0000","last_fullsized":"2024-05-08T06:06:32.480218+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:15.007996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480258+0000","last_change":"2024-05-08T06:05:51.422093+0000","last_active":"2024-05-08T06:06:32.480258+0000","last_peered":"2024-05-08T06:06:32.480258+0000","last_clean":"2024-05-08T06:06:32.480258+0000","last_became_active":"2024-05-08T06:05:51.421733+0000","last_became_peered":"2024-05-08T06:05:51.421733+0000","last_unstale":"2024-05-08T06:06:32.480258+0000","last_undegraded":"2024-05-08T06:06:32.480258+0000","last_fullsized":"2024-05-08T06:06:32.480258+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:35:55.950229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.485656+0000","last_change":"2024-05-08T06:05:51.434705+0000","last_active":"2024-05-08T06:06:32.485656+0000","last_peered":"2024-05-08T06:06:32.485656+0000","last_clean":"2024-05-08T06:06:32.485656+0000","last_became_active":"2024-05-08T06:05:51.434324+0000","last_became_peered":"2024-05-08T06:05:51.434324+0000","last_unstale":"2024-05-08T06:06:32.485656+0000","last_undegraded":"2024-05-08T06:06:32.485656+0000","last_fullsized":"2024-05-08T06:06:32.485656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:38:23.101247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.499837+0000","last_change":"2024-05-08T06:06:32.499837+0000","last_active":"2024-05-08T06:06:32.499837+0000","last_peered":"2024-05-08T06:06:32.499837+0000","last_clean":"2024-05-08T06:06:32.499837+0000","last_became_active":"2024-05-08T06:06:32.499026+0000","last_became_peered":"2024-05-08T06:06:32.499026+0000","last_unstale":"2024-05-08T06:06:32.499837+0000","last_undegraded":"2024-05-08T06:06:32.499837+0000","last_fullsized":"2024-05-08T06:06:32.499837+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:48:51.910384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.477627+0000","last_change":"2024-05-08T06:05:51.412370+0000","last_active":"2024-05-08T06:06:32.477627+0000","last_peered":"2024-05-08T06:06:32.477627+0000","last_clean":"2024-05-08T06:06:32.477627+0000","last_became_active":"2024-05-08T06:05:51.412057+0000","last_became_peered":"2024-05-08T06:05:51.412057+0000","last_unstale":"2024-05-08T06:06:32.477627+0000","last_undegraded":"2024-05-08T06:06:32.477627+0000","last_fullsized":"2024-05-08T06:06:32.477627+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:20:51.549926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.481074+0000","last_change":"2024-05-08T06:05:51.408393+0000","last_active":"2024-05-08T06:06:32.481074+0000","last_peered":"2024-05-08T06:06:32.481074+0000","last_clean":"2024-05-08T06:06:32.481074+0000","last_became_active":"2024-05-08T06:05:51.408023+0000","last_became_peered":"2024-05-08T06:05:51.408023+0000","last_unstale":"2024-05-08T06:06:32.481074+0000","last_undegraded":"2024-05-08T06:06:32.481074+0000","last_fullsized":"2024-05-08T06:06:32.481074+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:53.898929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.489822+0000","last_change":"2024-05-08T06:05:51.418055+0000","last_active":"2024-05-08T06:06:32.489822+0000","last_peered":"2024-05-08T06:06:32.489822+0000","last_clean":"2024-05-08T06:06:32.489822+0000","last_became_active":"2024-05-08T06:05:51.417067+0000","last_became_peered":"2024-05-08T06:05:51.417067+0000","last_unstale":"2024-05-08T06:06:32.489822+0000","last_undegraded":"2024-05-08T06:06:32.489822+0000","last_fullsized":"2024-05-08T06:06:32.489822+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:07:35.413672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.595724+0000","last_change":"2024-05-08T06:05:51.421025+0000","last_active":"2024-05-08T06:06:32.595724+0000","last_peered":"2024-05-08T06:06:32.595724+0000","last_clean":"2024-05-08T06:06:32.595724+0000","last_became_active":"2024-05-08T06:05:51.420794+0000","last_became_peered":"2024-05-08T06:05:51.420794+0000","last_unstale":"2024-05-08T06:06:32.595724+0000","last_undegraded":"2024-05-08T06:06:32.595724+0000","last_fullsized":"2024-05-08T06:06:32.595724+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:45:01.852405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.489148+0000","last_change":"2024-05-08T06:05:51.417709+0000","last_active":"2024-05-08T06:06:32.489148+0000","last_peered":"2024-05-08T06:06:32.489148+0000","last_clean":"2024-05-08T06:06:32.489148+0000","last_became_active":"2024-05-08T06:05:51.417337+0000","last_became_peered":"2024-05-08T06:05:51.417337+0000","last_unstale":"2024-05-08T06:06:32.489148+0000","last_undegraded":"2024-05-08T06:06:32.489148+0000","last_fullsized":"2024-05-08T06:06:32.489148+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:32:24.551768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.596197+0000","last_change":"2024-05-08T06:05:51.425318+0000","last_active":"2024-05-08T06:06:32.596197+0000","last_peered":"2024-05-08T06:06:32.596197+0000","last_clean":"2024-05-08T06:06:32.596197+0000","last_became_active":"2024-05-08T06:05:51.424931+0000","last_became_peered":"2024-05-08T06:05:51.424931+0000","last_unstale":"2024-05-08T06:06:32.596197+0000","last_undegraded":"2024-05-08T06:06:32.596197+0000","last_fullsized":"2024-05-08T06:06:32.596197+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:59:45.110201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.492024+0000","last_change":"2024-05-08T06:05:51.433529+0000","last_active":"2024-05-08T06:06:32.492024+0000","last_peered":"2024-05-08T06:06:32.492024+0000","last_clean":"2024-05-08T06:06:32.492024+0000","last_became_active":"2024-05-08T06:05:51.433173+0000","last_became_peered":"2024-05-08T06:05:51.433173+0000","last_unstale":"2024-05-08T06:06:32.492024+0000","last_undegraded":"2024-05-08T06:06:32.492024+0000","last_fullsized":"2024-05-08T06:06:32.492024+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:00:29.042188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.477396+0000","last_change":"2024-05-08T06:05:51.411418+0000","last_active":"2024-05-08T06:06:32.477396+0000","last_peered":"2024-05-08T06:06:32.477396+0000","last_clean":"2024-05-08T06:06:32.477396+0000","last_became_active":"2024-05-08T06:05:51.411084+0000","last_became_peered":"2024-05-08T06:05:51.411084+0000","last_unstale":"2024-05-08T06:06:32.477396+0000","last_undegraded":"2024-05-08T06:06:32.477396+0000","last_fullsized":"2024-05-08T06:06:32.477396+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:08:11.522282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282383,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67073,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":275,"num_read_kb":207,"num_write":166,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":202357,"data_compressed":5508,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645343,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":431859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841572,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":1086747,"data_compressed":37744,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005098,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":897024,"data_stored":432039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201329,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":892928,"data_stored":431937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397557,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":1023889,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561083,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":431428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":494293,"data_compressed":1836,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":432699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":496747,"data_compressed":1836,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117069,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":431739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":432256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757313,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":1023182,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34281,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34281,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34281,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":63072,"data_compressed":1836,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":65526,"data_compressed":1836,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":65041,"data_compressed":1836,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T06:07:02.758 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-08T06:07:02.758 INFO:tasks.cephadm:Setting up client nodes... 2024-05-08T06:07:02.759 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-08T06:07:03.238 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:07:04.368 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:04 smithi123 ceph-mon[29787]: pgmap v358: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-05-08T06:07:04.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:04 smithi163 ceph-mon[34520]: pgmap v358: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-05-08T06:07:04.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:04 smithi192 ceph-mon[34664]: pgmap v358: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-05-08T06:07:05.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.661+0000 7f0671000700 1 -- 172.21.15.123:0/3745137652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f066c104560 msgr2=0x7f066c1049c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:05.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.661+0000 7f0671000700 1 --2- 172.21.15.123:0/3745137652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f066c104560 0x7f066c1049c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f06580099c0 tx=0x7f0658009cd0 comp rx=0 tx=0).stop 2024-05-08T06:07:05.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.661+0000 7f0671000700 1 -- 172.21.15.123:0/3745137652 shutdown_connections 2024-05-08T06:07:05.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.661+0000 7f0671000700 1 --2- 172.21.15.123:0/3745137652 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f066c104f00 0x7f066c10b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:05.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.661+0000 7f0671000700 1 --2- 172.21.15.123:0/3745137652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f066c103360 0x7f066c103780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:05.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.661+0000 7f0671000700 1 --2- 172.21.15.123:0/3745137652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f066c104560 0x7f066c1049c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:05.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.661+0000 7f0671000700 1 -- 172.21.15.123:0/3745137652 >> 172.21.15.123:0/3745137652 conn(0x7f066c0fe8e0 msgr2=0x7f066c100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:07:05.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.661+0000 7f0671000700 1 -- 172.21.15.123:0/3745137652 shutdown_connections 2024-05-08T06:07:05.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.661+0000 7f0671000700 1 -- 172.21.15.123:0/3745137652 wait complete. 2024-05-08T06:07:05.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.662+0000 7f0671000700 1 Processor -- start 2024-05-08T06:07:05.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.662+0000 7f0671000700 1 -- start start 2024-05-08T06:07:05.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.663+0000 7f0671000700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f066c103360 0x7f066c19b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:05.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.663+0000 7f0671000700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f066c104560 0x7f066c19bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:05.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.663+0000 7f0671000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f066c104f00 0x7f066c1a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:05.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.663+0000 7f0671000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f066c10ee30 con 0x7f066c104f00 2024-05-08T06:07:05.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.663+0000 7f0671000700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f066c10ecb0 con 0x7f066c104560 2024-05-08T06:07:05.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.663+0000 7f0671000700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f066c10efb0 con 0x7f066c103360 2024-05-08T06:07:05.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.663+0000 7f066b400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f066c103360 0x7f066c19b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:05.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.663+0000 7f066be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f066c104f00 0x7f066c1a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:05.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.663+0000 7f066b400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f066c103360 0x7f066c19b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:46282/0 (socket says 172.21.15.123:46282) 2024-05-08T06:07:05.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.663+0000 7f066b400700 1 -- 172.21.15.123:0/4064223793 learned_addr learned my addr 172.21.15.123:0/4064223793 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:07:05.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.663+0000 7f066aa00700 1 --2- 172.21.15.123:0/4064223793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f066c104560 0x7f066c19bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:05.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.664+0000 7f066b400700 1 -- 172.21.15.123:0/4064223793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f066c104560 msgr2=0x7f066c19bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:05.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.664+0000 7f066b400700 1 --2- 172.21.15.123:0/4064223793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f066c104560 0x7f066c19bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:05.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.664+0000 7f066b400700 1 -- 172.21.15.123:0/4064223793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f066c104f00 msgr2=0x7f066c1a2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:05.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.664+0000 7f066b400700 1 --2- 172.21.15.123:0/4064223793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f066c104f00 0x7f066c1a2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:05.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.664+0000 7f066b400700 1 -- 172.21.15.123:0/4064223793 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f066c1a3230 con 0x7f066c103360 2024-05-08T06:07:05.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.664+0000 7f066b400700 1 --2- 172.21.15.123:0/4064223793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f066c103360 0x7f066c19b5d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f065c00e9a0 tx=0x7f065c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:05.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.664+0000 7f0663e00700 1 -- 172.21.15.123:0/4064223793 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c00cbd0 con 0x7f066c103360 2024-05-08T06:07:05.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.664+0000 7f0663e00700 1 -- 172.21.15.123:0/4064223793 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f065c0044d0 con 0x7f066c103360 2024-05-08T06:07:05.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.664+0000 7f0671000700 1 -- 172.21.15.123:0/4064223793 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f066c1a3520 con 0x7f066c103360 2024-05-08T06:07:05.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.664+0000 7f0663e00700 1 -- 172.21.15.123:0/4064223793 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c003d80 con 0x7f066c103360 2024-05-08T06:07:05.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.664+0000 7f0671000700 1 -- 172.21.15.123:0/4064223793 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f066c1a3a60 con 0x7f066c103360 2024-05-08T06:07:05.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.666+0000 7f0671000700 1 -- 172.21.15.123:0/4064223793 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f066c064b00 con 0x7f066c103360 2024-05-08T06:07:05.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.668+0000 7f0663e00700 1 -- 172.21.15.123:0/4064223793 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f065c003f60 con 0x7f066c103360 2024-05-08T06:07:05.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.669+0000 7f0663e00700 1 --2- 172.21.15.123:0/4064223793 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f06500b29b0 0x7f06500b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:05.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.669+0000 7f0663e00700 1 -- 172.21.15.123:0/4064223793 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f065c0e2720 con 0x7f066c103360 2024-05-08T06:07:05.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.669+0000 7f066aa00700 1 --2- 172.21.15.123:0/4064223793 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f06500b29b0 0x7f06500b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:05.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.670+0000 7f066aa00700 1 --2- 172.21.15.123:0/4064223793 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f06500b29b0 0x7f06500b4e70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0658000c00 tx=0x7f0658005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:05.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.674+0000 7f0663e00700 1 -- 172.21.15.123:0/4064223793 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f065c0aa290 con 0x7f066c103360 2024-05-08T06:07:05.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:05.998+0000 7f0671000700 1 -- 172.21.15.123:0/4064223793 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f066c1a3f50 con 0x7f066c103360 2024-05-08T06:07:06.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:06.007+0000 7f0663e00700 1 -- 172.21.15.123:0/4064223793 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f065c0a9b90 con 0x7f066c103360 2024-05-08T06:07:06.007 INFO:teuthology.orchestra.run.smithi123.stdout:[client.0] 2024-05-08T06:07:06.007 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCKFjtmp/oMABAAA7klg0CkfoyzplGHX9g/pA== 2024-05-08T06:07:06.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:06.010+0000 7f0671000700 1 -- 172.21.15.123:0/4064223793 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f06500b29b0 msgr2=0x7f06500b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:06.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:06.010+0000 7f0671000700 1 --2- 172.21.15.123:0/4064223793 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f06500b29b0 0x7f06500b4e70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0658000c00 tx=0x7f0658005fb0 comp rx=0 tx=0).stop 2024-05-08T06:07:06.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:06.010+0000 7f0671000700 1 -- 172.21.15.123:0/4064223793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f066c103360 msgr2=0x7f066c19b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:06.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:06.010+0000 7f0671000700 1 --2- 172.21.15.123:0/4064223793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f066c103360 0x7f066c19b5d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f065c00e9a0 tx=0x7f065c00ecb0 comp rx=0 tx=0).stop 2024-05-08T06:07:06.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:06.011+0000 7f0671000700 1 -- 172.21.15.123:0/4064223793 shutdown_connections 2024-05-08T06:07:06.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:06.011+0000 7f0671000700 1 --2- 172.21.15.123:0/4064223793 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f06500b29b0 0x7f06500b4e70 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:06.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:06.011+0000 7f0671000700 1 --2- 172.21.15.123:0/4064223793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f066c104f00 0x7f066c1a2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:06.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:06.011+0000 7f0671000700 1 --2- 172.21.15.123:0/4064223793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f066c103360 0x7f066c19b5d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:06.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:06.011+0000 7f0671000700 1 --2- 172.21.15.123:0/4064223793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f066c104560 0x7f066c19bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:06.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:06.011+0000 7f0671000700 1 -- 172.21.15.123:0/4064223793 >> 172.21.15.123:0/4064223793 conn(0x7f066c0fe8e0 msgr2=0x7f066c0756d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:07:06.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:06.011+0000 7f0671000700 1 -- 172.21.15.123:0/4064223793 shutdown_connections 2024-05-08T06:07:06.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:06.011+0000 7f0671000700 1 -- 172.21.15.123:0/4064223793 wait complete. 2024-05-08T06:07:06.445 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:06 smithi123 ceph-mon[29787]: pgmap v359: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-08T06:07:06.445 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:06 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/4064223793' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T06:07:06.445 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:06 smithi123 ceph-mon[29787]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T06:07:06.445 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:06 smithi123 ceph-mon[29787]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-08T06:07:06.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:06 smithi163 ceph-mon[34520]: pgmap v359: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-08T06:07:06.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:06 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/4064223793' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T06:07:06.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:06 smithi163 ceph-mon[34520]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T06:07:06.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:06 smithi163 ceph-mon[34520]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-08T06:07:06.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:06 smithi192 ceph-mon[34664]: pgmap v359: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-08T06:07:06.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:06 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/4064223793' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T06:07:06.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:06 smithi192 ceph-mon[34664]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T06:07:06.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:06 smithi192 ceph-mon[34664]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-08T06:07:07.024 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-05-08T06:07:07.024 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-08T06:07:07.024 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-08T06:07:07.155 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-08T06:07:07.156 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-08T06:07:07.156 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph mgr dump --format=json 2024-05-08T06:07:07.642 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:07:08.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:08 smithi123 ceph-mon[29787]: pgmap v360: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-08T06:07:08.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:08 smithi163 ceph-mon[34520]: pgmap v360: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-08T06:07:08.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:08 smithi192 ceph-mon[34664]: pgmap v360: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-08T06:07:09.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.943+0000 7fa7b5200700 1 -- 172.21.15.123:0/3147954455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7b006ba10 msgr2=0x7fa7b006be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:09.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.943+0000 7fa7b5200700 1 --2- 172.21.15.123:0/3147954455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7b006ba10 0x7fa7b006be90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa79c009990 tx=0x7fa79c009ca0 comp rx=0 tx=0).stop 2024-05-08T06:07:09.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.943+0000 7fa7b5200700 1 -- 172.21.15.123:0/3147954455 shutdown_connections 2024-05-08T06:07:09.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.943+0000 7fa7b5200700 1 --2- 172.21.15.123:0/3147954455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7b006c3d0 0x7fa7b0110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:09.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.943+0000 7fa7b5200700 1 --2- 172.21.15.123:0/3147954455 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7b0109500 0x7fa7b01098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:09.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.943+0000 7fa7b5200700 1 --2- 172.21.15.123:0/3147954455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7b006ba10 0x7fa7b006be90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:09.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.943+0000 7fa7b5200700 1 -- 172.21.15.123:0/3147954455 >> 172.21.15.123:0/3147954455 conn(0x7fa7b0079310 msgr2=0x7fa7b0079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:07:09.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.944+0000 7fa7b5200700 1 -- 172.21.15.123:0/3147954455 shutdown_connections 2024-05-08T06:07:09.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.944+0000 7fa7b5200700 1 -- 172.21.15.123:0/3147954455 wait complete. 2024-05-08T06:07:09.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.945+0000 7fa7b5200700 1 Processor -- start 2024-05-08T06:07:09.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.945+0000 7fa7b5200700 1 -- start start 2024-05-08T06:07:09.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.945+0000 7fa7b5200700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7b006ba10 0x7fa7b0102460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:09.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.945+0000 7fa7b5200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7b006c3d0 0x7fa7b01029a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:09.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.945+0000 7fa7b5200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7b0109500 0x7fa7b0102ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:09.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.945+0000 7fa7b5200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b0113660 con 0x7fa7b0109500 2024-05-08T06:07:09.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.945+0000 7fa7b5200700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b01134e0 con 0x7fa7b006c3d0 2024-05-08T06:07:09.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.945+0000 7fa7b5200700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b01137e0 con 0x7fa7b006ba10 2024-05-08T06:07:09.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.945+0000 7fa7ae800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7b0109500 0x7fa7b0102ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:09.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.945+0000 7fa7ae800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7b0109500 0x7fa7b0102ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55608/0 (socket says 172.21.15.123:55608) 2024-05-08T06:07:09.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.945+0000 7fa7ae800700 1 -- 172.21.15.123:0/1534064784 learned_addr learned my addr 172.21.15.123:0/1534064784 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:07:09.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.945+0000 7fa7ad400700 1 --2- 172.21.15.123:0/1534064784 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7b006c3d0 0x7fa7b01029a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:09.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.945+0000 7fa7ade00700 1 --2- 172.21.15.123:0/1534064784 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7b006ba10 0x7fa7b0102460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:09.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.946+0000 7fa7ae800700 1 -- 172.21.15.123:0/1534064784 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7b006ba10 msgr2=0x7fa7b0102460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:09.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.946+0000 7fa7ae800700 1 --2- 172.21.15.123:0/1534064784 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7b006ba10 0x7fa7b0102460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:09.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.946+0000 7fa7ae800700 1 -- 172.21.15.123:0/1534064784 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7b006c3d0 msgr2=0x7fa7b01029a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:09.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.946+0000 7fa7ae800700 1 --2- 172.21.15.123:0/1534064784 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7b006c3d0 0x7fa7b01029a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:09.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.946+0000 7fa7ae800700 1 -- 172.21.15.123:0/1534064784 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b01a8180 con 0x7fa7b0109500 2024-05-08T06:07:09.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.946+0000 7fa7ad400700 1 --2- 172.21.15.123:0/1534064784 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7b006c3d0 0x7fa7b01029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:07:09.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.946+0000 7fa7ade00700 1 --2- 172.21.15.123:0/1534064784 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7b006ba10 0x7fa7b0102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:07:09.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.946+0000 7fa7ae800700 1 --2- 172.21.15.123:0/1534064784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7b0109500 0x7fa7b0102ee0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa7a800ef30 tx=0x7fa7a800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:09.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.947+0000 7fa7a6a00700 1 -- 172.21.15.123:0/1534064784 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a80092e0 con 0x7fa7b0109500 2024-05-08T06:07:09.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.947+0000 7fa7a6a00700 1 -- 172.21.15.123:0/1534064784 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7a8004bb0 con 0x7fa7b0109500 2024-05-08T06:07:09.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.947+0000 7fa7a6a00700 1 -- 172.21.15.123:0/1534064784 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8004020 con 0x7fa7b0109500 2024-05-08T06:07:09.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.947+0000 7fa7b5200700 1 -- 172.21.15.123:0/1534064784 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7b01a8410 con 0x7fa7b0109500 2024-05-08T06:07:09.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.947+0000 7fa7b5200700 1 -- 172.21.15.123:0/1534064784 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7b01a8900 con 0x7fa7b0109500 2024-05-08T06:07:09.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.948+0000 7fa7b5200700 1 -- 172.21.15.123:0/1534064784 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7b006a190 con 0x7fa7b0109500 2024-05-08T06:07:09.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.950+0000 7fa7a6a00700 1 -- 172.21.15.123:0/1534064784 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fa7a801b030 con 0x7fa7b0109500 2024-05-08T06:07:09.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.952+0000 7fa7a6a00700 1 --2- 172.21.15.123:0/1534064784 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa7940b2990 0x7fa7940b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:09.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.952+0000 7fa7a6a00700 1 -- 172.21.15.123:0/1534064784 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fa7a80e1e20 con 0x7fa7b0109500 2024-05-08T06:07:09.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.952+0000 7fa7ade00700 1 --2- 172.21.15.123:0/1534064784 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa7940b2990 0x7fa7940b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:09.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.953+0000 7fa7ade00700 1 --2- 172.21.15.123:0/1534064784 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa7940b2990 0x7fa7940b4e50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0005950 tx=0x7fa7a000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:09.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:09.957+0000 7fa7a6a00700 1 -- 172.21.15.123:0/1534064784 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa7a8022020 con 0x7fa7b0109500 2024-05-08T06:07:10.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:10.246+0000 7fa7b5200700 1 -- 172.21.15.123:0/1534064784 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fa7b002e510 con 0x7fa7b0109500 2024-05-08T06:07:10.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:10.255+0000 7fa7a6a00700 1 -- 172.21.15.123:0/1534064784 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288656 (secure 0 0 0) 0x7fa7a8010040 con 0x7fa7b0109500 2024-05-08T06:07:10.259 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:07:10.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:10.260+0000 7fa7b5200700 1 -- 172.21.15.123:0/1534064784 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa7940b2990 msgr2=0x7fa7940b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:10.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:10.260+0000 7fa7b5200700 1 --2- 172.21.15.123:0/1534064784 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa7940b2990 0x7fa7940b4e50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0005950 tx=0x7fa7a000b410 comp rx=0 tx=0).stop 2024-05-08T06:07:10.261 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:10.260+0000 7fa7b5200700 1 -- 172.21.15.123:0/1534064784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7b0109500 msgr2=0x7fa7b0102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:10.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:10.260+0000 7fa7b5200700 1 --2- 172.21.15.123:0/1534064784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7b0109500 0x7fa7b0102ee0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa7a800ef30 tx=0x7fa7a800c3a0 comp rx=0 tx=0).stop 2024-05-08T06:07:10.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:10.260+0000 7fa7b5200700 1 -- 172.21.15.123:0/1534064784 shutdown_connections 2024-05-08T06:07:10.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:10.260+0000 7fa7b5200700 1 --2- 172.21.15.123:0/1534064784 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa7940b2990 0x7fa7940b4e50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:10.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:10.260+0000 7fa7b5200700 1 --2- 172.21.15.123:0/1534064784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7b0109500 0x7fa7b0102ee0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:10.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:10.260+0000 7fa7b5200700 1 --2- 172.21.15.123:0/1534064784 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7b006ba10 0x7fa7b0102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:10.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:10.260+0000 7fa7b5200700 1 --2- 172.21.15.123:0/1534064784 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7b006c3d0 0x7fa7b01029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:10.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:10.260+0000 7fa7b5200700 1 -- 172.21.15.123:0/1534064784 >> 172.21.15.123:0/1534064784 conn(0x7fa7b0079310 msgr2=0x7fa7b01011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:07:10.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:10.260+0000 7fa7b5200700 1 -- 172.21.15.123:0/1534064784 shutdown_connections 2024-05-08T06:07:10.262 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:10.260+0000 7fa7b5200700 1 -- 172.21.15.123:0/1534064784 wait complete. 2024-05-08T06:07:10.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:10 smithi123 ceph-mon[29787]: pgmap v361: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-08T06:07:10.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:10 smithi163 ceph-mon[34520]: pgmap v361: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-08T06:07:10.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:10 smithi192 ceph-mon[34664]: pgmap v361: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-08T06:07:11.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:11 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/1534064784' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-08T06:07:11.473 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":4169246006},{"type":"v1","addr":"172.21.15.123:6801","nonce":4169246006}]},"active_addr":"172.21.15.123:6801/4169246006","active_change":"2024-05-08T05:57:30.888145+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.123:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":1310876594}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":3375959212}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":2973550986}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":2128370455}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":2890250242}]}]} 2024-05-08T06:07:11.479 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-08T06:07:11.479 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-08T06:07:11.480 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T06:07:11.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:11 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/1534064784' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-08T06:07:11.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:11 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/1534064784' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-08T06:07:11.925 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:07:12.336 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:12 smithi123 ceph-mon[29787]: pgmap v362: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:07:12.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:12 smithi163 ceph-mon[34520]: pgmap v362: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:07:12.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:12 smithi192 ceph-mon[34664]: pgmap v362: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:07:14.450 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:14 smithi123 ceph-mon[29787]: pgmap v363: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-08T06:07:14.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.466+0000 7f8218a00700 1 -- 172.21.15.123:0/3790601782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8214102f30 msgr2=0x7f8214106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:14.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.466+0000 7f8218a00700 1 --2- 172.21.15.123:0/3790601782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8214102f30 0x7f8214106f80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f81fc0099c0 tx=0x7f81fc009cd0 comp rx=0 tx=0).stop 2024-05-08T06:07:14.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.466+0000 7f8218a00700 1 -- 172.21.15.123:0/3790601782 shutdown_connections 2024-05-08T06:07:14.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.466+0000 7f8218a00700 1 --2- 172.21.15.123:0/3790601782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8214107660 0x7f821410f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:14.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.466+0000 7f8218a00700 1 --2- 172.21.15.123:0/3790601782 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8214102580 0x7f8214102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:14.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.466+0000 7f8218a00700 1 --2- 172.21.15.123:0/3790601782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8214102f30 0x7f8214106f80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:14.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.466+0000 7f8218a00700 1 -- 172.21.15.123:0/3790601782 >> 172.21.15.123:0/3790601782 conn(0x7f82140fde30 msgr2=0x7f8214100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:07:14.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.466+0000 7f8218a00700 1 -- 172.21.15.123:0/3790601782 shutdown_connections 2024-05-08T06:07:14.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.466+0000 7f8218a00700 1 -- 172.21.15.123:0/3790601782 wait complete. 2024-05-08T06:07:14.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.468+0000 7f8218a00700 1 Processor -- start 2024-05-08T06:07:14.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.468+0000 7f8218a00700 1 -- start start 2024-05-08T06:07:14.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.468+0000 7f8218a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8214102580 0x7f821419f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:14.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.468+0000 7f8218a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8214102f30 0x7f821419fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:14.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.468+0000 7f8218a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8214107660 0x7f82141999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:14.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.468+0000 7f8218a00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8214112520 con 0x7f8214107660 2024-05-08T06:07:14.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.468+0000 7f8218a00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f82141123a0 con 0x7f8214102f30 2024-05-08T06:07:14.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.468+0000 7f8218a00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f82141126a0 con 0x7f8214102580 2024-05-08T06:07:14.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.469+0000 7f8213200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8214107660 0x7f82141999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:14.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.469+0000 7f8212800700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8214102580 0x7f821419f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:14.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.469+0000 7f8213200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8214107660 0x7f82141999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55636/0 (socket says 172.21.15.123:55636) 2024-05-08T06:07:14.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.469+0000 7f8213200700 1 -- 172.21.15.123:0/3723550327 learned_addr learned my addr 172.21.15.123:0/3723550327 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:07:14.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.469+0000 7f820be00700 1 --2- 172.21.15.123:0/3723550327 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8214102f30 0x7f821419fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:14.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.469+0000 7f8212800700 1 -- 172.21.15.123:0/3723550327 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8214102f30 msgr2=0x7f821419fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:14.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.469+0000 7f8212800700 1 --2- 172.21.15.123:0/3723550327 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8214102f30 0x7f821419fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:14.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.469+0000 7f8212800700 1 -- 172.21.15.123:0/3723550327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8214107660 msgr2=0x7f82141999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:14.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.469+0000 7f8212800700 1 --2- 172.21.15.123:0/3723550327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8214107660 0x7f82141999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:14.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.469+0000 7f8212800700 1 -- 172.21.15.123:0/3723550327 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f821419a130 con 0x7f8214102580 2024-05-08T06:07:14.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.469+0000 7f8213200700 1 --2- 172.21.15.123:0/3723550327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8214107660 0x7f82141999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:07:14.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.470+0000 7f8212800700 1 --2- 172.21.15.123:0/3723550327 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8214102580 0x7f821419f7c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f820400e9f0 tx=0x7f820400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:14.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.470+0000 7f820b400700 1 -- 172.21.15.123:0/3723550327 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820400c9c0 con 0x7f8214102580 2024-05-08T06:07:14.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.470+0000 7f820b400700 1 -- 172.21.15.123:0/3723550327 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f820400cb60 con 0x7f8214102580 2024-05-08T06:07:14.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.470+0000 7f8218a00700 1 -- 172.21.15.123:0/3723550327 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f821419a420 con 0x7f8214102580 2024-05-08T06:07:14.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.470+0000 7f820b400700 1 -- 172.21.15.123:0/3723550327 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8204018680 con 0x7f8214102580 2024-05-08T06:07:14.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.470+0000 7f8218a00700 1 -- 172.21.15.123:0/3723550327 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82141a6690 con 0x7f8214102580 2024-05-08T06:07:14.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.471+0000 7f8218a00700 1 -- 172.21.15.123:0/3723550327 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f821406a190 con 0x7f8214102580 2024-05-08T06:07:14.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.474+0000 7f820b400700 1 -- 172.21.15.123:0/3723550327 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f8204010430 con 0x7f8214102580 2024-05-08T06:07:14.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.475+0000 7f820b400700 1 --2- 172.21.15.123:0/3723550327 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f81f80b2910 0x7f81f80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:14.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.475+0000 7f820b400700 1 -- 172.21.15.123:0/3723550327 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f82040e28c0 con 0x7f8214102580 2024-05-08T06:07:14.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.475+0000 7f820be00700 1 --2- 172.21.15.123:0/3723550327 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f81f80b2910 0x7f81f80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:14.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.476+0000 7f820be00700 1 --2- 172.21.15.123:0/3723550327 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f81f80b2910 0x7f81f80b4dd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f81fc000c00 tx=0x7f81fc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:14.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.477+0000 7f820b400700 1 -- 172.21.15.123:0/3723550327 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f82040aa430 con 0x7f8214102580 2024-05-08T06:07:14.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:14 smithi163 ceph-mon[34520]: pgmap v363: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-08T06:07:14.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:14 smithi192 ceph-mon[34664]: pgmap v363: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-08T06:07:14.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.710+0000 7f8218a00700 1 -- 172.21.15.123:0/3723550327 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f821402e510 con 0x7f8214102580 2024-05-08T06:07:14.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.712+0000 7f820b400700 1 -- 172.21.15.123:0/3723550327 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22719 (secure 0 0 0) 0x7f82040a9d30 con 0x7f8214102580 2024-05-08T06:07:14.712 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:07:14.713 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":82,"fsid":"85cfb2e6-0cff-11ef-bc97-c7b262605968","created":"2024-05-08T05:56:12.544750+0000","modified":"2024-05-08T06:06:32.471993+0000","last_up_change":"2024-05-08T06:04:33.654727+0000","last_in_change":"2024-05-08T06:04:13.809050+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T06:00:17.050518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T06:05:44.815334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T06:05:49.604457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a14fde9b-bfb1-40a7-a105-305e8981a778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6803","nonce":3608618058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6805","nonce":3608618058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6809","nonce":3608618058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6807","nonce":3608618058}]},"public_addr":"172.21.15.123:6803/3608618058","cluster_addr":"172.21.15.123:6805/3608618058","heartbeat_back_addr":"172.21.15.123:6809/3608618058","heartbeat_front_addr":"172.21.15.123:6807/3608618058","state":["exists","up"]},{"osd":1,"uuid":"e023ae3d-a82e-4bb6-8fef-889dd07fe183","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6801","nonce":1884753449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6803","nonce":1884753449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6807","nonce":1884753449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6805","nonce":1884753449}]},"public_addr":"172.21.15.163:6801/1884753449","cluster_addr":"172.21.15.163:6803/1884753449","heartbeat_back_addr":"172.21.15.163:6807/1884753449","heartbeat_front_addr":"172.21.15.163:6805/1884753449","state":["exists","up"]},{"osd":2,"uuid":"69f4cfca-63cb-4862-9480-d79891b0f338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6801","nonce":4161570404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6803","nonce":4161570404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6807","nonce":4161570404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6805","nonce":4161570404}]},"public_addr":"172.21.15.192:6801/4161570404","cluster_addr":"172.21.15.192:6803/4161570404","heartbeat_back_addr":"172.21.15.192:6807/4161570404","heartbeat_front_addr":"172.21.15.192:6805/4161570404","state":["exists","up"]},{"osd":3,"uuid":"71bb5ecd-c88c-45fb-8b51-0eefb53941c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6811","nonce":1243003258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6813","nonce":1243003258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6817","nonce":1243003258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6815","nonce":1243003258}]},"public_addr":"172.21.15.123:6811/1243003258","cluster_addr":"172.21.15.123:6813/1243003258","heartbeat_back_addr":"172.21.15.123:6817/1243003258","heartbeat_front_addr":"172.21.15.123:6815/1243003258","state":["exists","up"]},{"osd":4,"uuid":"5a7a3462-e74f-4d15-a85c-709b5d423745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6809","nonce":591680273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6811","nonce":591680273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6815","nonce":591680273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6813","nonce":591680273}]},"public_addr":"172.21.15.163:6809/591680273","cluster_addr":"172.21.15.163:6811/591680273","heartbeat_back_addr":"172.21.15.163:6815/591680273","heartbeat_front_addr":"172.21.15.163:6813/591680273","state":["exists","up"]},{"osd":5,"uuid":"85e3f1ab-f8c6-4025-bab9-9ddb6bd3e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6809","nonce":1734821180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6811","nonce":1734821180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6815","nonce":1734821180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6813","nonce":1734821180}]},"public_addr":"172.21.15.192:6809/1734821180","cluster_addr":"172.21.15.192:6811/1734821180","heartbeat_back_addr":"172.21.15.192:6815/1734821180","heartbeat_front_addr":"172.21.15.192:6813/1734821180","state":["exists","up"]},{"osd":6,"uuid":"f21a5624-6f8f-48be-bc6d-0a883d6b7b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6819","nonce":1002138878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6821","nonce":1002138878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6825","nonce":1002138878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6823","nonce":1002138878}]},"public_addr":"172.21.15.123:6819/1002138878","cluster_addr":"172.21.15.123:6821/1002138878","heartbeat_back_addr":"172.21.15.123:6825/1002138878","heartbeat_front_addr":"172.21.15.123:6823/1002138878","state":["exists","up"]},{"osd":7,"uuid":"6eb16729-d260-4041-9829-3e868795b111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6817","nonce":354411978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6819","nonce":354411978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6823","nonce":354411978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6821","nonce":354411978}]},"public_addr":"172.21.15.163:6817/354411978","cluster_addr":"172.21.15.163:6819/354411978","heartbeat_back_addr":"172.21.15.163:6823/354411978","heartbeat_front_addr":"172.21.15.163:6821/354411978","state":["exists","up"]},{"osd":8,"uuid":"122a281e-18b8-4728-97db-458e7432b2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6817","nonce":3669753689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6819","nonce":3669753689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6823","nonce":3669753689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6821","nonce":3669753689}]},"public_addr":"172.21.15.192:6817/3669753689","cluster_addr":"172.21.15.192:6819/3669753689","heartbeat_back_addr":"172.21.15.192:6823/3669753689","heartbeat_front_addr":"172.21.15.192:6821/3669753689","state":["exists","up"]},{"osd":9,"uuid":"9fa022a9-b873-4e07-806c-641a706a150f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6827","nonce":3104436473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6829","nonce":3104436473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6833","nonce":3104436473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6831","nonce":3104436473}]},"public_addr":"172.21.15.123:6827/3104436473","cluster_addr":"172.21.15.123:6829/3104436473","heartbeat_back_addr":"172.21.15.123:6833/3104436473","heartbeat_front_addr":"172.21.15.123:6831/3104436473","state":["exists","up"]},{"osd":10,"uuid":"fb59339b-e11b-4ffc-b202-7f67cb21da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6825","nonce":2628487883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6827","nonce":2628487883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6831","nonce":2628487883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6829","nonce":2628487883}]},"public_addr":"172.21.15.163:6825/2628487883","cluster_addr":"172.21.15.163:6827/2628487883","heartbeat_back_addr":"172.21.15.163:6831/2628487883","heartbeat_front_addr":"172.21.15.163:6829/2628487883","state":["exists","up"]},{"osd":11,"uuid":"be7a5f36-c1be-41d7-80ee-004d0546eb9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6825","nonce":1339159347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6827","nonce":1339159347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6831","nonce":1339159347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6829","nonce":1339159347}]},"public_addr":"172.21.15.192:6825/1339159347","cluster_addr":"172.21.15.192:6827/1339159347","heartbeat_back_addr":"172.21.15.192:6831/1339159347","heartbeat_front_addr":"172.21.15.192:6829/1339159347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:59:24.923234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:59:49.867071+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:00:13.454062+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:00:46.983594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:01:12.820452+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:01:37.451382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:02:13.325181+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:02:39.453436+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:03:03.837509+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:03:39.460579+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:04:06.314082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:04:31.723154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6801/3057688649":"2024-05-09T05:57:30.887936+0000","172.21.15.123:6800/3057688649":"2024-05-09T05:57:30.887936+0000","172.21.15.123:6800/2769724312":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/399884043":"2024-05-09T05:57:30.887936+0000","172.21.15.123:0/3406061897":"2024-05-09T05:57:30.887936+0000","172.21.15.123:0/644457268":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/3055092197":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/343390357":"2024-05-09T05:57:30.887936+0000","172.21.15.123:6801/2769724312":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/2263171562":"2024-05-09T05:56:55.849036+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T06:07:14.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.715+0000 7f8218a00700 1 -- 172.21.15.123:0/3723550327 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f81f80b2910 msgr2=0x7f81f80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:14.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.715+0000 7f8218a00700 1 --2- 172.21.15.123:0/3723550327 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f81f80b2910 0x7f81f80b4dd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f81fc000c00 tx=0x7f81fc005fb0 comp rx=0 tx=0).stop 2024-05-08T06:07:14.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.715+0000 7f8218a00700 1 -- 172.21.15.123:0/3723550327 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8214102580 msgr2=0x7f821419f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:14.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.715+0000 7f8218a00700 1 --2- 172.21.15.123:0/3723550327 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8214102580 0x7f821419f7c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f820400e9f0 tx=0x7f820400ed00 comp rx=0 tx=0).stop 2024-05-08T06:07:14.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.715+0000 7f8218a00700 1 -- 172.21.15.123:0/3723550327 shutdown_connections 2024-05-08T06:07:14.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.715+0000 7f8218a00700 1 --2- 172.21.15.123:0/3723550327 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f81f80b2910 0x7f81f80b4dd0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:14.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.715+0000 7f8218a00700 1 --2- 172.21.15.123:0/3723550327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8214107660 0x7f82141999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:14.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.715+0000 7f8218a00700 1 --2- 172.21.15.123:0/3723550327 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8214102580 0x7f821419f7c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:14.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.715+0000 7f8218a00700 1 --2- 172.21.15.123:0/3723550327 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8214102f30 0x7f821419fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:14.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.715+0000 7f8218a00700 1 -- 172.21.15.123:0/3723550327 >> 172.21.15.123:0/3723550327 conn(0x7f82140fde30 msgr2=0x7f821406bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:07:14.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.716+0000 7f8218a00700 1 -- 172.21.15.123:0/3723550327 shutdown_connections 2024-05-08T06:07:14.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:14.716+0000 7f8218a00700 1 -- 172.21.15.123:0/3723550327 wait complete. 2024-05-08T06:07:15.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:15 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/3723550327' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:07:15.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:15 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/3723550327' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:07:15.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:15 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/3723550327' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:07:15.717 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-08T06:07:15.718 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T06:07:16.157 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:07:16.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:16 smithi123 ceph-mon[29787]: pgmap v364: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:07:16.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:16 smithi163 ceph-mon[34520]: pgmap v364: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:07:16.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:16 smithi192 ceph-mon[34664]: pgmap v364: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:07:18.370 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:18 smithi123 ceph-mon[29787]: pgmap v365: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:07:18.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:18 smithi163 ceph-mon[34520]: pgmap v365: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:07:18.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:18 smithi192 ceph-mon[34664]: pgmap v365: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:07:18.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.671+0000 7f484a400700 1 -- 172.21.15.123:0/3224445695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48441056c0 msgr2=0x7f4844105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:18.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.671+0000 7f484a400700 1 --2- 172.21.15.123:0/3224445695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48441056c0 0x7f4844105b40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4830009990 tx=0x7f4830009ca0 comp rx=0 tx=0).stop 2024-05-08T06:07:18.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.671+0000 7f484a400700 1 -- 172.21.15.123:0/3224445695 shutdown_connections 2024-05-08T06:07:18.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.671+0000 7f484a400700 1 --2- 172.21.15.123:0/3224445695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4844106080 0x7f4844112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:18.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.671+0000 7f484a400700 1 --2- 172.21.15.123:0/3224445695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f484410b6c0 0x7f484410baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:18.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.671+0000 7f484a400700 1 --2- 172.21.15.123:0/3224445695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48441056c0 0x7f4844105b40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:18.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.671+0000 7f484a400700 1 -- 172.21.15.123:0/3224445695 >> 172.21.15.123:0/3224445695 conn(0x7f48441011b0 msgr2=0x7f48441035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:07:18.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.671+0000 7f484a400700 1 -- 172.21.15.123:0/3224445695 shutdown_connections 2024-05-08T06:07:18.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.672+0000 7f484a400700 1 -- 172.21.15.123:0/3224445695 wait complete. 2024-05-08T06:07:18.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.673+0000 7f484a400700 1 Processor -- start 2024-05-08T06:07:18.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.673+0000 7f484a400700 1 -- start start 2024-05-08T06:07:18.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.674+0000 7f484a400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48441056c0 0x7f484419bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:18.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.674+0000 7f484a400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4844106080 0x7f484419c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:18.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.674+0000 7f484a400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f484410b6c0 0x7f48441a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:18.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.674+0000 7f484a400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4844115900 con 0x7f484410b6c0 2024-05-08T06:07:18.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.674+0000 7f484a400700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4844115780 con 0x7f48441056c0 2024-05-08T06:07:18.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.674+0000 7f484a400700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4844115a80 con 0x7f4844106080 2024-05-08T06:07:18.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.674+0000 7f4842a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4844106080 0x7f484419c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:18.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.674+0000 7f4843e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f484410b6c0 0x7f48441a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:18.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.674+0000 7f4842a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4844106080 0x7f484419c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:58256/0 (socket says 172.21.15.123:58256) 2024-05-08T06:07:18.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.674+0000 7f4842a00700 1 -- 172.21.15.123:0/533524182 learned_addr learned my addr 172.21.15.123:0/533524182 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:07:18.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.674+0000 7f4843400700 1 --2- 172.21.15.123:0/533524182 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48441056c0 0x7f484419bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:18.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.674+0000 7f4842a00700 1 -- 172.21.15.123:0/533524182 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48441056c0 msgr2=0x7f484419bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:18.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.674+0000 7f4842a00700 1 --2- 172.21.15.123:0/533524182 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48441056c0 0x7f484419bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:18.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.674+0000 7f4842a00700 1 -- 172.21.15.123:0/533524182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f484410b6c0 msgr2=0x7f48441a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:18.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.674+0000 7f4842a00700 1 --2- 172.21.15.123:0/533524182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f484410b6c0 0x7f48441a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:18.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.674+0000 7f4842a00700 1 -- 172.21.15.123:0/533524182 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48441a0d70 con 0x7f4844106080 2024-05-08T06:07:18.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.675+0000 7f4842a00700 1 --2- 172.21.15.123:0/533524182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4844106080 0x7f484419c2f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4830000c00 tx=0x7f483000bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:18.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.675+0000 7f483be00700 1 -- 172.21.15.123:0/533524182 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4830027070 con 0x7f4844106080 2024-05-08T06:07:18.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.675+0000 7f484a400700 1 -- 172.21.15.123:0/533524182 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48441a1000 con 0x7f4844106080 2024-05-08T06:07:18.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.675+0000 7f483be00700 1 -- 172.21.15.123:0/533524182 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4830022470 con 0x7f4844106080 2024-05-08T06:07:18.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.675+0000 7f483be00700 1 -- 172.21.15.123:0/533524182 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4830017650 con 0x7f4844106080 2024-05-08T06:07:18.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.675+0000 7f484a400700 1 -- 172.21.15.123:0/533524182 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48441a88e0 con 0x7f4844106080 2024-05-08T06:07:18.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.677+0000 7f484a400700 1 -- 172.21.15.123:0/533524182 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f484410dc50 con 0x7f4844106080 2024-05-08T06:07:18.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.679+0000 7f483be00700 1 -- 172.21.15.123:0/533524182 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f483002a070 con 0x7f4844106080 2024-05-08T06:07:18.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.680+0000 7f483be00700 1 --2- 172.21.15.123:0/533524182 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f48280b2910 0x7f48280b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:18.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.680+0000 7f483be00700 1 -- 172.21.15.123:0/533524182 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f48300e2fe0 con 0x7f4844106080 2024-05-08T06:07:18.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.680+0000 7f4843400700 1 --2- 172.21.15.123:0/533524182 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f48280b2910 0x7f48280b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:18.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.681+0000 7f4843400700 1 --2- 172.21.15.123:0/533524182 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f48280b2910 0x7f48280b4dd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4834009be0 tx=0x7f4834009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:18.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.684+0000 7f483be00700 1 -- 172.21.15.123:0/533524182 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4830013070 con 0x7f4844106080 2024-05-08T06:07:18.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.920+0000 7f484a400700 1 -- 172.21.15.123:0/533524182 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f484419d0c0 con 0x7f4844106080 2024-05-08T06:07:18.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.922+0000 7f483be00700 1 -- 172.21.15.123:0/533524182 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22719 (secure 0 0 0) 0x7f48300aac40 con 0x7f4844106080 2024-05-08T06:07:18.923 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:07:18.923 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":82,"fsid":"85cfb2e6-0cff-11ef-bc97-c7b262605968","created":"2024-05-08T05:56:12.544750+0000","modified":"2024-05-08T06:06:32.471993+0000","last_up_change":"2024-05-08T06:04:33.654727+0000","last_in_change":"2024-05-08T06:04:13.809050+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T06:00:17.050518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T06:05:44.815334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T06:05:49.604457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a14fde9b-bfb1-40a7-a105-305e8981a778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6803","nonce":3608618058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6805","nonce":3608618058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6809","nonce":3608618058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6807","nonce":3608618058}]},"public_addr":"172.21.15.123:6803/3608618058","cluster_addr":"172.21.15.123:6805/3608618058","heartbeat_back_addr":"172.21.15.123:6809/3608618058","heartbeat_front_addr":"172.21.15.123:6807/3608618058","state":["exists","up"]},{"osd":1,"uuid":"e023ae3d-a82e-4bb6-8fef-889dd07fe183","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6801","nonce":1884753449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6803","nonce":1884753449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6807","nonce":1884753449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6805","nonce":1884753449}]},"public_addr":"172.21.15.163:6801/1884753449","cluster_addr":"172.21.15.163:6803/1884753449","heartbeat_back_addr":"172.21.15.163:6807/1884753449","heartbeat_front_addr":"172.21.15.163:6805/1884753449","state":["exists","up"]},{"osd":2,"uuid":"69f4cfca-63cb-4862-9480-d79891b0f338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6801","nonce":4161570404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6803","nonce":4161570404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6807","nonce":4161570404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6805","nonce":4161570404}]},"public_addr":"172.21.15.192:6801/4161570404","cluster_addr":"172.21.15.192:6803/4161570404","heartbeat_back_addr":"172.21.15.192:6807/4161570404","heartbeat_front_addr":"172.21.15.192:6805/4161570404","state":["exists","up"]},{"osd":3,"uuid":"71bb5ecd-c88c-45fb-8b51-0eefb53941c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6811","nonce":1243003258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6813","nonce":1243003258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6817","nonce":1243003258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6815","nonce":1243003258}]},"public_addr":"172.21.15.123:6811/1243003258","cluster_addr":"172.21.15.123:6813/1243003258","heartbeat_back_addr":"172.21.15.123:6817/1243003258","heartbeat_front_addr":"172.21.15.123:6815/1243003258","state":["exists","up"]},{"osd":4,"uuid":"5a7a3462-e74f-4d15-a85c-709b5d423745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6809","nonce":591680273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6811","nonce":591680273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6815","nonce":591680273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6813","nonce":591680273}]},"public_addr":"172.21.15.163:6809/591680273","cluster_addr":"172.21.15.163:6811/591680273","heartbeat_back_addr":"172.21.15.163:6815/591680273","heartbeat_front_addr":"172.21.15.163:6813/591680273","state":["exists","up"]},{"osd":5,"uuid":"85e3f1ab-f8c6-4025-bab9-9ddb6bd3e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6809","nonce":1734821180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6811","nonce":1734821180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6815","nonce":1734821180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6813","nonce":1734821180}]},"public_addr":"172.21.15.192:6809/1734821180","cluster_addr":"172.21.15.192:6811/1734821180","heartbeat_back_addr":"172.21.15.192:6815/1734821180","heartbeat_front_addr":"172.21.15.192:6813/1734821180","state":["exists","up"]},{"osd":6,"uuid":"f21a5624-6f8f-48be-bc6d-0a883d6b7b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6819","nonce":1002138878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6821","nonce":1002138878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6825","nonce":1002138878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6823","nonce":1002138878}]},"public_addr":"172.21.15.123:6819/1002138878","cluster_addr":"172.21.15.123:6821/1002138878","heartbeat_back_addr":"172.21.15.123:6825/1002138878","heartbeat_front_addr":"172.21.15.123:6823/1002138878","state":["exists","up"]},{"osd":7,"uuid":"6eb16729-d260-4041-9829-3e868795b111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6817","nonce":354411978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6819","nonce":354411978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6823","nonce":354411978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6821","nonce":354411978}]},"public_addr":"172.21.15.163:6817/354411978","cluster_addr":"172.21.15.163:6819/354411978","heartbeat_back_addr":"172.21.15.163:6823/354411978","heartbeat_front_addr":"172.21.15.163:6821/354411978","state":["exists","up"]},{"osd":8,"uuid":"122a281e-18b8-4728-97db-458e7432b2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6817","nonce":3669753689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6819","nonce":3669753689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6823","nonce":3669753689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6821","nonce":3669753689}]},"public_addr":"172.21.15.192:6817/3669753689","cluster_addr":"172.21.15.192:6819/3669753689","heartbeat_back_addr":"172.21.15.192:6823/3669753689","heartbeat_front_addr":"172.21.15.192:6821/3669753689","state":["exists","up"]},{"osd":9,"uuid":"9fa022a9-b873-4e07-806c-641a706a150f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6827","nonce":3104436473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6829","nonce":3104436473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6833","nonce":3104436473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6831","nonce":3104436473}]},"public_addr":"172.21.15.123:6827/3104436473","cluster_addr":"172.21.15.123:6829/3104436473","heartbeat_back_addr":"172.21.15.123:6833/3104436473","heartbeat_front_addr":"172.21.15.123:6831/3104436473","state":["exists","up"]},{"osd":10,"uuid":"fb59339b-e11b-4ffc-b202-7f67cb21da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6825","nonce":2628487883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6827","nonce":2628487883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6831","nonce":2628487883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6829","nonce":2628487883}]},"public_addr":"172.21.15.163:6825/2628487883","cluster_addr":"172.21.15.163:6827/2628487883","heartbeat_back_addr":"172.21.15.163:6831/2628487883","heartbeat_front_addr":"172.21.15.163:6829/2628487883","state":["exists","up"]},{"osd":11,"uuid":"be7a5f36-c1be-41d7-80ee-004d0546eb9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6825","nonce":1339159347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6827","nonce":1339159347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6831","nonce":1339159347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6829","nonce":1339159347}]},"public_addr":"172.21.15.192:6825/1339159347","cluster_addr":"172.21.15.192:6827/1339159347","heartbeat_back_addr":"172.21.15.192:6831/1339159347","heartbeat_front_addr":"172.21.15.192:6829/1339159347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:59:24.923234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:59:49.867071+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:00:13.454062+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:00:46.983594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:01:12.820452+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:01:37.451382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:02:13.325181+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:02:39.453436+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:03:03.837509+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:03:39.460579+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:04:06.314082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:04:31.723154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6801/3057688649":"2024-05-09T05:57:30.887936+0000","172.21.15.123:6800/3057688649":"2024-05-09T05:57:30.887936+0000","172.21.15.123:6800/2769724312":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/399884043":"2024-05-09T05:57:30.887936+0000","172.21.15.123:0/3406061897":"2024-05-09T05:57:30.887936+0000","172.21.15.123:0/644457268":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/3055092197":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/343390357":"2024-05-09T05:57:30.887936+0000","172.21.15.123:6801/2769724312":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/2263171562":"2024-05-09T05:56:55.849036+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T06:07:18.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.925+0000 7f484a400700 1 -- 172.21.15.123:0/533524182 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f48280b2910 msgr2=0x7f48280b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:18.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.925+0000 7f484a400700 1 --2- 172.21.15.123:0/533524182 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f48280b2910 0x7f48280b4dd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4834009be0 tx=0x7f4834009260 comp rx=0 tx=0).stop 2024-05-08T06:07:18.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.925+0000 7f484a400700 1 -- 172.21.15.123:0/533524182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4844106080 msgr2=0x7f484419c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:18.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.925+0000 7f484a400700 1 --2- 172.21.15.123:0/533524182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4844106080 0x7f484419c2f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4830000c00 tx=0x7f483000bfd0 comp rx=0 tx=0).stop 2024-05-08T06:07:18.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.925+0000 7f484a400700 1 -- 172.21.15.123:0/533524182 shutdown_connections 2024-05-08T06:07:18.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.925+0000 7f484a400700 1 --2- 172.21.15.123:0/533524182 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f48280b2910 0x7f48280b4dd0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:18.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.925+0000 7f484a400700 1 --2- 172.21.15.123:0/533524182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f484410b6c0 0x7f48441a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:18.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.925+0000 7f484a400700 1 --2- 172.21.15.123:0/533524182 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48441056c0 0x7f484419bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:18.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.925+0000 7f484a400700 1 --2- 172.21.15.123:0/533524182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4844106080 0x7f484419c2f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:18.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.926+0000 7f484a400700 1 -- 172.21.15.123:0/533524182 >> 172.21.15.123:0/533524182 conn(0x7f48441011b0 msgr2=0x7f48441019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:07:18.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.926+0000 7f484a400700 1 -- 172.21.15.123:0/533524182 shutdown_connections 2024-05-08T06:07:18.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:18.926+0000 7f484a400700 1 -- 172.21.15.123:0/533524182 wait complete. 2024-05-08T06:07:19.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:19 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/533524182' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:07:19.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:19 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/533524182' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:07:19.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:19 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/533524182' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:07:20.000 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-08T06:07:20.001 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-08T06:07:20.001 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-08T06:07:20.001 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-08T06:07:20.002 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-08T06:07:20.002 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-08T06:07:20.003 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-08T06:07:20.003 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-08T06:07:20.003 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-08T06:07:20.004 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-08T06:07:20.004 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-08T06:07:20.005 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-08T06:07:20.456 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:20 smithi123 ceph-mon[29787]: pgmap v366: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:07:20.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:20 smithi163 ceph-mon[34520]: pgmap v366: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:07:20.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:20 smithi192 ceph-mon[34664]: pgmap v366: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:07:20.915 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:07:21.013 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:07:21.351 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:07:21.353 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:07:21.356 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:07:21.833 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:07:21.839 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:07:21.840 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:07:22.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:22 smithi163 ceph-mon[34520]: pgmap v367: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:07:22.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:22 smithi192 ceph-mon[34664]: pgmap v367: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:07:22.559 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:22 smithi123 ceph-mon[29787]: pgmap v367: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:07:24.485 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:24 smithi123 ceph-mon[29787]: pgmap v368: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:07:24.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:24 smithi163 ceph-mon[34520]: pgmap v368: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:07:24.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:24 smithi192 ceph-mon[34664]: pgmap v368: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:07:26.510 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:26 smithi123 ceph-mon[29787]: pgmap v369: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:07:26.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:26 smithi163 ceph-mon[34520]: pgmap v369: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:07:26.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:26 smithi192 ceph-mon[34664]: pgmap v369: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:07:28.500 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:28 smithi123 ceph-mon[29787]: pgmap v370: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:07:28.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:28 smithi163 ceph-mon[34520]: pgmap v370: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:07:28.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:28 smithi192 ceph-mon[34664]: pgmap v370: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:07:28.921 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:07:28.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.922+0000 7f544ee00700 1 -- 172.21.15.123:0/1890064023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5448109460 msgr2=0x7f5448109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:28.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.922+0000 7f544ee00700 1 --2- 172.21.15.123:0/1890064023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5448109460 0x7f5448109840 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f543c0098f0 tx=0x7f543c009c00 comp rx=0 tx=0).stop 2024-05-08T06:07:28.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.922+0000 7f544ee00700 1 -- 172.21.15.123:0/1890064023 shutdown_connections 2024-05-08T06:07:28.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.922+0000 7f544ee00700 1 --2- 172.21.15.123:0/1890064023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5448103d90 0x7f54481102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:28.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.922+0000 7f544ee00700 1 --2- 172.21.15.123:0/1890064023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5448109460 0x7f5448109840 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:28.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.922+0000 7f544ee00700 1 --2- 172.21.15.123:0/1890064023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54481033d0 0x7f5448103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:28.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.922+0000 7f544ee00700 1 -- 172.21.15.123:0/1890064023 >> 172.21.15.123:0/1890064023 conn(0x7f54480fef20 msgr2=0x7f5448101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:07:28.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.922+0000 7f544ee00700 1 -- 172.21.15.123:0/1890064023 shutdown_connections 2024-05-08T06:07:28.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.922+0000 7f544ee00700 1 -- 172.21.15.123:0/1890064023 wait complete. 2024-05-08T06:07:28.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.923+0000 7f544ee00700 1 Processor -- start 2024-05-08T06:07:28.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.924+0000 7f544ee00700 1 -- start start 2024-05-08T06:07:28.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.924+0000 7f544ee00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54481033d0 0x7f54481a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:28.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.924+0000 7f544ee00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5448103d90 0x7f54481a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:28.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.924+0000 7f544ee00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5448109460 0x7f544819bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:28.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.924+0000 7f544ee00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f54481135c0 con 0x7f5448109460 2024-05-08T06:07:28.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.924+0000 7f544ee00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5448113440 con 0x7f54481033d0 2024-05-08T06:07:28.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.924+0000 7f544ee00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5448113740 con 0x7f5448103d90 2024-05-08T06:07:28.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.924+0000 7f544e400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5448109460 0x7f544819bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:28.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.924+0000 7f544d000700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5448103d90 0x7f54481a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:28.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.925+0000 7f544d000700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5448103d90 0x7f54481a1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:47610/0 (socket says 172.21.15.123:47610) 2024-05-08T06:07:28.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.925+0000 7f544d000700 1 -- 172.21.15.123:0/1051582331 learned_addr learned my addr 172.21.15.123:0/1051582331 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:07:28.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.925+0000 7f544da00700 1 --2- 172.21.15.123:0/1051582331 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54481033d0 0x7f54481a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:28.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.925+0000 7f544d000700 1 -- 172.21.15.123:0/1051582331 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54481033d0 msgr2=0x7f54481a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:28.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.925+0000 7f544d000700 1 --2- 172.21.15.123:0/1051582331 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54481033d0 0x7f54481a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:28.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.925+0000 7f544d000700 1 -- 172.21.15.123:0/1051582331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5448109460 msgr2=0x7f544819bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:28.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.925+0000 7f544d000700 1 --2- 172.21.15.123:0/1051582331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5448109460 0x7f544819bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:28.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.925+0000 7f544d000700 1 -- 172.21.15.123:0/1051582331 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f544819c260 con 0x7f5448103d90 2024-05-08T06:07:28.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.925+0000 7f544d000700 1 --2- 172.21.15.123:0/1051582331 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5448103d90 0x7f54481a1e80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f543400e9a0 tx=0x7f543400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:28.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.925+0000 7f5446000700 1 -- 172.21.15.123:0/1051582331 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543400ca10 con 0x7f5448103d90 2024-05-08T06:07:28.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.926+0000 7f544ee00700 1 -- 172.21.15.123:0/1051582331 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f544819c550 con 0x7f5448103d90 2024-05-08T06:07:28.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.926+0000 7f5446000700 1 -- 172.21.15.123:0/1051582331 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f543400cbb0 con 0x7f5448103d90 2024-05-08T06:07:28.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.926+0000 7f5446000700 1 -- 172.21.15.123:0/1051582331 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5434018680 con 0x7f5448103d90 2024-05-08T06:07:28.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.926+0000 7f544ee00700 1 -- 172.21.15.123:0/1051582331 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54481a8810 con 0x7f5448103d90 2024-05-08T06:07:28.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.927+0000 7f544ee00700 1 -- 172.21.15.123:0/1051582331 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f54481007a0 con 0x7f5448103d90 2024-05-08T06:07:28.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.929+0000 7f5446000700 1 -- 172.21.15.123:0/1051582331 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f5434010430 con 0x7f5448103d90 2024-05-08T06:07:28.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.931+0000 7f5446000700 1 --2- 172.21.15.123:0/1051582331 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f54300b2910 0x7f54300b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:28.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.931+0000 7f5446000700 1 -- 172.21.15.123:0/1051582331 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f54340e2b70 con 0x7f5448103d90 2024-05-08T06:07:28.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.931+0000 7f544da00700 1 --2- 172.21.15.123:0/1051582331 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f54300b2910 0x7f54300b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:28.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.931+0000 7f5446000700 1 --2- 172.21.15.123:0/1051582331 >> [v2:172.21.15.123:6810/1243003258,v1:172.21.15.123:6811/1243003258] conn(0x7f54300bc450 0x7f54300be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:28.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.931+0000 7f5446000700 1 -- 172.21.15.123:0/1051582331 --> [v2:172.21.15.123:6810/1243003258,v1:172.21.15.123:6811/1243003258] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5434024e70 con 0x7f54300bc450 2024-05-08T06:07:28.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.931+0000 7f5446000700 1 -- 172.21.15.123:0/1051582331 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f54340e2f20 con 0x7f5448103d90 2024-05-08T06:07:28.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.932+0000 7f544e400700 1 --2- 172.21.15.123:0/1051582331 >> [v2:172.21.15.123:6810/1243003258,v1:172.21.15.123:6811/1243003258] conn(0x7f54300bc450 0x7f54300be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:28.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.932+0000 7f544da00700 1 --2- 172.21.15.123:0/1051582331 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f54300b2910 0x7f54300b4dd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f543c0098c0 tx=0x7f543c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:28.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.932+0000 7f544e400700 1 --2- 172.21.15.123:0/1051582331 >> [v2:172.21.15.123:6810/1243003258,v1:172.21.15.123:6811/1243003258] conn(0x7f54300bc450 0x7f54300be8f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:28.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.933+0000 7f5446000700 1 -- 172.21.15.123:0/1051582331 <== osd.3 v2:172.21.15.123:6810/1243003258 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5434024e70 con 0x7f54300bc450 2024-05-08T06:07:28.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.952+0000 7f544ee00700 1 -- 172.21.15.123:0/1051582331 --> [v2:172.21.15.123:6810/1243003258,v1:172.21.15.123:6811/1243003258] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5448104b10 con 0x7f54300bc450 2024-05-08T06:07:28.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.953+0000 7f5446000700 1 -- 172.21.15.123:0/1051582331 <== osd.3 v2:172.21.15.123:6810/1243003258 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5448104b10 con 0x7f54300bc450 2024-05-08T06:07:28.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.953+0000 7f544ee00700 1 -- 172.21.15.123:0/1051582331 >> [v2:172.21.15.123:6810/1243003258,v1:172.21.15.123:6811/1243003258] conn(0x7f54300bc450 msgr2=0x7f54300be8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:28.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.953+0000 7f544ee00700 1 --2- 172.21.15.123:0/1051582331 >> [v2:172.21.15.123:6810/1243003258,v1:172.21.15.123:6811/1243003258] conn(0x7f54300bc450 0x7f54300be8f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:28.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.953+0000 7f544ee00700 1 -- 172.21.15.123:0/1051582331 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f54300b2910 msgr2=0x7f54300b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:28.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.953+0000 7f544ee00700 1 --2- 172.21.15.123:0/1051582331 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f54300b2910 0x7f54300b4dd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f543c0098c0 tx=0x7f543c019040 comp rx=0 tx=0).stop 2024-05-08T06:07:28.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.954+0000 7f544ee00700 1 -- 172.21.15.123:0/1051582331 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5448103d90 msgr2=0x7f54481a1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:28.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.954+0000 7f544ee00700 1 --2- 172.21.15.123:0/1051582331 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5448103d90 0x7f54481a1e80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f543400e9a0 tx=0x7f543400ecb0 comp rx=0 tx=0).stop 2024-05-08T06:07:28.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.954+0000 7f544e400700 1 -- 172.21.15.123:0/1051582331 reap_dead start 2024-05-08T06:07:28.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.954+0000 7f544ee00700 1 -- 172.21.15.123:0/1051582331 shutdown_connections 2024-05-08T06:07:28.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.954+0000 7f544ee00700 1 -- 172.21.15.123:0/1051582331 >> 172.21.15.123:0/1051582331 conn(0x7f54480fef20 msgr2=0x7f544810eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:07:28.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.954+0000 7f544ee00700 1 -- 172.21.15.123:0/1051582331 shutdown_connections 2024-05-08T06:07:28.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:28.954+0000 7f544ee00700 1 -- 172.21.15.123:0/1051582331 wait complete. 2024-05-08T06:07:30.505 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:30 smithi123 ceph-mon[29787]: pgmap v371: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:07:30.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:30 smithi163 ceph-mon[34520]: pgmap v371: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:07:30.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:30 smithi192 ceph-mon[34664]: pgmap v371: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:07:31.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.226+0000 7fcd5da00700 1 -- 172.21.15.123:0/2574642731 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd5806ba10 msgr2=0x7fcd5806be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:31.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.226+0000 7fcd5da00700 1 --2- 172.21.15.123:0/2574642731 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd5806ba10 0x7fcd5806be90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcd440098f0 tx=0x7fcd44009c00 comp rx=0 tx=0).stop 2024-05-08T06:07:31.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.226+0000 7fcd5da00700 1 -- 172.21.15.123:0/2574642731 shutdown_connections 2024-05-08T06:07:31.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.226+0000 7fcd5da00700 1 --2- 172.21.15.123:0/2574642731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd58109560 0x7fcd58109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:31.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.226+0000 7fcd5da00700 1 --2- 172.21.15.123:0/2574642731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd5806c3d0 0x7fcd581102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:31.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.226+0000 7fcd5da00700 1 --2- 172.21.15.123:0/2574642731 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd5806ba10 0x7fcd5806be90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:31.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.226+0000 7fcd5da00700 1 -- 172.21.15.123:0/2574642731 >> 172.21.15.123:0/2574642731 conn(0x7fcd580791e0 msgr2=0x7fcd580795f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:07:31.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.226+0000 7fcd5da00700 1 -- 172.21.15.123:0/2574642731 shutdown_connections 2024-05-08T06:07:31.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.226+0000 7fcd5da00700 1 -- 172.21.15.123:0/2574642731 wait complete. 2024-05-08T06:07:31.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.228+0000 7fcd5da00700 1 Processor -- start 2024-05-08T06:07:31.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.228+0000 7fcd5da00700 1 -- start start 2024-05-08T06:07:31.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.228+0000 7fcd5da00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd5806ba10 0x7fcd5819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:31.329 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.228+0000 7fcd5da00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd5806c3d0 0x7fcd5819fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:31.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.228+0000 7fcd5da00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd58109560 0x7fcd58199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:31.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.228+0000 7fcd5da00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcd58113580 con 0x7fcd5806ba10 2024-05-08T06:07:31.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.228+0000 7fcd5da00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fcd58113400 con 0x7fcd58109560 2024-05-08T06:07:31.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.228+0000 7fcd5da00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcd58113700 con 0x7fcd5806c3d0 2024-05-08T06:07:31.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.229+0000 7fcd57e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd5806ba10 0x7fcd5819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:31.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.229+0000 7fcd57e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd5806ba10 0x7fcd5819f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36070/0 (socket says 172.21.15.123:36070) 2024-05-08T06:07:31.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.229+0000 7fcd57e00700 1 -- 172.21.15.123:0/4209659437 learned_addr learned my addr 172.21.15.123:0/4209659437 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:07:31.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.229+0000 7fcd57400700 1 --2- 172.21.15.123:0/4209659437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd5806c3d0 0x7fcd5819fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:31.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.229+0000 7fcd5d000700 1 --2- 172.21.15.123:0/4209659437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd58109560 0x7fcd58199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:31.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.229+0000 7fcd57e00700 1 -- 172.21.15.123:0/4209659437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd5806c3d0 msgr2=0x7fcd5819fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:31.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.229+0000 7fcd57e00700 1 --2- 172.21.15.123:0/4209659437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd5806c3d0 0x7fcd5819fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:31.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.229+0000 7fcd57e00700 1 -- 172.21.15.123:0/4209659437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd58109560 msgr2=0x7fcd58199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:31.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.229+0000 7fcd57e00700 1 --2- 172.21.15.123:0/4209659437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd58109560 0x7fcd58199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:31.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.229+0000 7fcd57e00700 1 -- 172.21.15.123:0/4209659437 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd5819a000 con 0x7fcd5806ba10 2024-05-08T06:07:31.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.229+0000 7fcd57400700 1 --2- 172.21.15.123:0/4209659437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd5806c3d0 0x7fcd5819fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:07:31.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.230+0000 7fcd57e00700 1 --2- 172.21.15.123:0/4209659437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd5806ba10 0x7fcd5819f6a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcd4800e9c0 tx=0x7fcd4800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:31.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.230+0000 7fcd54c00700 1 -- 172.21.15.123:0/4209659437 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4800cbd0 con 0x7fcd5806ba10 2024-05-08T06:07:31.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.230+0000 7fcd5da00700 1 -- 172.21.15.123:0/4209659437 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd5819a2f0 con 0x7fcd5806ba10 2024-05-08T06:07:31.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.230+0000 7fcd54c00700 1 -- 172.21.15.123:0/4209659437 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd4800cd70 con 0x7fcd5806ba10 2024-05-08T06:07:31.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.230+0000 7fcd54c00700 1 -- 172.21.15.123:0/4209659437 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd48005550 con 0x7fcd5806ba10 2024-05-08T06:07:31.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.230+0000 7fcd5da00700 1 -- 172.21.15.123:0/4209659437 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd581a6500 con 0x7fcd5806ba10 2024-05-08T06:07:31.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.231+0000 7fcd5da00700 1 -- 172.21.15.123:0/4209659437 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcd58063c30 con 0x7fcd5806ba10 2024-05-08T06:07:31.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.234+0000 7fcd54c00700 1 -- 172.21.15.123:0/4209659437 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fcd48004750 con 0x7fcd5806ba10 2024-05-08T06:07:31.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.235+0000 7fcd54c00700 1 --2- 172.21.15.123:0/4209659437 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fcd3c0b2910 0x7fcd3c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:31.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.235+0000 7fcd54c00700 1 -- 172.21.15.123:0/4209659437 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fcd480e2a10 con 0x7fcd5806ba10 2024-05-08T06:07:31.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.235+0000 7fcd57400700 1 --2- 172.21.15.123:0/4209659437 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fcd3c0b2910 0x7fcd3c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:31.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.236+0000 7fcd54c00700 1 --2- 172.21.15.123:0/4209659437 >> [v2:172.21.15.163:6808/591680273,v1:172.21.15.163:6809/591680273] conn(0x7fcd3c0bc450 0x7fcd3c0be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:31.331 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.236+0000 7fcd54c00700 1 -- 172.21.15.123:0/4209659437 --> [v2:172.21.15.163:6808/591680273,v1:172.21.15.163:6809/591680273] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcd48016ec0 con 0x7fcd3c0bc450 2024-05-08T06:07:31.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.236+0000 7fcd54c00700 1 -- 172.21.15.123:0/4209659437 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fcd480e2dc0 con 0x7fcd5806ba10 2024-05-08T06:07:31.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.236+0000 7fcd5d000700 1 --2- 172.21.15.123:0/4209659437 >> [v2:172.21.15.163:6808/591680273,v1:172.21.15.163:6809/591680273] conn(0x7fcd3c0bc450 0x7fcd3c0be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:31.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.236+0000 7fcd57400700 1 --2- 172.21.15.123:0/4209659437 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fcd3c0b2910 0x7fcd3c0b4dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcd44009fd0 tx=0x7fcd44019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:31.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.237+0000 7fcd5d000700 1 --2- 172.21.15.123:0/4209659437 >> [v2:172.21.15.163:6808/591680273,v1:172.21.15.163:6809/591680273] conn(0x7fcd3c0bc450 0x7fcd3c0be8f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:31.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.238+0000 7fcd54c00700 1 -- 172.21.15.123:0/4209659437 <== osd.4 v2:172.21.15.163:6808/591680273 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcd48016ec0 con 0x7fcd3c0bc450 2024-05-08T06:07:31.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.256+0000 7fcd5da00700 1 -- 172.21.15.123:0/4209659437 --> [v2:172.21.15.163:6808/591680273,v1:172.21.15.163:6809/591680273] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcd580640b0 con 0x7fcd3c0bc450 2024-05-08T06:07:31.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.257+0000 7fcd54c00700 1 -- 172.21.15.123:0/4209659437 <== osd.4 v2:172.21.15.163:6808/591680273 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcd580640b0 con 0x7fcd3c0bc450 2024-05-08T06:07:31.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.257+0000 7fcd5da00700 1 -- 172.21.15.123:0/4209659437 >> [v2:172.21.15.163:6808/591680273,v1:172.21.15.163:6809/591680273] conn(0x7fcd3c0bc450 msgr2=0x7fcd3c0be8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:31.332 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.257+0000 7fcd5da00700 1 --2- 172.21.15.123:0/4209659437 >> [v2:172.21.15.163:6808/591680273,v1:172.21.15.163:6809/591680273] conn(0x7fcd3c0bc450 0x7fcd3c0be8f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:31.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.257+0000 7fcd5da00700 1 -- 172.21.15.123:0/4209659437 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fcd3c0b2910 msgr2=0x7fcd3c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:31.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.257+0000 7fcd5da00700 1 --2- 172.21.15.123:0/4209659437 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fcd3c0b2910 0x7fcd3c0b4dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcd44009fd0 tx=0x7fcd44019040 comp rx=0 tx=0).stop 2024-05-08T06:07:31.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.257+0000 7fcd5da00700 1 -- 172.21.15.123:0/4209659437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd5806ba10 msgr2=0x7fcd5819f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:31.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.257+0000 7fcd5da00700 1 --2- 172.21.15.123:0/4209659437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd5806ba10 0x7fcd5819f6a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcd4800e9c0 tx=0x7fcd4800ecd0 comp rx=0 tx=0).stop 2024-05-08T06:07:31.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.257+0000 7fcd5d000700 1 -- 172.21.15.123:0/4209659437 reap_dead start 2024-05-08T06:07:31.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.257+0000 7fcd5da00700 1 -- 172.21.15.123:0/4209659437 shutdown_connections 2024-05-08T06:07:31.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.257+0000 7fcd5da00700 1 -- 172.21.15.123:0/4209659437 >> 172.21.15.123:0/4209659437 conn(0x7fcd580791e0 msgr2=0x7fcd58100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:07:31.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.258+0000 7fcd5da00700 1 -- 172.21.15.123:0/4209659437 shutdown_connections 2024-05-08T06:07:31.333 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:31.258+0000 7fcd5da00700 1 -- 172.21.15.123:0/4209659437 wait complete. 2024-05-08T06:07:31.337 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:07:32.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.198+0000 7fe1dd200700 1 -- 172.21.15.123:0/1136160226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1d8109460 msgr2=0x7fe1d8109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:32.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.198+0000 7fe1dd200700 1 --2- 172.21.15.123:0/1136160226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1d8109460 0x7fe1d8109840 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8009990 tx=0x7fe1c8009ca0 comp rx=0 tx=0).stop 2024-05-08T06:07:32.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.198+0000 7fe1dd200700 1 -- 172.21.15.123:0/1136160226 shutdown_connections 2024-05-08T06:07:32.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.198+0000 7fe1dd200700 1 --2- 172.21.15.123:0/1136160226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1d8109460 0x7fe1d8109840 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:32.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.198+0000 7fe1dd200700 1 --2- 172.21.15.123:0/1136160226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1d8103d90 0x7fe1d81102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:32.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.198+0000 7fe1dd200700 1 --2- 172.21.15.123:0/1136160226 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1d81033d0 0x7fe1d8103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:32.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.198+0000 7fe1dd200700 1 -- 172.21.15.123:0/1136160226 >> 172.21.15.123:0/1136160226 conn(0x7fe1d80fef20 msgr2=0x7fe1d8101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:07:32.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.198+0000 7fe1dd200700 1 -- 172.21.15.123:0/1136160226 shutdown_connections 2024-05-08T06:07:32.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.198+0000 7fe1dd200700 1 -- 172.21.15.123:0/1136160226 wait complete. 2024-05-08T06:07:32.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.199+0000 7fe1dd200700 1 Processor -- start 2024-05-08T06:07:32.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.199+0000 7fe1dd200700 1 -- start start 2024-05-08T06:07:32.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.199+0000 7fe1dd200700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1d81033d0 0x7fe1d81a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:32.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.199+0000 7fe1dd200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1d8103d90 0x7fe1d81a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:32.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.199+0000 7fe1dd200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1d8109460 0x7fe1d819bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:32.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.200+0000 7fe1dd200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d8113590 con 0x7fe1d8103d90 2024-05-08T06:07:32.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.200+0000 7fe1dd200700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d8113410 con 0x7fe1d8109460 2024-05-08T06:07:32.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.200+0000 7fe1dd200700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d8113710 con 0x7fe1d81033d0 2024-05-08T06:07:32.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.200+0000 7fe1d6a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1d8103d90 0x7fe1d81a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:32.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.200+0000 7fe1d7400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1d81033d0 0x7fe1d81a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:32.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.200+0000 7fe1d6a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1d8103d90 0x7fe1d81a1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36090/0 (socket says 172.21.15.123:36090) 2024-05-08T06:07:32.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.200+0000 7fe1d6a00700 1 -- 172.21.15.123:0/211054343 learned_addr learned my addr 172.21.15.123:0/211054343 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:07:32.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.200+0000 7fe1d7e00700 1 --2- 172.21.15.123:0/211054343 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1d8109460 0x7fe1d819bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:32.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.200+0000 7fe1d6a00700 1 -- 172.21.15.123:0/211054343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1d81033d0 msgr2=0x7fe1d81a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:32.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.200+0000 7fe1d6a00700 1 --2- 172.21.15.123:0/211054343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1d81033d0 0x7fe1d81a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:32.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.200+0000 7fe1d6a00700 1 -- 172.21.15.123:0/211054343 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1d8109460 msgr2=0x7fe1d819bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:32.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.200+0000 7fe1d6a00700 1 --2- 172.21.15.123:0/211054343 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1d8109460 0x7fe1d819bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:32.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.200+0000 7fe1d6a00700 1 -- 172.21.15.123:0/211054343 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1d819c210 con 0x7fe1d8103d90 2024-05-08T06:07:32.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.200+0000 7fe1d7400700 1 --2- 172.21.15.123:0/211054343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1d81033d0 0x7fe1d81a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:07:32.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.201+0000 7fe1d6a00700 1 --2- 172.21.15.123:0/211054343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1d8103d90 0x7fe1d81a1e30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe1c400e9a0 tx=0x7fe1c400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:32.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.201+0000 7fe1cfe00700 1 -- 172.21.15.123:0/211054343 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c400ca10 con 0x7fe1d8103d90 2024-05-08T06:07:32.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.201+0000 7fe1cfe00700 1 -- 172.21.15.123:0/211054343 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1c400cbb0 con 0x7fe1d8103d90 2024-05-08T06:07:32.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.201+0000 7fe1dd200700 1 -- 172.21.15.123:0/211054343 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1d819c500 con 0x7fe1d8103d90 2024-05-08T06:07:32.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.201+0000 7fe1cfe00700 1 -- 172.21.15.123:0/211054343 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c4018680 con 0x7fe1d8103d90 2024-05-08T06:07:32.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.201+0000 7fe1dd200700 1 -- 172.21.15.123:0/211054343 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1d81a87c0 con 0x7fe1d8103d90 2024-05-08T06:07:32.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.202+0000 7fe1dd200700 1 -- 172.21.15.123:0/211054343 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe1d8100760 con 0x7fe1d8103d90 2024-05-08T06:07:32.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.205+0000 7fe1cfe00700 1 -- 172.21.15.123:0/211054343 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fe1c4010430 con 0x7fe1d8103d90 2024-05-08T06:07:32.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.207+0000 7fe1cfe00700 1 --2- 172.21.15.123:0/211054343 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe1bc0b2990 0x7fe1bc0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:32.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.207+0000 7fe1cfe00700 1 -- 172.21.15.123:0/211054343 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fe1c40e2af0 con 0x7fe1d8103d90 2024-05-08T06:07:32.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.207+0000 7fe1d7400700 1 --2- 172.21.15.123:0/211054343 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe1bc0b2990 0x7fe1bc0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:32.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.207+0000 7fe1cfe00700 1 --2- 172.21.15.123:0/211054343 >> [v2:172.21.15.192:6800/4161570404,v1:172.21.15.192:6801/4161570404] conn(0x7fe1bc0bc4d0 0x7fe1bc0be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:32.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.207+0000 7fe1cfe00700 1 -- 172.21.15.123:0/211054343 --> [v2:172.21.15.192:6800/4161570404,v1:172.21.15.192:6801/4161570404] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe1c4024e70 con 0x7fe1bc0bc4d0 2024-05-08T06:07:32.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.207+0000 7fe1cfe00700 1 -- 172.21.15.123:0/211054343 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fe1c40e2ea0 con 0x7fe1d8103d90 2024-05-08T06:07:32.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.208+0000 7fe1d7e00700 1 --2- 172.21.15.123:0/211054343 >> [v2:172.21.15.192:6800/4161570404,v1:172.21.15.192:6801/4161570404] conn(0x7fe1bc0bc4d0 0x7fe1bc0be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:32.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.208+0000 7fe1d7400700 1 --2- 172.21.15.123:0/211054343 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe1bc0b2990 0x7fe1bc0b4e50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe1c800b2e0 tx=0x7fe1c801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:32.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.208+0000 7fe1d7e00700 1 --2- 172.21.15.123:0/211054343 >> [v2:172.21.15.192:6800/4161570404,v1:172.21.15.192:6801/4161570404] conn(0x7fe1bc0bc4d0 0x7fe1bc0be970 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:32.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.209+0000 7fe1cfe00700 1 -- 172.21.15.123:0/211054343 <== osd.2 v2:172.21.15.192:6800/4161570404 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe1c4024e70 con 0x7fe1bc0bc4d0 2024-05-08T06:07:32.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.227+0000 7fe1dd200700 1 -- 172.21.15.123:0/211054343 --> [v2:172.21.15.192:6800/4161570404,v1:172.21.15.192:6801/4161570404] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe1d8104b10 con 0x7fe1bc0bc4d0 2024-05-08T06:07:32.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.228+0000 7fe1cfe00700 1 -- 172.21.15.123:0/211054343 <== osd.2 v2:172.21.15.192:6800/4161570404 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe1d8104b10 con 0x7fe1bc0bc4d0 2024-05-08T06:07:32.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.228+0000 7fe1dd200700 1 -- 172.21.15.123:0/211054343 >> [v2:172.21.15.192:6800/4161570404,v1:172.21.15.192:6801/4161570404] conn(0x7fe1bc0bc4d0 msgr2=0x7fe1bc0be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:32.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.228+0000 7fe1dd200700 1 --2- 172.21.15.123:0/211054343 >> [v2:172.21.15.192:6800/4161570404,v1:172.21.15.192:6801/4161570404] conn(0x7fe1bc0bc4d0 0x7fe1bc0be970 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:32.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.229+0000 7fe1dd200700 1 -- 172.21.15.123:0/211054343 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe1bc0b2990 msgr2=0x7fe1bc0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:32.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.229+0000 7fe1dd200700 1 --2- 172.21.15.123:0/211054343 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe1bc0b2990 0x7fe1bc0b4e50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe1c800b2e0 tx=0x7fe1c801a040 comp rx=0 tx=0).stop 2024-05-08T06:07:32.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.229+0000 7fe1dd200700 1 -- 172.21.15.123:0/211054343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1d8103d90 msgr2=0x7fe1d81a1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:32.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.229+0000 7fe1dd200700 1 --2- 172.21.15.123:0/211054343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1d8103d90 0x7fe1d81a1e30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe1c400e9a0 tx=0x7fe1c400ecb0 comp rx=0 tx=0).stop 2024-05-08T06:07:32.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.229+0000 7fe1d7e00700 1 -- 172.21.15.123:0/211054343 reap_dead start 2024-05-08T06:07:32.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.229+0000 7fe1dd200700 1 -- 172.21.15.123:0/211054343 shutdown_connections 2024-05-08T06:07:32.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.229+0000 7fe1dd200700 1 -- 172.21.15.123:0/211054343 >> 172.21.15.123:0/211054343 conn(0x7fe1d80fef20 msgr2=0x7fe1d810eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:07:32.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.229+0000 7fe1dd200700 1 -- 172.21.15.123:0/211054343 shutdown_connections 2024-05-08T06:07:32.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.229+0000 7fe1dd200700 1 -- 172.21.15.123:0/211054343 wait complete. 2024-05-08T06:07:32.345 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:32 smithi123 ceph-mon[29787]: pgmap v372: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:07:32.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:32 smithi163 ceph-mon[34520]: pgmap v372: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:07:32.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:32 smithi192 ceph-mon[34664]: pgmap v372: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:07:32.726 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:07:32.726 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:07:34.366 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:34 smithi123 ceph-mon[29787]: pgmap v373: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:07:34.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:34 smithi163 ceph-mon[34520]: pgmap v373: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:07:34.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:34 smithi192 ceph-mon[34664]: pgmap v373: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:07:35.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.345+0000 7f5900e00700 1 -- 172.21.15.123:0/2185532116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58fc076160 msgr2=0x7f58fc113ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:35.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.345+0000 7f5900e00700 1 --2- 172.21.15.123:0/2185532116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58fc076160 0x7f58fc113ea0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f58e4009990 tx=0x7f58e4009ca0 comp rx=0 tx=0).stop 2024-05-08T06:07:35.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.345+0000 7f5900e00700 1 -- 172.21.15.123:0/2185532116 shutdown_connections 2024-05-08T06:07:35.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.345+0000 7f5900e00700 1 --2- 172.21.15.123:0/2185532116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58fc1143e0 0x7f58fc1167d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:35.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.345+0000 7f5900e00700 1 --2- 172.21.15.123:0/2185532116 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58fc075840 0x7f58fc075c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:35.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.345+0000 7f5900e00700 1 --2- 172.21.15.123:0/2185532116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58fc076160 0x7f58fc113ea0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:35.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.345+0000 7f5900e00700 1 -- 172.21.15.123:0/2185532116 >> 172.21.15.123:0/2185532116 conn(0x7f58fc0fef20 msgr2=0x7f58fc101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:07:35.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.345+0000 7f5900e00700 1 -- 172.21.15.123:0/2185532116 shutdown_connections 2024-05-08T06:07:35.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.345+0000 7f5900e00700 1 -- 172.21.15.123:0/2185532116 wait complete. 2024-05-08T06:07:35.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.346+0000 7f5900e00700 1 Processor -- start 2024-05-08T06:07:35.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.347+0000 7f5900e00700 1 -- start start 2024-05-08T06:07:35.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.347+0000 7f5900e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58fc075840 0x7f58fc1a00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:35.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.347+0000 7f5900e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58fc076160 0x7f58fc1a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:35.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.347+0000 7f5900e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58fc1143e0 0x7f58fc1a49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:35.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.347+0000 7f5900e00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f58fc119c20 con 0x7f58fc076160 2024-05-08T06:07:35.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.347+0000 7f5900e00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f58fc119aa0 con 0x7f58fc075840 2024-05-08T06:07:35.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.347+0000 7f5900e00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f58fc119da0 con 0x7f58fc1143e0 2024-05-08T06:07:35.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.347+0000 7f58f3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58fc076160 0x7f58fc1a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:35.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.347+0000 7f58fb400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58fc075840 0x7f58fc1a00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:35.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.347+0000 7f58f3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58fc076160 0x7f58fc1a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36110/0 (socket says 172.21.15.123:36110) 2024-05-08T06:07:35.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.348+0000 7f58f3e00700 1 -- 172.21.15.123:0/3030006591 learned_addr learned my addr 172.21.15.123:0/3030006591 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:07:35.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.347+0000 7f58fb400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58fc075840 0x7f58fc1a00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:41630/0 (socket says 172.21.15.123:41630) 2024-05-08T06:07:35.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.348+0000 7f58f3e00700 1 -- 172.21.15.123:0/3030006591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58fc1143e0 msgr2=0x7f58fc1a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:35.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.348+0000 7f58f3e00700 1 --2- 172.21.15.123:0/3030006591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58fc1143e0 0x7f58fc1a49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:35.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.348+0000 7f58f3e00700 1 -- 172.21.15.123:0/3030006591 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58fc075840 msgr2=0x7f58fc1a00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:35.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.348+0000 7f58f3e00700 1 --2- 172.21.15.123:0/3030006591 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58fc075840 0x7f58fc1a00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:35.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.348+0000 7f58f3e00700 1 -- 172.21.15.123:0/3030006591 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58fc1a50e0 con 0x7f58fc076160 2024-05-08T06:07:35.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.348+0000 7f58f3e00700 1 --2- 172.21.15.123:0/3030006591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58fc076160 0x7f58fc1a0610 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f58e4005910 tx=0x7f58e400b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:35.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.349+0000 7f58f8c00700 1 -- 172.21.15.123:0/3030006591 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e4027070 con 0x7f58fc076160 2024-05-08T06:07:35.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.349+0000 7f58f8c00700 1 -- 172.21.15.123:0/3030006591 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58e4022470 con 0x7f58fc076160 2024-05-08T06:07:35.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.349+0000 7f58f8c00700 1 -- 172.21.15.123:0/3030006591 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e4017650 con 0x7f58fc076160 2024-05-08T06:07:35.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.349+0000 7f5900e00700 1 -- 172.21.15.123:0/3030006591 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58fc1a5370 con 0x7f58fc076160 2024-05-08T06:07:35.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.349+0000 7f5900e00700 1 -- 172.21.15.123:0/3030006591 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58fc1acc50 con 0x7f58fc076160 2024-05-08T06:07:35.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.350+0000 7f5900e00700 1 -- 172.21.15.123:0/3030006591 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f58fc063c30 con 0x7f58fc076160 2024-05-08T06:07:35.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.352+0000 7f58f8c00700 1 -- 172.21.15.123:0/3030006591 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f58e402a070 con 0x7f58fc076160 2024-05-08T06:07:35.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.354+0000 7f58f8c00700 1 --2- 172.21.15.123:0/3030006591 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f58e00b2990 0x7f58e00b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:35.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.354+0000 7f58f8c00700 1 -- 172.21.15.123:0/3030006591 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f58e40e3490 con 0x7f58fc076160 2024-05-08T06:07:35.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.354+0000 7f58fb400700 1 --2- 172.21.15.123:0/3030006591 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f58e00b2990 0x7f58e00b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:35.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.354+0000 7f58f8c00700 1 --2- 172.21.15.123:0/3030006591 >> [v2:172.21.15.192:6824/1339159347,v1:172.21.15.192:6825/1339159347] conn(0x7f58e00bc4d0 0x7f58e00be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:35.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.354+0000 7f58f8c00700 1 -- 172.21.15.123:0/3030006591 --> [v2:172.21.15.192:6824/1339159347,v1:172.21.15.192:6825/1339159347] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f58e400b0b0 con 0x7f58e00bc4d0 2024-05-08T06:07:35.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.354+0000 7f58f8c00700 1 -- 172.21.15.123:0/3030006591 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f58e40e38e0 con 0x7f58fc076160 2024-05-08T06:07:35.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.355+0000 7f58fbe00700 1 --2- 172.21.15.123:0/3030006591 >> [v2:172.21.15.192:6824/1339159347,v1:172.21.15.192:6825/1339159347] conn(0x7f58e00bc4d0 0x7f58e00be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:35.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.355+0000 7f58fb400700 1 --2- 172.21.15.123:0/3030006591 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f58e00b2990 0x7f58e00b4e50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f58ec009c60 tx=0x7f58ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:35.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.355+0000 7f58fbe00700 1 --2- 172.21.15.123:0/3030006591 >> [v2:172.21.15.192:6824/1339159347,v1:172.21.15.192:6825/1339159347] conn(0x7f58e00bc4d0 0x7f58e00be970 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:35.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.356+0000 7f58f8c00700 1 -- 172.21.15.123:0/3030006591 <== osd.11 v2:172.21.15.192:6824/1339159347 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f58e400b0b0 con 0x7f58e00bc4d0 2024-05-08T06:07:35.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.374+0000 7f5900e00700 1 -- 172.21.15.123:0/3030006591 --> [v2:172.21.15.192:6824/1339159347,v1:172.21.15.192:6825/1339159347] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f58fc064030 con 0x7f58e00bc4d0 2024-05-08T06:07:35.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.375+0000 7f58f8c00700 1 -- 172.21.15.123:0/3030006591 <== osd.11 v2:172.21.15.192:6824/1339159347 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f58fc064030 con 0x7f58e00bc4d0 2024-05-08T06:07:35.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.375+0000 7f5900e00700 1 -- 172.21.15.123:0/3030006591 >> [v2:172.21.15.192:6824/1339159347,v1:172.21.15.192:6825/1339159347] conn(0x7f58e00bc4d0 msgr2=0x7f58e00be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:35.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.375+0000 7f5900e00700 1 --2- 172.21.15.123:0/3030006591 >> [v2:172.21.15.192:6824/1339159347,v1:172.21.15.192:6825/1339159347] conn(0x7f58e00bc4d0 0x7f58e00be970 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:35.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.375+0000 7f5900e00700 1 -- 172.21.15.123:0/3030006591 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f58e00b2990 msgr2=0x7f58e00b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:35.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.375+0000 7f5900e00700 1 --2- 172.21.15.123:0/3030006591 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f58e00b2990 0x7f58e00b4e50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f58ec009c60 tx=0x7f58ec009260 comp rx=0 tx=0).stop 2024-05-08T06:07:35.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.375+0000 7f5900e00700 1 -- 172.21.15.123:0/3030006591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58fc076160 msgr2=0x7f58fc1a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:35.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.375+0000 7f5900e00700 1 --2- 172.21.15.123:0/3030006591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58fc076160 0x7f58fc1a0610 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f58e4005910 tx=0x7f58e400b670 comp rx=0 tx=0).stop 2024-05-08T06:07:35.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.375+0000 7f58fbe00700 1 -- 172.21.15.123:0/3030006591 reap_dead start 2024-05-08T06:07:35.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.376+0000 7f5900e00700 1 -- 172.21.15.123:0/3030006591 shutdown_connections 2024-05-08T06:07:35.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.376+0000 7f5900e00700 1 -- 172.21.15.123:0/3030006591 >> 172.21.15.123:0/3030006591 conn(0x7f58fc0fef20 msgr2=0x7f58fc100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:07:35.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.376+0000 7f5900e00700 1 -- 172.21.15.123:0/3030006591 shutdown_connections 2024-05-08T06:07:35.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:32.376+0000 7f5900e00700 1 -- 172.21.15.123:0/3030006591 wait complete. 2024-05-08T06:07:36.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:36 smithi163 ceph-mon[34520]: pgmap v374: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:07:36.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:36 smithi192 ceph-mon[34664]: pgmap v374: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:07:36.577 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:36 smithi123 ceph-mon[29787]: pgmap v374: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:07:37.286 INFO:teuthology.orchestra.run.smithi123.stdout:309237645350 2024-05-08T06:07:37.287 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-08T06:07:38.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:38 smithi163 ceph-mon[34520]: pgmap v375: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:07:38.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:38 smithi192 ceph-mon[34664]: pgmap v375: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:07:38.613 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:38 smithi123 ceph-mon[29787]: pgmap v375: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:07:38.759 INFO:teuthology.orchestra.run.smithi123.stdout:141733920846 2024-05-08T06:07:38.759 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-08T06:07:39.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.275+0000 7feac9a00700 1 -- 172.21.15.123:0/486024063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feac41099e0 msgr2=0x7feac4109e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:39.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.275+0000 7feac9a00700 1 --2- 172.21.15.123:0/486024063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feac41099e0 0x7feac4109e60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7feab0009990 tx=0x7feab0009ca0 comp rx=0 tx=0).stop 2024-05-08T06:07:39.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.275+0000 7feac9a00700 1 -- 172.21.15.123:0/486024063 shutdown_connections 2024-05-08T06:07:39.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.275+0000 7feac9a00700 1 --2- 172.21.15.123:0/486024063 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feac410a3a0 0x7feac4116840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:39.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.275+0000 7feac9a00700 1 --2- 172.21.15.123:0/486024063 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feac410e8b0 0x7feac410ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:39.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.275+0000 7feac9a00700 1 --2- 172.21.15.123:0/486024063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feac41099e0 0x7feac4109e60 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:39.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.275+0000 7feac9a00700 1 -- 172.21.15.123:0/486024063 >> 172.21.15.123:0/486024063 conn(0x7feac4079230 msgr2=0x7feac41082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:07:39.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.275+0000 7feac9a00700 1 -- 172.21.15.123:0/486024063 shutdown_connections 2024-05-08T06:07:39.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.275+0000 7feac9a00700 1 -- 172.21.15.123:0/486024063 wait complete. 2024-05-08T06:07:39.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.277+0000 7feac9a00700 1 Processor -- start 2024-05-08T06:07:39.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.277+0000 7feac9a00700 1 -- start start 2024-05-08T06:07:39.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.277+0000 7feac9a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feac41099e0 0x7feac41a0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:39.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.277+0000 7feac9a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feac410a3a0 0x7feac41a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:39.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.277+0000 7feac9a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feac410e8b0 0x7feac41a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:39.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.277+0000 7feac9a00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feac4119be0 con 0x7feac41099e0 2024-05-08T06:07:39.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.277+0000 7feac9a00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7feac4119a60 con 0x7feac410a3a0 2024-05-08T06:07:39.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.277+0000 7feac9a00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7feac4119d60 con 0x7feac410e8b0 2024-05-08T06:07:39.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.278+0000 7feac3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feac41099e0 0x7feac41a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:39.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.278+0000 7feac3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feac41099e0 0x7feac41a0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53308/0 (socket says 172.21.15.123:53308) 2024-05-08T06:07:39.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.278+0000 7feac3e00700 1 -- 172.21.15.123:0/4213467739 learned_addr learned my addr 172.21.15.123:0/4213467739 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:07:39.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.278+0000 7feac3400700 1 --2- 172.21.15.123:0/4213467739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feac410a3a0 0x7feac41a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:39.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.278+0000 7feac9000700 1 --2- 172.21.15.123:0/4213467739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feac410e8b0 0x7feac41a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:39.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.278+0000 7feac3e00700 1 -- 172.21.15.123:0/4213467739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feac410e8b0 msgr2=0x7feac41a4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:39.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.278+0000 7feac3e00700 1 --2- 172.21.15.123:0/4213467739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feac410e8b0 0x7feac41a4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:39.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.278+0000 7feac3e00700 1 -- 172.21.15.123:0/4213467739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feac410a3a0 msgr2=0x7feac41a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:39.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.278+0000 7feac3e00700 1 --2- 172.21.15.123:0/4213467739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feac410a3a0 0x7feac41a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:39.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.278+0000 7feac3e00700 1 -- 172.21.15.123:0/4213467739 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac41a5080 con 0x7feac41099e0 2024-05-08T06:07:39.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.278+0000 7feac3e00700 1 --2- 172.21.15.123:0/4213467739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feac41099e0 0x7feac41a0070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7feab400e9a0 tx=0x7feab400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:39.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.279+0000 7feac0c00700 1 -- 172.21.15.123:0/4213467739 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab400ca10 con 0x7feac41099e0 2024-05-08T06:07:39.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.279+0000 7feac0c00700 1 -- 172.21.15.123:0/4213467739 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feab400cbb0 con 0x7feac41099e0 2024-05-08T06:07:39.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.279+0000 7feac9a00700 1 -- 172.21.15.123:0/4213467739 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feac41a5370 con 0x7feac41099e0 2024-05-08T06:07:39.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.279+0000 7feac0c00700 1 -- 172.21.15.123:0/4213467739 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab4018680 con 0x7feac41099e0 2024-05-08T06:07:39.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.279+0000 7feac9a00700 1 -- 172.21.15.123:0/4213467739 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feac41acbf0 con 0x7feac41099e0 2024-05-08T06:07:39.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.280+0000 7feac9a00700 1 -- 172.21.15.123:0/4213467739 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7feac4063c30 con 0x7feac41099e0 2024-05-08T06:07:39.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.282+0000 7feac0c00700 1 -- 172.21.15.123:0/4213467739 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7feab4010430 con 0x7feac41099e0 2024-05-08T06:07:39.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.284+0000 7feac0c00700 1 --2- 172.21.15.123:0/4213467739 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7feaa80b2990 0x7feaa80b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:39.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.284+0000 7feac0c00700 1 -- 172.21.15.123:0/4213467739 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7feab40e2a90 con 0x7feac41099e0 2024-05-08T06:07:39.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.284+0000 7feac3400700 1 --2- 172.21.15.123:0/4213467739 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7feaa80b2990 0x7feaa80b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:39.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.284+0000 7feac0c00700 1 --2- 172.21.15.123:0/4213467739 >> [v2:172.21.15.163:6824/2628487883,v1:172.21.15.163:6825/2628487883] conn(0x7feaa80bc4d0 0x7feaa80be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:39.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.284+0000 7feac0c00700 1 -- 172.21.15.123:0/4213467739 --> [v2:172.21.15.163:6824/2628487883,v1:172.21.15.163:6825/2628487883] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feab4024e70 con 0x7feaa80bc4d0 2024-05-08T06:07:39.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.284+0000 7feac0c00700 1 -- 172.21.15.123:0/4213467739 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7feab40e2e40 con 0x7feac41099e0 2024-05-08T06:07:39.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.285+0000 7feac9000700 1 --2- 172.21.15.123:0/4213467739 >> [v2:172.21.15.163:6824/2628487883,v1:172.21.15.163:6825/2628487883] conn(0x7feaa80bc4d0 0x7feaa80be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:39.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.285+0000 7feac3400700 1 --2- 172.21.15.123:0/4213467739 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7feaa80b2990 0x7feaa80b4e50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7feac4109590 tx=0x7feab000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:39.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.285+0000 7feac9000700 1 --2- 172.21.15.123:0/4213467739 >> [v2:172.21.15.163:6824/2628487883,v1:172.21.15.163:6825/2628487883] conn(0x7feaa80bc4d0 0x7feaa80be970 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:39.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.286+0000 7feac0c00700 1 -- 172.21.15.123:0/4213467739 <== osd.10 v2:172.21.15.163:6824/2628487883 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feab4024e70 con 0x7feaa80bc4d0 2024-05-08T06:07:39.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.304+0000 7feac9a00700 1 -- 172.21.15.123:0/4213467739 --> [v2:172.21.15.163:6824/2628487883,v1:172.21.15.163:6825/2628487883] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7feac4064030 con 0x7feaa80bc4d0 2024-05-08T06:07:39.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.305+0000 7feac0c00700 1 -- 172.21.15.123:0/4213467739 <== osd.10 v2:172.21.15.163:6824/2628487883 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7feac4064030 con 0x7feaa80bc4d0 2024-05-08T06:07:39.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.305+0000 7feac9a00700 1 -- 172.21.15.123:0/4213467739 >> [v2:172.21.15.163:6824/2628487883,v1:172.21.15.163:6825/2628487883] conn(0x7feaa80bc4d0 msgr2=0x7feaa80be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:39.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.305+0000 7feac9a00700 1 --2- 172.21.15.123:0/4213467739 >> [v2:172.21.15.163:6824/2628487883,v1:172.21.15.163:6825/2628487883] conn(0x7feaa80bc4d0 0x7feaa80be970 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:39.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.305+0000 7feac9a00700 1 -- 172.21.15.123:0/4213467739 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7feaa80b2990 msgr2=0x7feaa80b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:39.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.305+0000 7feac9a00700 1 --2- 172.21.15.123:0/4213467739 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7feaa80b2990 0x7feaa80b4e50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7feac4109590 tx=0x7feab000b260 comp rx=0 tx=0).stop 2024-05-08T06:07:39.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.305+0000 7feac9a00700 1 -- 172.21.15.123:0/4213467739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feac41099e0 msgr2=0x7feac41a0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:39.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.305+0000 7feac9a00700 1 --2- 172.21.15.123:0/4213467739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feac41099e0 0x7feac41a0070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7feab400e9a0 tx=0x7feab400ecb0 comp rx=0 tx=0).stop 2024-05-08T06:07:39.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.305+0000 7feac9000700 1 -- 172.21.15.123:0/4213467739 reap_dead start 2024-05-08T06:07:39.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.305+0000 7feac9a00700 1 -- 172.21.15.123:0/4213467739 shutdown_connections 2024-05-08T06:07:39.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.305+0000 7feac9a00700 1 -- 172.21.15.123:0/4213467739 >> 172.21.15.123:0/4213467739 conn(0x7feac4079230 msgr2=0x7feac40798b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:07:39.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.306+0000 7feac9a00700 1 -- 172.21.15.123:0/4213467739 shutdown_connections 2024-05-08T06:07:39.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:39.306+0000 7feac9a00700 1 -- 172.21.15.123:0/4213467739 wait complete. 2024-05-08T06:07:39.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.741+0000 7f29b0a00700 1 -- 172.21.15.123:0/3893982662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29ac1023a0 msgr2=0x7f29ac102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:39.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.741+0000 7f29b0a00700 1 --2- 172.21.15.123:0/3893982662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29ac1023a0 0x7f29ac102820 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2998009990 tx=0x7f2998009ca0 comp rx=0 tx=0).stop 2024-05-08T06:07:39.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.741+0000 7f29b0a00700 1 -- 172.21.15.123:0/3893982662 shutdown_connections 2024-05-08T06:07:39.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.741+0000 7f29b0a00700 1 --2- 172.21.15.123:0/3893982662 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29ac102d60 0x7f29ac10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:39.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.741+0000 7f29b0a00700 1 --2- 172.21.15.123:0/3893982662 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29ac108410 0x7f29ac1087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:39.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.741+0000 7f29b0a00700 1 --2- 172.21.15.123:0/3893982662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29ac1023a0 0x7f29ac102820 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:39.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.741+0000 7f29b0a00700 1 -- 172.21.15.123:0/3893982662 >> 172.21.15.123:0/3893982662 conn(0x7f29ac079350 msgr2=0x7f29ac079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:07:39.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.741+0000 7f29b0a00700 1 -- 172.21.15.123:0/3893982662 shutdown_connections 2024-05-08T06:07:39.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.741+0000 7f29b0a00700 1 -- 172.21.15.123:0/3893982662 wait complete. 2024-05-08T06:07:39.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.743+0000 7f29b0a00700 1 Processor -- start 2024-05-08T06:07:39.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.744+0000 7f29b0a00700 1 -- start start 2024-05-08T06:07:39.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.744+0000 7f29b0a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29ac1023a0 0x7f29ac19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:39.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.744+0000 7f29b0a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29ac102d60 0x7f29ac19fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:39.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.744+0000 7f29b0a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29ac108410 0x7f29ac199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:39.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.744+0000 7f29b0a00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f29ac112580 con 0x7f29ac102d60 2024-05-08T06:07:39.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.744+0000 7f29b0a00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f29ac112400 con 0x7f29ac108410 2024-05-08T06:07:39.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.744+0000 7f29b0a00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f29ac112700 con 0x7f29ac1023a0 2024-05-08T06:07:39.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.745+0000 7f29a9400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29ac102d60 0x7f29ac19fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:39.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.745+0000 7f29a9e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29ac1023a0 0x7f29ac19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:39.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.745+0000 7f29a9400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29ac102d60 0x7f29ac19fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53292/0 (socket says 172.21.15.123:53292) 2024-05-08T06:07:39.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.745+0000 7f29a9400700 1 -- 172.21.15.123:0/4233587589 learned_addr learned my addr 172.21.15.123:0/4233587589 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:07:39.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.745+0000 7f29aa800700 1 --2- 172.21.15.123:0/4233587589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29ac108410 0x7f29ac199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:39.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.745+0000 7f29a9e00700 1 -- 172.21.15.123:0/4233587589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29ac108410 msgr2=0x7f29ac199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:39.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.745+0000 7f29a9e00700 1 --2- 172.21.15.123:0/4233587589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29ac108410 0x7f29ac199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:39.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.745+0000 7f29a9e00700 1 -- 172.21.15.123:0/4233587589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29ac102d60 msgr2=0x7f29ac19fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:39.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.745+0000 7f29a9e00700 1 --2- 172.21.15.123:0/4233587589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29ac102d60 0x7f29ac19fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:39.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.745+0000 7f29a9e00700 1 -- 172.21.15.123:0/4233587589 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29ac19a0f0 con 0x7f29ac1023a0 2024-05-08T06:07:39.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.746+0000 7f29a9e00700 1 --2- 172.21.15.123:0/4233587589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29ac1023a0 0x7f29ac19f7d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f299c00e9f0 tx=0x7f299c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:39.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.746+0000 7f29a2a00700 1 -- 172.21.15.123:0/4233587589 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299c00cbd0 con 0x7f29ac1023a0 2024-05-08T06:07:39.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.746+0000 7f29a2a00700 1 -- 172.21.15.123:0/4233587589 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f299c00cd70 con 0x7f29ac1023a0 2024-05-08T06:07:39.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.746+0000 7f29b0a00700 1 -- 172.21.15.123:0/4233587589 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29ac19a3e0 con 0x7f29ac1023a0 2024-05-08T06:07:39.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.746+0000 7f29a2a00700 1 -- 172.21.15.123:0/4233587589 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299c018680 con 0x7f29ac1023a0 2024-05-08T06:07:39.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.746+0000 7f29b0a00700 1 -- 172.21.15.123:0/4233587589 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29ac103fb0 con 0x7f29ac1023a0 2024-05-08T06:07:39.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.747+0000 7f29b0a00700 1 -- 172.21.15.123:0/4233587589 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f29ac063c30 con 0x7f29ac1023a0 2024-05-08T06:07:39.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.749+0000 7f29a2a00700 1 -- 172.21.15.123:0/4233587589 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f299c010430 con 0x7f29ac1023a0 2024-05-08T06:07:39.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.751+0000 7f29a2a00700 1 --2- 172.21.15.123:0/4233587589 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f29900b2990 0x7f29900b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:39.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.751+0000 7f29a2a00700 1 -- 172.21.15.123:0/4233587589 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f299c0e2fe0 con 0x7f29ac1023a0 2024-05-08T06:07:39.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.751+0000 7f29a9400700 1 --2- 172.21.15.123:0/4233587589 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f29900b2990 0x7f29900b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:39.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.751+0000 7f29a2a00700 1 --2- 172.21.15.123:0/4233587589 >> [v2:172.21.15.192:6808/1734821180,v1:172.21.15.192:6809/1734821180] conn(0x7f29900bc4d0 0x7f29900be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:39.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.751+0000 7f29a2a00700 1 -- 172.21.15.123:0/4233587589 --> [v2:172.21.15.192:6808/1734821180,v1:172.21.15.192:6809/1734821180] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f299c0090b0 con 0x7f29900bc4d0 2024-05-08T06:07:39.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.751+0000 7f29a2a00700 1 -- 172.21.15.123:0/4233587589 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f299c0e33f0 con 0x7f29ac1023a0 2024-05-08T06:07:39.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.752+0000 7f29aa800700 1 --2- 172.21.15.123:0/4233587589 >> [v2:172.21.15.192:6808/1734821180,v1:172.21.15.192:6809/1734821180] conn(0x7f29900bc4d0 0x7f29900be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:39.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.752+0000 7f29a9400700 1 --2- 172.21.15.123:0/4233587589 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f29900b2990 0x7f29900b4e50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f29ac0798c0 tx=0x7f299800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:39.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.752+0000 7f29aa800700 1 --2- 172.21.15.123:0/4233587589 >> [v2:172.21.15.192:6808/1734821180,v1:172.21.15.192:6809/1734821180] conn(0x7f29900bc4d0 0x7f29900be970 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:39.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.753+0000 7f29a2a00700 1 -- 172.21.15.123:0/4233587589 <== osd.5 v2:172.21.15.192:6808/1734821180 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f299c0090b0 con 0x7f29900bc4d0 2024-05-08T06:07:39.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.791+0000 7f29b0a00700 1 -- 172.21.15.123:0/4233587589 --> [v2:172.21.15.192:6808/1734821180,v1:172.21.15.192:6809/1734821180] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f29ac064030 con 0x7f29900bc4d0 2024-05-08T06:07:39.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.792+0000 7f29a2a00700 1 -- 172.21.15.123:0/4233587589 <== osd.5 v2:172.21.15.192:6808/1734821180 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f29ac064030 con 0x7f29900bc4d0 2024-05-08T06:07:39.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.792+0000 7f29b0a00700 1 -- 172.21.15.123:0/4233587589 >> [v2:172.21.15.192:6808/1734821180,v1:172.21.15.192:6809/1734821180] conn(0x7f29900bc4d0 msgr2=0x7f29900be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:39.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.792+0000 7f29b0a00700 1 --2- 172.21.15.123:0/4233587589 >> [v2:172.21.15.192:6808/1734821180,v1:172.21.15.192:6809/1734821180] conn(0x7f29900bc4d0 0x7f29900be970 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:39.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.793+0000 7f29b0a00700 1 -- 172.21.15.123:0/4233587589 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f29900b2990 msgr2=0x7f29900b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:39.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.793+0000 7f29b0a00700 1 --2- 172.21.15.123:0/4233587589 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f29900b2990 0x7f29900b4e50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f29ac0798c0 tx=0x7f299800b260 comp rx=0 tx=0).stop 2024-05-08T06:07:39.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.793+0000 7f29b0a00700 1 -- 172.21.15.123:0/4233587589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29ac1023a0 msgr2=0x7f29ac19f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:39.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.793+0000 7f29b0a00700 1 --2- 172.21.15.123:0/4233587589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29ac1023a0 0x7f29ac19f7d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f299c00e9f0 tx=0x7f299c00ed00 comp rx=0 tx=0).stop 2024-05-08T06:07:39.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.793+0000 7f29aa800700 1 -- 172.21.15.123:0/4233587589 reap_dead start 2024-05-08T06:07:39.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.793+0000 7f29b0a00700 1 -- 172.21.15.123:0/4233587589 shutdown_connections 2024-05-08T06:07:39.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.793+0000 7f29b0a00700 1 -- 172.21.15.123:0/4233587589 >> 172.21.15.123:0/4233587589 conn(0x7f29ac079350 msgr2=0x7f29ac1016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:07:39.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.793+0000 7f29b0a00700 1 -- 172.21.15.123:0/4233587589 shutdown_connections 2024-05-08T06:07:39.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:36.793+0000 7f29b0a00700 1 -- 172.21.15.123:0/4233587589 wait complete. 2024-05-08T06:07:40.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:40 smithi163 ceph-mon[34520]: pgmap v376: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:07:40.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:40 smithi192 ceph-mon[34664]: pgmap v376: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:07:40.640 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:40 smithi123 ceph-mon[29787]: pgmap v376: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:07:42.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.323+0000 7ffa73400700 1 -- 172.21.15.123:0/723808685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa6c06bb40 msgr2=0x7ffa6c110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:42.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.323+0000 7ffa73400700 1 --2- 172.21.15.123:0/723808685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa6c06bb40 0x7ffa6c110330 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ffa64009990 tx=0x7ffa64009ca0 comp rx=0 tx=0).stop 2024-05-08T06:07:42.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.323+0000 7ffa73400700 1 -- 172.21.15.123:0/723808685 shutdown_connections 2024-05-08T06:07:42.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.323+0000 7ffa73400700 1 --2- 172.21.15.123:0/723808685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa6c06bb40 0x7ffa6c110330 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:42.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.323+0000 7ffa73400700 1 --2- 172.21.15.123:0/723808685 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa6c109510 0x7ffa6c1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:42.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.323+0000 7ffa73400700 1 --2- 172.21.15.123:0/723808685 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa6c06b180 0x7ffa6c06b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:42.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.323+0000 7ffa73400700 1 -- 172.21.15.123:0/723808685 >> 172.21.15.123:0/723808685 conn(0x7ffa6c079350 msgr2=0x7ffa6c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:07:42.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.323+0000 7ffa73400700 1 -- 172.21.15.123:0/723808685 shutdown_connections 2024-05-08T06:07:42.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.323+0000 7ffa73400700 1 -- 172.21.15.123:0/723808685 wait complete. 2024-05-08T06:07:42.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.324+0000 7ffa73400700 1 Processor -- start 2024-05-08T06:07:42.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.324+0000 7ffa73400700 1 -- start start 2024-05-08T06:07:42.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.325+0000 7ffa73400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa6c06b180 0x7ffa6c19bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:42.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.325+0000 7ffa73400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa6c06bb40 0x7ffa6c19c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:42.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.325+0000 7ffa73400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa6c109510 0x7ffa6c1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:42.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.325+0000 7ffa73400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6c113690 con 0x7ffa6c06bb40 2024-05-08T06:07:42.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.325+0000 7ffa73400700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6c113510 con 0x7ffa6c109510 2024-05-08T06:07:42.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.325+0000 7ffa73400700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6c113810 con 0x7ffa6c06b180 2024-05-08T06:07:42.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.325+0000 7ffa6be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa6c06bb40 0x7ffa6c19c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:42.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.325+0000 7ffa70a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa6c06b180 0x7ffa6c19bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:42.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.325+0000 7ffa6be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa6c06bb40 0x7ffa6c19c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53336/0 (socket says 172.21.15.123:53336) 2024-05-08T06:07:42.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.325+0000 7ffa6be00700 1 -- 172.21.15.123:0/3895791167 learned_addr learned my addr 172.21.15.123:0/3895791167 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:07:42.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.325+0000 7ffa71400700 1 --2- 172.21.15.123:0/3895791167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa6c109510 0x7ffa6c1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:42.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.326+0000 7ffa6be00700 1 -- 172.21.15.123:0/3895791167 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa6c06b180 msgr2=0x7ffa6c19bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:42.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.326+0000 7ffa6be00700 1 --2- 172.21.15.123:0/3895791167 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa6c06b180 0x7ffa6c19bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:42.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.326+0000 7ffa6be00700 1 -- 172.21.15.123:0/3895791167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa6c109510 msgr2=0x7ffa6c1a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:42.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.326+0000 7ffa6be00700 1 --2- 172.21.15.123:0/3895791167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa6c109510 0x7ffa6c1a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:42.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.326+0000 7ffa6be00700 1 -- 172.21.15.123:0/3895791167 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa6c1a0df0 con 0x7ffa6c06bb40 2024-05-08T06:07:42.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.326+0000 7ffa70a00700 1 --2- 172.21.15.123:0/3895791167 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa6c06b180 0x7ffa6c19bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:07:42.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.326+0000 7ffa6be00700 1 --2- 172.21.15.123:0/3895791167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa6c06bb40 0x7ffa6c19c320 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ffa5800e9a0 tx=0x7ffa5800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:42.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.326+0000 7ffa69600700 1 -- 172.21.15.123:0/3895791167 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5800ca10 con 0x7ffa6c06bb40 2024-05-08T06:07:42.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.326+0000 7ffa69600700 1 -- 172.21.15.123:0/3895791167 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa5800cbb0 con 0x7ffa6c06bb40 2024-05-08T06:07:42.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.326+0000 7ffa73400700 1 -- 172.21.15.123:0/3895791167 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa6c1a10e0 con 0x7ffa6c06bb40 2024-05-08T06:07:42.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.326+0000 7ffa69600700 1 -- 172.21.15.123:0/3895791167 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa58018680 con 0x7ffa6c06bb40 2024-05-08T06:07:42.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.326+0000 7ffa73400700 1 -- 172.21.15.123:0/3895791167 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa6c1a8960 con 0x7ffa6c06bb40 2024-05-08T06:07:42.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.327+0000 7ffa73400700 1 -- 172.21.15.123:0/3895791167 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffa6c10f060 con 0x7ffa6c06bb40 2024-05-08T06:07:42.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.330+0000 7ffa69600700 1 -- 172.21.15.123:0/3895791167 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7ffa58010430 con 0x7ffa6c06bb40 2024-05-08T06:07:42.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.331+0000 7ffa69600700 1 --2- 172.21.15.123:0/3895791167 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7ffa540b2910 0x7ffa540b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:42.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.331+0000 7ffa69600700 1 -- 172.21.15.123:0/3895791167 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7ffa580e2b70 con 0x7ffa6c06bb40 2024-05-08T06:07:42.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.332+0000 7ffa70a00700 1 --2- 172.21.15.123:0/3895791167 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7ffa540b2910 0x7ffa540b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:42.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.332+0000 7ffa69600700 1 --2- 172.21.15.123:0/3895791167 >> [v2:172.21.15.163:6800/1884753449,v1:172.21.15.163:6801/1884753449] conn(0x7ffa540bc450 0x7ffa540be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:42.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.332+0000 7ffa69600700 1 -- 172.21.15.123:0/3895791167 --> [v2:172.21.15.163:6800/1884753449,v1:172.21.15.163:6801/1884753449] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffa58024e70 con 0x7ffa540bc450 2024-05-08T06:07:42.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.332+0000 7ffa69600700 1 -- 172.21.15.123:0/3895791167 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7ffa580e2f20 con 0x7ffa6c06bb40 2024-05-08T06:07:42.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.332+0000 7ffa70a00700 1 --2- 172.21.15.123:0/3895791167 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7ffa540b2910 0x7ffa540b4dd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ffa60003e30 tx=0x7ffa6000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:42.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.332+0000 7ffa71400700 1 --2- 172.21.15.123:0/3895791167 >> [v2:172.21.15.163:6800/1884753449,v1:172.21.15.163:6801/1884753449] conn(0x7ffa540bc450 0x7ffa540be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:42.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.333+0000 7ffa71400700 1 --2- 172.21.15.123:0/3895791167 >> [v2:172.21.15.163:6800/1884753449,v1:172.21.15.163:6801/1884753449] conn(0x7ffa540bc450 0x7ffa540be8f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:42.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.334+0000 7ffa69600700 1 -- 172.21.15.123:0/3895791167 <== osd.1 v2:172.21.15.163:6800/1884753449 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa58024e70 con 0x7ffa540bc450 2024-05-08T06:07:42.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.352+0000 7ffa73400700 1 -- 172.21.15.123:0/3895791167 --> [v2:172.21.15.163:6800/1884753449,v1:172.21.15.163:6801/1884753449] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffa6c104bd0 con 0x7ffa540bc450 2024-05-08T06:07:42.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.352+0000 7ffa69600700 1 -- 172.21.15.123:0/3895791167 <== osd.1 v2:172.21.15.163:6800/1884753449 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ffa6c104bd0 con 0x7ffa540bc450 2024-05-08T06:07:42.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.353+0000 7ffa73400700 1 -- 172.21.15.123:0/3895791167 >> [v2:172.21.15.163:6800/1884753449,v1:172.21.15.163:6801/1884753449] conn(0x7ffa540bc450 msgr2=0x7ffa540be8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:42.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.353+0000 7ffa73400700 1 --2- 172.21.15.123:0/3895791167 >> [v2:172.21.15.163:6800/1884753449,v1:172.21.15.163:6801/1884753449] conn(0x7ffa540bc450 0x7ffa540be8f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:42.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.353+0000 7ffa73400700 1 -- 172.21.15.123:0/3895791167 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7ffa540b2910 msgr2=0x7ffa540b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:42.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.353+0000 7ffa73400700 1 --2- 172.21.15.123:0/3895791167 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7ffa540b2910 0x7ffa540b4dd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ffa60003e30 tx=0x7ffa6000b040 comp rx=0 tx=0).stop 2024-05-08T06:07:42.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.353+0000 7ffa73400700 1 -- 172.21.15.123:0/3895791167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa6c06bb40 msgr2=0x7ffa6c19c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:42.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.353+0000 7ffa73400700 1 --2- 172.21.15.123:0/3895791167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa6c06bb40 0x7ffa6c19c320 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ffa5800e9a0 tx=0x7ffa5800ecb0 comp rx=0 tx=0).stop 2024-05-08T06:07:42.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.353+0000 7ffa71400700 1 -- 172.21.15.123:0/3895791167 reap_dead start 2024-05-08T06:07:42.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.353+0000 7ffa73400700 1 -- 172.21.15.123:0/3895791167 shutdown_connections 2024-05-08T06:07:42.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.353+0000 7ffa73400700 1 -- 172.21.15.123:0/3895791167 >> 172.21.15.123:0/3895791167 conn(0x7ffa6c079350 msgr2=0x7ffa6c10e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:07:42.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.354+0000 7ffa73400700 1 -- 172.21.15.123:0/3895791167 shutdown_connections 2024-05-08T06:07:42.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:42.354+0000 7ffa73400700 1 -- 172.21.15.123:0/3895791167 wait complete. 2024-05-08T06:07:42.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:42 smithi163 ceph-mon[34520]: pgmap v377: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-05-08T06:07:42.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:42 smithi192 ceph-mon[34664]: pgmap v377: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-05-08T06:07:42.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:42 smithi123 ceph-mon[29787]: pgmap v377: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-05-08T06:07:42.995 INFO:teuthology.orchestra.run.smithi123.stdout:115964117075 2024-05-08T06:07:42.996 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-08T06:07:44.356 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:44 smithi123 ceph-mon[29787]: pgmap v378: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T06:07:44.357 INFO:teuthology.orchestra.run.smithi123.stdout:283467841581 2024-05-08T06:07:44.358 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-08T06:07:44.364 INFO:teuthology.orchestra.run.smithi123.stdout:163208757321 2024-05-08T06:07:44.364 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-08T06:07:44.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:44 smithi163 ceph-mon[34520]: pgmap v378: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T06:07:44.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:44 smithi192 ceph-mon[34664]: pgmap v378: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T06:07:45.253 INFO:teuthology.orchestra.run.smithi123.stdout:77309411418 2024-05-08T06:07:45.254 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-08T06:07:45.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.059+0000 7f4cb1400700 1 -- 172.21.15.123:0/2026934616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4cac1033d0 msgr2=0x7f4cac103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:45.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.059+0000 7f4cb1400700 1 --2- 172.21.15.123:0/2026934616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4cac1033d0 0x7f4cac103850 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4c980098f0 tx=0x7f4c98009c00 comp rx=0 tx=0).stop 2024-05-08T06:07:45.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.059+0000 7f4cb1400700 1 -- 172.21.15.123:0/2026934616 shutdown_connections 2024-05-08T06:07:45.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.059+0000 7f4cb1400700 1 --2- 172.21.15.123:0/2026934616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cac109460 0x7f4cac109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:45.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.059+0000 7f4cb1400700 1 --2- 172.21.15.123:0/2026934616 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cac103d90 0x7f4cac1102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:45.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.059+0000 7f4cb1400700 1 --2- 172.21.15.123:0/2026934616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4cac1033d0 0x7f4cac103850 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:45.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.059+0000 7f4cb1400700 1 -- 172.21.15.123:0/2026934616 >> 172.21.15.123:0/2026934616 conn(0x7f4cac0fef20 msgr2=0x7f4cac101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:07:45.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.059+0000 7f4cb1400700 1 -- 172.21.15.123:0/2026934616 shutdown_connections 2024-05-08T06:07:45.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.059+0000 7f4cb1400700 1 -- 172.21.15.123:0/2026934616 wait complete. 2024-05-08T06:07:45.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.060+0000 7f4cb1400700 1 Processor -- start 2024-05-08T06:07:45.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.060+0000 7f4cb1400700 1 -- start start 2024-05-08T06:07:45.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.061+0000 7f4cb1400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4cac1033d0 0x7f4cac1a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:45.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.061+0000 7f4cb1400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cac103d90 0x7f4cac1a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:45.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.061+0000 7f4cb1400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cac109460 0x7f4cac19bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:45.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.061+0000 7f4cb1400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac113570 con 0x7f4cac103d90 2024-05-08T06:07:45.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.061+0000 7f4cb1400700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac1133f0 con 0x7f4cac109460 2024-05-08T06:07:45.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.061+0000 7f4cb1400700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac1136f0 con 0x7f4cac1033d0 2024-05-08T06:07:45.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.061+0000 7f4cab400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cac103d90 0x7f4cac1a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:45.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.061+0000 7f4cabe00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4cac1033d0 0x7f4cac1a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:45.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.061+0000 7f4cab400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cac103d90 0x7f4cac1a1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53352/0 (socket says 172.21.15.123:53352) 2024-05-08T06:07:45.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.061+0000 7f4cab400700 1 -- 172.21.15.123:0/2843547503 learned_addr learned my addr 172.21.15.123:0/2843547503 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:07:45.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.061+0000 7f4cb0a00700 1 --2- 172.21.15.123:0/2843547503 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cac109460 0x7f4cac19bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:45.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.061+0000 7f4cab400700 1 -- 172.21.15.123:0/2843547503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4cac1033d0 msgr2=0x7f4cac1a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:45.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.061+0000 7f4cab400700 1 --2- 172.21.15.123:0/2843547503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4cac1033d0 0x7f4cac1a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:45.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.061+0000 7f4cab400700 1 -- 172.21.15.123:0/2843547503 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cac109460 msgr2=0x7f4cac19bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:45.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.061+0000 7f4cab400700 1 --2- 172.21.15.123:0/2843547503 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cac109460 0x7f4cac19bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:45.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.061+0000 7f4cab400700 1 -- 172.21.15.123:0/2843547503 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cac19c210 con 0x7f4cac103d90 2024-05-08T06:07:45.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.061+0000 7f4cabe00700 1 --2- 172.21.15.123:0/2843547503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4cac1033d0 0x7f4cac1a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:07:45.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.062+0000 7f4cab400700 1 --2- 172.21.15.123:0/2843547503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cac103d90 0x7f4cac1a1e30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4c9800f990 tx=0x7f4c9800f9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:45.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.062+0000 7f4ca8c00700 1 -- 172.21.15.123:0/2843547503 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c98027070 con 0x7f4cac103d90 2024-05-08T06:07:45.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.062+0000 7f4cb1400700 1 -- 172.21.15.123:0/2843547503 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cac19c4a0 con 0x7f4cac103d90 2024-05-08T06:07:45.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.062+0000 7f4ca8c00700 1 -- 172.21.15.123:0/2843547503 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c98005570 con 0x7f4cac103d90 2024-05-08T06:07:45.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.062+0000 7f4ca8c00700 1 -- 172.21.15.123:0/2843547503 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c98004930 con 0x7f4cac103d90 2024-05-08T06:07:45.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.062+0000 7f4cb1400700 1 -- 172.21.15.123:0/2843547503 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cac1a87c0 con 0x7f4cac103d90 2024-05-08T06:07:45.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.063+0000 7f4cb1400700 1 -- 172.21.15.123:0/2843547503 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4cac1007b0 con 0x7f4cac103d90 2024-05-08T06:07:45.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.066+0000 7f4ca8c00700 1 -- 172.21.15.123:0/2843547503 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f4c9802a070 con 0x7f4cac103d90 2024-05-08T06:07:45.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.068+0000 7f4ca8c00700 1 --2- 172.21.15.123:0/2843547503 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f4c900b2910 0x7f4c900b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:45.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.068+0000 7f4ca8c00700 1 -- 172.21.15.123:0/2843547503 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f4c980e37b0 con 0x7f4cac103d90 2024-05-08T06:07:45.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.068+0000 7f4cabe00700 1 --2- 172.21.15.123:0/2843547503 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f4c900b2910 0x7f4c900b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:45.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.068+0000 7f4ca8c00700 1 --2- 172.21.15.123:0/2843547503 >> [v2:172.21.15.123:6826/3104436473,v1:172.21.15.123:6827/3104436473] conn(0x7f4c900bc450 0x7f4c900be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:45.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.068+0000 7f4ca8c00700 1 -- 172.21.15.123:0/2843547503 --> [v2:172.21.15.123:6826/3104436473,v1:172.21.15.123:6827/3104436473] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4c9801ce50 con 0x7f4c900bc450 2024-05-08T06:07:45.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.068+0000 7f4ca8c00700 1 -- 172.21.15.123:0/2843547503 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f4c980e3b60 con 0x7f4cac103d90 2024-05-08T06:07:45.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.069+0000 7f4cb0a00700 1 --2- 172.21.15.123:0/2843547503 >> [v2:172.21.15.123:6826/3104436473,v1:172.21.15.123:6827/3104436473] conn(0x7f4c900bc450 0x7f4c900be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:45.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.069+0000 7f4cabe00700 1 --2- 172.21.15.123:0/2843547503 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f4c900b2910 0x7f4c900b4dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4cac19cbf0 tx=0x7f4c9c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:45.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.069+0000 7f4cb0a00700 1 --2- 172.21.15.123:0/2843547503 >> [v2:172.21.15.123:6826/3104436473,v1:172.21.15.123:6827/3104436473] conn(0x7f4c900bc450 0x7f4c900be8f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:45.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.070+0000 7f4ca8c00700 1 -- 172.21.15.123:0/2843547503 <== osd.9 v2:172.21.15.123:6826/3104436473 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4c9801ce50 con 0x7f4c900bc450 2024-05-08T06:07:45.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.088+0000 7f4cb1400700 1 -- 172.21.15.123:0/2843547503 --> [v2:172.21.15.123:6826/3104436473,v1:172.21.15.123:6827/3104436473] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4cac104b10 con 0x7f4c900bc450 2024-05-08T06:07:45.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.089+0000 7f4ca8c00700 1 -- 172.21.15.123:0/2843547503 <== osd.9 v2:172.21.15.123:6826/3104436473 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4cac104b10 con 0x7f4c900bc450 2024-05-08T06:07:45.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.089+0000 7f4cb1400700 1 -- 172.21.15.123:0/2843547503 >> [v2:172.21.15.123:6826/3104436473,v1:172.21.15.123:6827/3104436473] conn(0x7f4c900bc450 msgr2=0x7f4c900be8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:45.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.089+0000 7f4cb1400700 1 --2- 172.21.15.123:0/2843547503 >> [v2:172.21.15.123:6826/3104436473,v1:172.21.15.123:6827/3104436473] conn(0x7f4c900bc450 0x7f4c900be8f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:45.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.089+0000 7f4cb1400700 1 -- 172.21.15.123:0/2843547503 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f4c900b2910 msgr2=0x7f4c900b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:45.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.089+0000 7f4cb1400700 1 --2- 172.21.15.123:0/2843547503 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f4c900b2910 0x7f4c900b4dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4cac19cbf0 tx=0x7f4c9c009310 comp rx=0 tx=0).stop 2024-05-08T06:07:45.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.089+0000 7f4cb1400700 1 -- 172.21.15.123:0/2843547503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cac103d90 msgr2=0x7f4cac1a1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:45.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.089+0000 7f4cb1400700 1 --2- 172.21.15.123:0/2843547503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cac103d90 0x7f4cac1a1e30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4c9800f990 tx=0x7f4c9800f9c0 comp rx=0 tx=0).stop 2024-05-08T06:07:45.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.089+0000 7f4cb0a00700 1 -- 172.21.15.123:0/2843547503 reap_dead start 2024-05-08T06:07:45.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.089+0000 7f4cb1400700 1 -- 172.21.15.123:0/2843547503 shutdown_connections 2024-05-08T06:07:45.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.089+0000 7f4cb1400700 1 -- 172.21.15.123:0/2843547503 >> 172.21.15.123:0/2843547503 conn(0x7f4cac0fef20 msgr2=0x7f4cac10eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:07:45.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.089+0000 7f4cb1400700 1 -- 172.21.15.123:0/2843547503 shutdown_connections 2024-05-08T06:07:45.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:44.090+0000 7f4cb1400700 1 -- 172.21.15.123:0/2843547503 wait complete. 2024-05-08T06:07:46.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:46 smithi163 ceph-mon[34520]: pgmap v379: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:07:46.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:46 smithi192 ceph-mon[34664]: pgmap v379: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:07:46.704 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:46 smithi123 ceph-mon[29787]: pgmap v379: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:07:46.764 INFO:teuthology.orchestra.run.smithi123.stdout:55834574945 2024-05-08T06:07:46.764 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-08T06:07:47.525 INFO:teuthology.orchestra.run.smithi123.stdout:261993005108 2024-05-08T06:07:47.525 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-08T06:07:47.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:47 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:07:47.600 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:47 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:07:47.605 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:47 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:07:47.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.285+0000 7fec6a200700 1 -- 172.21.15.123:0/3012088553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec640756d0 msgr2=0x7fec64075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:47.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.285+0000 7fec6a200700 1 --2- 172.21.15.123:0/3012088553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec640756d0 0x7fec64075ab0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fec5400b0a0 tx=0x7fec5400b3b0 comp rx=0 tx=0).stop 2024-05-08T06:07:47.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.285+0000 7fec6a200700 1 -- 172.21.15.123:0/3012088553 shutdown_connections 2024-05-08T06:07:47.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.285+0000 7fec6a200700 1 --2- 172.21.15.123:0/3012088553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec640756d0 0x7fec64075ab0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:47.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.285+0000 7fec6a200700 1 --2- 172.21.15.123:0/3012088553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec6410fea0 0x7fec64112290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:47.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.285+0000 7fec6a200700 1 --2- 172.21.15.123:0/3012088553 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec64075ff0 0x7fec6410f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:47.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.285+0000 7fec6a200700 1 -- 172.21.15.123:0/3012088553 >> 172.21.15.123:0/3012088553 conn(0x7fec6407a600 msgr2=0x7fec6407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:07:47.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.286+0000 7fec6a200700 1 -- 172.21.15.123:0/3012088553 shutdown_connections 2024-05-08T06:07:47.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.286+0000 7fec6a200700 1 -- 172.21.15.123:0/3012088553 wait complete. 2024-05-08T06:07:47.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.287+0000 7fec6a200700 1 Processor -- start 2024-05-08T06:07:47.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.287+0000 7fec6a200700 1 -- start start 2024-05-08T06:07:47.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.287+0000 7fec6a200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec640756d0 0x7fec6419bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:47.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.287+0000 7fec6a200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec64075ff0 0x7fec6419c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:47.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.287+0000 7fec6a200700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec6410fea0 0x7fec641a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:47.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.287+0000 7fec6a200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fec64115870 con 0x7fec640756d0 2024-05-08T06:07:47.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.287+0000 7fec6a200700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fec641156f0 con 0x7fec64075ff0 2024-05-08T06:07:47.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.287+0000 7fec6a200700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fec641159f0 con 0x7fec6410fea0 2024-05-08T06:07:47.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.288+0000 7fec63400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec640756d0 0x7fec6419bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:47.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.288+0000 7fec63400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec640756d0 0x7fec6419bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37990/0 (socket says 172.21.15.123:37990) 2024-05-08T06:07:47.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.288+0000 7fec63400700 1 -- 172.21.15.123:0/3645706028 learned_addr learned my addr 172.21.15.123:0/3645706028 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:07:47.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.288+0000 7fec63e00700 1 --2- 172.21.15.123:0/3645706028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec6410fea0 0x7fec641a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:47.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.288+0000 7fec5aa00700 1 --2- 172.21.15.123:0/3645706028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec64075ff0 0x7fec6419c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:47.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.288+0000 7fec63400700 1 -- 172.21.15.123:0/3645706028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec6410fea0 msgr2=0x7fec641a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:47.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.288+0000 7fec63400700 1 --2- 172.21.15.123:0/3645706028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec6410fea0 0x7fec641a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:47.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.288+0000 7fec63400700 1 -- 172.21.15.123:0/3645706028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec64075ff0 msgr2=0x7fec6419c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:47.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.288+0000 7fec63400700 1 --2- 172.21.15.123:0/3645706028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec64075ff0 0x7fec6419c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:47.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.288+0000 7fec63400700 1 -- 172.21.15.123:0/3645706028 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec641a0dd0 con 0x7fec640756d0 2024-05-08T06:07:47.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.288+0000 7fec63400700 1 --2- 172.21.15.123:0/3645706028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec640756d0 0x7fec6419bdc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fec5400d010 tx=0x7fec54004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:47.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.291+0000 7fec60c00700 1 -- 172.21.15.123:0/3645706028 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec54021070 con 0x7fec640756d0 2024-05-08T06:07:47.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.291+0000 7fec60c00700 1 -- 172.21.15.123:0/3645706028 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec54007e30 con 0x7fec640756d0 2024-05-08T06:07:47.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.291+0000 7fec6a200700 1 -- 172.21.15.123:0/3645706028 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec641a1060 con 0x7fec640756d0 2024-05-08T06:07:47.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.291+0000 7fec60c00700 1 -- 172.21.15.123:0/3645706028 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec54004690 con 0x7fec640756d0 2024-05-08T06:07:47.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.291+0000 7fec6a200700 1 -- 172.21.15.123:0/3645706028 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec641a8940 con 0x7fec640756d0 2024-05-08T06:07:47.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.292+0000 7fec6a200700 1 -- 172.21.15.123:0/3645706028 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fec64063c30 con 0x7fec640756d0 2024-05-08T06:07:47.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.294+0000 7fec60c00700 1 -- 172.21.15.123:0/3645706028 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fec54007980 con 0x7fec640756d0 2024-05-08T06:07:47.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.296+0000 7fec60c00700 1 --2- 172.21.15.123:0/3645706028 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fec480b2910 0x7fec480b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:47.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.296+0000 7fec60c00700 1 -- 172.21.15.123:0/3645706028 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fec5400f020 con 0x7fec640756d0 2024-05-08T06:07:47.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.296+0000 7fec5aa00700 1 --2- 172.21.15.123:0/3645706028 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fec480b2910 0x7fec480b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:47.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.296+0000 7fec60c00700 1 --2- 172.21.15.123:0/3645706028 >> [v2:172.21.15.123:6802/3608618058,v1:172.21.15.123:6803/3608618058] conn(0x7fec480bc450 0x7fec480be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:47.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.296+0000 7fec60c00700 1 -- 172.21.15.123:0/3645706028 --> [v2:172.21.15.123:6802/3608618058,v1:172.21.15.123:6803/3608618058] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fec480befc0 con 0x7fec480bc450 2024-05-08T06:07:47.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.296+0000 7fec60c00700 1 -- 172.21.15.123:0/3645706028 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fec540e3e10 con 0x7fec640756d0 2024-05-08T06:07:47.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.297+0000 7fec63e00700 1 --2- 172.21.15.123:0/3645706028 >> [v2:172.21.15.123:6802/3608618058,v1:172.21.15.123:6803/3608618058] conn(0x7fec480bc450 0x7fec480be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:47.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.297+0000 7fec5aa00700 1 --2- 172.21.15.123:0/3645706028 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fec480b2910 0x7fec480b4dd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fec64111ca0 tx=0x7fec4c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:47.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.297+0000 7fec63e00700 1 --2- 172.21.15.123:0/3645706028 >> [v2:172.21.15.123:6802/3608618058,v1:172.21.15.123:6803/3608618058] conn(0x7fec480bc450 0x7fec480be8f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:47.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.298+0000 7fec60c00700 1 -- 172.21.15.123:0/3645706028 <== osd.0 v2:172.21.15.123:6802/3608618058 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fec480befc0 con 0x7fec480bc450 2024-05-08T06:07:47.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.316+0000 7fec6a200700 1 -- 172.21.15.123:0/3645706028 --> [v2:172.21.15.123:6802/3608618058,v1:172.21.15.123:6803/3608618058] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fec640640b0 con 0x7fec480bc450 2024-05-08T06:07:47.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.317+0000 7fec60c00700 1 -- 172.21.15.123:0/3645706028 <== osd.0 v2:172.21.15.123:6802/3608618058 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fec640640b0 con 0x7fec480bc450 2024-05-08T06:07:47.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.317+0000 7fec6a200700 1 -- 172.21.15.123:0/3645706028 >> [v2:172.21.15.123:6802/3608618058,v1:172.21.15.123:6803/3608618058] conn(0x7fec480bc450 msgr2=0x7fec480be8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:47.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.317+0000 7fec6a200700 1 --2- 172.21.15.123:0/3645706028 >> [v2:172.21.15.123:6802/3608618058,v1:172.21.15.123:6803/3608618058] conn(0x7fec480bc450 0x7fec480be8f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:47.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.318+0000 7fec6a200700 1 -- 172.21.15.123:0/3645706028 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fec480b2910 msgr2=0x7fec480b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:47.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.318+0000 7fec6a200700 1 --2- 172.21.15.123:0/3645706028 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fec480b2910 0x7fec480b4dd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fec64111ca0 tx=0x7fec4c0073d0 comp rx=0 tx=0).stop 2024-05-08T06:07:47.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.318+0000 7fec6a200700 1 -- 172.21.15.123:0/3645706028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec640756d0 msgr2=0x7fec6419bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:47.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.318+0000 7fec6a200700 1 --2- 172.21.15.123:0/3645706028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec640756d0 0x7fec6419bdc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fec5400d010 tx=0x7fec54004d00 comp rx=0 tx=0).stop 2024-05-08T06:07:47.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.318+0000 7fec63e00700 1 -- 172.21.15.123:0/3645706028 reap_dead start 2024-05-08T06:07:47.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.318+0000 7fec6a200700 1 -- 172.21.15.123:0/3645706028 shutdown_connections 2024-05-08T06:07:47.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.318+0000 7fec6a200700 1 -- 172.21.15.123:0/3645706028 >> 172.21.15.123:0/3645706028 conn(0x7fec6407a600 msgr2=0x7fec641007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:07:47.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.318+0000 7fec6a200700 1 -- 172.21.15.123:0/3645706028 shutdown_connections 2024-05-08T06:07:47.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:47.318+0000 7fec6a200700 1 -- 172.21.15.123:0/3645706028 wait complete. 2024-05-08T06:07:48.431 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:48 smithi123 ceph-mon[29787]: pgmap v380: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:07:48.431 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:48 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:07:48.431 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:48 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:07:48.431 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:48 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:07:48.431 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:48 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:07:48.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.533+0000 7f5c09200700 1 -- 172.21.15.123:0/4145495373 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c04103d90 msgr2=0x7f5c041102a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:48.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.533+0000 7f5c09200700 1 --2- 172.21.15.123:0/4145495373 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c04103d90 0x7f5c041102a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc00b0a0 tx=0x7f5bfc00b3b0 comp rx=0 tx=0).stop 2024-05-08T06:07:48.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.533+0000 7f5c09200700 1 -- 172.21.15.123:0/4145495373 shutdown_connections 2024-05-08T06:07:48.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.533+0000 7f5c09200700 1 --2- 172.21.15.123:0/4145495373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c04109460 0x7f5c04109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:48.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.533+0000 7f5c09200700 1 --2- 172.21.15.123:0/4145495373 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c04103d90 0x7f5c041102a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:48.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.533+0000 7f5c09200700 1 --2- 172.21.15.123:0/4145495373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c041033d0 0x7f5c04103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:48.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.533+0000 7f5c09200700 1 -- 172.21.15.123:0/4145495373 >> 172.21.15.123:0/4145495373 conn(0x7f5c040fef20 msgr2=0x7f5c04101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:07:48.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.533+0000 7f5c09200700 1 -- 172.21.15.123:0/4145495373 shutdown_connections 2024-05-08T06:07:48.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.533+0000 7f5c09200700 1 -- 172.21.15.123:0/4145495373 wait complete. 2024-05-08T06:07:48.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.534+0000 7f5c09200700 1 Processor -- start 2024-05-08T06:07:48.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.534+0000 7f5c09200700 1 -- start start 2024-05-08T06:07:48.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.535+0000 7f5c09200700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c041033d0 0x7f5c041a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:48.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.535+0000 7f5c09200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c04103d90 0x7f5c041a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:48.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.535+0000 7f5c09200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c04109460 0x7f5c0419ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:48.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.535+0000 7f5c09200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5c041135c0 con 0x7f5c04103d90 2024-05-08T06:07:48.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.535+0000 7f5c09200700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5c04113440 con 0x7f5c04109460 2024-05-08T06:07:48.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.535+0000 7f5c09200700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5c04113740 con 0x7f5c041033d0 2024-05-08T06:07:48.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.535+0000 7f5c03400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c041033d0 0x7f5c041a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:48.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.535+0000 7f5c03400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c041033d0 0x7f5c041a18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:50678/0 (socket says 172.21.15.123:50678) 2024-05-08T06:07:48.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.535+0000 7f5c03400700 1 -- 172.21.15.123:0/1946523377 learned_addr learned my addr 172.21.15.123:0/1946523377 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:07:48.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.535+0000 7f5c02a00700 1 --2- 172.21.15.123:0/1946523377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c04103d90 0x7f5c041a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:48.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.535+0000 7f5c03e00700 1 --2- 172.21.15.123:0/1946523377 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c04109460 0x7f5c0419ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:48.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.535+0000 7f5c03400700 1 -- 172.21.15.123:0/1946523377 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c04109460 msgr2=0x7f5c0419ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:48.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.535+0000 7f5c03400700 1 --2- 172.21.15.123:0/1946523377 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c04109460 0x7f5c0419ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:48.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.535+0000 7f5c03400700 1 -- 172.21.15.123:0/1946523377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c04103d90 msgr2=0x7f5c041a1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:48.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.535+0000 7f5c03400700 1 --2- 172.21.15.123:0/1946523377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c04103d90 0x7f5c041a1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:48.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.535+0000 7f5c03400700 1 -- 172.21.15.123:0/1946523377 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c0419c1c0 con 0x7f5c041033d0 2024-05-08T06:07:48.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.536+0000 7f5c03400700 1 --2- 172.21.15.123:0/1946523377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c041033d0 0x7f5c041a18a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5bf400eb30 tx=0x7f5bf400ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:48.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.536+0000 7f5bfbe00700 1 -- 172.21.15.123:0/1946523377 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf400cbd0 con 0x7f5c041033d0 2024-05-08T06:07:48.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.536+0000 7f5bfbe00700 1 -- 172.21.15.123:0/1946523377 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bf40044d0 con 0x7f5c041033d0 2024-05-08T06:07:48.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.536+0000 7f5c09200700 1 -- 172.21.15.123:0/1946523377 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c0419c4b0 con 0x7f5c041033d0 2024-05-08T06:07:48.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.536+0000 7f5bfbe00700 1 -- 172.21.15.123:0/1946523377 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf4003e30 con 0x7f5c041033d0 2024-05-08T06:07:48.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.536+0000 7f5c09200700 1 -- 172.21.15.123:0/1946523377 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c041a8770 con 0x7f5c041033d0 2024-05-08T06:07:48.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.537+0000 7f5c09200700 1 -- 172.21.15.123:0/1946523377 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5c041102c0 con 0x7f5c041033d0 2024-05-08T06:07:48.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.539+0000 7f5bfbe00700 1 -- 172.21.15.123:0/1946523377 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f5bf4004000 con 0x7f5c041033d0 2024-05-08T06:07:48.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.541+0000 7f5bfbe00700 1 --2- 172.21.15.123:0/1946523377 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5be80b2990 0x7f5be80b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:48.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.541+0000 7f5bfbe00700 1 -- 172.21.15.123:0/1946523377 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f5bf40e2aa0 con 0x7f5c041033d0 2024-05-08T06:07:48.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.541+0000 7f5c02a00700 1 --2- 172.21.15.123:0/1946523377 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5be80b2990 0x7f5be80b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:48.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.541+0000 7f5bfbe00700 1 --2- 172.21.15.123:0/1946523377 >> [v2:172.21.15.192:6816/3669753689,v1:172.21.15.192:6817/3669753689] conn(0x7f5be80bc400 0x7f5be80be8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:48.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.541+0000 7f5bfbe00700 1 -- 172.21.15.123:0/1946523377 --> [v2:172.21.15.192:6816/3669753689,v1:172.21.15.192:6817/3669753689] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5bf4018e50 con 0x7f5be80bc400 2024-05-08T06:07:48.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.541+0000 7f5bfbe00700 1 -- 172.21.15.123:0/1946523377 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f5bf40e2e90 con 0x7f5c041033d0 2024-05-08T06:07:48.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.541+0000 7f5c03e00700 1 --2- 172.21.15.123:0/1946523377 >> [v2:172.21.15.192:6816/3669753689,v1:172.21.15.192:6817/3669753689] conn(0x7f5be80bc400 0x7f5be80be8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:48.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.542+0000 7f5c02a00700 1 --2- 172.21.15.123:0/1946523377 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5be80b2990 0x7f5be80b4e50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5c0410f670 tx=0x7f5bf0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:48.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.542+0000 7f5c03e00700 1 --2- 172.21.15.123:0/1946523377 >> [v2:172.21.15.192:6816/3669753689,v1:172.21.15.192:6817/3669753689] conn(0x7f5be80bc400 0x7f5be80be8a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:48.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.543+0000 7f5bfbe00700 1 -- 172.21.15.123:0/1946523377 <== osd.8 v2:172.21.15.192:6816/3669753689 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5bf4018e50 con 0x7f5be80bc400 2024-05-08T06:07:48.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:48 smithi192 ceph-mon[34664]: pgmap v380: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:07:48.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:48 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:07:48.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:48 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:07:48.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:48 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:07:48.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:48 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:07:48.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.561+0000 7f5c09200700 1 -- 172.21.15.123:0/1946523377 --> [v2:172.21.15.192:6816/3669753689,v1:172.21.15.192:6817/3669753689] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5c04104b10 con 0x7f5be80bc400 2024-05-08T06:07:48.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.561+0000 7f5bfbe00700 1 -- 172.21.15.123:0/1946523377 <== osd.8 v2:172.21.15.192:6816/3669753689 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5c04104b10 con 0x7f5be80bc400 2024-05-08T06:07:48.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.562+0000 7f5c09200700 1 -- 172.21.15.123:0/1946523377 >> [v2:172.21.15.192:6816/3669753689,v1:172.21.15.192:6817/3669753689] conn(0x7f5be80bc400 msgr2=0x7f5be80be8a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:48.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.562+0000 7f5c09200700 1 --2- 172.21.15.123:0/1946523377 >> [v2:172.21.15.192:6816/3669753689,v1:172.21.15.192:6817/3669753689] conn(0x7f5be80bc400 0x7f5be80be8a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:48.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.562+0000 7f5c09200700 1 -- 172.21.15.123:0/1946523377 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5be80b2990 msgr2=0x7f5be80b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:48.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.562+0000 7f5c09200700 1 --2- 172.21.15.123:0/1946523377 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5be80b2990 0x7f5be80b4e50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5c0410f670 tx=0x7f5bf0009260 comp rx=0 tx=0).stop 2024-05-08T06:07:48.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.562+0000 7f5c09200700 1 -- 172.21.15.123:0/1946523377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c041033d0 msgr2=0x7f5c041a18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:48.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.562+0000 7f5c09200700 1 --2- 172.21.15.123:0/1946523377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c041033d0 0x7f5c041a18a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5bf400eb30 tx=0x7f5bf400ee40 comp rx=0 tx=0).stop 2024-05-08T06:07:48.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.562+0000 7f5c03e00700 1 -- 172.21.15.123:0/1946523377 reap_dead start 2024-05-08T06:07:48.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.562+0000 7f5c09200700 1 -- 172.21.15.123:0/1946523377 shutdown_connections 2024-05-08T06:07:48.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.562+0000 7f5c09200700 1 -- 172.21.15.123:0/1946523377 >> 172.21.15.123:0/1946523377 conn(0x7f5c040fef20 msgr2=0x7f5c04108720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:07:48.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.563+0000 7f5c09200700 1 -- 172.21.15.123:0/1946523377 shutdown_connections 2024-05-08T06:07:48.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:48.563+0000 7f5c09200700 1 -- 172.21.15.123:0/1946523377 wait complete. 2024-05-08T06:07:48.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:48 smithi163 ceph-mon[34520]: pgmap v380: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:07:48.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:48 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:07:48.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:48 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:07:48.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:48 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:07:48.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:48 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:07:49.740 INFO:teuthology.orchestra.run.smithi123.stdout:34359738471 2024-05-08T06:07:49.740 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-08T06:07:50.372 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:07:50.401 INFO:teuthology.orchestra.run.smithi123.stdout:236223201339 2024-05-08T06:07:50.401 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-08T06:07:50.611 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:50 smithi123 ceph-mon[29787]: pgmap v381: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-05-08T06:07:50.611 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:50 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:07:50.611 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:50 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:07:50.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:50 smithi163 ceph-mon[34520]: pgmap v381: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-05-08T06:07:50.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:50 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:07:50.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:50 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:07:50.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:50 smithi192 ceph-mon[34664]: pgmap v381: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-05-08T06:07:50.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:50 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:07:50.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:50 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:07:51.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.397+0000 7fd05fe00700 1 -- 172.21.15.123:0/4119424288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd06010a3a0 msgr2=0x7fd060116840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:51.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.397+0000 7fd05fe00700 1 --2- 172.21.15.123:0/4119424288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd06010a3a0 0x7fd060116840 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd05800b0a0 tx=0x7fd05800b3b0 comp rx=0 tx=0).stop 2024-05-08T06:07:51.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.397+0000 7fd05fe00700 1 -- 172.21.15.123:0/4119424288 shutdown_connections 2024-05-08T06:07:51.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.397+0000 7fd05fe00700 1 --2- 172.21.15.123:0/4119424288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd06010a3a0 0x7fd060116840 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:51.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.397+0000 7fd05fe00700 1 --2- 172.21.15.123:0/4119424288 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd06010e8b0 0x7fd06010ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:51.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.397+0000 7fd05fe00700 1 --2- 172.21.15.123:0/4119424288 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0601099e0 0x7fd060109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:51.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.397+0000 7fd05fe00700 1 -- 172.21.15.123:0/4119424288 >> 172.21.15.123:0/4119424288 conn(0x7fd060079230 msgr2=0x7fd0601082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:07:51.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.398+0000 7fd05fe00700 1 -- 172.21.15.123:0/4119424288 shutdown_connections 2024-05-08T06:07:51.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.398+0000 7fd05fe00700 1 -- 172.21.15.123:0/4119424288 wait complete. 2024-05-08T06:07:51.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.399+0000 7fd05fe00700 1 Processor -- start 2024-05-08T06:07:51.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.399+0000 7fd05fe00700 1 -- start start 2024-05-08T06:07:51.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.399+0000 7fd05fe00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0601099e0 0x7fd0601a0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:51.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.399+0000 7fd05fe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd06010a3a0 0x7fd0601a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:51.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.399+0000 7fd05fe00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd06010e8b0 0x7fd0601a4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:51.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.399+0000 7fd05fe00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd060119be0 con 0x7fd06010a3a0 2024-05-08T06:07:51.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.399+0000 7fd05fe00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd060119a60 con 0x7fd0601099e0 2024-05-08T06:07:51.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.399+0000 7fd05fe00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd060119d60 con 0x7fd06010e8b0 2024-05-08T06:07:51.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.399+0000 7fd05e000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd06010a3a0 0x7fd0601a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:51.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.399+0000 7fd05e000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd06010a3a0 0x7fd0601a0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38014/0 (socket says 172.21.15.123:38014) 2024-05-08T06:07:51.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.399+0000 7fd05e000700 1 -- 172.21.15.123:0/3991427820 learned_addr learned my addr 172.21.15.123:0/3991427820 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:07:51.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.400+0000 7fd05ea00700 1 --2- 172.21.15.123:0/3991427820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0601099e0 0x7fd0601a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:51.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.400+0000 7fd05f400700 1 --2- 172.21.15.123:0/3991427820 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd06010e8b0 0x7fd0601a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:51.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.400+0000 7fd05e000700 1 -- 172.21.15.123:0/3991427820 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd06010e8b0 msgr2=0x7fd0601a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:51.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.400+0000 7fd05e000700 1 --2- 172.21.15.123:0/3991427820 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd06010e8b0 0x7fd0601a4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:51.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.400+0000 7fd05e000700 1 -- 172.21.15.123:0/3991427820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0601099e0 msgr2=0x7fd0601a0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:51.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.400+0000 7fd05e000700 1 --2- 172.21.15.123:0/3991427820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0601099e0 0x7fd0601a0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:51.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.400+0000 7fd05e000700 1 -- 172.21.15.123:0/3991427820 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0601a5060 con 0x7fd06010a3a0 2024-05-08T06:07:51.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.400+0000 7fd05f400700 1 --2- 172.21.15.123:0/3991427820 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd06010e8b0 0x7fd0601a4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:07:51.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.400+0000 7fd05e000700 1 --2- 172.21.15.123:0/3991427820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd06010a3a0 0x7fd0601a0590 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00e9a0 tx=0x7fd04c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:51.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.400+0000 7fd057400700 1 -- 172.21.15.123:0/3991427820 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c00cbd0 con 0x7fd06010a3a0 2024-05-08T06:07:51.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.400+0000 7fd05fe00700 1 -- 172.21.15.123:0/3991427820 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0601a5350 con 0x7fd06010a3a0 2024-05-08T06:07:51.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.400+0000 7fd057400700 1 -- 172.21.15.123:0/3991427820 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd04c00cd70 con 0x7fd06010a3a0 2024-05-08T06:07:51.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.401+0000 7fd057400700 1 -- 172.21.15.123:0/3991427820 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c018680 con 0x7fd06010a3a0 2024-05-08T06:07:51.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.401+0000 7fd05fe00700 1 -- 172.21.15.123:0/3991427820 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0601acbd0 con 0x7fd06010a3a0 2024-05-08T06:07:51.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.401+0000 7fd05fe00700 1 -- 172.21.15.123:0/3991427820 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd060063c30 con 0x7fd06010a3a0 2024-05-08T06:07:51.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.405+0000 7fd057400700 1 -- 172.21.15.123:0/3991427820 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fd04c010430 con 0x7fd06010a3a0 2024-05-08T06:07:51.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.406+0000 7fd057400700 1 --2- 172.21.15.123:0/3991427820 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd0440b2910 0x7fd0440b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:51.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.406+0000 7fd057400700 1 -- 172.21.15.123:0/3991427820 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fd04c0e2b50 con 0x7fd06010a3a0 2024-05-08T06:07:51.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.406+0000 7fd05ea00700 1 --2- 172.21.15.123:0/3991427820 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd0440b2910 0x7fd0440b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:51.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.407+0000 7fd057400700 1 --2- 172.21.15.123:0/3991427820 >> [v2:172.21.15.163:6816/354411978,v1:172.21.15.163:6817/354411978] conn(0x7fd0440bc450 0x7fd0440be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:51.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.407+0000 7fd057400700 1 -- 172.21.15.123:0/3991427820 --> [v2:172.21.15.163:6816/354411978,v1:172.21.15.163:6817/354411978] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd04c00e1a0 con 0x7fd0440bc450 2024-05-08T06:07:51.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.407+0000 7fd057400700 1 -- 172.21.15.123:0/3991427820 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fd04c0e2f00 con 0x7fd06010a3a0 2024-05-08T06:07:51.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.407+0000 7fd05f400700 1 --2- 172.21.15.123:0/3991427820 >> [v2:172.21.15.163:6816/354411978,v1:172.21.15.163:6817/354411978] conn(0x7fd0440bc450 0x7fd0440be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:51.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.407+0000 7fd05ea00700 1 --2- 172.21.15.123:0/3991427820 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd0440b2910 0x7fd0440b4dd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd050005950 tx=0x7fd05000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:51.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.408+0000 7fd05f400700 1 --2- 172.21.15.123:0/3991427820 >> [v2:172.21.15.163:6816/354411978,v1:172.21.15.163:6817/354411978] conn(0x7fd0440bc450 0x7fd0440be8f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:51.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.409+0000 7fd057400700 1 -- 172.21.15.123:0/3991427820 <== osd.7 v2:172.21.15.163:6816/354411978 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd04c00e1a0 con 0x7fd0440bc450 2024-05-08T06:07:51.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.430+0000 7fd05fe00700 1 -- 172.21.15.123:0/3991427820 --> [v2:172.21.15.163:6816/354411978,v1:172.21.15.163:6817/354411978] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd0600640b0 con 0x7fd0440bc450 2024-05-08T06:07:51.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.430+0000 7fd057400700 1 -- 172.21.15.123:0/3991427820 <== osd.7 v2:172.21.15.163:6816/354411978 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd0600640b0 con 0x7fd0440bc450 2024-05-08T06:07:51.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.431+0000 7fd05fe00700 1 -- 172.21.15.123:0/3991427820 >> [v2:172.21.15.163:6816/354411978,v1:172.21.15.163:6817/354411978] conn(0x7fd0440bc450 msgr2=0x7fd0440be8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:51.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.431+0000 7fd05fe00700 1 --2- 172.21.15.123:0/3991427820 >> [v2:172.21.15.163:6816/354411978,v1:172.21.15.163:6817/354411978] conn(0x7fd0440bc450 0x7fd0440be8f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:51.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.431+0000 7fd05fe00700 1 -- 172.21.15.123:0/3991427820 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd0440b2910 msgr2=0x7fd0440b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:51.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.431+0000 7fd05fe00700 1 --2- 172.21.15.123:0/3991427820 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd0440b2910 0x7fd0440b4dd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd050005950 tx=0x7fd05000b410 comp rx=0 tx=0).stop 2024-05-08T06:07:51.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.431+0000 7fd05fe00700 1 -- 172.21.15.123:0/3991427820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd06010a3a0 msgr2=0x7fd0601a0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:51.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.431+0000 7fd05fe00700 1 --2- 172.21.15.123:0/3991427820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd06010a3a0 0x7fd0601a0590 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00e9a0 tx=0x7fd04c00ecb0 comp rx=0 tx=0).stop 2024-05-08T06:07:51.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.432+0000 7fd05f400700 1 -- 172.21.15.123:0/3991427820 reap_dead start 2024-05-08T06:07:51.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.432+0000 7fd05fe00700 1 -- 172.21.15.123:0/3991427820 shutdown_connections 2024-05-08T06:07:51.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.432+0000 7fd05fe00700 1 -- 172.21.15.123:0/3991427820 >> 172.21.15.123:0/3991427820 conn(0x7fd060079230 msgr2=0x7fd0600797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:07:51.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.432+0000 7fd05fe00700 1 -- 172.21.15.123:0/3991427820 shutdown_connections 2024-05-08T06:07:51.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:51.432+0000 7fd05fe00700 1 -- 172.21.15.123:0/3991427820 wait complete. 2024-05-08T06:07:52.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:52 smithi123 ceph-mon[29787]: pgmap v382: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:07:52.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:52 smithi163 ceph-mon[34520]: pgmap v382: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:07:52.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:52 smithi192 ceph-mon[34664]: pgmap v382: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:07:52.858 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:07:53.028 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:07:53.598 INFO:teuthology.orchestra.run.smithi123.stdout:210453397568 2024-05-08T06:07:53.599 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-08T06:07:54.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:54 smithi123 ceph-mon[29787]: pgmap v383: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-05-08T06:07:54.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.278+0000 7f1667400700 1 -- 172.21.15.123:0/4140660534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1660104100 msgr2=0x7f1660108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:54.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.278+0000 7f1667400700 1 --2- 172.21.15.123:0/4140660534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1660104100 0x7f1660108120 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f16480099c0 tx=0x7f1648009cd0 comp rx=0 tx=0).stop 2024-05-08T06:07:54.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.278+0000 7f1667400700 1 -- 172.21.15.123:0/4140660534 shutdown_connections 2024-05-08T06:07:54.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.278+0000 7f1667400700 1 --2- 172.21.15.123:0/4140660534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1660108850 0x7f16601103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:54.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.278+0000 7f1667400700 1 --2- 172.21.15.123:0/4140660534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1660103750 0x7f1660103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:54.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.278+0000 7f1667400700 1 --2- 172.21.15.123:0/4140660534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1660104100 0x7f1660108120 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:54.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.278+0000 7f1667400700 1 -- 172.21.15.123:0/4140660534 >> 172.21.15.123:0/4140660534 conn(0x7f166007a600 msgr2=0x7f166007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:07:54.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.278+0000 7f1667400700 1 -- 172.21.15.123:0/4140660534 shutdown_connections 2024-05-08T06:07:54.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.278+0000 7f1667400700 1 -- 172.21.15.123:0/4140660534 wait complete. 2024-05-08T06:07:54.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.279+0000 7f1667400700 1 Processor -- start 2024-05-08T06:07:54.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.280+0000 7f1667400700 1 -- start start 2024-05-08T06:07:54.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.280+0000 7f1667400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1660103750 0x7f16600757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:54.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.280+0000 7f1667400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1660104100 0x7f1660075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:54.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.280+0000 7f1667400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1660108850 0x7f1660076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:54.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.280+0000 7f1667400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1660113660 con 0x7f1660104100 2024-05-08T06:07:54.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.280+0000 7f1667400700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f16601134e0 con 0x7f1660108850 2024-05-08T06:07:54.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.280+0000 7f1667400700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f16601137e0 con 0x7f1660103750 2024-05-08T06:07:54.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.280+0000 7f1657e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1660104100 0x7f1660075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:54.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.280+0000 7f1657e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1660104100 0x7f1660075ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38036/0 (socket says 172.21.15.123:38036) 2024-05-08T06:07:54.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.280+0000 7f1664a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1660103750 0x7f16600757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:54.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.280+0000 7f1657e00700 1 -- 172.21.15.123:0/4017742302 learned_addr learned my addr 172.21.15.123:0/4017742302 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:07:54.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.281+0000 7f1665400700 1 --2- 172.21.15.123:0/4017742302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1660108850 0x7f1660076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:54.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.281+0000 7f1657e00700 1 -- 172.21.15.123:0/4017742302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1660103750 msgr2=0x7f16600757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:54.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.281+0000 7f1657e00700 1 --2- 172.21.15.123:0/4017742302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1660103750 0x7f16600757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:54.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.281+0000 7f1657e00700 1 -- 172.21.15.123:0/4017742302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1660108850 msgr2=0x7f1660076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:54.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.281+0000 7f1657e00700 1 --2- 172.21.15.123:0/4017742302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1660108850 0x7f1660076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:54.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.281+0000 7f1657e00700 1 -- 172.21.15.123:0/4017742302 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16601a8020 con 0x7f1660104100 2024-05-08T06:07:54.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.281+0000 7f1657e00700 1 --2- 172.21.15.123:0/4017742302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1660104100 0x7f1660075ce0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1648006010 tx=0x7f16480055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:54.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.281+0000 7f165e000700 1 -- 172.21.15.123:0/4017742302 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1648026070 con 0x7f1660104100 2024-05-08T06:07:54.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.281+0000 7f165e000700 1 -- 172.21.15.123:0/4017742302 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f164800f460 con 0x7f1660104100 2024-05-08T06:07:54.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.282+0000 7f165e000700 1 -- 172.21.15.123:0/4017742302 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1648009e60 con 0x7f1660104100 2024-05-08T06:07:54.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.282+0000 7f1667400700 1 -- 172.21.15.123:0/4017742302 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16601a82b0 con 0x7f1660104100 2024-05-08T06:07:54.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.282+0000 7f1667400700 1 -- 172.21.15.123:0/4017742302 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16601a86b0 con 0x7f1660104100 2024-05-08T06:07:54.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.283+0000 7f1667400700 1 -- 172.21.15.123:0/4017742302 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1660063c30 con 0x7f1660104100 2024-05-08T06:07:54.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.285+0000 7f165e000700 1 -- 172.21.15.123:0/4017742302 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f1648029070 con 0x7f1660104100 2024-05-08T06:07:54.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.286+0000 7f165e000700 1 --2- 172.21.15.123:0/4017742302 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f16440b2910 0x7f16440b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:54.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.287+0000 7f165e000700 1 -- 172.21.15.123:0/4017742302 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f16480e2ff0 con 0x7f1660104100 2024-05-08T06:07:54.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.287+0000 7f1664a00700 1 --2- 172.21.15.123:0/4017742302 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f16440b2910 0x7f16440b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:54.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.287+0000 7f165e000700 1 --2- 172.21.15.123:0/4017742302 >> [v2:172.21.15.123:6818/1002138878,v1:172.21.15.123:6819/1002138878] conn(0x7f16440bc450 0x7f16440be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:54.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.287+0000 7f165e000700 1 -- 172.21.15.123:0/4017742302 --> [v2:172.21.15.123:6818/1002138878,v1:172.21.15.123:6819/1002138878] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1648004ea0 con 0x7f16440bc450 2024-05-08T06:07:54.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.287+0000 7f165e000700 1 -- 172.21.15.123:0/4017742302 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f16480e3440 con 0x7f1660104100 2024-05-08T06:07:54.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.287+0000 7f1665400700 1 --2- 172.21.15.123:0/4017742302 >> [v2:172.21.15.123:6818/1002138878,v1:172.21.15.123:6819/1002138878] conn(0x7f16440bc450 0x7f16440be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:54.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.287+0000 7f1664a00700 1 --2- 172.21.15.123:0/4017742302 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f16440b2910 0x7f16440b4dd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1650009550 tx=0x7f16500091f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:54.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.288+0000 7f1665400700 1 --2- 172.21.15.123:0/4017742302 >> [v2:172.21.15.123:6818/1002138878,v1:172.21.15.123:6819/1002138878] conn(0x7f16440bc450 0x7f16440be8f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:54.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.289+0000 7f165e000700 1 -- 172.21.15.123:0/4017742302 <== osd.6 v2:172.21.15.123:6818/1002138878 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1648004ea0 con 0x7f16440bc450 2024-05-08T06:07:54.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.306+0000 7f1667400700 1 -- 172.21.15.123:0/4017742302 --> [v2:172.21.15.123:6818/1002138878,v1:172.21.15.123:6819/1002138878] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f16600640b0 con 0x7f16440bc450 2024-05-08T06:07:54.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.307+0000 7f165e000700 1 -- 172.21.15.123:0/4017742302 <== osd.6 v2:172.21.15.123:6818/1002138878 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f16600640b0 con 0x7f16440bc450 2024-05-08T06:07:54.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.307+0000 7f1667400700 1 -- 172.21.15.123:0/4017742302 >> [v2:172.21.15.123:6818/1002138878,v1:172.21.15.123:6819/1002138878] conn(0x7f16440bc450 msgr2=0x7f16440be8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:54.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.307+0000 7f1667400700 1 --2- 172.21.15.123:0/4017742302 >> [v2:172.21.15.123:6818/1002138878,v1:172.21.15.123:6819/1002138878] conn(0x7f16440bc450 0x7f16440be8f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:54.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.308+0000 7f1667400700 1 -- 172.21.15.123:0/4017742302 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f16440b2910 msgr2=0x7f16440b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:54.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.308+0000 7f1667400700 1 --2- 172.21.15.123:0/4017742302 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f16440b2910 0x7f16440b4dd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1650009550 tx=0x7f16500091f0 comp rx=0 tx=0).stop 2024-05-08T06:07:54.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.308+0000 7f1667400700 1 -- 172.21.15.123:0/4017742302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1660104100 msgr2=0x7f1660075ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:54.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.308+0000 7f1667400700 1 --2- 172.21.15.123:0/4017742302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1660104100 0x7f1660075ce0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1648006010 tx=0x7f16480055f0 comp rx=0 tx=0).stop 2024-05-08T06:07:54.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.308+0000 7f1665400700 1 -- 172.21.15.123:0/4017742302 reap_dead start 2024-05-08T06:07:54.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.308+0000 7f1667400700 1 -- 172.21.15.123:0/4017742302 shutdown_connections 2024-05-08T06:07:54.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.308+0000 7f1667400700 1 -- 172.21.15.123:0/4017742302 >> 172.21.15.123:0/4017742302 conn(0x7f166007a600 msgr2=0x7f1660101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:07:54.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.308+0000 7f1667400700 1 -- 172.21.15.123:0/4017742302 shutdown_connections 2024-05-08T06:07:54.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:53.308+0000 7f1667400700 1 -- 172.21.15.123:0/4017742302 wait complete. 2024-05-08T06:07:54.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:54 smithi163 ceph-mon[34520]: pgmap v383: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-05-08T06:07:54.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:54 smithi192 ceph-mon[34664]: pgmap v383: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-05-08T06:07:55.530 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:07:55.535 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:07:55.539 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:07:55.543 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:07:56.590 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:56 smithi123 ceph-mon[29787]: pgmap v384: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:07:56.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:56 smithi163 ceph-mon[34520]: pgmap v384: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:07:56.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:56 smithi192 ceph-mon[34664]: pgmap v384: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:07:58.192 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:07:58.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.341+0000 7f948c400700 1 -- 172.21.15.123:0/3592013242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94841056c0 msgr2=0x7f9484105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:58.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.341+0000 7f948c400700 1 --2- 172.21.15.123:0/3592013242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94841056c0 0x7f9484105b40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9470009990 tx=0x7f9470009ca0 comp rx=0 tx=0).stop 2024-05-08T06:07:58.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.341+0000 7f948c400700 1 -- 172.21.15.123:0/3592013242 shutdown_connections 2024-05-08T06:07:58.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.341+0000 7f948c400700 1 --2- 172.21.15.123:0/3592013242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9484106080 0x7f9484112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:58.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.341+0000 7f948c400700 1 --2- 172.21.15.123:0/3592013242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f948410b6c0 0x7f948410baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:58.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.341+0000 7f948c400700 1 --2- 172.21.15.123:0/3592013242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94841056c0 0x7f9484105b40 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:58.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.341+0000 7f948c400700 1 -- 172.21.15.123:0/3592013242 >> 172.21.15.123:0/3592013242 conn(0x7f94841011b0 msgr2=0x7f94841035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:07:58.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.341+0000 7f948c400700 1 -- 172.21.15.123:0/3592013242 shutdown_connections 2024-05-08T06:07:58.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.341+0000 7f948c400700 1 -- 172.21.15.123:0/3592013242 wait complete. 2024-05-08T06:07:58.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.342+0000 7f948c400700 1 Processor -- start 2024-05-08T06:07:58.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.343+0000 7f948c400700 1 -- start start 2024-05-08T06:07:58.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.343+0000 7f948c400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94841056c0 0x7f948419bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:58.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.343+0000 7f948c400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9484106080 0x7f948419c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:58.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.343+0000 7f948c400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f948410b6c0 0x7f94841a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:58.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.343+0000 7f948c400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9484115910 con 0x7f94841056c0 2024-05-08T06:07:58.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.343+0000 7f948c400700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9484115790 con 0x7f948410b6c0 2024-05-08T06:07:58.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.343+0000 7f948c400700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9484115a90 con 0x7f9484106080 2024-05-08T06:07:58.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.344+0000 7f9489a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94841056c0 0x7f948419bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:58.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.344+0000 7f9489000700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9484106080 0x7f948419c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:58.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.344+0000 7f9489000700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9484106080 0x7f948419c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:48368/0 (socket says 172.21.15.123:48368) 2024-05-08T06:07:58.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.344+0000 7f9489000700 1 -- 172.21.15.123:0/1013099787 learned_addr learned my addr 172.21.15.123:0/1013099787 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:07:58.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.344+0000 7f9489a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94841056c0 0x7f948419bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39010/0 (socket says 172.21.15.123:39010) 2024-05-08T06:07:58.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.344+0000 7f948a400700 1 --2- 172.21.15.123:0/1013099787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f948410b6c0 0x7f94841a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:58.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.344+0000 7f9489000700 1 -- 172.21.15.123:0/1013099787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f948410b6c0 msgr2=0x7f94841a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:58.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.344+0000 7f9489000700 1 --2- 172.21.15.123:0/1013099787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f948410b6c0 0x7f94841a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:58.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.344+0000 7f9489000700 1 -- 172.21.15.123:0/1013099787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94841056c0 msgr2=0x7f948419bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:58.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.344+0000 7f9489000700 1 --2- 172.21.15.123:0/1013099787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94841056c0 0x7f948419bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:58.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.344+0000 7f9489000700 1 -- 172.21.15.123:0/1013099787 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94841a0d90 con 0x7f9484106080 2024-05-08T06:07:58.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.344+0000 7f9489000700 1 --2- 172.21.15.123:0/1013099787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9484106080 0x7f948419c2c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9470005910 tx=0x7f947000b670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:58.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.345+0000 7f9482000700 1 -- 172.21.15.123:0/1013099787 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9470027070 con 0x7f9484106080 2024-05-08T06:07:58.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.345+0000 7f9482000700 1 -- 172.21.15.123:0/1013099787 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9470022470 con 0x7f9484106080 2024-05-08T06:07:58.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.345+0000 7f948c400700 1 -- 172.21.15.123:0/1013099787 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94841a1020 con 0x7f9484106080 2024-05-08T06:07:58.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.345+0000 7f9482000700 1 -- 172.21.15.123:0/1013099787 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9470017650 con 0x7f9484106080 2024-05-08T06:07:58.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.345+0000 7f948c400700 1 -- 172.21.15.123:0/1013099787 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94841a8900 con 0x7f9484106080 2024-05-08T06:07:58.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.346+0000 7f948c400700 1 -- 172.21.15.123:0/1013099787 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f948406a190 con 0x7f9484106080 2024-05-08T06:07:58.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.348+0000 7f9482000700 1 -- 172.21.15.123:0/1013099787 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f947002a070 con 0x7f9484106080 2024-05-08T06:07:58.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.350+0000 7f9482000700 1 --2- 172.21.15.123:0/1013099787 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f946c0b2990 0x7f946c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:07:58.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.350+0000 7f9482000700 1 -- 172.21.15.123:0/1013099787 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f94700e3270 con 0x7f9484106080 2024-05-08T06:07:58.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.350+0000 7f9489a00700 1 --2- 172.21.15.123:0/1013099787 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f946c0b2990 0x7f946c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:07:58.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.351+0000 7f9489a00700 1 --2- 172.21.15.123:0/1013099787 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f946c0b2990 0x7f946c0b4e50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9478009680 tx=0x7f9478009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:07:58.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.352+0000 7f9482000700 1 -- 172.21.15.123:0/1013099787 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9470013070 con 0x7f9484106080 2024-05-08T06:07:58.367 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:07:58.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.600+0000 7f948c400700 1 -- 172.21.15.123:0/1013099787 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f948419d0e0 con 0x7f9484106080 2024-05-08T06:07:58.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.601+0000 7f9482000700 1 -- 172.21.15.123:0/1013099787 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f94700aae70 con 0x7f9484106080 2024-05-08T06:07:58.602 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:58 smithi123 ceph-mon[29787]: pgmap v385: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:07:58.602 INFO:teuthology.orchestra.run.smithi123.stdout:309237645355 2024-05-08T06:07:58.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.604+0000 7f948c400700 1 -- 172.21.15.123:0/1013099787 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f946c0b2990 msgr2=0x7f946c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:58.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.604+0000 7f948c400700 1 --2- 172.21.15.123:0/1013099787 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f946c0b2990 0x7f946c0b4e50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9478009680 tx=0x7f9478009260 comp rx=0 tx=0).stop 2024-05-08T06:07:58.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.604+0000 7f948c400700 1 -- 172.21.15.123:0/1013099787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9484106080 msgr2=0x7f948419c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:07:58.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.604+0000 7f948c400700 1 --2- 172.21.15.123:0/1013099787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9484106080 0x7f948419c2c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9470005910 tx=0x7f947000b670 comp rx=0 tx=0).stop 2024-05-08T06:07:58.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.604+0000 7f948c400700 1 -- 172.21.15.123:0/1013099787 shutdown_connections 2024-05-08T06:07:58.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.604+0000 7f948c400700 1 --2- 172.21.15.123:0/1013099787 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f946c0b2990 0x7f946c0b4e50 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:58.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.604+0000 7f948c400700 1 --2- 172.21.15.123:0/1013099787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94841056c0 0x7f948419bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:58.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.604+0000 7f948c400700 1 --2- 172.21.15.123:0/1013099787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f948410b6c0 0x7f94841a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:58.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.604+0000 7f948c400700 1 --2- 172.21.15.123:0/1013099787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9484106080 0x7f948419c2c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:07:58.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.604+0000 7f948c400700 1 -- 172.21.15.123:0/1013099787 >> 172.21.15.123:0/1013099787 conn(0x7f94841011b0 msgr2=0x7f9484101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:07:58.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.605+0000 7f948c400700 1 -- 172.21.15.123:0/1013099787 shutdown_connections 2024-05-08T06:07:58.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:07:58.605+0000 7f948c400700 1 -- 172.21.15.123:0/1013099787 wait complete. 2024-05-08T06:07:58.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:58 smithi163 ceph-mon[34520]: pgmap v385: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:07:58.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:58 smithi192 ceph-mon[34664]: pgmap v385: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:07:59.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:07:59 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/1013099787' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-08T06:07:59.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:07:59 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/1013099787' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-08T06:07:59.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:07:59 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/1013099787' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-08T06:08:00.400 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:00 smithi123 ceph-mon[29787]: pgmap v386: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:08:00.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:00 smithi163 ceph-mon[34520]: pgmap v386: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:08:00.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:00 smithi192 ceph-mon[34664]: pgmap v386: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:08:02.599 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:02 smithi123 ceph-mon[29787]: pgmap v387: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:08:02.599 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:02 smithi123 ceph-mon[29787]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-08T06:08:02.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:02 smithi163 ceph-mon[34520]: pgmap v387: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:08:02.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:02 smithi163 ceph-mon[34520]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-08T06:08:02.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:02 smithi192 ceph-mon[34664]: pgmap v387: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:08:02.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:02 smithi192 ceph-mon[34664]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-08T06:08:03.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.429+0000 7fa89a600700 1 -- 172.21.15.123:0/1993468537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa89410a3a0 msgr2=0x7fa894116840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:03.429 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.429+0000 7fa89a600700 1 --2- 172.21.15.123:0/1993468537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa89410a3a0 0x7fa894116840 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa88c00b0a0 tx=0x7fa88c00b3b0 comp rx=0 tx=0).stop 2024-05-08T06:08:03.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.429+0000 7fa89a600700 1 -- 172.21.15.123:0/1993468537 shutdown_connections 2024-05-08T06:08:03.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.429+0000 7fa89a600700 1 --2- 172.21.15.123:0/1993468537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa89410a3a0 0x7fa894116840 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:03.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.429+0000 7fa89a600700 1 --2- 172.21.15.123:0/1993468537 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa89410e8b0 0x7fa89410ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:03.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.429+0000 7fa89a600700 1 --2- 172.21.15.123:0/1993468537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8941099e0 0x7fa894109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:03.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.429+0000 7fa89a600700 1 -- 172.21.15.123:0/1993468537 >> 172.21.15.123:0/1993468537 conn(0x7fa894079230 msgr2=0x7fa8941082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:08:03.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.429+0000 7fa89a600700 1 -- 172.21.15.123:0/1993468537 shutdown_connections 2024-05-08T06:08:03.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.429+0000 7fa89a600700 1 -- 172.21.15.123:0/1993468537 wait complete. 2024-05-08T06:08:03.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.430+0000 7fa89a600700 1 Processor -- start 2024-05-08T06:08:03.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.430+0000 7fa89a600700 1 -- start start 2024-05-08T06:08:03.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.431+0000 7fa89a600700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8941099e0 0x7fa8941a0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:03.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.431+0000 7fa89a600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa89410a3a0 0x7fa8941a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:03.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.431+0000 7fa89a600700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa89410e8b0 0x7fa8941a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:03.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.431+0000 7fa89a600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa894119b90 con 0x7fa89410a3a0 2024-05-08T06:08:03.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.431+0000 7fa89a600700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa894119a10 con 0x7fa8941099e0 2024-05-08T06:08:03.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.431+0000 7fa89a600700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa894119d10 con 0x7fa89410e8b0 2024-05-08T06:08:03.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.431+0000 7fa893e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa89410a3a0 0x7fa8941a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:03.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.431+0000 7fa893e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa89410a3a0 0x7fa8941a05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39030/0 (socket says 172.21.15.123:39030) 2024-05-08T06:08:03.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.431+0000 7fa893e00700 1 -- 172.21.15.123:0/3146340649 learned_addr learned my addr 172.21.15.123:0/3146340649 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:08:03.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.431+0000 7fa899200700 1 --2- 172.21.15.123:0/3146340649 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8941099e0 0x7fa8941a0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:03.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.431+0000 7fa899c00700 1 --2- 172.21.15.123:0/3146340649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa89410e8b0 0x7fa8941a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:03.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.432+0000 7fa893e00700 1 -- 172.21.15.123:0/3146340649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa89410e8b0 msgr2=0x7fa8941a4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:03.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.432+0000 7fa893e00700 1 --2- 172.21.15.123:0/3146340649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa89410e8b0 0x7fa8941a4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:03.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.432+0000 7fa893e00700 1 -- 172.21.15.123:0/3146340649 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8941099e0 msgr2=0x7fa8941a0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:03.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.432+0000 7fa893e00700 1 --2- 172.21.15.123:0/3146340649 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8941099e0 0x7fa8941a0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:03.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.432+0000 7fa893e00700 1 -- 172.21.15.123:0/3146340649 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8941a5040 con 0x7fa89410a3a0 2024-05-08T06:08:03.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.432+0000 7fa893e00700 1 --2- 172.21.15.123:0/3146340649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa89410a3a0 0x7fa8941a05c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa88000e9c0 tx=0x7fa88000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:08:03.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.432+0000 7fa891600700 1 -- 172.21.15.123:0/3146340649 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88000cbd0 con 0x7fa89410a3a0 2024-05-08T06:08:03.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.432+0000 7fa891600700 1 -- 172.21.15.123:0/3146340649 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa88000ee60 con 0x7fa89410a3a0 2024-05-08T06:08:03.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.432+0000 7fa891600700 1 -- 172.21.15.123:0/3146340649 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8800055c0 con 0x7fa89410a3a0 2024-05-08T06:08:03.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.432+0000 7fa89a600700 1 -- 172.21.15.123:0/3146340649 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8941a5330 con 0x7fa89410a3a0 2024-05-08T06:08:03.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.433+0000 7fa89a600700 1 -- 172.21.15.123:0/3146340649 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8941acbb0 con 0x7fa89410a3a0 2024-05-08T06:08:03.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.434+0000 7fa89a600700 1 -- 172.21.15.123:0/3146340649 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa89406a190 con 0x7fa89410a3a0 2024-05-08T06:08:03.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.436+0000 7fa891600700 1 -- 172.21.15.123:0/3146340649 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fa880004750 con 0x7fa89410a3a0 2024-05-08T06:08:03.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.437+0000 7fa891600700 1 --2- 172.21.15.123:0/3146340649 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa87c0b2900 0x7fa87c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:03.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.438+0000 7fa891600700 1 -- 172.21.15.123:0/3146340649 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fa8800e2460 con 0x7fa89410a3a0 2024-05-08T06:08:03.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.438+0000 7fa899200700 1 --2- 172.21.15.123:0/3146340649 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa87c0b2900 0x7fa87c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:03.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.438+0000 7fa899200700 1 --2- 172.21.15.123:0/3146340649 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa87c0b2900 0x7fa87c0b4dc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa888009be0 tx=0x7fa888009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:08:03.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.442+0000 7fa891600700 1 -- 172.21.15.123:0/3146340649 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa8800aa130 con 0x7fa89410a3a0 2024-05-08T06:08:03.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.689+0000 7fa89a600700 1 -- 172.21.15.123:0/3146340649 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fa8941a1390 con 0x7fa89410a3a0 2024-05-08T06:08:03.690 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.690+0000 7fa891600700 1 -- 172.21.15.123:0/3146340649 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa8800a9a30 con 0x7fa89410a3a0 2024-05-08T06:08:03.690 INFO:teuthology.orchestra.run.smithi123.stdout:163208757326 2024-05-08T06:08:03.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.693+0000 7fa89a600700 1 -- 172.21.15.123:0/3146340649 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa87c0b2900 msgr2=0x7fa87c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:03.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.693+0000 7fa89a600700 1 --2- 172.21.15.123:0/3146340649 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa87c0b2900 0x7fa87c0b4dc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa888009be0 tx=0x7fa888009260 comp rx=0 tx=0).stop 2024-05-08T06:08:03.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.693+0000 7fa89a600700 1 -- 172.21.15.123:0/3146340649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa89410a3a0 msgr2=0x7fa8941a05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:03.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.693+0000 7fa89a600700 1 --2- 172.21.15.123:0/3146340649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa89410a3a0 0x7fa8941a05c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa88000e9c0 tx=0x7fa88000ecd0 comp rx=0 tx=0).stop 2024-05-08T06:08:03.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.693+0000 7fa89a600700 1 -- 172.21.15.123:0/3146340649 shutdown_connections 2024-05-08T06:08:03.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.693+0000 7fa89a600700 1 --2- 172.21.15.123:0/3146340649 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa87c0b2900 0x7fa87c0b4dc0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:03.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.693+0000 7fa89a600700 1 --2- 172.21.15.123:0/3146340649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa89410e8b0 0x7fa8941a4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:03.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.693+0000 7fa89a600700 1 --2- 172.21.15.123:0/3146340649 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8941099e0 0x7fa8941a0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:03.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.693+0000 7fa89a600700 1 --2- 172.21.15.123:0/3146340649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa89410a3a0 0x7fa8941a05c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:03.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.693+0000 7fa89a600700 1 -- 172.21.15.123:0/3146340649 >> 172.21.15.123:0/3146340649 conn(0x7fa894079230 msgr2=0x7fa894079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:08:03.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.693+0000 7fa89a600700 1 -- 172.21.15.123:0/3146340649 shutdown_connections 2024-05-08T06:08:03.694 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:03.693+0000 7fa89a600700 1 -- 172.21.15.123:0/3146340649 wait complete. 2024-05-08T06:08:04.565 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645350 got 309237645355 for osd.11 2024-05-08T06:08:04.565 DEBUG:teuthology.parallel:result is None 2024-05-08T06:08:04.657 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:04 smithi123 ceph-mon[29787]: pgmap v388: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-08T06:08:04.657 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:04 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/3146340649' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-08T06:08:04.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:04 smithi163 ceph-mon[34520]: pgmap v388: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-08T06:08:04.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:04 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/3146340649' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-08T06:08:04.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:04 smithi192 ceph-mon[34664]: pgmap v388: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-08T06:08:04.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:04 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/3146340649' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-08T06:08:05.752 INFO:teuthology.orchestra.run.smithi123.stdout:188978561094 2024-05-08T06:08:05.753 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-08T06:08:06.654 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:06 smithi123 ceph-mon[29787]: pgmap v389: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-08T06:08:06.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:06 smithi163 ceph-mon[34520]: pgmap v389: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-08T06:08:06.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:06 smithi192 ceph-mon[34664]: pgmap v389: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-08T06:08:07.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.514+0000 7f7c6ba00700 1 -- 172.21.15.123:0/3240127266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c64105680 msgr2=0x7f7c64105b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:07.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.514+0000 7f7c6ba00700 1 --2- 172.21.15.123:0/3240127266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c64105680 0x7f7c64105b00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7c500098f0 tx=0x7f7c50009c00 comp rx=0 tx=0).stop 2024-05-08T06:08:07.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.514+0000 7f7c6ba00700 1 -- 172.21.15.123:0/3240127266 shutdown_connections 2024-05-08T06:08:07.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.514+0000 7f7c6ba00700 1 --2- 172.21.15.123:0/3240127266 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c64106040 0x7f7c64112550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:07.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.514+0000 7f7c6ba00700 1 --2- 172.21.15.123:0/3240127266 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c6410b680 0x7f7c6410ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:07.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.514+0000 7f7c6ba00700 1 --2- 172.21.15.123:0/3240127266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c64105680 0x7f7c64105b00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:07.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.514+0000 7f7c6ba00700 1 -- 172.21.15.123:0/3240127266 >> 172.21.15.123:0/3240127266 conn(0x7f7c64101150 msgr2=0x7f7c64103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:08:07.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.514+0000 7f7c6ba00700 1 -- 172.21.15.123:0/3240127266 shutdown_connections 2024-05-08T06:08:07.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.515+0000 7f7c6ba00700 1 -- 172.21.15.123:0/3240127266 wait complete. 2024-05-08T06:08:07.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.515+0000 7f7c6ba00700 1 Processor -- start 2024-05-08T06:08:07.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.516+0000 7f7c6ba00700 1 -- start start 2024-05-08T06:08:07.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.516+0000 7f7c6ba00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c64105680 0x7f7c6419bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:07.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.516+0000 7f7c6ba00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c64106040 0x7f7c6419c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:07.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.516+0000 7f7c6ba00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c6410b680 0x7f7c641a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:07.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.516+0000 7f7c6ba00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7c64115880 con 0x7f7c6410b680 2024-05-08T06:08:07.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.516+0000 7f7c6ba00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7c64115700 con 0x7f7c64106040 2024-05-08T06:08:07.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.516+0000 7f7c6ba00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7c64115a00 con 0x7f7c64105680 2024-05-08T06:08:07.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.516+0000 7f7c69000700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c64105680 0x7f7c6419bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:07.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.516+0000 7f7c63e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c64106040 0x7f7c6419c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:07.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.516+0000 7f7c69000700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c64105680 0x7f7c6419bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:41610/0 (socket says 172.21.15.123:41610) 2024-05-08T06:08:07.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.516+0000 7f7c69000700 1 -- 172.21.15.123:0/478435537 learned_addr learned my addr 172.21.15.123:0/478435537 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:08:07.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.517+0000 7f7c69a00700 1 --2- 172.21.15.123:0/478435537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c6410b680 0x7f7c641a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:07.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.517+0000 7f7c69000700 1 -- 172.21.15.123:0/478435537 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c64106040 msgr2=0x7f7c6419c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:07.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.517+0000 7f7c69000700 1 --2- 172.21.15.123:0/478435537 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c64106040 0x7f7c6419c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:07.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.517+0000 7f7c69000700 1 -- 172.21.15.123:0/478435537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c6410b680 msgr2=0x7f7c641a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:07.589 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.517+0000 7f7c69000700 1 --2- 172.21.15.123:0/478435537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c6410b680 0x7f7c641a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:07.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.517+0000 7f7c69000700 1 -- 172.21.15.123:0/478435537 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c641a0db0 con 0x7f7c64105680 2024-05-08T06:08:07.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.517+0000 7f7c69000700 1 --2- 172.21.15.123:0/478435537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c64105680 0x7f7c6419bdf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7c5800d950 tx=0x7f7c5800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:08:07.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.517+0000 7f7c61600700 1 -- 172.21.15.123:0/478435537 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c58016070 con 0x7f7c64105680 2024-05-08T06:08:07.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.517+0000 7f7c61600700 1 -- 172.21.15.123:0/478435537 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c5800fd40 con 0x7f7c64105680 2024-05-08T06:08:07.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.517+0000 7f7c6ba00700 1 -- 172.21.15.123:0/478435537 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c641a10a0 con 0x7f7c64105680 2024-05-08T06:08:07.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.518+0000 7f7c61600700 1 -- 172.21.15.123:0/478435537 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5800e3b0 con 0x7f7c64105680 2024-05-08T06:08:07.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.518+0000 7f7c6ba00700 1 -- 172.21.15.123:0/478435537 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c641a8920 con 0x7f7c64105680 2024-05-08T06:08:07.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.519+0000 7f7c6ba00700 1 -- 172.21.15.123:0/478435537 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c6406a190 con 0x7f7c64105680 2024-05-08T06:08:07.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.521+0000 7f7c61600700 1 -- 172.21.15.123:0/478435537 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f7c58019070 con 0x7f7c64105680 2024-05-08T06:08:07.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.523+0000 7f7c61600700 1 --2- 172.21.15.123:0/478435537 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f7c4c0b2880 0x7f7c4c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:07.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.523+0000 7f7c61600700 1 -- 172.21.15.123:0/478435537 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f7c580e1cb0 con 0x7f7c64105680 2024-05-08T06:08:07.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.523+0000 7f7c63e00700 1 --2- 172.21.15.123:0/478435537 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f7c4c0b2880 0x7f7c4c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:07.590 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.523+0000 7f7c63e00700 1 --2- 172.21.15.123:0/478435537 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f7c4c0b2880 0x7f7c4c0b4d40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7c50009fd0 tx=0x7f7c50019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:08:07.591 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.525+0000 7f7c61600700 1 -- 172.21.15.123:0/478435537 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7c580a9980 con 0x7f7c64105680 2024-05-08T06:08:07.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.754+0000 7f7c6ba00700 1 -- 172.21.15.123:0/478435537 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f7c6402e510 con 0x7f7c64105680 2024-05-08T06:08:07.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.755+0000 7f7c61600700 1 -- 172.21.15.123:0/478435537 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7c580a9280 con 0x7f7c64105680 2024-05-08T06:08:07.755 INFO:teuthology.orchestra.run.smithi123.stdout:236223201343 2024-05-08T06:08:07.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.759+0000 7f7c6ba00700 1 -- 172.21.15.123:0/478435537 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f7c4c0b2880 msgr2=0x7f7c4c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:07.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.759+0000 7f7c6ba00700 1 --2- 172.21.15.123:0/478435537 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f7c4c0b2880 0x7f7c4c0b4d40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7c50009fd0 tx=0x7f7c50019040 comp rx=0 tx=0).stop 2024-05-08T06:08:07.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.759+0000 7f7c6ba00700 1 -- 172.21.15.123:0/478435537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c64105680 msgr2=0x7f7c6419bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:07.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.759+0000 7f7c6ba00700 1 --2- 172.21.15.123:0/478435537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c64105680 0x7f7c6419bdf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7c5800d950 tx=0x7f7c5800dc60 comp rx=0 tx=0).stop 2024-05-08T06:08:07.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.759+0000 7f7c6ba00700 1 -- 172.21.15.123:0/478435537 shutdown_connections 2024-05-08T06:08:07.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.759+0000 7f7c6ba00700 1 --2- 172.21.15.123:0/478435537 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f7c4c0b2880 0x7f7c4c0b4d40 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:07.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.759+0000 7f7c6ba00700 1 --2- 172.21.15.123:0/478435537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c6410b680 0x7f7c641a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:07.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.759+0000 7f7c6ba00700 1 --2- 172.21.15.123:0/478435537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c64105680 0x7f7c6419bdf0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:07.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.759+0000 7f7c6ba00700 1 --2- 172.21.15.123:0/478435537 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c64106040 0x7f7c6419c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:07.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.759+0000 7f7c6ba00700 1 -- 172.21.15.123:0/478435537 >> 172.21.15.123:0/478435537 conn(0x7f7c64101150 msgr2=0x7f7c641019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:08:07.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.759+0000 7f7c6ba00700 1 -- 172.21.15.123:0/478435537 shutdown_connections 2024-05-08T06:08:07.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.759+0000 7f7c6ba00700 1 -- 172.21.15.123:0/478435537 wait complete. 2024-05-08T06:08:07.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.897+0000 7f1cad800700 1 -- 172.21.15.123:0/1384113176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ca81056c0 msgr2=0x7f1ca8105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:07.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.897+0000 7f1cad800700 1 --2- 172.21.15.123:0/1384113176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ca81056c0 0x7f1ca8105b40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1c90009990 tx=0x7f1c90009ca0 comp rx=0 tx=0).stop 2024-05-08T06:08:07.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.897+0000 7f1cad800700 1 -- 172.21.15.123:0/1384113176 shutdown_connections 2024-05-08T06:08:07.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.897+0000 7f1cad800700 1 --2- 172.21.15.123:0/1384113176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ca8106080 0x7f1ca8112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:07.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.897+0000 7f1cad800700 1 --2- 172.21.15.123:0/1384113176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ca810b6c0 0x7f1ca810baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:07.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.897+0000 7f1cad800700 1 --2- 172.21.15.123:0/1384113176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ca81056c0 0x7f1ca8105b40 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:07.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.897+0000 7f1cad800700 1 -- 172.21.15.123:0/1384113176 >> 172.21.15.123:0/1384113176 conn(0x7f1ca8101170 msgr2=0x7f1ca8103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:08:07.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.898+0000 7f1cad800700 1 -- 172.21.15.123:0/1384113176 shutdown_connections 2024-05-08T06:08:07.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.898+0000 7f1cad800700 1 -- 172.21.15.123:0/1384113176 wait complete. 2024-05-08T06:08:07.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.899+0000 7f1cad800700 1 Processor -- start 2024-05-08T06:08:07.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.899+0000 7f1cad800700 1 -- start start 2024-05-08T06:08:07.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.900+0000 7f1cad800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ca81056c0 0x7f1ca819bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:07.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.900+0000 7f1cad800700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ca8106080 0x7f1ca819c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:07.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.900+0000 7f1cad800700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ca810b6c0 0x7f1ca81a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:07.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.900+0000 7f1cad800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca8115950 con 0x7f1ca81056c0 2024-05-08T06:08:07.901 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.900+0000 7f1cad800700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca81157d0 con 0x7f1ca810b6c0 2024-05-08T06:08:07.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.900+0000 7f1cad800700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca8115ad0 con 0x7f1ca8106080 2024-05-08T06:08:07.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.900+0000 7f1c9ea00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ca81056c0 0x7f1ca819bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:07.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.900+0000 7f1c9ea00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ca81056c0 0x7f1ca819bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49390/0 (socket says 172.21.15.123:49390) 2024-05-08T06:08:07.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.900+0000 7f1c9ea00700 1 -- 172.21.15.123:0/1658370202 learned_addr learned my addr 172.21.15.123:0/1658370202 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:08:07.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.900+0000 7f1ca6a00700 1 --2- 172.21.15.123:0/1658370202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ca8106080 0x7f1ca819c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:07.902 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.900+0000 7f1ca7400700 1 --2- 172.21.15.123:0/1658370202 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ca810b6c0 0x7f1ca81a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:07.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.901+0000 7f1c9ea00700 1 -- 172.21.15.123:0/1658370202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ca8106080 msgr2=0x7f1ca819c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:07.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.901+0000 7f1c9ea00700 1 --2- 172.21.15.123:0/1658370202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ca8106080 0x7f1ca819c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:07.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.901+0000 7f1c9ea00700 1 -- 172.21.15.123:0/1658370202 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ca810b6c0 msgr2=0x7f1ca81a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:07.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.901+0000 7f1c9ea00700 1 --2- 172.21.15.123:0/1658370202 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ca810b6c0 0x7f1ca81a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:07.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.901+0000 7f1c9ea00700 1 -- 172.21.15.123:0/1658370202 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca81a0db0 con 0x7f1ca81056c0 2024-05-08T06:08:07.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.901+0000 7f1c9ea00700 1 --2- 172.21.15.123:0/1658370202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ca81056c0 0x7f1ca819bda0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1c9800e9a0 tx=0x7f1c9800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:08:07.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.901+0000 7f1c9fe00700 1 -- 172.21.15.123:0/1658370202 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9800ca10 con 0x7f1ca81056c0 2024-05-08T06:08:07.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.901+0000 7f1cad800700 1 -- 172.21.15.123:0/1658370202 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ca81a10a0 con 0x7f1ca81056c0 2024-05-08T06:08:07.903 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.901+0000 7f1c9fe00700 1 -- 172.21.15.123:0/1658370202 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c9800cbb0 con 0x7f1ca81056c0 2024-05-08T06:08:07.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.901+0000 7f1c9fe00700 1 -- 172.21.15.123:0/1658370202 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c98018680 con 0x7f1ca81056c0 2024-05-08T06:08:07.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.901+0000 7f1cad800700 1 -- 172.21.15.123:0/1658370202 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ca81a8920 con 0x7f1ca81056c0 2024-05-08T06:08:07.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.903+0000 7f1cad800700 1 -- 172.21.15.123:0/1658370202 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ca806a190 con 0x7f1ca81056c0 2024-05-08T06:08:07.905 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.905+0000 7f1c9fe00700 1 -- 172.21.15.123:0/1658370202 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f1c98010430 con 0x7f1ca81056c0 2024-05-08T06:08:07.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.906+0000 7f1c9fe00700 1 --2- 172.21.15.123:0/1658370202 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f1c8c0b2900 0x7f1c8c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:07.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.906+0000 7f1c9fe00700 1 -- 172.21.15.123:0/1658370202 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f1c980e22e0 con 0x7f1ca81056c0 2024-05-08T06:08:07.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.906+0000 7f1ca6a00700 1 --2- 172.21.15.123:0/1658370202 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f1c8c0b2900 0x7f1c8c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:07.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.907+0000 7f1ca6a00700 1 --2- 172.21.15.123:0/1658370202 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f1c8c0b2900 0x7f1c8c0b4dc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8103860 tx=0x7f1c9000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:08:07.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:07.911+0000 7f1c9fe00700 1 -- 172.21.15.123:0/1658370202 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1c980a9fb0 con 0x7f1ca81056c0 2024-05-08T06:08:08.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:08.158+0000 7f1cad800700 1 -- 172.21.15.123:0/1658370202 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f1ca819d100 con 0x7f1ca81056c0 2024-05-08T06:08:08.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:08.159+0000 7f1c9fe00700 1 -- 172.21.15.123:0/1658370202 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1c980a98b0 con 0x7f1ca81056c0 2024-05-08T06:08:08.160 INFO:teuthology.orchestra.run.smithi123.stdout:115964117083 2024-05-08T06:08:08.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:08.162+0000 7f1cad800700 1 -- 172.21.15.123:0/1658370202 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f1c8c0b2900 msgr2=0x7f1c8c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:08.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:08.162+0000 7f1cad800700 1 --2- 172.21.15.123:0/1658370202 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f1c8c0b2900 0x7f1c8c0b4dc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8103860 tx=0x7f1c9000b260 comp rx=0 tx=0).stop 2024-05-08T06:08:08.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:08.162+0000 7f1cad800700 1 -- 172.21.15.123:0/1658370202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ca81056c0 msgr2=0x7f1ca819bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:08.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:08.162+0000 7f1cad800700 1 --2- 172.21.15.123:0/1658370202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ca81056c0 0x7f1ca819bda0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1c9800e9a0 tx=0x7f1c9800ecb0 comp rx=0 tx=0).stop 2024-05-08T06:08:08.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:08.162+0000 7f1cad800700 1 -- 172.21.15.123:0/1658370202 shutdown_connections 2024-05-08T06:08:08.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:08.162+0000 7f1cad800700 1 --2- 172.21.15.123:0/1658370202 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f1c8c0b2900 0x7f1c8c0b4dc0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:08.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:08.162+0000 7f1cad800700 1 --2- 172.21.15.123:0/1658370202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ca81056c0 0x7f1ca819bda0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:08.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:08.162+0000 7f1cad800700 1 --2- 172.21.15.123:0/1658370202 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ca810b6c0 0x7f1ca81a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:08.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:08.162+0000 7f1cad800700 1 --2- 172.21.15.123:0/1658370202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ca8106080 0x7f1ca819c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:08.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:08.162+0000 7f1cad800700 1 -- 172.21.15.123:0/1658370202 >> 172.21.15.123:0/1658370202 conn(0x7f1ca8101170 msgr2=0x7f1ca8101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:08:08.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:08.162+0000 7f1cad800700 1 -- 172.21.15.123:0/1658370202 shutdown_connections 2024-05-08T06:08:08.164 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:08.162+0000 7f1cad800700 1 -- 172.21.15.123:0/1658370202 wait complete. 2024-05-08T06:08:08.707 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:08 smithi123 ceph-mon[29787]: pgmap v390: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-08T06:08:08.707 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:08 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/478435537' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-08T06:08:08.707 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:08 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/1658370202' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-08T06:08:08.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:08 smithi163 ceph-mon[34520]: pgmap v390: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-08T06:08:08.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:08 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/478435537' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-08T06:08:08.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:08 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/1658370202' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-08T06:08:08.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:08 smithi192 ceph-mon[34664]: pgmap v390: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-08T06:08:08.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:08 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/478435537' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-08T06:08:08.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:08 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/1658370202' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-08T06:08:10.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:10 smithi123 ceph-mon[29787]: pgmap v391: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-08T06:08:10.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:10 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/1533422979' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-08T06:08:10.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:10 smithi163 ceph-mon[34520]: pgmap v391: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-08T06:08:10.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:10 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/1533422979' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-08T06:08:10.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:10 smithi192 ceph-mon[34664]: pgmap v391: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-08T06:08:10.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:10 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/1533422979' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-08T06:08:10.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.263+0000 7f225b200700 1 -- 172.21.15.123:0/3588316540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f225c1099e0 msgr2=0x7f225c109e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:10.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.263+0000 7f225b200700 1 --2- 172.21.15.123:0/3588316540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f225c1099e0 0x7f225c109e60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2244009990 tx=0x7f2244009ca0 comp rx=0 tx=0).stop 2024-05-08T06:08:10.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.263+0000 7f225b200700 1 -- 172.21.15.123:0/3588316540 shutdown_connections 2024-05-08T06:08:10.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.263+0000 7f225b200700 1 --2- 172.21.15.123:0/3588316540 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f225c10a3a0 0x7f225c116840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:10.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.263+0000 7f225b200700 1 --2- 172.21.15.123:0/3588316540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f225c10e8b0 0x7f225c10ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:10.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.263+0000 7f225b200700 1 --2- 172.21.15.123:0/3588316540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f225c1099e0 0x7f225c109e60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:10.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.263+0000 7f225b200700 1 -- 172.21.15.123:0/3588316540 >> 172.21.15.123:0/3588316540 conn(0x7f225c079230 msgr2=0x7f225c1082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:08:10.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.263+0000 7f225b200700 1 -- 172.21.15.123:0/3588316540 shutdown_connections 2024-05-08T06:08:10.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.264+0000 7f225b200700 1 -- 172.21.15.123:0/3588316540 wait complete. 2024-05-08T06:08:10.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.265+0000 7f225b200700 1 Processor -- start 2024-05-08T06:08:10.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.265+0000 7f225b200700 1 -- start start 2024-05-08T06:08:10.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.265+0000 7f225b200700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f225c1099e0 0x7f225c1a0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:10.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.265+0000 7f225b200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f225c10a3a0 0x7f225c1a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:10.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.265+0000 7f225b200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f225c10e8b0 0x7f225c1a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:10.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.265+0000 7f225b200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f225c119be0 con 0x7f225c10a3a0 2024-05-08T06:08:10.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.265+0000 7f225b200700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f225c119a60 con 0x7f225c10e8b0 2024-05-08T06:08:10.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.265+0000 7f225b200700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f225c119d60 con 0x7f225c1099e0 2024-05-08T06:08:10.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.265+0000 7f2253e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f225c10a3a0 0x7f225c1a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:10.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.265+0000 7f2253e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f225c10a3a0 0x7f225c1a05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49402/0 (socket says 172.21.15.123:49402) 2024-05-08T06:08:10.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.265+0000 7f2253e00700 1 -- 172.21.15.123:0/1533422979 learned_addr learned my addr 172.21.15.123:0/1533422979 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:08:10.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.265+0000 7f225a800700 1 --2- 172.21.15.123:0/1533422979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f225c10e8b0 0x7f225c1a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:10.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.265+0000 7f2259e00700 1 --2- 172.21.15.123:0/1533422979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f225c1099e0 0x7f225c1a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:10.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.266+0000 7f2253e00700 1 -- 172.21.15.123:0/1533422979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f225c1099e0 msgr2=0x7f225c1a0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:10.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.266+0000 7f2253e00700 1 --2- 172.21.15.123:0/1533422979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f225c1099e0 0x7f225c1a0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:10.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.266+0000 7f2253e00700 1 -- 172.21.15.123:0/1533422979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f225c10e8b0 msgr2=0x7f225c1a4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:10.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.266+0000 7f2253e00700 1 --2- 172.21.15.123:0/1533422979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f225c10e8b0 0x7f225c1a4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:10.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.266+0000 7f2253e00700 1 -- 172.21.15.123:0/1533422979 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225c1a5080 con 0x7f225c10a3a0 2024-05-08T06:08:10.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.266+0000 7f2253e00700 1 --2- 172.21.15.123:0/1533422979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f225c10a3a0 0x7f225c1a05b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2244000c00 tx=0x7f22440057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:08:10.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.266+0000 7f2252a00700 1 -- 172.21.15.123:0/1533422979 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2244027070 con 0x7f225c10a3a0 2024-05-08T06:08:10.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.266+0000 7f225b200700 1 -- 172.21.15.123:0/1533422979 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f225c1a5310 con 0x7f225c10a3a0 2024-05-08T06:08:10.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.266+0000 7f2252a00700 1 -- 172.21.15.123:0/1533422979 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2244022470 con 0x7f225c10a3a0 2024-05-08T06:08:10.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.267+0000 7f2252a00700 1 -- 172.21.15.123:0/1533422979 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2244017650 con 0x7f225c10a3a0 2024-05-08T06:08:10.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.267+0000 7f225b200700 1 -- 172.21.15.123:0/1533422979 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f225c1acbf0 con 0x7f225c10a3a0 2024-05-08T06:08:10.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.268+0000 7f225b200700 1 -- 172.21.15.123:0/1533422979 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f225c06a190 con 0x7f225c10a3a0 2024-05-08T06:08:10.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.271+0000 7f2252a00700 1 -- 172.21.15.123:0/1533422979 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f224402a070 con 0x7f225c10a3a0 2024-05-08T06:08:10.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.273+0000 7f2252a00700 1 --2- 172.21.15.123:0/1533422979 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f22400b2880 0x7f22400b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:10.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.273+0000 7f2252a00700 1 -- 172.21.15.123:0/1533422979 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f22440e2fe0 con 0x7f225c10a3a0 2024-05-08T06:08:10.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.273+0000 7f2259e00700 1 --2- 172.21.15.123:0/1533422979 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f22400b2880 0x7f22400b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:10.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.274+0000 7f2259e00700 1 --2- 172.21.15.123:0/1533422979 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f22400b2880 0x7f22400b4d40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f224c009680 tx=0x7f224c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:08:10.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.276+0000 7f2252a00700 1 -- 172.21.15.123:0/1533422979 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2244013070 con 0x7f225c10a3a0 2024-05-08T06:08:10.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.515+0000 7f225b200700 1 -- 172.21.15.123:0/1533422979 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f225c02e510 con 0x7f225c10a3a0 2024-05-08T06:08:10.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.515+0000 7f2252a00700 1 -- 172.21.15.123:0/1533422979 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f22440aada0 con 0x7f225c10a3a0 2024-05-08T06:08:10.976 INFO:teuthology.orchestra.run.smithi123.stdout:283467841587 2024-05-08T06:08:10.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.519+0000 7f225b200700 1 -- 172.21.15.123:0/1533422979 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f22400b2880 msgr2=0x7f22400b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:10.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.519+0000 7f225b200700 1 --2- 172.21.15.123:0/1533422979 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f22400b2880 0x7f22400b4d40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f224c009680 tx=0x7f224c009260 comp rx=0 tx=0).stop 2024-05-08T06:08:10.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.519+0000 7f225b200700 1 -- 172.21.15.123:0/1533422979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f225c10a3a0 msgr2=0x7f225c1a05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:10.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.519+0000 7f225b200700 1 --2- 172.21.15.123:0/1533422979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f225c10a3a0 0x7f225c1a05b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2244000c00 tx=0x7f22440057c0 comp rx=0 tx=0).stop 2024-05-08T06:08:10.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.519+0000 7f225b200700 1 -- 172.21.15.123:0/1533422979 shutdown_connections 2024-05-08T06:08:10.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.519+0000 7f225b200700 1 --2- 172.21.15.123:0/1533422979 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f22400b2880 0x7f22400b4d40 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:10.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.519+0000 7f225b200700 1 --2- 172.21.15.123:0/1533422979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f225c10e8b0 0x7f225c1a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:10.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.519+0000 7f225b200700 1 --2- 172.21.15.123:0/1533422979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f225c1099e0 0x7f225c1a0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:10.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.519+0000 7f225b200700 1 --2- 172.21.15.123:0/1533422979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f225c10a3a0 0x7f225c1a05b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:10.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.519+0000 7f225b200700 1 -- 172.21.15.123:0/1533422979 >> 172.21.15.123:0/1533422979 conn(0x7f225c079230 msgr2=0x7f225c0798b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:08:10.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.519+0000 7f225b200700 1 -- 172.21.15.123:0/1533422979 shutdown_connections 2024-05-08T06:08:10.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:09.519+0000 7f225b200700 1 -- 172.21.15.123:0/1533422979 wait complete. 2024-05-08T06:08:11.702 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757321 got 163208757326 for osd.5 2024-05-08T06:08:11.702 DEBUG:teuthology.parallel:result is None 2024-05-08T06:08:12.591 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:12 smithi123 ceph-mon[29787]: pgmap v392: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-08T06:08:12.625 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:08:12.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:12 smithi163 ceph-mon[34520]: pgmap v392: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-08T06:08:12.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:12 smithi192 ceph-mon[34664]: pgmap v392: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-08T06:08:13.448 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841581 got 283467841587 for osd.10 2024-05-08T06:08:13.449 DEBUG:teuthology.parallel:result is None 2024-05-08T06:08:13.574 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201339 got 236223201343 for osd.8 2024-05-08T06:08:13.574 DEBUG:teuthology.parallel:result is None 2024-05-08T06:08:14.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:14 smithi123 ceph-mon[29787]: pgmap v393: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-08T06:08:14.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:14 smithi163 ceph-mon[34520]: pgmap v393: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-08T06:08:14.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:14 smithi192 ceph-mon[34664]: pgmap v393: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-08T06:08:15.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.603+0000 7f9f23200700 1 -- 172.21.15.123:0/971378013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f1c104eb0 msgr2=0x7f9f1c10d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:15.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.603+0000 7f9f23200700 1 --2- 172.21.15.123:0/971378013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f1c104eb0 0x7f9f1c10d3a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c009990 tx=0x7f9f0c009ca0 comp rx=0 tx=0).stop 2024-05-08T06:08:15.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.603+0000 7f9f23200700 1 -- 172.21.15.123:0/971378013 shutdown_connections 2024-05-08T06:08:15.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.603+0000 7f9f23200700 1 --2- 172.21.15.123:0/971378013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f1c10d9c0 0x7f9f1c10fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:15.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.603+0000 7f9f23200700 1 --2- 172.21.15.123:0/971378013 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f1c104590 0x7f9f1c104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:15.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.603+0000 7f9f23200700 1 --2- 172.21.15.123:0/971378013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f1c104eb0 0x7f9f1c10d3a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:15.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.603+0000 7f9f23200700 1 -- 172.21.15.123:0/971378013 >> 172.21.15.123:0/971378013 conn(0x7f9f1c0fddc0 msgr2=0x7f9f1c1001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:08:15.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.603+0000 7f9f23200700 1 -- 172.21.15.123:0/971378013 shutdown_connections 2024-05-08T06:08:15.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.603+0000 7f9f23200700 1 -- 172.21.15.123:0/971378013 wait complete. 2024-05-08T06:08:15.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.604+0000 7f9f23200700 1 Processor -- start 2024-05-08T06:08:15.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.604+0000 7f9f23200700 1 -- start start 2024-05-08T06:08:15.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.605+0000 7f9f23200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f1c104590 0x7f9f1c19bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:15.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.605+0000 7f9f23200700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f1c104eb0 0x7f9f1c19c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:15.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.605+0000 7f9f23200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f1c10d9c0 0x7f9f1c1a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:15.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.605+0000 7f9f23200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1c113600 con 0x7f9f1c10d9c0 2024-05-08T06:08:15.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.605+0000 7f9f23200700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1c113480 con 0x7f9f1c104590 2024-05-08T06:08:15.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.605+0000 7f9f23200700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1c113780 con 0x7f9f1c104eb0 2024-05-08T06:08:15.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.605+0000 7f9f22800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f1c10d9c0 0x7f9f1c1a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:15.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.605+0000 7f9f21e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f1c104590 0x7f9f1c19bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:15.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.605+0000 7f9f22800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f1c10d9c0 0x7f9f1c1a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49438/0 (socket says 172.21.15.123:49438) 2024-05-08T06:08:15.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.605+0000 7f9f22800700 1 -- 172.21.15.123:0/1424140265 learned_addr learned my addr 172.21.15.123:0/1424140265 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:08:15.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.605+0000 7f9f21400700 1 --2- 172.21.15.123:0/1424140265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f1c104eb0 0x7f9f1c19c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:15.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.605+0000 7f9f22800700 1 -- 172.21.15.123:0/1424140265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f1c104eb0 msgr2=0x7f9f1c19c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:15.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.605+0000 7f9f22800700 1 --2- 172.21.15.123:0/1424140265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f1c104eb0 0x7f9f1c19c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:15.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.605+0000 7f9f22800700 1 -- 172.21.15.123:0/1424140265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f1c104590 msgr2=0x7f9f1c19bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:15.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.605+0000 7f9f22800700 1 --2- 172.21.15.123:0/1424140265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f1c104590 0x7f9f1c19bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:15.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.605+0000 7f9f22800700 1 -- 172.21.15.123:0/1424140265 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f1c1a0dc0 con 0x7f9f1c10d9c0 2024-05-08T06:08:15.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.606+0000 7f9f21e00700 1 --2- 172.21.15.123:0/1424140265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f1c104590 0x7f9f1c19bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:08:15.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.606+0000 7f9f21400700 1 --2- 172.21.15.123:0/1424140265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f1c104eb0 0x7f9f1c19c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:08:15.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.606+0000 7f9f22800700 1 --2- 172.21.15.123:0/1424140265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f1c10d9c0 0x7f9f1c1a0680 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9f18007ee0 tx=0x7f9f1800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:08:15.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.606+0000 7f9f16a00700 1 -- 172.21.15.123:0/1424140265 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f18004bb0 con 0x7f9f1c10d9c0 2024-05-08T06:08:15.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.606+0000 7f9f16a00700 1 -- 172.21.15.123:0/1424140265 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f18004d50 con 0x7f9f1c10d9c0 2024-05-08T06:08:15.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.606+0000 7f9f23200700 1 -- 172.21.15.123:0/1424140265 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f1c1a10b0 con 0x7f9f1c10d9c0 2024-05-08T06:08:15.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.606+0000 7f9f16a00700 1 -- 172.21.15.123:0/1424140265 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f18005920 con 0x7f9f1c10d9c0 2024-05-08T06:08:15.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.606+0000 7f9f23200700 1 -- 172.21.15.123:0/1424140265 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f1c1a8930 con 0x7f9f1c10d9c0 2024-05-08T06:08:15.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.607+0000 7f9f23200700 1 -- 172.21.15.123:0/1424140265 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f1c06a190 con 0x7f9f1c10d9c0 2024-05-08T06:08:15.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.610+0000 7f9f16a00700 1 -- 172.21.15.123:0/1424140265 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f9f1801a020 con 0x7f9f1c10d9c0 2024-05-08T06:08:15.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.612+0000 7f9f16a00700 1 --2- 172.21.15.123:0/1424140265 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9f040b2900 0x7f9f040b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:15.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.613+0000 7f9f16a00700 1 -- 172.21.15.123:0/1424140265 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f9f180e1e60 con 0x7f9f1c10d9c0 2024-05-08T06:08:15.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.613+0000 7f9f21e00700 1 --2- 172.21.15.123:0/1424140265 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9f040b2900 0x7f9f040b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:15.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.613+0000 7f9f21e00700 1 --2- 172.21.15.123:0/1424140265 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9f040b2900 0x7f9f040b4dc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9f10011490 tx=0x7f9f10009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:08:15.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.615+0000 7f9f16a00700 1 -- 172.21.15.123:0/1424140265 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9f18010070 con 0x7f9f1c10d9c0 2024-05-08T06:08:15.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.860+0000 7f9f23200700 1 -- 172.21.15.123:0/1424140265 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f9f1c1a8b00 con 0x7f9f1c10d9c0 2024-05-08T06:08:15.245 INFO:teuthology.orchestra.run.smithi123.stdout:77309411426 2024-05-08T06:08:15.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.861+0000 7f9f16a00700 1 -- 172.21.15.123:0/1424140265 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9f180a9b30 con 0x7f9f1c10d9c0 2024-05-08T06:08:15.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.864+0000 7f9f23200700 1 -- 172.21.15.123:0/1424140265 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9f040b2900 msgr2=0x7f9f040b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:15.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.864+0000 7f9f23200700 1 --2- 172.21.15.123:0/1424140265 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9f040b2900 0x7f9f040b4dc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9f10011490 tx=0x7f9f10009260 comp rx=0 tx=0).stop 2024-05-08T06:08:15.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.864+0000 7f9f23200700 1 -- 172.21.15.123:0/1424140265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f1c10d9c0 msgr2=0x7f9f1c1a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:15.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.864+0000 7f9f23200700 1 --2- 172.21.15.123:0/1424140265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f1c10d9c0 0x7f9f1c1a0680 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9f18007ee0 tx=0x7f9f1800d3b0 comp rx=0 tx=0).stop 2024-05-08T06:08:15.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.865+0000 7f9f23200700 1 -- 172.21.15.123:0/1424140265 shutdown_connections 2024-05-08T06:08:15.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.865+0000 7f9f23200700 1 --2- 172.21.15.123:0/1424140265 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9f040b2900 0x7f9f040b4dc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:15.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.865+0000 7f9f23200700 1 --2- 172.21.15.123:0/1424140265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f1c10d9c0 0x7f9f1c1a0680 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:15.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.865+0000 7f9f23200700 1 --2- 172.21.15.123:0/1424140265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f1c104590 0x7f9f1c19bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:15.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.865+0000 7f9f23200700 1 --2- 172.21.15.123:0/1424140265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f1c104eb0 0x7f9f1c19c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:15.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.865+0000 7f9f23200700 1 -- 172.21.15.123:0/1424140265 >> 172.21.15.123:0/1424140265 conn(0x7f9f1c0fddc0 msgr2=0x7f9f1c1001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:08:15.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.865+0000 7f9f23200700 1 -- 172.21.15.123:0/1424140265 shutdown_connections 2024-05-08T06:08:15.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.865+0000 7f9f23200700 1 -- 172.21.15.123:0/1424140265 wait complete. 2024-05-08T06:08:15.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:15 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/4189439183' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-08T06:08:15.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:15 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/1424140265' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-08T06:08:15.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:15 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/662727699' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-08T06:08:15.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.254+0000 7f152c400700 1 -- 172.21.15.123:0/3260555397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1524101b80 msgr2=0x7f1524102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:15.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.254+0000 7f152c400700 1 --2- 172.21.15.123:0/3260555397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1524101b80 0x7f1524102000 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f15100099c0 tx=0x7f1510009cd0 comp rx=0 tx=0).stop 2024-05-08T06:08:15.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.255+0000 7f152c400700 1 -- 172.21.15.123:0/3260555397 shutdown_connections 2024-05-08T06:08:15.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.255+0000 7f152c400700 1 --2- 172.21.15.123:0/3260555397 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1524102540 0x7f152410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:15.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.255+0000 7f152c400700 1 --2- 172.21.15.123:0/3260555397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1524107be0 0x7f1524107fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:15.746 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.255+0000 7f152c400700 1 --2- 172.21.15.123:0/3260555397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1524101b80 0x7f1524102000 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:15.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.255+0000 7f152c400700 1 -- 172.21.15.123:0/3260555397 >> 172.21.15.123:0/3260555397 conn(0x7f1524078a10 msgr2=0x7f15240ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:08:15.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.255+0000 7f152c400700 1 -- 172.21.15.123:0/3260555397 shutdown_connections 2024-05-08T06:08:15.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.255+0000 7f152c400700 1 -- 172.21.15.123:0/3260555397 wait complete. 2024-05-08T06:08:15.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.256+0000 7f152c400700 1 Processor -- start 2024-05-08T06:08:15.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.256+0000 7f152c400700 1 -- start start 2024-05-08T06:08:15.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.257+0000 7f152c400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1524101b80 0x7f152419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:15.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.257+0000 7f152c400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1524102540 0x7f152419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:15.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.257+0000 7f152c400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1524107be0 0x7f1524199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:15.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.257+0000 7f152c400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f15241125c0 con 0x7f1524107be0 2024-05-08T06:08:15.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.257+0000 7f152c400700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1524112440 con 0x7f1524101b80 2024-05-08T06:08:15.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.257+0000 7f152c400700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1524112740 con 0x7f1524102540 2024-05-08T06:08:15.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.257+0000 7f152a400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1524107be0 0x7f1524199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:15.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.257+0000 7f152a400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1524107be0 0x7f1524199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49426/0 (socket says 172.21.15.123:49426) 2024-05-08T06:08:15.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.257+0000 7f152a400700 1 -- 172.21.15.123:0/4189439183 learned_addr learned my addr 172.21.15.123:0/4189439183 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:08:15.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.257+0000 7f1529000700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1524102540 0x7f152419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:15.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.257+0000 7f1529a00700 1 --2- 172.21.15.123:0/4189439183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1524101b80 0x7f152419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:15.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.258+0000 7f152a400700 1 -- 172.21.15.123:0/4189439183 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1524102540 msgr2=0x7f152419fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:15.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.258+0000 7f152a400700 1 --2- 172.21.15.123:0/4189439183 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1524102540 0x7f152419fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:15.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.258+0000 7f152a400700 1 -- 172.21.15.123:0/4189439183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1524101b80 msgr2=0x7f152419f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:15.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.258+0000 7f152a400700 1 --2- 172.21.15.123:0/4189439183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1524101b80 0x7f152419f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:15.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.258+0000 7f152a400700 1 -- 172.21.15.123:0/4189439183 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f152419a060 con 0x7f1524107be0 2024-05-08T06:08:15.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.258+0000 7f1529a00700 1 --2- 172.21.15.123:0/4189439183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1524101b80 0x7f152419f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:08:15.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.258+0000 7f152a400700 1 --2- 172.21.15.123:0/4189439183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1524107be0 0x7f1524199900 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f151c00c370 tx=0x7f151c00c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:08:15.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.258+0000 7f1522000700 1 -- 172.21.15.123:0/4189439183 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c0092e0 con 0x7f1524107be0 2024-05-08T06:08:15.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.258+0000 7f1522000700 1 -- 172.21.15.123:0/4189439183 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f151c003a00 con 0x7f1524107be0 2024-05-08T06:08:15.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.258+0000 7f152c400700 1 -- 172.21.15.123:0/4189439183 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f152419a2f0 con 0x7f1524107be0 2024-05-08T06:08:15.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.258+0000 7f1522000700 1 -- 172.21.15.123:0/4189439183 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c0046f0 con 0x7f1524107be0 2024-05-08T06:08:15.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.259+0000 7f152c400700 1 -- 172.21.15.123:0/4189439183 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15241a65c0 con 0x7f1524107be0 2024-05-08T06:08:15.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.260+0000 7f152c400700 1 -- 172.21.15.123:0/4189439183 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f152406a190 con 0x7f1524107be0 2024-05-08T06:08:15.752 INFO:teuthology.orchestra.run.smithi123.stdout:261993005114 2024-05-08T06:08:15.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.262+0000 7f1522000700 1 -- 172.21.15.123:0/4189439183 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f151c01a020 con 0x7f1524107be0 2024-05-08T06:08:15.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.263+0000 7f1522000700 1 --2- 172.21.15.123:0/4189439183 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f150c0b2900 0x7f150c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:15.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.264+0000 7f1522000700 1 -- 172.21.15.123:0/4189439183 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f151c0e1a80 con 0x7f1524107be0 2024-05-08T06:08:15.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.264+0000 7f1529a00700 1 --2- 172.21.15.123:0/4189439183 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f150c0b2900 0x7f150c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:15.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.264+0000 7f1529a00700 1 --2- 172.21.15.123:0/4189439183 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f150c0b2900 0x7f150c0b4dc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1524106bf0 tx=0x7f1518009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:08:15.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.268+0000 7f1522000700 1 -- 172.21.15.123:0/4189439183 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f151c0e3e20 con 0x7f1524107be0 2024-05-08T06:08:15.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.511+0000 7f152c400700 1 -- 172.21.15.123:0/4189439183 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f15241a6790 con 0x7f1524107be0 2024-05-08T06:08:15.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.511+0000 7f1522000700 1 -- 172.21.15.123:0/4189439183 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f151c010040 con 0x7f1524107be0 2024-05-08T06:08:15.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.515+0000 7f152c400700 1 -- 172.21.15.123:0/4189439183 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f150c0b2900 msgr2=0x7f150c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:15.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.515+0000 7f152c400700 1 --2- 172.21.15.123:0/4189439183 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f150c0b2900 0x7f150c0b4dc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1524106bf0 tx=0x7f1518009260 comp rx=0 tx=0).stop 2024-05-08T06:08:15.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.515+0000 7f152c400700 1 -- 172.21.15.123:0/4189439183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1524107be0 msgr2=0x7f1524199900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:15.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.515+0000 7f152c400700 1 --2- 172.21.15.123:0/4189439183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1524107be0 0x7f1524199900 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f151c00c370 tx=0x7f151c00c680 comp rx=0 tx=0).stop 2024-05-08T06:08:15.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.515+0000 7f152c400700 1 -- 172.21.15.123:0/4189439183 shutdown_connections 2024-05-08T06:08:15.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.515+0000 7f152c400700 1 --2- 172.21.15.123:0/4189439183 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f150c0b2900 0x7f150c0b4dc0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:15.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.515+0000 7f152c400700 1 --2- 172.21.15.123:0/4189439183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1524107be0 0x7f1524199900 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:15.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.515+0000 7f152c400700 1 --2- 172.21.15.123:0/4189439183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1524101b80 0x7f152419f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:15.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.515+0000 7f152c400700 1 --2- 172.21.15.123:0/4189439183 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1524102540 0x7f152419fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:15.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.515+0000 7f152c400700 1 -- 172.21.15.123:0/4189439183 >> 172.21.15.123:0/4189439183 conn(0x7f1524078a10 msgr2=0x7f15241007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:08:15.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.515+0000 7f152c400700 1 -- 172.21.15.123:0/4189439183 shutdown_connections 2024-05-08T06:08:15.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:14.515+0000 7f152c400700 1 -- 172.21.15.123:0/4189439183 wait complete. 2024-05-08T06:08:15.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:15 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/4189439183' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-08T06:08:15.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:15 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/1424140265' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-08T06:08:15.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:15 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/662727699' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-08T06:08:15.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:15 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/4189439183' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-08T06:08:15.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:15 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/1424140265' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-08T06:08:15.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:15 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/662727699' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-08T06:08:16.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.115+0000 7f2cf7600700 1 -- 172.21.15.123:0/3031873909 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cf0103750 msgr2=0x7f2cf0103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:16.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.115+0000 7f2cf7600700 1 --2- 172.21.15.123:0/3031873909 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cf0103750 0x7f2cf0103b30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009920 tx=0x7f2ce4009c30 comp rx=0 tx=0).stop 2024-05-08T06:08:16.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.115+0000 7f2cf7600700 1 -- 172.21.15.123:0/3031873909 shutdown_connections 2024-05-08T06:08:16.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.115+0000 7f2cf7600700 1 --2- 172.21.15.123:0/3031873909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2cf0108850 0x7f2cf01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:16.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.115+0000 7f2cf7600700 1 --2- 172.21.15.123:0/3031873909 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cf0103750 0x7f2cf0103b30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:16.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.115+0000 7f2cf7600700 1 --2- 172.21.15.123:0/3031873909 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cf0104100 0x7f2cf0108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:16.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.115+0000 7f2cf7600700 1 -- 172.21.15.123:0/3031873909 >> 172.21.15.123:0/3031873909 conn(0x7f2cf007a600 msgr2=0x7f2cf007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:08:16.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.115+0000 7f2cf7600700 1 -- 172.21.15.123:0/3031873909 shutdown_connections 2024-05-08T06:08:16.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.115+0000 7f2cf7600700 1 -- 172.21.15.123:0/3031873909 wait complete. 2024-05-08T06:08:16.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.116+0000 7f2cf7600700 1 Processor -- start 2024-05-08T06:08:16.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.117+0000 7f2cf7600700 1 -- start start 2024-05-08T06:08:16.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.117+0000 7f2cf7600700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cf0103750 0x7f2cf019bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:16.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.117+0000 7f2cf7600700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cf0104100 0x7f2cf019c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:16.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.117+0000 7f2cf7600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2cf0108850 0x7f2cf01a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:16.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.117+0000 7f2cf7600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf0113670 con 0x7f2cf0108850 2024-05-08T06:08:16.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.117+0000 7f2cf7600700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf01134f0 con 0x7f2cf0104100 2024-05-08T06:08:16.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.117+0000 7f2cf7600700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf01137f0 con 0x7f2cf0103750 2024-05-08T06:08:16.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.117+0000 7f2cf5600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2cf0108850 0x7f2cf01a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:16.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.117+0000 7f2cf5600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2cf0108850 0x7f2cf01a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41614/0 (socket says 172.21.15.123:41614) 2024-05-08T06:08:16.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.117+0000 7f2cf5600700 1 -- 172.21.15.123:0/662727699 learned_addr learned my addr 172.21.15.123:0/662727699 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:08:16.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.117+0000 7f2cf4c00700 1 --2- 172.21.15.123:0/662727699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cf0103750 0x7f2cf019bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:16.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.117+0000 7f2cefe00700 1 --2- 172.21.15.123:0/662727699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cf0104100 0x7f2cf019c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:16.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.118+0000 7f2cf5600700 1 -- 172.21.15.123:0/662727699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cf0103750 msgr2=0x7f2cf019bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:16.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.118+0000 7f2cf5600700 1 --2- 172.21.15.123:0/662727699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cf0103750 0x7f2cf019bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:16.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.118+0000 7f2cf5600700 1 -- 172.21.15.123:0/662727699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cf0104100 msgr2=0x7f2cf019c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:16.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.118+0000 7f2cf5600700 1 --2- 172.21.15.123:0/662727699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cf0104100 0x7f2cf019c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:16.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.118+0000 7f2cf5600700 1 -- 172.21.15.123:0/662727699 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cf01a0df0 con 0x7f2cf0108850 2024-05-08T06:08:16.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.118+0000 7f2cf5600700 1 --2- 172.21.15.123:0/662727699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2cf0108850 0x7f2cf01a06b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2ce800ef30 tx=0x7f2ce800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:08:16.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.118+0000 7f2ced600700 1 -- 172.21.15.123:0/662727699 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce80092e0 con 0x7f2cf0108850 2024-05-08T06:08:16.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.118+0000 7f2cf7600700 1 -- 172.21.15.123:0/662727699 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cf01a1080 con 0x7f2cf0108850 2024-05-08T06:08:16.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.118+0000 7f2ced600700 1 -- 172.21.15.123:0/662727699 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ce8004bb0 con 0x7f2cf0108850 2024-05-08T06:08:16.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.118+0000 7f2ced600700 1 -- 172.21.15.123:0/662727699 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce8004020 con 0x7f2cf0108850 2024-05-08T06:08:16.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.119+0000 7f2cf7600700 1 -- 172.21.15.123:0/662727699 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cf01a8960 con 0x7f2cf0108850 2024-05-08T06:08:16.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.120+0000 7f2cf7600700 1 -- 172.21.15.123:0/662727699 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cf006a190 con 0x7f2cf0108850 2024-05-08T06:08:16.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.122+0000 7f2ced600700 1 -- 172.21.15.123:0/662727699 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2ce801b030 con 0x7f2cf0108850 2024-05-08T06:08:16.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.123+0000 7f2ced600700 1 --2- 172.21.15.123:0/662727699 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f2cd80b2880 0x7f2cd80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:16.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.124+0000 7f2ced600700 1 -- 172.21.15.123:0/662727699 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f2ce80e1d70 con 0x7f2cf0108850 2024-05-08T06:08:16.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.124+0000 7f2cf4c00700 1 --2- 172.21.15.123:0/662727699 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f2cd80b2880 0x7f2cd80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:16.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.124+0000 7f2cf4c00700 1 --2- 172.21.15.123:0/662727699 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f2cd80b2880 0x7f2cd80b4d40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009680 tx=0x7f2ce4019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:08:16.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.128+0000 7f2ced600700 1 -- 172.21.15.123:0/662727699 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2ce8022020 con 0x7f2cf0108850 2024-05-08T06:08:16.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.361+0000 7f2cf7600700 1 -- 172.21.15.123:0/662727699 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f2cf01a8b30 con 0x7f2cf0108850 2024-05-08T06:08:16.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.362+0000 7f2ced600700 1 -- 172.21.15.123:0/662727699 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2ce8010040 con 0x7f2cf0108850 2024-05-08T06:08:16.108 INFO:teuthology.orchestra.run.smithi123.stdout:141733920854 2024-05-08T06:08:16.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.365+0000 7f2cf7600700 1 -- 172.21.15.123:0/662727699 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f2cd80b2880 msgr2=0x7f2cd80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:16.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.365+0000 7f2cf7600700 1 --2- 172.21.15.123:0/662727699 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f2cd80b2880 0x7f2cd80b4d40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009680 tx=0x7f2ce4019040 comp rx=0 tx=0).stop 2024-05-08T06:08:16.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.365+0000 7f2cf7600700 1 -- 172.21.15.123:0/662727699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2cf0108850 msgr2=0x7f2cf01a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:16.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.365+0000 7f2cf7600700 1 --2- 172.21.15.123:0/662727699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2cf0108850 0x7f2cf01a06b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2ce800ef30 tx=0x7f2ce800c3a0 comp rx=0 tx=0).stop 2024-05-08T06:08:16.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.366+0000 7f2cf7600700 1 -- 172.21.15.123:0/662727699 shutdown_connections 2024-05-08T06:08:16.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.366+0000 7f2cf7600700 1 --2- 172.21.15.123:0/662727699 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f2cd80b2880 0x7f2cd80b4d40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:16.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.366+0000 7f2cf7600700 1 --2- 172.21.15.123:0/662727699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2cf0108850 0x7f2cf01a06b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:16.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.366+0000 7f2cf7600700 1 --2- 172.21.15.123:0/662727699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cf0103750 0x7f2cf019bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:16.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.366+0000 7f2cf7600700 1 --2- 172.21.15.123:0/662727699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cf0104100 0x7f2cf019c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:16.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.366+0000 7f2cf7600700 1 -- 172.21.15.123:0/662727699 >> 172.21.15.123:0/662727699 conn(0x7f2cf007a600 msgr2=0x7f2cf0101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:08:16.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.366+0000 7f2cf7600700 1 -- 172.21.15.123:0/662727699 shutdown_connections 2024-05-08T06:08:16.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:15.366+0000 7f2cf7600700 1 -- 172.21.15.123:0/662727699 wait complete. 2024-05-08T06:08:16.640 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117075 got 115964117083 for osd.3 2024-05-08T06:08:16.640 DEBUG:teuthology.parallel:result is None 2024-05-08T06:08:16.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:16 smithi123 ceph-mon[29787]: pgmap v394: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-08T06:08:16.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:16 smithi163 ceph-mon[34520]: pgmap v394: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-08T06:08:16.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:16 smithi192 ceph-mon[34664]: pgmap v394: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-08T06:08:16.868 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:08:17.717 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411418 got 77309411426 for osd.2 2024-05-08T06:08:17.718 DEBUG:teuthology.parallel:result is None 2024-05-08T06:08:18.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:18 smithi123 ceph-mon[29787]: pgmap v395: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-08T06:08:18.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:18 smithi163 ceph-mon[34520]: pgmap v395: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-08T06:08:18.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:18 smithi192 ceph-mon[34664]: pgmap v395: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-08T06:08:20.463 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005108 got 261993005114 for osd.9 2024-05-08T06:08:20.463 DEBUG:teuthology.parallel:result is None 2024-05-08T06:08:20.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:20 smithi123 ceph-mon[29787]: pgmap v396: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:08:20.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:20 smithi163 ceph-mon[34520]: pgmap v396: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:08:20.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:20 smithi192 ceph-mon[34664]: pgmap v396: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:08:20.979 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920846 got 141733920854 for osd.4 2024-05-08T06:08:20.980 DEBUG:teuthology.parallel:result is None 2024-05-08T06:08:21.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.190+0000 7f2e2ca00700 1 -- 172.21.15.123:0/936634699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e281045e0 msgr2=0x7f2e281049c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:21.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.190+0000 7f2e2ca00700 1 --2- 172.21.15.123:0/936634699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e281045e0 0x7f2e281049c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2e18009990 tx=0x7f2e18009ca0 comp rx=0 tx=0).stop 2024-05-08T06:08:21.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.190+0000 7f2e2ca00700 1 -- 172.21.15.123:0/936634699 shutdown_connections 2024-05-08T06:08:21.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.190+0000 7f2e2ca00700 1 --2- 172.21.15.123:0/936634699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e281045e0 0x7f2e281049c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:21.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.190+0000 7f2e2ca00700 1 --2- 172.21.15.123:0/936634699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e2810da10 0x7f2e2810fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:21.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.190+0000 7f2e2ca00700 1 --2- 172.21.15.123:0/936634699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e28104f00 0x7f2e2810d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:21.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.190+0000 7f2e2ca00700 1 -- 172.21.15.123:0/936634699 >> 172.21.15.123:0/936634699 conn(0x7f2e280fde30 msgr2=0x7f2e28100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:08:21.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.191+0000 7f2e2ca00700 1 -- 172.21.15.123:0/936634699 shutdown_connections 2024-05-08T06:08:21.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.191+0000 7f2e2ca00700 1 -- 172.21.15.123:0/936634699 wait complete. 2024-05-08T06:08:21.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.192+0000 7f2e2ca00700 1 Processor -- start 2024-05-08T06:08:21.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.192+0000 7f2e2ca00700 1 -- start start 2024-05-08T06:08:21.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.193+0000 7f2e2ca00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e281045e0 0x7f2e2819bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:21.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.193+0000 7f2e2ca00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e28104f00 0x7f2e2819c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:21.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.193+0000 7f2e2ca00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e2810da10 0x7f2e281a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:21.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.193+0000 7f2e2ca00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2e28113670 con 0x7f2e281045e0 2024-05-08T06:08:21.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.193+0000 7f2e2ca00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2e281134f0 con 0x7f2e2810da10 2024-05-08T06:08:21.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.193+0000 7f2e2ca00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2e281137f0 con 0x7f2e28104f00 2024-05-08T06:08:21.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.193+0000 7f2e26800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e281045e0 0x7f2e2819bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:21.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.193+0000 7f2e1fe00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e28104f00 0x7f2e2819c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:21.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.193+0000 7f2e26800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e281045e0 0x7f2e2819bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41626/0 (socket says 172.21.15.123:41626) 2024-05-08T06:08:21.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.193+0000 7f2e26800700 1 -- 172.21.15.123:0/1071356917 learned_addr learned my addr 172.21.15.123:0/1071356917 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:08:21.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.193+0000 7f2e27200700 1 --2- 172.21.15.123:0/1071356917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e2810da10 0x7f2e281a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:21.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.194+0000 7f2e26800700 1 -- 172.21.15.123:0/1071356917 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e28104f00 msgr2=0x7f2e2819c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:21.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.194+0000 7f2e26800700 1 --2- 172.21.15.123:0/1071356917 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e28104f00 0x7f2e2819c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:21.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.194+0000 7f2e26800700 1 -- 172.21.15.123:0/1071356917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e2810da10 msgr2=0x7f2e281a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:21.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.194+0000 7f2e26800700 1 --2- 172.21.15.123:0/1071356917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e2810da10 0x7f2e281a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:21.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.194+0000 7f2e26800700 1 -- 172.21.15.123:0/1071356917 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e281a0d90 con 0x7f2e281045e0 2024-05-08T06:08:21.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.194+0000 7f2e26800700 1 --2- 172.21.15.123:0/1071356917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e281045e0 0x7f2e2819bd80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2e18000c00 tx=0x7f2e180057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:08:21.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.194+0000 7f2e1f400700 1 -- 172.21.15.123:0/1071356917 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e18027070 con 0x7f2e281045e0 2024-05-08T06:08:21.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.194+0000 7f2e1f400700 1 -- 172.21.15.123:0/1071356917 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e18022470 con 0x7f2e281045e0 2024-05-08T06:08:21.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.194+0000 7f2e2ca00700 1 -- 172.21.15.123:0/1071356917 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e281a1020 con 0x7f2e281045e0 2024-05-08T06:08:21.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.195+0000 7f2e1f400700 1 -- 172.21.15.123:0/1071356917 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e18017650 con 0x7f2e281045e0 2024-05-08T06:08:21.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.195+0000 7f2e2ca00700 1 -- 172.21.15.123:0/1071356917 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e281a8900 con 0x7f2e281045e0 2024-05-08T06:08:21.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.196+0000 7f2e2ca00700 1 -- 172.21.15.123:0/1071356917 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e2806a190 con 0x7f2e281045e0 2024-05-08T06:08:21.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.198+0000 7f2e1f400700 1 -- 172.21.15.123:0/1071356917 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2e1802a070 con 0x7f2e281045e0 2024-05-08T06:08:21.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.201+0000 7f2e1f400700 1 --2- 172.21.15.123:0/1071356917 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f2e0c0b2900 0x7f2e0c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:21.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.201+0000 7f2e1f400700 1 -- 172.21.15.123:0/1071356917 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f2e180e3230 con 0x7f2e281045e0 2024-05-08T06:08:21.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.201+0000 7f2e1fe00700 1 --2- 172.21.15.123:0/1071356917 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f2e0c0b2900 0x7f2e0c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:21.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.202+0000 7f2e1fe00700 1 --2- 172.21.15.123:0/1071356917 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f2e0c0b2900 0x7f2e0c0b4dc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2e28110300 tx=0x7f2e10007320 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:08:21.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.204+0000 7f2e1f400700 1 -- 172.21.15.123:0/1071356917 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2e18013070 con 0x7f2e281045e0 2024-05-08T06:08:21.422 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:08:21.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.464+0000 7f2e2ca00700 1 -- 172.21.15.123:0/1071356917 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f2e281a8ad0 con 0x7f2e281045e0 2024-05-08T06:08:21.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.464+0000 7f2e1f400700 1 -- 172.21.15.123:0/1071356917 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2e180aaf90 con 0x7f2e281045e0 2024-05-08T06:08:21.464 INFO:teuthology.orchestra.run.smithi123.stdout:34359738478 2024-05-08T06:08:21.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.468+0000 7f2e2ca00700 1 -- 172.21.15.123:0/1071356917 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f2e0c0b2900 msgr2=0x7f2e0c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:21.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.468+0000 7f2e2ca00700 1 --2- 172.21.15.123:0/1071356917 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f2e0c0b2900 0x7f2e0c0b4dc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2e28110300 tx=0x7f2e10007320 comp rx=0 tx=0).stop 2024-05-08T06:08:21.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.468+0000 7f2e2ca00700 1 -- 172.21.15.123:0/1071356917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e281045e0 msgr2=0x7f2e2819bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:21.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.468+0000 7f2e2ca00700 1 --2- 172.21.15.123:0/1071356917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e281045e0 0x7f2e2819bd80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2e18000c00 tx=0x7f2e180057c0 comp rx=0 tx=0).stop 2024-05-08T06:08:21.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.468+0000 7f2e2ca00700 1 -- 172.21.15.123:0/1071356917 shutdown_connections 2024-05-08T06:08:21.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.468+0000 7f2e2ca00700 1 --2- 172.21.15.123:0/1071356917 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f2e0c0b2900 0x7f2e0c0b4dc0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:21.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.468+0000 7f2e2ca00700 1 --2- 172.21.15.123:0/1071356917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e281045e0 0x7f2e2819bd80 secure :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2e18000c00 tx=0x7f2e180057c0 comp rx=0 tx=0).stop 2024-05-08T06:08:21.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.468+0000 7f2e2ca00700 1 --2- 172.21.15.123:0/1071356917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e2810da10 0x7f2e281a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:21.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.468+0000 7f2e2ca00700 1 --2- 172.21.15.123:0/1071356917 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e28104f00 0x7f2e2819c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:21.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.468+0000 7f2e2ca00700 1 -- 172.21.15.123:0/1071356917 >> 172.21.15.123:0/1071356917 conn(0x7f2e280fde30 msgr2=0x7f2e28100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:08:21.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.469+0000 7f2e2ca00700 1 -- 172.21.15.123:0/1071356917 shutdown_connections 2024-05-08T06:08:21.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:21.469+0000 7f2e2ca00700 1 -- 172.21.15.123:0/1071356917 wait complete. 2024-05-08T06:08:22.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.312+0000 7fd9a1e00700 1 -- 172.21.15.123:0/3047096657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd99c06b6f0 msgr2=0x7fd99c06bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:22.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.312+0000 7fd9a1e00700 1 --2- 172.21.15.123:0/3047096657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd99c06b6f0 0x7fd99c06bb50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd988009990 tx=0x7fd988009ca0 comp rx=0 tx=0).stop 2024-05-08T06:08:22.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.313+0000 7fd9a1e00700 1 -- 172.21.15.123:0/3047096657 shutdown_connections 2024-05-08T06:08:22.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.313+0000 7fd9a1e00700 1 --2- 172.21.15.123:0/3047096657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd99c105ee0 0x7fd99c106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:22.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.313+0000 7fd9a1e00700 1 --2- 172.21.15.123:0/3047096657 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd99c1083d0 0x7fd99c06b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:22.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.313+0000 7fd9a1e00700 1 --2- 172.21.15.123:0/3047096657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd99c06b6f0 0x7fd99c06bb50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:22.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.313+0000 7fd9a1e00700 1 -- 172.21.15.123:0/3047096657 >> 172.21.15.123:0/3047096657 conn(0x7fd99c079350 msgr2=0x7fd99c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:08:22.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.313+0000 7fd9a1e00700 1 -- 172.21.15.123:0/3047096657 shutdown_connections 2024-05-08T06:08:22.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.313+0000 7fd9a1e00700 1 -- 172.21.15.123:0/3047096657 wait complete. 2024-05-08T06:08:22.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.314+0000 7fd9a1e00700 1 Processor -- start 2024-05-08T06:08:22.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.314+0000 7fd9a1e00700 1 -- start start 2024-05-08T06:08:22.314 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.315+0000 7fd9a1e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd99c06b6f0 0x7fd99c1a0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:22.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.315+0000 7fd9a1e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd99c105ee0 0x7fd99c1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:22.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.315+0000 7fd9a1e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd99c1083d0 0x7fd99c1a49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:22.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.315+0000 7fd9a1e00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c119c30 con 0x7fd99c06b6f0 2024-05-08T06:08:22.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.315+0000 7fd9a1e00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c119ab0 con 0x7fd99c105ee0 2024-05-08T06:08:22.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.315+0000 7fd9a1e00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c119db0 con 0x7fd99c1083d0 2024-05-08T06:08:22.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.315+0000 7fd99aa00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd99c06b6f0 0x7fd99c1a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:22.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.315+0000 7fd99b400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd99c1083d0 0x7fd99c1a49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:22.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.315+0000 7fd99aa00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd99c06b6f0 0x7fd99c1a0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41634/0 (socket says 172.21.15.123:41634) 2024-05-08T06:08:22.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.315+0000 7fd99aa00700 1 -- 172.21.15.123:0/4039723272 learned_addr learned my addr 172.21.15.123:0/4039723272 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:08:22.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.315+0000 7fd99a000700 1 --2- 172.21.15.123:0/4039723272 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd99c105ee0 0x7fd99c1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:22.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.316+0000 7fd99b400700 1 -- 172.21.15.123:0/4039723272 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd99c105ee0 msgr2=0x7fd99c1a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:22.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.316+0000 7fd99b400700 1 --2- 172.21.15.123:0/4039723272 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd99c105ee0 0x7fd99c1a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:22.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.316+0000 7fd99b400700 1 -- 172.21.15.123:0/4039723272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd99c06b6f0 msgr2=0x7fd99c1a0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:22.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.316+0000 7fd99b400700 1 --2- 172.21.15.123:0/4039723272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd99c06b6f0 0x7fd99c1a0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:22.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.316+0000 7fd99b400700 1 -- 172.21.15.123:0/4039723272 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd99c1a5110 con 0x7fd99c1083d0 2024-05-08T06:08:22.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.316+0000 7fd99b400700 1 --2- 172.21.15.123:0/4039723272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd99c1083d0 0x7fd99c1a49d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd99400efc0 tx=0x7fd99400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:08:22.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.316+0000 7fd9a0a00700 1 -- 172.21.15.123:0/4039723272 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9940092e0 con 0x7fd99c1083d0 2024-05-08T06:08:22.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.316+0000 7fd9a1e00700 1 -- 172.21.15.123:0/4039723272 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd99c1a53a0 con 0x7fd99c1083d0 2024-05-08T06:08:22.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.317+0000 7fd9a0a00700 1 -- 172.21.15.123:0/4039723272 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd994004bb0 con 0x7fd99c1083d0 2024-05-08T06:08:22.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.317+0000 7fd9a0a00700 1 -- 172.21.15.123:0/4039723272 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd994004020 con 0x7fd99c1083d0 2024-05-08T06:08:22.317 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.317+0000 7fd9a1e00700 1 -- 172.21.15.123:0/4039723272 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd99c1acc80 con 0x7fd99c1083d0 2024-05-08T06:08:22.318 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.318+0000 7fd9a1e00700 1 -- 172.21.15.123:0/4039723272 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd99c06a190 con 0x7fd99c1083d0 2024-05-08T06:08:22.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.320+0000 7fd9a0a00700 1 -- 172.21.15.123:0/4039723272 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fd99401b030 con 0x7fd99c1083d0 2024-05-08T06:08:22.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.322+0000 7fd9a0a00700 1 --2- 172.21.15.123:0/4039723272 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd9800b2880 0x7fd9800b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:22.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.322+0000 7fd9a0a00700 1 -- 172.21.15.123:0/4039723272 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fd9940e1d10 con 0x7fd99c1083d0 2024-05-08T06:08:22.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.322+0000 7fd99aa00700 1 --2- 172.21.15.123:0/4039723272 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd9800b2880 0x7fd9800b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:22.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.322+0000 7fd99aa00700 1 --2- 172.21.15.123:0/4039723272 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd9800b2880 0x7fd9800b4d40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd98c005950 tx=0x7fd98c00b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:08:22.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.326+0000 7fd9a0a00700 1 -- 172.21.15.123:0/4039723272 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd9940e7020 con 0x7fd99c1083d0 2024-05-08T06:08:22.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.561+0000 7fd9a1e00700 1 -- 172.21.15.123:0/4039723272 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fd99c1acf10 con 0x7fd99c1083d0 2024-05-08T06:08:22.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.562+0000 7fd9a0a00700 1 -- 172.21.15.123:0/4039723272 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd994022020 con 0x7fd99c1083d0 2024-05-08T06:08:22.562 INFO:teuthology.orchestra.run.smithi123.stdout:55834574952 2024-05-08T06:08:22.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.565+0000 7fd9a1e00700 1 -- 172.21.15.123:0/4039723272 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd9800b2880 msgr2=0x7fd9800b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:22.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.565+0000 7fd9a1e00700 1 --2- 172.21.15.123:0/4039723272 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd9800b2880 0x7fd9800b4d40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd98c005950 tx=0x7fd98c00b500 comp rx=0 tx=0).stop 2024-05-08T06:08:22.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.565+0000 7fd9a1e00700 1 -- 172.21.15.123:0/4039723272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd99c1083d0 msgr2=0x7fd99c1a49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:22.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.565+0000 7fd9a1e00700 1 --2- 172.21.15.123:0/4039723272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd99c1083d0 0x7fd99c1a49d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd99400efc0 tx=0x7fd99400c3a0 comp rx=0 tx=0).stop 2024-05-08T06:08:22.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.565+0000 7fd9a1e00700 1 -- 172.21.15.123:0/4039723272 shutdown_connections 2024-05-08T06:08:22.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.565+0000 7fd9a1e00700 1 --2- 172.21.15.123:0/4039723272 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd9800b2880 0x7fd9800b4d40 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:22.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.565+0000 7fd9a1e00700 1 --2- 172.21.15.123:0/4039723272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd99c06b6f0 0x7fd99c1a0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:22.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.565+0000 7fd9a1e00700 1 --2- 172.21.15.123:0/4039723272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd99c1083d0 0x7fd99c1a49d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:22.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.565+0000 7fd9a1e00700 1 --2- 172.21.15.123:0/4039723272 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd99c105ee0 0x7fd99c1a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:22.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.565+0000 7fd9a1e00700 1 -- 172.21.15.123:0/4039723272 >> 172.21.15.123:0/4039723272 conn(0x7fd99c079350 msgr2=0x7fd99c104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:08:22.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.565+0000 7fd9a1e00700 1 -- 172.21.15.123:0/4039723272 shutdown_connections 2024-05-08T06:08:22.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:22.565+0000 7fd9a1e00700 1 -- 172.21.15.123:0/4039723272 wait complete. 2024-05-08T06:08:22.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:22 smithi163 ceph-mon[34520]: pgmap v397: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:08:22.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:22 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/1071356917' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-08T06:08:22.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:22 smithi192 ceph-mon[34664]: pgmap v397: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:08:22.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:22 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/1071356917' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-08T06:08:22.818 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:22 smithi123 ceph-mon[29787]: pgmap v397: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:08:22.818 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:22 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/1071356917' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-08T06:08:23.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.521+0000 7fb67e000700 1 -- 172.21.15.123:0/2610132403 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb678109600 msgr2=0x7fb6781099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:23.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.521+0000 7fb67e000700 1 --2- 172.21.15.123:0/2610132403 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb678109600 0x7fb6781099e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb668009990 tx=0x7fb668009ca0 comp rx=0 tx=0).stop 2024-05-08T06:08:23.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.521+0000 7fb67e000700 1 -- 172.21.15.123:0/2610132403 shutdown_connections 2024-05-08T06:08:23.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.521+0000 7fb67e000700 1 --2- 172.21.15.123:0/2610132403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb678103eb0 0x7fb678110390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:23.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.521+0000 7fb67e000700 1 --2- 172.21.15.123:0/2610132403 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb678109600 0x7fb6781099e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:23.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.521+0000 7fb67e000700 1 --2- 172.21.15.123:0/2610132403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6781034f0 0x7fb678103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:23.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.521+0000 7fb67e000700 1 -- 172.21.15.123:0/2610132403 >> 172.21.15.123:0/2610132403 conn(0x7fb67807a650 msgr2=0x7fb67807aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:08:23.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.522+0000 7fb67e000700 1 -- 172.21.15.123:0/2610132403 shutdown_connections 2024-05-08T06:08:23.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.522+0000 7fb67e000700 1 -- 172.21.15.123:0/2610132403 wait complete. 2024-05-08T06:08:23.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.522+0000 7fb67e000700 1 Processor -- start 2024-05-08T06:08:23.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.523+0000 7fb67e000700 1 -- start start 2024-05-08T06:08:23.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.523+0000 7fb67e000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6781034f0 0x7fb67819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:23.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.523+0000 7fb67e000700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb678103eb0 0x7fb67819fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:23.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.523+0000 7fb67e000700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb678109600 0x7fb6781998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:23.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.523+0000 7fb67e000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb678113650 con 0x7fb6781034f0 2024-05-08T06:08:23.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.523+0000 7fb67e000700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb6781134d0 con 0x7fb678109600 2024-05-08T06:08:23.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.523+0000 7fb67e000700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb6781137d0 con 0x7fb678103eb0 2024-05-08T06:08:23.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.523+0000 7fb677400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6781034f0 0x7fb67819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:23.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.523+0000 7fb677400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6781034f0 0x7fb67819f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41644/0 (socket says 172.21.15.123:41644) 2024-05-08T06:08:23.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.523+0000 7fb677400700 1 -- 172.21.15.123:0/1636461584 learned_addr learned my addr 172.21.15.123:0/1636461584 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:08:23.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.524+0000 7fb677e00700 1 --2- 172.21.15.123:0/1636461584 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb678109600 0x7fb6781998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:23.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.524+0000 7fb676a00700 1 --2- 172.21.15.123:0/1636461584 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb678103eb0 0x7fb67819fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:23.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.524+0000 7fb677400700 1 -- 172.21.15.123:0/1636461584 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb678103eb0 msgr2=0x7fb67819fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:23.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.524+0000 7fb677400700 1 --2- 172.21.15.123:0/1636461584 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb678103eb0 0x7fb67819fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:23.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.524+0000 7fb677400700 1 -- 172.21.15.123:0/1636461584 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb678109600 msgr2=0x7fb6781998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:23.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.524+0000 7fb677400700 1 --2- 172.21.15.123:0/1636461584 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb678109600 0x7fb6781998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:23.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.524+0000 7fb677400700 1 -- 172.21.15.123:0/1636461584 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb67819a030 con 0x7fb6781034f0 2024-05-08T06:08:23.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.524+0000 7fb677400700 1 --2- 172.21.15.123:0/1636461584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6781034f0 0x7fb67819f710 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb6680057f0 tx=0x7fb66800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:08:23.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.525+0000 7fb66fe00700 1 -- 172.21.15.123:0/1636461584 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb668026070 con 0x7fb6781034f0 2024-05-08T06:08:23.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.525+0000 7fb66fe00700 1 -- 172.21.15.123:0/1636461584 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb668021410 con 0x7fb6781034f0 2024-05-08T06:08:23.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.525+0000 7fb67e000700 1 -- 172.21.15.123:0/1636461584 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb67819a2c0 con 0x7fb6781034f0 2024-05-08T06:08:23.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.525+0000 7fb66fe00700 1 -- 172.21.15.123:0/1636461584 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb668017650 con 0x7fb6781034f0 2024-05-08T06:08:23.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.525+0000 7fb67e000700 1 -- 172.21.15.123:0/1636461584 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6780756d0 con 0x7fb6781034f0 2024-05-08T06:08:23.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.526+0000 7fb67e000700 1 -- 172.21.15.123:0/1636461584 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb658005330 con 0x7fb6781034f0 2024-05-08T06:08:23.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.528+0000 7fb66fe00700 1 -- 172.21.15.123:0/1636461584 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fb668029070 con 0x7fb6781034f0 2024-05-08T06:08:23.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.530+0000 7fb66fe00700 1 --2- 172.21.15.123:0/1636461584 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fb65c0b2880 0x7fb65c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:23.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.530+0000 7fb66fe00700 1 -- 172.21.15.123:0/1636461584 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fb6680e2fe0 con 0x7fb6781034f0 2024-05-08T06:08:23.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.530+0000 7fb676a00700 1 --2- 172.21.15.123:0/1636461584 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fb65c0b2880 0x7fb65c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:23.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.531+0000 7fb676a00700 1 --2- 172.21.15.123:0/1636461584 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fb65c0b2880 0x7fb65c0b4d40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb67807abe0 tx=0x7fb664012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:08:23.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.533+0000 7fb66fe00700 1 -- 172.21.15.123:0/1636461584 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb668030080 con 0x7fb6781034f0 2024-05-08T06:08:23.739 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:23 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/4039723272' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-08T06:08:23.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.784+0000 7fb67e000700 1 -- 172.21.15.123:0/1636461584 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fb6580058f0 con 0x7fb6781034f0 2024-05-08T06:08:23.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.785+0000 7fb66fe00700 1 -- 172.21.15.123:0/1636461584 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb668013070 con 0x7fb6781034f0 2024-05-08T06:08:23.785 INFO:teuthology.orchestra.run.smithi123.stdout:210453397575 2024-05-08T06:08:23.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:23 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/4039723272' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-08T06:08:23.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.788+0000 7fb67e000700 1 -- 172.21.15.123:0/1636461584 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fb65c0b2880 msgr2=0x7fb65c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:23.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.788+0000 7fb67e000700 1 --2- 172.21.15.123:0/1636461584 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fb65c0b2880 0x7fb65c0b4d40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb67807abe0 tx=0x7fb664012040 comp rx=0 tx=0).stop 2024-05-08T06:08:23.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.789+0000 7fb67e000700 1 -- 172.21.15.123:0/1636461584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6781034f0 msgr2=0x7fb67819f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:23.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.789+0000 7fb67e000700 1 --2- 172.21.15.123:0/1636461584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6781034f0 0x7fb67819f710 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb6680057f0 tx=0x7fb66800bfd0 comp rx=0 tx=0).stop 2024-05-08T06:08:23.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.789+0000 7fb67e000700 1 -- 172.21.15.123:0/1636461584 shutdown_connections 2024-05-08T06:08:23.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.789+0000 7fb67e000700 1 --2- 172.21.15.123:0/1636461584 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fb65c0b2880 0x7fb65c0b4d40 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:23.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.789+0000 7fb67e000700 1 --2- 172.21.15.123:0/1636461584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6781034f0 0x7fb67819f710 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:23.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.789+0000 7fb67e000700 1 --2- 172.21.15.123:0/1636461584 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb678109600 0x7fb6781998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:23.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.789+0000 7fb67e000700 1 --2- 172.21.15.123:0/1636461584 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb678103eb0 0x7fb67819fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:23.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.789+0000 7fb67e000700 1 -- 172.21.15.123:0/1636461584 >> 172.21.15.123:0/1636461584 conn(0x7fb67807a650 msgr2=0x7fb678102070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:08:23.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.789+0000 7fb67e000700 1 -- 172.21.15.123:0/1636461584 shutdown_connections 2024-05-08T06:08:23.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:23.789+0000 7fb67e000700 1 -- 172.21.15.123:0/1636461584 wait complete. 2024-05-08T06:08:23.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:23 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/4039723272' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-08T06:08:24.327 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738471 got 34359738478 for osd.0 2024-05-08T06:08:24.327 DEBUG:teuthology.parallel:result is None 2024-05-08T06:08:24.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:24 smithi123 ceph-mon[29787]: pgmap v398: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:08:24.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:24 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:08:24.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:24 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:08:24.579 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:24 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/1636461584' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-08T06:08:24.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:24 smithi163 ceph-mon[34520]: pgmap v398: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:08:24.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:24 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:08:24.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:24 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:08:24.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:24 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/1636461584' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-08T06:08:24.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:24 smithi192 ceph-mon[34664]: pgmap v398: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:08:24.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:24 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:08:24.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:24 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:08:24.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:24 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/1636461584' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-08T06:08:25.156 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574945 got 55834574952 for osd.1 2024-05-08T06:08:25.156 DEBUG:teuthology.parallel:result is None 2024-05-08T06:08:25.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:25 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:08:25.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:25 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:08:25.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:25 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:08:25.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:25 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:08:25.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:25 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:08:25.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:25 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:08:25.820 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:25 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:08:25.820 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:25 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:08:25.820 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:25 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:08:26.787 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397568 got 210453397575 for osd.7 2024-05-08T06:08:26.788 DEBUG:teuthology.parallel:result is None 2024-05-08T06:08:26.917 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:26 smithi123 ceph-mon[29787]: pgmap v399: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:08:27.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:26 smithi163 ceph-mon[34520]: pgmap v399: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:08:27.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:26 smithi192 ceph-mon[34664]: pgmap v399: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:08:27.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.473+0000 7f2070c00700 1 -- 172.21.15.123:0/2450772069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f206c06b180 msgr2=0x7f206c06b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:27.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.473+0000 7f2070c00700 1 --2- 172.21.15.123:0/2450772069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f206c06b180 0x7f206c06b600 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2058009990 tx=0x7f2058009ca0 comp rx=0 tx=0).stop 2024-05-08T06:08:27.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.473+0000 7f2070c00700 1 -- 172.21.15.123:0/2450772069 shutdown_connections 2024-05-08T06:08:27.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.473+0000 7f2070c00700 1 --2- 172.21.15.123:0/2450772069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f206c06bb40 0x7f206c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:27.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.473+0000 7f2070c00700 1 --2- 172.21.15.123:0/2450772069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f206c109510 0x7f206c1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:27.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.473+0000 7f2070c00700 1 --2- 172.21.15.123:0/2450772069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f206c06b180 0x7f206c06b600 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:27.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.473+0000 7f2070c00700 1 -- 172.21.15.123:0/2450772069 >> 172.21.15.123:0/2450772069 conn(0x7f206c079350 msgr2=0x7f206c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:08:27.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.473+0000 7f2070c00700 1 -- 172.21.15.123:0/2450772069 shutdown_connections 2024-05-08T06:08:27.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.473+0000 7f2070c00700 1 -- 172.21.15.123:0/2450772069 wait complete. 2024-05-08T06:08:27.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.474+0000 7f2070c00700 1 Processor -- start 2024-05-08T06:08:27.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.475+0000 7f2070c00700 1 -- start start 2024-05-08T06:08:27.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.475+0000 7f2070c00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f206c06b180 0x7f206c102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:27.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.475+0000 7f2070c00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f206c06bb40 0x7f206c1029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:27.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.475+0000 7f2070c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f206c109510 0x7f206c102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:27.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.475+0000 7f2070c00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f206c113670 con 0x7f206c109510 2024-05-08T06:08:27.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.475+0000 7f2070c00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f206c1134f0 con 0x7f206c06b180 2024-05-08T06:08:27.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.475+0000 7f2070c00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f206c1137f0 con 0x7f206c06bb40 2024-05-08T06:08:27.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.475+0000 7f206a800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f206c109510 0x7f206c102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:27.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.475+0000 7f206a800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f206c109510 0x7f206c102ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50406/0 (socket says 172.21.15.123:50406) 2024-05-08T06:08:27.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.475+0000 7f206a800700 1 -- 172.21.15.123:0/3198147267 learned_addr learned my addr 172.21.15.123:0/3198147267 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:08:27.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.475+0000 7f2069e00700 1 --2- 172.21.15.123:0/3198147267 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f206c06b180 0x7f206c102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:27.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.476+0000 7f2069400700 1 --2- 172.21.15.123:0/3198147267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f206c06bb40 0x7f206c1029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:27.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.476+0000 7f2069e00700 1 -- 172.21.15.123:0/3198147267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f206c06bb40 msgr2=0x7f206c1029b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:27.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.476+0000 7f2069e00700 1 --2- 172.21.15.123:0/3198147267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f206c06bb40 0x7f206c1029b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:27.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.476+0000 7f2069e00700 1 -- 172.21.15.123:0/3198147267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f206c109510 msgr2=0x7f206c102ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:27.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.476+0000 7f2069e00700 1 --2- 172.21.15.123:0/3198147267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f206c109510 0x7f206c102ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:27.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.476+0000 7f2069e00700 1 -- 172.21.15.123:0/3198147267 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f206c1a80d0 con 0x7f206c06b180 2024-05-08T06:08:27.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.476+0000 7f2069e00700 1 --2- 172.21.15.123:0/3198147267 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f206c06b180 0x7f206c102470 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f205c00e9f0 tx=0x7f205c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:08:27.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.476+0000 7f2062a00700 1 -- 172.21.15.123:0/3198147267 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205c00c9c0 con 0x7f206c06b180 2024-05-08T06:08:27.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.477+0000 7f2070c00700 1 -- 172.21.15.123:0/3198147267 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f206c1a83c0 con 0x7f206c06b180 2024-05-08T06:08:27.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.477+0000 7f2062a00700 1 -- 172.21.15.123:0/3198147267 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f205c00cb60 con 0x7f206c06b180 2024-05-08T06:08:27.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.477+0000 7f2062a00700 1 -- 172.21.15.123:0/3198147267 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205c018680 con 0x7f206c06b180 2024-05-08T06:08:27.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.477+0000 7f2070c00700 1 -- 172.21.15.123:0/3198147267 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f206c1a8950 con 0x7f206c06b180 2024-05-08T06:08:27.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.478+0000 7f2070c00700 1 -- 172.21.15.123:0/3198147267 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f206c06a190 con 0x7f206c06b180 2024-05-08T06:08:27.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.480+0000 7f2062a00700 1 -- 172.21.15.123:0/3198147267 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f205c04aab0 con 0x7f206c06b180 2024-05-08T06:08:27.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.482+0000 7f2062a00700 1 --2- 172.21.15.123:0/3198147267 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f20500b2900 0x7f20500b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:27.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.482+0000 7f2062a00700 1 -- 172.21.15.123:0/3198147267 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f205c0e2450 con 0x7f206c06b180 2024-05-08T06:08:27.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.482+0000 7f2069400700 1 --2- 172.21.15.123:0/3198147267 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f20500b2900 0x7f20500b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:27.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.483+0000 7f2069400700 1 --2- 172.21.15.123:0/3198147267 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f20500b2900 0x7f20500b4dc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f206c079a30 tx=0x7f205800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:08:27.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.484+0000 7f2062a00700 1 -- 172.21.15.123:0/3198147267 <== mon.2 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f205c0aa120 con 0x7f206c06b180 2024-05-08T06:08:27.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.718+0000 7f2070c00700 1 -- 172.21.15.123:0/3198147267 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f206c02e510 con 0x7f206c06b180 2024-05-08T06:08:27.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.718+0000 7f2062a00700 1 -- 172.21.15.123:0/3198147267 <== mon.2 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f205c0b21a0 con 0x7f206c06b180 2024-05-08T06:08:27.719 INFO:teuthology.orchestra.run.smithi123.stdout:188978561100 2024-05-08T06:08:27.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.722+0000 7f2070c00700 1 -- 172.21.15.123:0/3198147267 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f20500b2900 msgr2=0x7f20500b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:27.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.722+0000 7f2070c00700 1 --2- 172.21.15.123:0/3198147267 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f20500b2900 0x7f20500b4dc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f206c079a30 tx=0x7f205800b260 comp rx=0 tx=0).stop 2024-05-08T06:08:27.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.722+0000 7f2070c00700 1 -- 172.21.15.123:0/3198147267 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f206c06b180 msgr2=0x7f206c102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:27.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.722+0000 7f2070c00700 1 --2- 172.21.15.123:0/3198147267 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f206c06b180 0x7f206c102470 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f205c00e9f0 tx=0x7f205c00ed00 comp rx=0 tx=0).stop 2024-05-08T06:08:27.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.722+0000 7f2070c00700 1 -- 172.21.15.123:0/3198147267 shutdown_connections 2024-05-08T06:08:27.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.722+0000 7f2070c00700 1 --2- 172.21.15.123:0/3198147267 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f20500b2900 0x7f20500b4dc0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:27.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.722+0000 7f2070c00700 1 --2- 172.21.15.123:0/3198147267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f206c109510 0x7f206c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:27.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.722+0000 7f2070c00700 1 --2- 172.21.15.123:0/3198147267 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f206c06b180 0x7f206c102470 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:27.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.722+0000 7f2070c00700 1 --2- 172.21.15.123:0/3198147267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f206c06bb40 0x7f206c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:27.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.722+0000 7f2070c00700 1 -- 172.21.15.123:0/3198147267 >> 172.21.15.123:0/3198147267 conn(0x7f206c079350 msgr2=0x7f206c101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:08:27.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.722+0000 7f2070c00700 1 -- 172.21.15.123:0/3198147267 shutdown_connections 2024-05-08T06:08:27.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:27.722+0000 7f2070c00700 1 -- 172.21.15.123:0/3198147267 wait complete. 2024-05-08T06:08:28.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:27 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/3198147267' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-08T06:08:28.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:27 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/3198147267' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-08T06:08:28.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:27 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/3198147267' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-08T06:08:28.836 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561094 got 188978561100 for osd.6 2024-05-08T06:08:28.836 DEBUG:teuthology.parallel:result is None 2024-05-08T06:08:28.836 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-08T06:08:28.836 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T06:08:29.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:28 smithi192 ceph-mon[34664]: pgmap v400: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:08:29.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:28 smithi123 ceph-mon[29787]: pgmap v400: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:08:29.278 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:08:29.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:28 smithi163 ceph-mon[34520]: pgmap v400: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:08:31.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:30 smithi192 ceph-mon[34664]: pgmap v401: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:08:31.194 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:30 smithi123 ceph-mon[29787]: pgmap v401: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:08:31.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:30 smithi163 ceph-mon[34520]: pgmap v401: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:08:31.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.732+0000 7f00f1000700 1 -- 172.21.15.123:0/1622369766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00ec108850 msgr2=0x7f00ec1103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:31.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.732+0000 7f00f1000700 1 --2- 172.21.15.123:0/1622369766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00ec108850 0x7f00ec1103d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f00e400b0d0 tx=0x7f00e400b3e0 comp rx=0 tx=0).stop 2024-05-08T06:08:31.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.732+0000 7f00f1000700 1 -- 172.21.15.123:0/1622369766 shutdown_connections 2024-05-08T06:08:31.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.732+0000 7f00f1000700 1 --2- 172.21.15.123:0/1622369766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00ec108850 0x7f00ec1103d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:31.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.732+0000 7f00f1000700 1 --2- 172.21.15.123:0/1622369766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00ec103750 0x7f00ec103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:31.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.732+0000 7f00f1000700 1 --2- 172.21.15.123:0/1622369766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00ec104100 0x7f00ec108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:31.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.732+0000 7f00f1000700 1 -- 172.21.15.123:0/1622369766 >> 172.21.15.123:0/1622369766 conn(0x7f00ec07a600 msgr2=0x7f00ec07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:08:31.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.732+0000 7f00f1000700 1 -- 172.21.15.123:0/1622369766 shutdown_connections 2024-05-08T06:08:31.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.732+0000 7f00f1000700 1 -- 172.21.15.123:0/1622369766 wait complete. 2024-05-08T06:08:31.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.734+0000 7f00f1000700 1 Processor -- start 2024-05-08T06:08:31.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.734+0000 7f00f1000700 1 -- start start 2024-05-08T06:08:31.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.734+0000 7f00f1000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00ec103750 0x7f00ec1a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:31.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.735+0000 7f00f1000700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00ec104100 0x7f00ec1a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:31.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.735+0000 7f00f1000700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00ec108850 0x7f00ec19bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:31.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.735+0000 7f00f1000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec113640 con 0x7f00ec103750 2024-05-08T06:08:31.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.735+0000 7f00f1000700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec1134c0 con 0x7f00ec104100 2024-05-08T06:08:31.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.735+0000 7f00f1000700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec1137c0 con 0x7f00ec108850 2024-05-08T06:08:31.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.735+0000 7f00e9e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00ec103750 0x7f00ec1a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:31.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.735+0000 7f00e9400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00ec104100 0x7f00ec1a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:31.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.735+0000 7f00e9e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00ec103750 0x7f00ec1a1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50420/0 (socket says 172.21.15.123:50420) 2024-05-08T06:08:31.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.735+0000 7f00e9e00700 1 -- 172.21.15.123:0/3639009220 learned_addr learned my addr 172.21.15.123:0/3639009220 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:08:31.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.735+0000 7f00ea800700 1 --2- 172.21.15.123:0/3639009220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00ec108850 0x7f00ec19bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:31.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.735+0000 7f00e9e00700 1 -- 172.21.15.123:0/3639009220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00ec108850 msgr2=0x7f00ec19bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:31.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.735+0000 7f00e9e00700 1 --2- 172.21.15.123:0/3639009220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00ec108850 0x7f00ec19bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:31.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.735+0000 7f00e9e00700 1 -- 172.21.15.123:0/3639009220 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00ec104100 msgr2=0x7f00ec1a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:31.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.735+0000 7f00e9e00700 1 --2- 172.21.15.123:0/3639009220 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00ec104100 0x7f00ec1a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:31.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.735+0000 7f00e9e00700 1 -- 172.21.15.123:0/3639009220 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00ec19c260 con 0x7f00ec103750 2024-05-08T06:08:31.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.736+0000 7f00e9e00700 1 --2- 172.21.15.123:0/3639009220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00ec103750 0x7f00ec1a1940 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f00dc00e9c0 tx=0x7f00dc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:08:31.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.736+0000 7f00e2a00700 1 -- 172.21.15.123:0/3639009220 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc00cbd0 con 0x7f00ec103750 2024-05-08T06:08:31.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.736+0000 7f00e2a00700 1 -- 172.21.15.123:0/3639009220 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00dc00ee60 con 0x7f00ec103750 2024-05-08T06:08:31.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.736+0000 7f00f1000700 1 -- 172.21.15.123:0/3639009220 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00ec19c550 con 0x7f00ec103750 2024-05-08T06:08:31.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.736+0000 7f00e2a00700 1 -- 172.21.15.123:0/3639009220 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc0055c0 con 0x7f00ec103750 2024-05-08T06:08:31.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.736+0000 7f00f1000700 1 -- 172.21.15.123:0/3639009220 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00ec1a8810 con 0x7f00ec103750 2024-05-08T06:08:31.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.737+0000 7f00f1000700 1 -- 172.21.15.123:0/3639009220 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00ec06a190 con 0x7f00ec103750 2024-05-08T06:08:31.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.739+0000 7f00e2a00700 1 -- 172.21.15.123:0/3639009220 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f00dc004750 con 0x7f00ec103750 2024-05-08T06:08:31.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.741+0000 7f00e2a00700 1 --2- 172.21.15.123:0/3639009220 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f00d00b2900 0x7f00d00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:31.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.741+0000 7f00e2a00700 1 -- 172.21.15.123:0/3639009220 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f00dc0e2460 con 0x7f00ec103750 2024-05-08T06:08:31.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.741+0000 7f00e9400700 1 --2- 172.21.15.123:0/3639009220 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f00d00b2900 0x7f00d00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:31.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.742+0000 7f00e9400700 1 --2- 172.21.15.123:0/3639009220 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f00d00b2900 0x7f00d00b4dc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f00ec103410 tx=0x7f00d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:08:31.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.746+0000 7f00e2a00700 1 -- 172.21.15.123:0/3639009220 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f00dc0aa130 con 0x7f00ec103750 2024-05-08T06:08:31.973 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.973+0000 7f00f1000700 1 -- 172.21.15.123:0/3639009220 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f00ec10f6c0 con 0x7f00d00b2900 2024-05-08T06:08:31.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:31.996+0000 7f00e2a00700 1 -- 172.21.15.123:0/3639009220 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259047 (secure 0 0 0) 0x7f00ec10f6c0 con 0x7f00d00b2900 2024-05-08T06:08:32.001 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:08:32.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:32.002+0000 7f00f1000700 1 -- 172.21.15.123:0/3639009220 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f00d00b2900 msgr2=0x7f00d00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:32.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:32.002+0000 7f00f1000700 1 --2- 172.21.15.123:0/3639009220 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f00d00b2900 0x7f00d00b4dc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f00ec103410 tx=0x7f00d8009260 comp rx=0 tx=0).stop 2024-05-08T06:08:32.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:32.002+0000 7f00f1000700 1 -- 172.21.15.123:0/3639009220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00ec103750 msgr2=0x7f00ec1a1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:32.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:32.002+0000 7f00f1000700 1 --2- 172.21.15.123:0/3639009220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00ec103750 0x7f00ec1a1940 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f00dc00e9c0 tx=0x7f00dc00ecd0 comp rx=0 tx=0).stop 2024-05-08T06:08:32.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:32.002+0000 7f00f1000700 1 -- 172.21.15.123:0/3639009220 shutdown_connections 2024-05-08T06:08:32.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:32.002+0000 7f00f1000700 1 --2- 172.21.15.123:0/3639009220 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f00d00b2900 0x7f00d00b4dc0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:32.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:32.002+0000 7f00f1000700 1 --2- 172.21.15.123:0/3639009220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00ec103750 0x7f00ec1a1940 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:32.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:32.002+0000 7f00f1000700 1 --2- 172.21.15.123:0/3639009220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00ec108850 0x7f00ec19bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:32.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:32.002+0000 7f00f1000700 1 --2- 172.21.15.123:0/3639009220 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00ec104100 0x7f00ec1a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:32.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:32.002+0000 7f00f1000700 1 -- 172.21.15.123:0/3639009220 >> 172.21.15.123:0/3639009220 conn(0x7f00ec07a600 msgr2=0x7f00ec101880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:08:32.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:32.003+0000 7f00f1000700 1 -- 172.21.15.123:0/3639009220 shutdown_connections 2024-05-08T06:08:32.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:32.003+0000 7f00f1000700 1 -- 172.21.15.123:0/3639009220 wait complete. 2024-05-08T06:08:32.005 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-05-08T06:08:32.975 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":402,"stamp":"2024-05-08T06:08:31.106357+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659203,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":893,"num_read_kb":545,"num_write":300,"num_write_kb":2274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":2951896,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":743,"ondisk_log_size":743,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372480,"kb_used_data":10848,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569344,"statfs":{"total":1151940427776,"available":1151559008256,"internally_reserved":0,"allocated":11108352,"data_stored":7152261,"data_compressed":113232,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004788"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.064044+0000","last_change":"2024-05-08T06:05:51.431744+0000","last_active":"2024-05-08T06:08:01.064044+0000","last_peered":"2024-05-08T06:08:01.064044+0000","last_clean":"2024-05-08T06:08:01.064044+0000","last_became_active":"2024-05-08T06:05:51.431405+0000","last_became_peered":"2024-05-08T06:05:51.431405+0000","last_unstale":"2024-05-08T06:08:01.064044+0000","last_undegraded":"2024-05-08T06:08:01.064044+0000","last_fullsized":"2024-05-08T06:08:01.064044+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:56:40.503607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.056293+0000","last_change":"2024-05-08T06:05:51.426473+0000","last_active":"2024-05-08T06:08:01.056293+0000","last_peered":"2024-05-08T06:08:01.056293+0000","last_clean":"2024-05-08T06:08:01.056293+0000","last_became_active":"2024-05-08T06:05:51.424477+0000","last_became_peered":"2024-05-08T06:05:51.424477+0000","last_unstale":"2024-05-08T06:08:01.056293+0000","last_undegraded":"2024-05-08T06:08:01.056293+0000","last_fullsized":"2024-05-08T06:08:01.056293+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:26:55.747808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.060523+0000","last_change":"2024-05-08T06:05:51.423767+0000","last_active":"2024-05-08T06:08:01.060523+0000","last_peered":"2024-05-08T06:08:01.060523+0000","last_clean":"2024-05-08T06:08:01.060523+0000","last_became_active":"2024-05-08T06:05:51.423273+0000","last_became_peered":"2024-05-08T06:05:51.423273+0000","last_unstale":"2024-05-08T06:08:01.060523+0000","last_undegraded":"2024-05-08T06:08:01.060523+0000","last_fullsized":"2024-05-08T06:08:01.060523+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:24:44.099743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.047123+0000","last_change":"2024-05-08T06:05:51.416604+0000","last_active":"2024-05-08T06:08:01.047123+0000","last_peered":"2024-05-08T06:08:01.047123+0000","last_clean":"2024-05-08T06:08:01.047123+0000","last_became_active":"2024-05-08T06:05:51.416284+0000","last_became_peered":"2024-05-08T06:05:51.416284+0000","last_unstale":"2024-05-08T06:08:01.047123+0000","last_undegraded":"2024-05-08T06:08:01.047123+0000","last_fullsized":"2024-05-08T06:08:01.047123+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:05:26.014323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.059697+0000","last_change":"2024-05-08T06:05:51.423915+0000","last_active":"2024-05-08T06:08:01.059697+0000","last_peered":"2024-05-08T06:08:01.059697+0000","last_clean":"2024-05-08T06:08:01.059697+0000","last_became_active":"2024-05-08T06:05:51.423534+0000","last_became_peered":"2024-05-08T06:05:51.423534+0000","last_unstale":"2024-05-08T06:08:01.059697+0000","last_undegraded":"2024-05-08T06:08:01.059697+0000","last_fullsized":"2024-05-08T06:08:01.059697+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:06:46.197071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.041181+0000","last_change":"2024-05-08T06:05:51.411102+0000","last_active":"2024-05-08T06:08:01.041181+0000","last_peered":"2024-05-08T06:08:01.041181+0000","last_clean":"2024-05-08T06:08:01.041181+0000","last_became_active":"2024-05-08T06:05:51.410791+0000","last_became_peered":"2024-05-08T06:05:51.410791+0000","last_unstale":"2024-05-08T06:08:01.041181+0000","last_undegraded":"2024-05-08T06:08:01.041181+0000","last_fullsized":"2024-05-08T06:08:01.041181+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:31:47.861663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.041524+0000","last_change":"2024-05-08T06:05:51.417756+0000","last_active":"2024-05-08T06:08:01.041524+0000","last_peered":"2024-05-08T06:08:01.041524+0000","last_clean":"2024-05-08T06:08:01.041524+0000","last_became_active":"2024-05-08T06:05:51.417373+0000","last_became_peered":"2024-05-08T06:05:51.417373+0000","last_unstale":"2024-05-08T06:08:01.041524+0000","last_undegraded":"2024-05-08T06:08:01.041524+0000","last_fullsized":"2024-05-08T06:08:01.041524+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:07:48.330401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.063489+0000","last_change":"2024-05-08T06:05:51.413883+0000","last_active":"2024-05-08T06:08:01.063489+0000","last_peered":"2024-05-08T06:08:01.063489+0000","last_clean":"2024-05-08T06:08:01.063489+0000","last_became_active":"2024-05-08T06:05:51.413565+0000","last_became_peered":"2024-05-08T06:05:51.413565+0000","last_unstale":"2024-05-08T06:08:01.063489+0000","last_undegraded":"2024-05-08T06:08:01.063489+0000","last_fullsized":"2024-05-08T06:08:01.063489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:06:38.186225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.053054+0000","last_change":"2024-05-08T06:05:51.416300+0000","last_active":"2024-05-08T06:08:01.053054+0000","last_peered":"2024-05-08T06:08:01.053054+0000","last_clean":"2024-05-08T06:08:01.053054+0000","last_became_active":"2024-05-08T06:05:51.415970+0000","last_became_peered":"2024-05-08T06:05:51.415970+0000","last_unstale":"2024-05-08T06:08:01.053054+0000","last_undegraded":"2024-05-08T06:08:01.053054+0000","last_fullsized":"2024-05-08T06:08:01.053054+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:44:51.853339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.039789+0000","last_change":"2024-05-08T06:05:51.410396+0000","last_active":"2024-05-08T06:08:01.039789+0000","last_peered":"2024-05-08T06:08:01.039789+0000","last_clean":"2024-05-08T06:08:01.039789+0000","last_became_active":"2024-05-08T06:05:51.410067+0000","last_became_peered":"2024-05-08T06:05:51.410067+0000","last_unstale":"2024-05-08T06:08:01.039789+0000","last_undegraded":"2024-05-08T06:08:01.039789+0000","last_fullsized":"2024-05-08T06:08:01.039789+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:32:00.217217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.052539+0000","last_change":"2024-05-08T06:05:51.409276+0000","last_active":"2024-05-08T06:08:01.052539+0000","last_peered":"2024-05-08T06:08:01.052539+0000","last_clean":"2024-05-08T06:08:01.052539+0000","last_became_active":"2024-05-08T06:05:51.408768+0000","last_became_peered":"2024-05-08T06:05:51.408768+0000","last_unstale":"2024-05-08T06:08:01.052539+0000","last_undegraded":"2024-05-08T06:08:01.052539+0000","last_fullsized":"2024-05-08T06:08:01.052539+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:22:57.483944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.048049+0000","last_change":"2024-05-08T06:05:51.419083+0000","last_active":"2024-05-08T06:08:01.048049+0000","last_peered":"2024-05-08T06:08:01.048049+0000","last_clean":"2024-05-08T06:08:01.048049+0000","last_became_active":"2024-05-08T06:05:51.418685+0000","last_became_peered":"2024-05-08T06:05:51.418685+0000","last_unstale":"2024-05-08T06:08:01.048049+0000","last_undegraded":"2024-05-08T06:08:01.048049+0000","last_fullsized":"2024-05-08T06:08:01.048049+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:25:49.256242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.057514+0000","last_change":"2024-05-08T06:05:51.420399+0000","last_active":"2024-05-08T06:08:01.057514+0000","last_peered":"2024-05-08T06:08:01.057514+0000","last_clean":"2024-05-08T06:08:01.057514+0000","last_became_active":"2024-05-08T06:05:51.420076+0000","last_became_peered":"2024-05-08T06:05:51.420076+0000","last_unstale":"2024-05-08T06:08:01.057514+0000","last_undegraded":"2024-05-08T06:08:01.057514+0000","last_fullsized":"2024-05-08T06:08:01.057514+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:51:38.579280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.059465+0000","last_change":"2024-05-08T06:05:51.431518+0000","last_active":"2024-05-08T06:08:01.059465+0000","last_peered":"2024-05-08T06:08:01.059465+0000","last_clean":"2024-05-08T06:08:01.059465+0000","last_became_active":"2024-05-08T06:05:51.431029+0000","last_became_peered":"2024-05-08T06:05:51.431029+0000","last_unstale":"2024-05-08T06:08:01.059465+0000","last_undegraded":"2024-05-08T06:08:01.059465+0000","last_fullsized":"2024-05-08T06:08:01.059465+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:11:09.596221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.061333+0000","last_change":"2024-05-08T06:06:32.498002+0000","last_active":"2024-05-08T06:08:01.061333+0000","last_peered":"2024-05-08T06:08:01.061333+0000","last_clean":"2024-05-08T06:08:01.061333+0000","last_became_active":"2024-05-08T06:06:32.497367+0000","last_became_peered":"2024-05-08T06:06:32.497367+0000","last_unstale":"2024-05-08T06:08:01.061333+0000","last_undegraded":"2024-05-08T06:08:01.061333+0000","last_fullsized":"2024-05-08T06:08:01.061333+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:32:36.461777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.064064+0000","last_change":"2024-05-08T06:05:51.423857+0000","last_active":"2024-05-08T06:08:01.064064+0000","last_peered":"2024-05-08T06:08:01.064064+0000","last_clean":"2024-05-08T06:08:01.064064+0000","last_became_active":"2024-05-08T06:05:51.423487+0000","last_became_peered":"2024-05-08T06:05:51.423487+0000","last_unstale":"2024-05-08T06:08:01.064064+0000","last_undegraded":"2024-05-08T06:08:01.064064+0000","last_fullsized":"2024-05-08T06:08:01.064064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:17:33.269354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.064245+0000","last_change":"2024-05-08T06:05:51.431795+0000","last_active":"2024-05-08T06:08:01.064245+0000","last_peered":"2024-05-08T06:08:01.064245+0000","last_clean":"2024-05-08T06:08:01.064245+0000","last_became_active":"2024-05-08T06:05:51.431432+0000","last_became_peered":"2024-05-08T06:05:51.431432+0000","last_unstale":"2024-05-08T06:08:01.064245+0000","last_undegraded":"2024-05-08T06:08:01.064245+0000","last_fullsized":"2024-05-08T06:08:01.064245+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:11:16.561478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.048666+0000","last_change":"2024-05-08T06:05:51.414805+0000","last_active":"2024-05-08T06:08:01.048666+0000","last_peered":"2024-05-08T06:08:01.048666+0000","last_clean":"2024-05-08T06:08:01.048666+0000","last_became_active":"2024-05-08T06:05:51.414482+0000","last_became_peered":"2024-05-08T06:05:51.414482+0000","last_unstale":"2024-05-08T06:08:01.048666+0000","last_undegraded":"2024-05-08T06:08:01.048666+0000","last_fullsized":"2024-05-08T06:08:01.048666+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:08:13.444591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.064433+0000","last_change":"2024-05-08T06:05:51.418516+0000","last_active":"2024-05-08T06:08:01.064433+0000","last_peered":"2024-05-08T06:08:01.064433+0000","last_clean":"2024-05-08T06:08:01.064433+0000","last_became_active":"2024-05-08T06:05:51.418182+0000","last_became_peered":"2024-05-08T06:05:51.418182+0000","last_unstale":"2024-05-08T06:08:01.064433+0000","last_undegraded":"2024-05-08T06:08:01.064433+0000","last_fullsized":"2024-05-08T06:08:01.064433+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:32:33.338311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482461+0000","last_change":"2024-05-08T06:05:46.350509+0000","last_active":"2024-05-08T06:06:32.482461+0000","last_peered":"2024-05-08T06:06:32.482461+0000","last_clean":"2024-05-08T06:06:32.482461+0000","last_became_active":"2024-05-08T06:05:46.349986+0000","last_became_peered":"2024-05-08T06:05:46.349986+0000","last_unstale":"2024-05-08T06:06:32.482461+0000","last_undegraded":"2024-05-08T06:06:32.482461+0000","last_fullsized":"2024-05-08T06:06:32.482461+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:53:00.928971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.061004+0000","last_change":"2024-05-08T06:05:51.413621+0000","last_active":"2024-05-08T06:08:01.061004+0000","last_peered":"2024-05-08T06:08:01.061004+0000","last_clean":"2024-05-08T06:08:01.061004+0000","last_became_active":"2024-05-08T06:05:51.413188+0000","last_became_peered":"2024-05-08T06:05:51.413188+0000","last_unstale":"2024-05-08T06:08:01.061004+0000","last_undegraded":"2024-05-08T06:08:01.061004+0000","last_fullsized":"2024-05-08T06:08:01.061004+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:13:21.413696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.492763+0000","last_change":"2024-05-08T06:05:46.552444+0000","last_active":"2024-05-08T06:06:32.492763+0000","last_peered":"2024-05-08T06:06:32.492763+0000","last_clean":"2024-05-08T06:06:32.492763+0000","last_became_active":"2024-05-08T06:05:46.551310+0000","last_became_peered":"2024-05-08T06:05:46.551310+0000","last_unstale":"2024-05-08T06:06:32.492763+0000","last_undegraded":"2024-05-08T06:06:32.492763+0000","last_fullsized":"2024-05-08T06:06:32.492763+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:58:17.632832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.062343+0000","last_change":"2024-05-08T06:05:51.425649+0000","last_active":"2024-05-08T06:08:01.062343+0000","last_peered":"2024-05-08T06:08:01.062343+0000","last_clean":"2024-05-08T06:08:01.062343+0000","last_became_active":"2024-05-08T06:05:51.425288+0000","last_became_peered":"2024-05-08T06:05:51.425288+0000","last_unstale":"2024-05-08T06:08:01.062343+0000","last_undegraded":"2024-05-08T06:08:01.062343+0000","last_fullsized":"2024-05-08T06:08:01.062343+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:42:40.580641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480655+0000","last_change":"2024-05-08T06:05:46.350300+0000","last_active":"2024-05-08T06:06:32.480655+0000","last_peered":"2024-05-08T06:06:32.480655+0000","last_clean":"2024-05-08T06:06:32.480655+0000","last_became_active":"2024-05-08T06:05:46.349918+0000","last_became_peered":"2024-05-08T06:05:46.349918+0000","last_unstale":"2024-05-08T06:06:32.480655+0000","last_undegraded":"2024-05-08T06:06:32.480655+0000","last_fullsized":"2024-05-08T06:06:32.480655+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:32:41.952772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.057546+0000","last_change":"2024-05-08T06:05:51.430130+0000","last_active":"2024-05-08T06:08:01.057546+0000","last_peered":"2024-05-08T06:08:01.057546+0000","last_clean":"2024-05-08T06:08:01.057546+0000","last_became_active":"2024-05-08T06:05:51.429794+0000","last_became_peered":"2024-05-08T06:05:51.429794+0000","last_unstale":"2024-05-08T06:08:01.057546+0000","last_undegraded":"2024-05-08T06:08:01.057546+0000","last_fullsized":"2024-05-08T06:08:01.057546+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:21:55.773309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480539+0000","last_change":"2024-05-08T06:05:46.351664+0000","last_active":"2024-05-08T06:06:32.480539+0000","last_peered":"2024-05-08T06:06:32.480539+0000","last_clean":"2024-05-08T06:06:32.480539+0000","last_became_active":"2024-05-08T06:05:46.351305+0000","last_became_peered":"2024-05-08T06:05:46.351305+0000","last_unstale":"2024-05-08T06:06:32.480539+0000","last_undegraded":"2024-05-08T06:06:32.480539+0000","last_fullsized":"2024-05-08T06:06:32.480539+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:24:36.164315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.043418+0000","last_change":"2024-05-08T06:05:51.414793+0000","last_active":"2024-05-08T06:08:01.043418+0000","last_peered":"2024-05-08T06:08:01.043418+0000","last_clean":"2024-05-08T06:08:01.043418+0000","last_became_active":"2024-05-08T06:05:51.414317+0000","last_became_peered":"2024-05-08T06:05:51.414317+0000","last_unstale":"2024-05-08T06:08:01.043418+0000","last_undegraded":"2024-05-08T06:08:01.043418+0000","last_fullsized":"2024-05-08T06:08:01.043418+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:57:56.152836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":165,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:27.577714+0000","last_change":"2024-05-08T06:05:46.352683+0000","last_active":"2024-05-08T06:08:27.577714+0000","last_peered":"2024-05-08T06:08:27.577714+0000","last_clean":"2024-05-08T06:08:27.577714+0000","last_became_active":"2024-05-08T06:05:46.352278+0000","last_became_peered":"2024-05-08T06:05:46.352278+0000","last_unstale":"2024-05-08T06:08:27.577714+0000","last_undegraded":"2024-05-08T06:08:27.577714+0000","last_fullsized":"2024-05-08T06:08:27.577714+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:39:22.655516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":133,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.037458+0000","last_change":"2024-05-08T06:05:51.415822+0000","last_active":"2024-05-08T06:08:01.037458+0000","last_peered":"2024-05-08T06:08:01.037458+0000","last_clean":"2024-05-08T06:08:01.037458+0000","last_became_active":"2024-05-08T06:05:51.415449+0000","last_became_peered":"2024-05-08T06:05:51.415449+0000","last_unstale":"2024-05-08T06:08:01.037458+0000","last_undegraded":"2024-05-08T06:08:01.037458+0000","last_fullsized":"2024-05-08T06:08:01.037458+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:44:49.127335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.478073+0000","last_change":"2024-05-08T06:05:46.361288+0000","last_active":"2024-05-08T06:06:32.478073+0000","last_peered":"2024-05-08T06:06:32.478073+0000","last_clean":"2024-05-08T06:06:32.478073+0000","last_became_active":"2024-05-08T06:05:46.360761+0000","last_became_peered":"2024-05-08T06:05:46.360761+0000","last_unstale":"2024-05-08T06:06:32.478073+0000","last_undegraded":"2024-05-08T06:06:32.478073+0000","last_fullsized":"2024-05-08T06:06:32.478073+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:14:59.858935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.063490+0000","last_change":"2024-05-08T06:05:51.418547+0000","last_active":"2024-05-08T06:08:01.063490+0000","last_peered":"2024-05-08T06:08:01.063490+0000","last_clean":"2024-05-08T06:08:01.063490+0000","last_became_active":"2024-05-08T06:05:51.417588+0000","last_became_peered":"2024-05-08T06:05:51.417588+0000","last_unstale":"2024-05-08T06:08:01.063490+0000","last_undegraded":"2024-05-08T06:08:01.063490+0000","last_fullsized":"2024-05-08T06:08:01.063490+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:16:41.451351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.492080+0000","last_change":"2024-05-08T06:05:46.362077+0000","last_active":"2024-05-08T06:06:32.492080+0000","last_peered":"2024-05-08T06:06:32.492080+0000","last_clean":"2024-05-08T06:06:32.492080+0000","last_became_active":"2024-05-08T06:05:46.360823+0000","last_became_peered":"2024-05-08T06:05:46.360823+0000","last_unstale":"2024-05-08T06:06:32.492080+0000","last_undegraded":"2024-05-08T06:06:32.492080+0000","last_fullsized":"2024-05-08T06:06:32.492080+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:14.169822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.063350+0000","last_change":"2024-05-08T06:05:51.423687+0000","last_active":"2024-05-08T06:08:01.063350+0000","last_peered":"2024-05-08T06:08:01.063350+0000","last_clean":"2024-05-08T06:08:01.063350+0000","last_became_active":"2024-05-08T06:05:51.417656+0000","last_became_peered":"2024-05-08T06:05:51.417656+0000","last_unstale":"2024-05-08T06:08:01.063350+0000","last_undegraded":"2024-05-08T06:08:01.063350+0000","last_fullsized":"2024-05-08T06:08:01.063350+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:59:31.319915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.595803+0000","last_change":"2024-05-08T06:05:46.351892+0000","last_active":"2024-05-08T06:06:32.595803+0000","last_peered":"2024-05-08T06:06:32.595803+0000","last_clean":"2024-05-08T06:06:32.595803+0000","last_became_active":"2024-05-08T06:05:46.351523+0000","last_became_peered":"2024-05-08T06:05:46.351523+0000","last_unstale":"2024-05-08T06:06:32.595803+0000","last_undegraded":"2024-05-08T06:06:32.595803+0000","last_fullsized":"2024-05-08T06:06:32.595803+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:11:23.794313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.063933+0000","last_change":"2024-05-08T06:05:51.407681+0000","last_active":"2024-05-08T06:08:01.063933+0000","last_peered":"2024-05-08T06:08:01.063933+0000","last_clean":"2024-05-08T06:08:01.063933+0000","last_became_active":"2024-05-08T06:05:51.407243+0000","last_became_peered":"2024-05-08T06:05:51.407243+0000","last_unstale":"2024-05-08T06:08:01.063933+0000","last_undegraded":"2024-05-08T06:08:01.063933+0000","last_fullsized":"2024-05-08T06:08:01.063933+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:06:30.490268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'3","reported_seq":160,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:26.575359+0000","last_change":"2024-05-08T06:05:46.356675+0000","last_active":"2024-05-08T06:08:26.575359+0000","last_peered":"2024-05-08T06:08:26.575359+0000","last_clean":"2024-05-08T06:08:26.575359+0000","last_became_active":"2024-05-08T06:05:46.356360+0000","last_became_peered":"2024-05-08T06:05:46.356360+0000","last_unstale":"2024-05-08T06:08:26.575359+0000","last_undegraded":"2024-05-08T06:08:26.575359+0000","last_fullsized":"2024-05-08T06:08:26.575359+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:46:48.131551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":130,"num_read_kb":130,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.047752+0000","last_change":"2024-05-08T06:05:51.421325+0000","last_active":"2024-05-08T06:08:01.047752+0000","last_peered":"2024-05-08T06:08:01.047752+0000","last_clean":"2024-05-08T06:08:01.047752+0000","last_became_active":"2024-05-08T06:05:51.420905+0000","last_became_peered":"2024-05-08T06:05:51.420905+0000","last_unstale":"2024-05-08T06:08:01.047752+0000","last_undegraded":"2024-05-08T06:08:01.047752+0000","last_fullsized":"2024-05-08T06:08:01.047752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:37:27.324920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482723+0000","last_change":"2024-05-08T06:05:46.351958+0000","last_active":"2024-05-08T06:06:32.482723+0000","last_peered":"2024-05-08T06:06:32.482723+0000","last_clean":"2024-05-08T06:06:32.482723+0000","last_became_active":"2024-05-08T06:05:46.351561+0000","last_became_peered":"2024-05-08T06:05:46.351561+0000","last_unstale":"2024-05-08T06:06:32.482723+0000","last_undegraded":"2024-05-08T06:06:32.482723+0000","last_fullsized":"2024-05-08T06:06:32.482723+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:11:27.727029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.061468+0000","last_change":"2024-05-08T06:05:51.431809+0000","last_active":"2024-05-08T06:08:01.061468+0000","last_peered":"2024-05-08T06:08:01.061468+0000","last_clean":"2024-05-08T06:08:01.061468+0000","last_became_active":"2024-05-08T06:05:51.431416+0000","last_became_peered":"2024-05-08T06:05:51.431416+0000","last_unstale":"2024-05-08T06:08:01.061468+0000","last_undegraded":"2024-05-08T06:08:01.061468+0000","last_fullsized":"2024-05-08T06:08:01.061468+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:48:16.704134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.596352+0000","last_change":"2024-05-08T06:05:46.550763+0000","last_active":"2024-05-08T06:06:32.596352+0000","last_peered":"2024-05-08T06:06:32.596352+0000","last_clean":"2024-05-08T06:06:32.596352+0000","last_became_active":"2024-05-08T06:05:46.550297+0000","last_became_peered":"2024-05-08T06:05:46.550297+0000","last_unstale":"2024-05-08T06:06:32.596352+0000","last_undegraded":"2024-05-08T06:06:32.596352+0000","last_fullsized":"2024-05-08T06:06:32.596352+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:32:19.907220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.051960+0000","last_change":"2024-05-08T06:05:51.414738+0000","last_active":"2024-05-08T06:08:01.051960+0000","last_peered":"2024-05-08T06:08:01.051960+0000","last_clean":"2024-05-08T06:08:01.051960+0000","last_became_active":"2024-05-08T06:05:51.414369+0000","last_became_peered":"2024-05-08T06:05:51.414369+0000","last_unstale":"2024-05-08T06:08:01.051960+0000","last_undegraded":"2024-05-08T06:08:01.051960+0000","last_fullsized":"2024-05-08T06:08:01.051960+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:57:44.215343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'35","reported_seq":333,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:27.576858+0000","last_change":"2024-05-08T06:05:46.349825+0000","last_active":"2024-05-08T06:08:27.576858+0000","last_peered":"2024-05-08T06:08:27.576858+0000","last_clean":"2024-05-08T06:08:27.576858+0000","last_became_active":"2024-05-08T06:05:46.349425+0000","last_became_peered":"2024-05-08T06:05:46.349425+0000","last_unstale":"2024-05-08T06:08:27.576858+0000","last_undegraded":"2024-05-08T06:08:27.576858+0000","last_fullsized":"2024-05-08T06:08:27.576858+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:29:39.123817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64459,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":271,"num_read_kb":199,"num_write":36,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.022668+0000","last_change":"2024-05-08T06:05:51.411341+0000","last_active":"2024-05-08T06:08:01.022668+0000","last_peered":"2024-05-08T06:08:01.022668+0000","last_clean":"2024-05-08T06:08:01.022668+0000","last_became_active":"2024-05-08T06:05:51.411002+0000","last_became_peered":"2024-05-08T06:05:51.411002+0000","last_unstale":"2024-05-08T06:08:01.022668+0000","last_undegraded":"2024-05-08T06:08:01.022668+0000","last_fullsized":"2024-05-08T06:08:01.022668+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:27:07.013809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.509330+0000","last_change":"2024-05-08T06:06:32.508904+0000","last_active":"2024-05-08T06:06:32.509330+0000","last_peered":"2024-05-08T06:06:32.509330+0000","last_clean":"2024-05-08T06:06:32.509330+0000","last_became_active":"2024-05-08T06:06:32.496001+0000","last_became_peered":"2024-05-08T06:06:32.496001+0000","last_unstale":"2024-05-08T06:06:32.509330+0000","last_undegraded":"2024-05-08T06:06:32.509330+0000","last_fullsized":"2024-05-08T06:06:32.509330+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:34:32.417105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.065212+0000","last_change":"2024-05-08T06:05:51.418395+0000","last_active":"2024-05-08T06:08:01.065212+0000","last_peered":"2024-05-08T06:08:01.065212+0000","last_clean":"2024-05-08T06:08:01.065212+0000","last_became_active":"2024-05-08T06:05:51.418035+0000","last_became_peered":"2024-05-08T06:05:51.418035+0000","last_unstale":"2024-05-08T06:08:01.065212+0000","last_undegraded":"2024-05-08T06:08:01.065212+0000","last_fullsized":"2024-05-08T06:08:01.065212+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:58:16.529144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.595996+0000","last_change":"2024-05-08T06:05:46.363088+0000","last_active":"2024-05-08T06:06:32.595996+0000","last_peered":"2024-05-08T06:06:32.595996+0000","last_clean":"2024-05-08T06:06:32.595996+0000","last_became_active":"2024-05-08T06:05:46.362488+0000","last_became_peered":"2024-05-08T06:05:46.362488+0000","last_unstale":"2024-05-08T06:06:32.595996+0000","last_undegraded":"2024-05-08T06:06:32.595996+0000","last_fullsized":"2024-05-08T06:06:32.595996+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:28:10.039647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.059203+0000","last_change":"2024-05-08T06:05:51.415567+0000","last_active":"2024-05-08T06:08:01.059203+0000","last_peered":"2024-05-08T06:08:01.059203+0000","last_clean":"2024-05-08T06:08:01.059203+0000","last_became_active":"2024-05-08T06:05:51.415148+0000","last_became_peered":"2024-05-08T06:05:51.415148+0000","last_unstale":"2024-05-08T06:08:01.059203+0000","last_undegraded":"2024-05-08T06:08:01.059203+0000","last_fullsized":"2024-05-08T06:08:01.059203+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:51:48.871149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":169,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:29.677457+0000","last_change":"2024-05-08T06:05:46.348769+0000","last_active":"2024-05-08T06:08:29.677457+0000","last_peered":"2024-05-08T06:08:29.677457+0000","last_clean":"2024-05-08T06:08:29.677457+0000","last_became_active":"2024-05-08T06:05:46.348072+0000","last_became_peered":"2024-05-08T06:05:46.348072+0000","last_unstale":"2024-05-08T06:08:29.677457+0000","last_undegraded":"2024-05-08T06:08:29.677457+0000","last_fullsized":"2024-05-08T06:08:29.677457+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:45.292436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":138,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.049147+0000","last_change":"2024-05-08T06:05:51.414623+0000","last_active":"2024-05-08T06:08:01.049147+0000","last_peered":"2024-05-08T06:08:01.049147+0000","last_clean":"2024-05-08T06:08:01.049147+0000","last_became_active":"2024-05-08T06:05:51.414258+0000","last_became_peered":"2024-05-08T06:05:51.414258+0000","last_unstale":"2024-05-08T06:08:01.049147+0000","last_undegraded":"2024-05-08T06:08:01.049147+0000","last_fullsized":"2024-05-08T06:08:01.049147+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:38:48.297900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":43,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480030+0000","last_change":"2024-05-08T06:05:46.546936+0000","last_active":"2024-05-08T06:06:32.480030+0000","last_peered":"2024-05-08T06:06:32.480030+0000","last_clean":"2024-05-08T06:06:32.480030+0000","last_became_active":"2024-05-08T06:05:46.546574+0000","last_became_peered":"2024-05-08T06:05:46.546574+0000","last_unstale":"2024-05-08T06:06:32.480030+0000","last_undegraded":"2024-05-08T06:06:32.480030+0000","last_fullsized":"2024-05-08T06:06:32.480030+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:16:29.720304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.029792+0000","last_change":"2024-05-08T06:05:51.418012+0000","last_active":"2024-05-08T06:08:01.029792+0000","last_peered":"2024-05-08T06:08:01.029792+0000","last_clean":"2024-05-08T06:08:01.029792+0000","last_became_active":"2024-05-08T06:05:51.417646+0000","last_became_peered":"2024-05-08T06:05:51.417646+0000","last_unstale":"2024-05-08T06:08:01.029792+0000","last_undegraded":"2024-05-08T06:08:01.029792+0000","last_fullsized":"2024-05-08T06:08:01.029792+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:43:02.733967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480265+0000","last_change":"2024-05-08T06:05:46.347748+0000","last_active":"2024-05-08T06:06:32.480265+0000","last_peered":"2024-05-08T06:06:32.480265+0000","last_clean":"2024-05-08T06:06:32.480265+0000","last_became_active":"2024-05-08T06:05:46.347367+0000","last_became_peered":"2024-05-08T06:05:46.347367+0000","last_unstale":"2024-05-08T06:06:32.480265+0000","last_undegraded":"2024-05-08T06:06:32.480265+0000","last_fullsized":"2024-05-08T06:06:32.480265+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:51:50.973855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.038216+0000","last_change":"2024-05-08T06:06:32.597018+0000","last_active":"2024-05-08T06:08:01.038216+0000","last_peered":"2024-05-08T06:08:01.038216+0000","last_clean":"2024-05-08T06:08:01.038216+0000","last_became_active":"2024-05-08T06:06:32.596677+0000","last_became_peered":"2024-05-08T06:06:32.596677+0000","last_unstale":"2024-05-08T06:08:01.038216+0000","last_undegraded":"2024-05-08T06:08:01.038216+0000","last_fullsized":"2024-05-08T06:08:01.038216+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:02:46.223949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.481700+0000","last_change":"2024-05-08T06:05:46.554443+0000","last_active":"2024-05-08T06:06:32.481700+0000","last_peered":"2024-05-08T06:06:32.481700+0000","last_clean":"2024-05-08T06:06:32.481700+0000","last_became_active":"2024-05-08T06:05:46.553226+0000","last_became_peered":"2024-05-08T06:05:46.553226+0000","last_unstale":"2024-05-08T06:06:32.481700+0000","last_undegraded":"2024-05-08T06:06:32.481700+0000","last_fullsized":"2024-05-08T06:06:32.481700+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:56:39.784652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.060786+0000","last_change":"2024-05-08T06:05:51.413121+0000","last_active":"2024-05-08T06:08:01.060786+0000","last_peered":"2024-05-08T06:08:01.060786+0000","last_clean":"2024-05-08T06:08:01.060786+0000","last_became_active":"2024-05-08T06:05:51.412767+0000","last_became_peered":"2024-05-08T06:05:51.412767+0000","last_unstale":"2024-05-08T06:08:01.060786+0000","last_undegraded":"2024-05-08T06:08:01.060786+0000","last_fullsized":"2024-05-08T06:08:01.060786+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:59:13.416981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482457+0000","last_change":"2024-05-08T06:05:46.361349+0000","last_active":"2024-05-08T06:06:32.482457+0000","last_peered":"2024-05-08T06:06:32.482457+0000","last_clean":"2024-05-08T06:06:32.482457+0000","last_became_active":"2024-05-08T06:05:46.360654+0000","last_became_peered":"2024-05-08T06:05:46.360654+0000","last_unstale":"2024-05-08T06:06:32.482457+0000","last_undegraded":"2024-05-08T06:06:32.482457+0000","last_fullsized":"2024-05-08T06:06:32.482457+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:19:22.144874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.062066+0000","last_change":"2024-05-08T06:05:51.423836+0000","last_active":"2024-05-08T06:08:01.062066+0000","last_peered":"2024-05-08T06:08:01.062066+0000","last_clean":"2024-05-08T06:08:01.062066+0000","last_became_active":"2024-05-08T06:05:51.423424+0000","last_became_peered":"2024-05-08T06:05:51.423424+0000","last_unstale":"2024-05-08T06:08:01.062066+0000","last_undegraded":"2024-05-08T06:08:01.062066+0000","last_fullsized":"2024-05-08T06:08:01.062066+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:56:14.779765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.047365+0000","last_change":"2024-05-08T06:06:32.598978+0000","last_active":"2024-05-08T06:08:01.047365+0000","last_peered":"2024-05-08T06:08:01.047365+0000","last_clean":"2024-05-08T06:08:01.047365+0000","last_became_active":"2024-05-08T06:06:32.598158+0000","last_became_peered":"2024-05-08T06:06:32.598158+0000","last_unstale":"2024-05-08T06:08:01.047365+0000","last_undegraded":"2024-05-08T06:08:01.047365+0000","last_fullsized":"2024-05-08T06:08:01.047365+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:51:09.270066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":165,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:29.676599+0000","last_change":"2024-05-08T06:05:46.547450+0000","last_active":"2024-05-08T06:08:29.676599+0000","last_peered":"2024-05-08T06:08:29.676599+0000","last_clean":"2024-05-08T06:08:29.676599+0000","last_became_active":"2024-05-08T06:05:46.546444+0000","last_became_peered":"2024-05-08T06:05:46.546444+0000","last_unstale":"2024-05-08T06:08:29.676599+0000","last_undegraded":"2024-05-08T06:08:29.676599+0000","last_fullsized":"2024-05-08T06:08:29.676599+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:17:14.124224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":133,"num_read_kb":133,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.065170+0000","last_change":"2024-05-08T06:05:51.423191+0000","last_active":"2024-05-08T06:08:01.065170+0000","last_peered":"2024-05-08T06:08:01.065170+0000","last_clean":"2024-05-08T06:08:01.065170+0000","last_became_active":"2024-05-08T06:05:51.420912+0000","last_became_peered":"2024-05-08T06:05:51.420912+0000","last_unstale":"2024-05-08T06:08:01.065170+0000","last_undegraded":"2024-05-08T06:08:01.065170+0000","last_fullsized":"2024-05-08T06:08:01.065170+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:27:57.678527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.597388+0000","last_change":"2024-05-08T06:05:46.348650+0000","last_active":"2024-05-08T06:06:32.597388+0000","last_peered":"2024-05-08T06:06:32.597388+0000","last_clean":"2024-05-08T06:06:32.597388+0000","last_became_active":"2024-05-08T06:05:46.347266+0000","last_became_peered":"2024-05-08T06:05:46.347266+0000","last_unstale":"2024-05-08T06:06:32.597388+0000","last_undegraded":"2024-05-08T06:06:32.597388+0000","last_fullsized":"2024-05-08T06:06:32.597388+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:37:30.623580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":100,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482311+0000","last_change":"2024-05-08T06:04:09.404663+0000","last_active":"2024-05-08T06:06:32.482311+0000","last_peered":"2024-05-08T06:06:32.482311+0000","last_clean":"2024-05-08T06:06:32.482311+0000","last_became_active":"2024-05-08T06:04:09.400115+0000","last_became_peered":"2024-05-08T06:04:09.400115+0000","last_unstale":"2024-05-08T06:06:32.482311+0000","last_undegraded":"2024-05-08T06:06:32.482311+0000","last_fullsized":"2024-05-08T06:06:32.482311+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:00:17.419712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:00:17.419712+0000","last_clean_scrub_stamp":"2024-05-08T06:00:17.419712+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:27:39.154694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.053460+0000","last_change":"2024-05-08T06:05:51.421826+0000","last_active":"2024-05-08T06:08:01.053460+0000","last_peered":"2024-05-08T06:08:01.053460+0000","last_clean":"2024-05-08T06:08:01.053460+0000","last_became_active":"2024-05-08T06:05:51.420706+0000","last_became_peered":"2024-05-08T06:05:51.420706+0000","last_unstale":"2024-05-08T06:08:01.053460+0000","last_undegraded":"2024-05-08T06:08:01.053460+0000","last_fullsized":"2024-05-08T06:08:01.053460+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:06:16.242332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482383+0000","last_change":"2024-05-08T06:05:46.351365+0000","last_active":"2024-05-08T06:06:32.482383+0000","last_peered":"2024-05-08T06:06:32.482383+0000","last_clean":"2024-05-08T06:06:32.482383+0000","last_became_active":"2024-05-08T06:05:46.351022+0000","last_became_peered":"2024-05-08T06:05:46.351022+0000","last_unstale":"2024-05-08T06:06:32.482383+0000","last_undegraded":"2024-05-08T06:06:32.482383+0000","last_fullsized":"2024-05-08T06:06:32.482383+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:32:44.134994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.501668+0000","last_change":"2024-05-08T06:06:32.501263+0000","last_active":"2024-05-08T06:06:32.501668+0000","last_peered":"2024-05-08T06:06:32.501668+0000","last_clean":"2024-05-08T06:06:32.501668+0000","last_became_active":"2024-05-08T06:06:32.493795+0000","last_became_peered":"2024-05-08T06:06:32.493795+0000","last_unstale":"2024-05-08T06:06:32.501668+0000","last_undegraded":"2024-05-08T06:06:32.501668+0000","last_fullsized":"2024-05-08T06:06:32.501668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:49:03.898162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.049894+0000","last_change":"2024-05-08T06:05:51.428382+0000","last_active":"2024-05-08T06:08:01.049894+0000","last_peered":"2024-05-08T06:08:01.049894+0000","last_clean":"2024-05-08T06:08:01.049894+0000","last_became_active":"2024-05-08T06:05:51.421443+0000","last_became_peered":"2024-05-08T06:05:51.421443+0000","last_unstale":"2024-05-08T06:08:01.049894+0000","last_undegraded":"2024-05-08T06:08:01.049894+0000","last_fullsized":"2024-05-08T06:08:01.049894+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:26:10.215020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'13","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:07:56.949283+0000","last_change":"2024-05-08T06:05:46.364306+0000","last_active":"2024-05-08T06:07:56.949283+0000","last_peered":"2024-05-08T06:07:56.949283+0000","last_clean":"2024-05-08T06:07:56.949283+0000","last_became_active":"2024-05-08T06:05:46.363871+0000","last_became_peered":"2024-05-08T06:05:46.363871+0000","last_unstale":"2024-05-08T06:07:56.949283+0000","last_undegraded":"2024-05-08T06:07:56.949283+0000","last_fullsized":"2024-05-08T06:07:56.949283+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:38:34.531162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.061822+0000","last_change":"2024-05-08T06:05:51.415020+0000","last_active":"2024-05-08T06:08:01.061822+0000","last_peered":"2024-05-08T06:08:01.061822+0000","last_clean":"2024-05-08T06:08:01.061822+0000","last_became_active":"2024-05-08T06:05:51.413690+0000","last_became_peered":"2024-05-08T06:05:51.413690+0000","last_unstale":"2024-05-08T06:08:01.061822+0000","last_undegraded":"2024-05-08T06:08:01.061822+0000","last_fullsized":"2024-05-08T06:08:01.061822+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:55:38.121492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.597283+0000","last_change":"2024-05-08T06:05:46.359100+0000","last_active":"2024-05-08T06:06:32.597283+0000","last_peered":"2024-05-08T06:06:32.597283+0000","last_clean":"2024-05-08T06:06:32.597283+0000","last_became_active":"2024-05-08T06:05:46.358815+0000","last_became_peered":"2024-05-08T06:05:46.358815+0000","last_unstale":"2024-05-08T06:06:32.597283+0000","last_undegraded":"2024-05-08T06:06:32.597283+0000","last_fullsized":"2024-05-08T06:06:32.597283+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:22:58.532358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.064793+0000","last_change":"2024-05-08T06:05:51.429222+0000","last_active":"2024-05-08T06:08:01.064793+0000","last_peered":"2024-05-08T06:08:01.064793+0000","last_clean":"2024-05-08T06:08:01.064793+0000","last_became_active":"2024-05-08T06:05:51.428845+0000","last_became_peered":"2024-05-08T06:05:51.428845+0000","last_unstale":"2024-05-08T06:08:01.064793+0000","last_undegraded":"2024-05-08T06:08:01.064793+0000","last_fullsized":"2024-05-08T06:08:01.064793+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:36:41.435666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.596086+0000","last_change":"2024-05-08T06:05:46.547931+0000","last_active":"2024-05-08T06:06:32.596086+0000","last_peered":"2024-05-08T06:06:32.596086+0000","last_clean":"2024-05-08T06:06:32.596086+0000","last_became_active":"2024-05-08T06:05:46.547547+0000","last_became_peered":"2024-05-08T06:05:46.547547+0000","last_unstale":"2024-05-08T06:06:32.596086+0000","last_undegraded":"2024-05-08T06:06:32.596086+0000","last_fullsized":"2024-05-08T06:06:32.596086+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:27:03.502153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.015273+0000","last_change":"2024-05-08T06:05:51.413259+0000","last_active":"2024-05-08T06:08:01.015273+0000","last_peered":"2024-05-08T06:08:01.015273+0000","last_clean":"2024-05-08T06:08:01.015273+0000","last_became_active":"2024-05-08T06:05:51.412890+0000","last_became_peered":"2024-05-08T06:05:51.412890+0000","last_unstale":"2024-05-08T06:08:01.015273+0000","last_undegraded":"2024-05-08T06:08:01.015273+0000","last_fullsized":"2024-05-08T06:08:01.015273+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:04:56.690098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.489841+0000","last_change":"2024-05-08T06:05:46.365409+0000","last_active":"2024-05-08T06:06:32.489841+0000","last_peered":"2024-05-08T06:06:32.489841+0000","last_clean":"2024-05-08T06:06:32.489841+0000","last_became_active":"2024-05-08T06:05:46.361277+0000","last_became_peered":"2024-05-08T06:05:46.361277+0000","last_unstale":"2024-05-08T06:06:32.489841+0000","last_undegraded":"2024-05-08T06:06:32.489841+0000","last_fullsized":"2024-05-08T06:06:32.489841+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:06.927385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.063724+0000","last_change":"2024-05-08T06:05:51.423749+0000","last_active":"2024-05-08T06:08:01.063724+0000","last_peered":"2024-05-08T06:08:01.063724+0000","last_clean":"2024-05-08T06:08:01.063724+0000","last_became_active":"2024-05-08T06:05:51.423386+0000","last_became_peered":"2024-05-08T06:05:51.423386+0000","last_unstale":"2024-05-08T06:08:01.063724+0000","last_undegraded":"2024-05-08T06:08:01.063724+0000","last_fullsized":"2024-05-08T06:08:01.063724+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:41:26.261683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480440+0000","last_change":"2024-05-08T06:05:46.362497+0000","last_active":"2024-05-08T06:06:32.480440+0000","last_peered":"2024-05-08T06:06:32.480440+0000","last_clean":"2024-05-08T06:06:32.480440+0000","last_became_active":"2024-05-08T06:05:46.362110+0000","last_became_peered":"2024-05-08T06:05:46.362110+0000","last_unstale":"2024-05-08T06:06:32.480440+0000","last_undegraded":"2024-05-08T06:06:32.480440+0000","last_fullsized":"2024-05-08T06:06:32.480440+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:25:06.205380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.060664+0000","last_change":"2024-05-08T06:06:32.495078+0000","last_active":"2024-05-08T06:08:01.060664+0000","last_peered":"2024-05-08T06:08:01.060664+0000","last_clean":"2024-05-08T06:08:01.060664+0000","last_became_active":"2024-05-08T06:06:32.493855+0000","last_became_peered":"2024-05-08T06:06:32.493855+0000","last_unstale":"2024-05-08T06:08:01.060664+0000","last_undegraded":"2024-05-08T06:08:01.060664+0000","last_fullsized":"2024-05-08T06:08:01.060664+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:56:54.143080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.602199+0000","last_change":"2024-05-08T06:06:32.601826+0000","last_active":"2024-05-08T06:06:32.602199+0000","last_peered":"2024-05-08T06:06:32.602199+0000","last_clean":"2024-05-08T06:06:32.602199+0000","last_became_active":"2024-05-08T06:06:32.597823+0000","last_became_peered":"2024-05-08T06:06:32.597823+0000","last_unstale":"2024-05-08T06:06:32.602199+0000","last_undegraded":"2024-05-08T06:06:32.602199+0000","last_fullsized":"2024-05-08T06:06:32.602199+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:59:09.741096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.057017+0000","last_change":"2024-05-08T06:05:51.413744+0000","last_active":"2024-05-08T06:08:01.057017+0000","last_peered":"2024-05-08T06:08:01.057017+0000","last_clean":"2024-05-08T06:08:01.057017+0000","last_became_active":"2024-05-08T06:05:51.413346+0000","last_became_peered":"2024-05-08T06:05:51.413346+0000","last_unstale":"2024-05-08T06:08:01.057017+0000","last_undegraded":"2024-05-08T06:08:01.057017+0000","last_fullsized":"2024-05-08T06:08:01.057017+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:46:56.109755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480947+0000","last_change":"2024-05-08T06:05:46.552604+0000","last_active":"2024-05-08T06:06:32.480947+0000","last_peered":"2024-05-08T06:06:32.480947+0000","last_clean":"2024-05-08T06:06:32.480947+0000","last_became_active":"2024-05-08T06:05:46.552248+0000","last_became_peered":"2024-05-08T06:05:46.552248+0000","last_unstale":"2024-05-08T06:06:32.480947+0000","last_undegraded":"2024-05-08T06:06:32.480947+0000","last_fullsized":"2024-05-08T06:06:32.480947+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:39:10.306692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.054186+0000","last_change":"2024-05-08T06:05:51.404988+0000","last_active":"2024-05-08T06:08:01.054186+0000","last_peered":"2024-05-08T06:08:01.054186+0000","last_clean":"2024-05-08T06:08:01.054186+0000","last_became_active":"2024-05-08T06:05:51.404625+0000","last_became_peered":"2024-05-08T06:05:51.404625+0000","last_unstale":"2024-05-08T06:08:01.054186+0000","last_undegraded":"2024-05-08T06:08:01.054186+0000","last_fullsized":"2024-05-08T06:08:01.054186+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:33:27.919929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.596178+0000","last_change":"2024-05-08T06:05:46.361466+0000","last_active":"2024-05-08T06:06:32.596178+0000","last_peered":"2024-05-08T06:06:32.596178+0000","last_clean":"2024-05-08T06:06:32.596178+0000","last_became_active":"2024-05-08T06:05:46.361107+0000","last_became_peered":"2024-05-08T06:05:46.361107+0000","last_unstale":"2024-05-08T06:06:32.596178+0000","last_undegraded":"2024-05-08T06:06:32.596178+0000","last_fullsized":"2024-05-08T06:06:32.596178+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:15:31.314226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.062827+0000","last_change":"2024-05-08T06:05:51.416437+0000","last_active":"2024-05-08T06:08:01.062827+0000","last_peered":"2024-05-08T06:08:01.062827+0000","last_clean":"2024-05-08T06:08:01.062827+0000","last_became_active":"2024-05-08T06:05:51.416070+0000","last_became_peered":"2024-05-08T06:05:51.416070+0000","last_unstale":"2024-05-08T06:08:01.062827+0000","last_undegraded":"2024-05-08T06:08:01.062827+0000","last_fullsized":"2024-05-08T06:08:01.062827+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:35:23.652373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.493014+0000","last_change":"2024-05-08T06:05:46.362475+0000","last_active":"2024-05-08T06:06:32.493014+0000","last_peered":"2024-05-08T06:06:32.493014+0000","last_clean":"2024-05-08T06:06:32.493014+0000","last_became_active":"2024-05-08T06:05:46.361852+0000","last_became_peered":"2024-05-08T06:05:46.361852+0000","last_unstale":"2024-05-08T06:06:32.493014+0000","last_undegraded":"2024-05-08T06:06:32.493014+0000","last_fullsized":"2024-05-08T06:06:32.493014+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:16:54.536330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.049399+0000","last_change":"2024-05-08T06:05:51.410718+0000","last_active":"2024-05-08T06:08:01.049399+0000","last_peered":"2024-05-08T06:08:01.049399+0000","last_clean":"2024-05-08T06:08:01.049399+0000","last_became_active":"2024-05-08T06:05:51.408902+0000","last_became_peered":"2024-05-08T06:05:51.408902+0000","last_unstale":"2024-05-08T06:08:01.049399+0000","last_undegraded":"2024-05-08T06:08:01.049399+0000","last_fullsized":"2024-05-08T06:08:01.049399+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:23:33.857677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.038950+0000","last_change":"2024-05-08T06:05:51.414362+0000","last_active":"2024-05-08T06:08:01.038950+0000","last_peered":"2024-05-08T06:08:01.038950+0000","last_clean":"2024-05-08T06:08:01.038950+0000","last_became_active":"2024-05-08T06:05:51.413998+0000","last_became_peered":"2024-05-08T06:05:51.413998+0000","last_unstale":"2024-05-08T06:08:01.038950+0000","last_undegraded":"2024-05-08T06:08:01.038950+0000","last_fullsized":"2024-05-08T06:08:01.038950+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:38:22.198219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.032457+0000","last_change":"2024-05-08T06:05:51.415508+0000","last_active":"2024-05-08T06:08:01.032457+0000","last_peered":"2024-05-08T06:08:01.032457+0000","last_clean":"2024-05-08T06:08:01.032457+0000","last_became_active":"2024-05-08T06:05:51.415137+0000","last_became_peered":"2024-05-08T06:05:51.415137+0000","last_unstale":"2024-05-08T06:08:01.032457+0000","last_undegraded":"2024-05-08T06:08:01.032457+0000","last_fullsized":"2024-05-08T06:08:01.032457+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:15.007996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.064306+0000","last_change":"2024-05-08T06:05:51.422093+0000","last_active":"2024-05-08T06:08:01.064306+0000","last_peered":"2024-05-08T06:08:01.064306+0000","last_clean":"2024-05-08T06:08:01.064306+0000","last_became_active":"2024-05-08T06:05:51.421733+0000","last_became_peered":"2024-05-08T06:05:51.421733+0000","last_unstale":"2024-05-08T06:08:01.064306+0000","last_undegraded":"2024-05-08T06:08:01.064306+0000","last_fullsized":"2024-05-08T06:08:01.064306+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:35:55.950229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.060949+0000","last_change":"2024-05-08T06:05:51.434705+0000","last_active":"2024-05-08T06:08:01.060949+0000","last_peered":"2024-05-08T06:08:01.060949+0000","last_clean":"2024-05-08T06:08:01.060949+0000","last_became_active":"2024-05-08T06:05:51.434324+0000","last_became_peered":"2024-05-08T06:05:51.434324+0000","last_unstale":"2024-05-08T06:08:01.060949+0000","last_undegraded":"2024-05-08T06:08:01.060949+0000","last_fullsized":"2024-05-08T06:08:01.060949+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:38:23.101247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.063946+0000","last_change":"2024-05-08T06:06:32.499837+0000","last_active":"2024-05-08T06:08:01.063946+0000","last_peered":"2024-05-08T06:08:01.063946+0000","last_clean":"2024-05-08T06:08:01.063946+0000","last_became_active":"2024-05-08T06:06:32.499026+0000","last_became_peered":"2024-05-08T06:06:32.499026+0000","last_unstale":"2024-05-08T06:08:01.063946+0000","last_undegraded":"2024-05-08T06:08:01.063946+0000","last_fullsized":"2024-05-08T06:08:01.063946+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:48:51.910384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.038524+0000","last_change":"2024-05-08T06:05:51.412370+0000","last_active":"2024-05-08T06:08:01.038524+0000","last_peered":"2024-05-08T06:08:01.038524+0000","last_clean":"2024-05-08T06:08:01.038524+0000","last_became_active":"2024-05-08T06:05:51.412057+0000","last_became_peered":"2024-05-08T06:05:51.412057+0000","last_unstale":"2024-05-08T06:08:01.038524+0000","last_undegraded":"2024-05-08T06:08:01.038524+0000","last_fullsized":"2024-05-08T06:08:01.038524+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:20:51.549926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.048004+0000","last_change":"2024-05-08T06:05:51.408393+0000","last_active":"2024-05-08T06:08:01.048004+0000","last_peered":"2024-05-08T06:08:01.048004+0000","last_clean":"2024-05-08T06:08:01.048004+0000","last_became_active":"2024-05-08T06:05:51.408023+0000","last_became_peered":"2024-05-08T06:05:51.408023+0000","last_unstale":"2024-05-08T06:08:01.048004+0000","last_undegraded":"2024-05-08T06:08:01.048004+0000","last_fullsized":"2024-05-08T06:08:01.048004+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:53.898929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.059503+0000","last_change":"2024-05-08T06:05:51.418055+0000","last_active":"2024-05-08T06:08:01.059503+0000","last_peered":"2024-05-08T06:08:01.059503+0000","last_clean":"2024-05-08T06:08:01.059503+0000","last_became_active":"2024-05-08T06:05:51.417067+0000","last_became_peered":"2024-05-08T06:05:51.417067+0000","last_unstale":"2024-05-08T06:08:01.059503+0000","last_undegraded":"2024-05-08T06:08:01.059503+0000","last_fullsized":"2024-05-08T06:08:01.059503+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:07:35.413672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.063735+0000","last_change":"2024-05-08T06:05:51.421025+0000","last_active":"2024-05-08T06:08:01.063735+0000","last_peered":"2024-05-08T06:08:01.063735+0000","last_clean":"2024-05-08T06:08:01.063735+0000","last_became_active":"2024-05-08T06:05:51.420794+0000","last_became_peered":"2024-05-08T06:05:51.420794+0000","last_unstale":"2024-05-08T06:08:01.063735+0000","last_undegraded":"2024-05-08T06:08:01.063735+0000","last_fullsized":"2024-05-08T06:08:01.063735+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:45:01.852405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.060497+0000","last_change":"2024-05-08T06:05:51.417709+0000","last_active":"2024-05-08T06:08:01.060497+0000","last_peered":"2024-05-08T06:08:01.060497+0000","last_clean":"2024-05-08T06:08:01.060497+0000","last_became_active":"2024-05-08T06:05:51.417337+0000","last_became_peered":"2024-05-08T06:05:51.417337+0000","last_unstale":"2024-05-08T06:08:01.060497+0000","last_undegraded":"2024-05-08T06:08:01.060497+0000","last_fullsized":"2024-05-08T06:08:01.060497+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:32:24.551768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.065666+0000","last_change":"2024-05-08T06:05:51.425318+0000","last_active":"2024-05-08T06:08:01.065666+0000","last_peered":"2024-05-08T06:08:01.065666+0000","last_clean":"2024-05-08T06:08:01.065666+0000","last_became_active":"2024-05-08T06:05:51.424931+0000","last_became_peered":"2024-05-08T06:05:51.424931+0000","last_unstale":"2024-05-08T06:08:01.065666+0000","last_undegraded":"2024-05-08T06:08:01.065666+0000","last_fullsized":"2024-05-08T06:08:01.065666+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:59:45.110201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.064188+0000","last_change":"2024-05-08T06:05:51.433529+0000","last_active":"2024-05-08T06:08:01.064188+0000","last_peered":"2024-05-08T06:08:01.064188+0000","last_clean":"2024-05-08T06:08:01.064188+0000","last_became_active":"2024-05-08T06:05:51.433173+0000","last_became_peered":"2024-05-08T06:05:51.433173+0000","last_unstale":"2024-05-08T06:08:01.064188+0000","last_undegraded":"2024-05-08T06:08:01.064188+0000","last_fullsized":"2024-05-08T06:08:01.064188+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:00:29.042188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.045237+0000","last_change":"2024-05-08T06:05:51.411418+0000","last_active":"2024-05-08T06:08:01.045237+0000","last_peered":"2024-05-08T06:08:01.045237+0000","last_clean":"2024-05-08T06:08:01.045237+0000","last_became_active":"2024-05-08T06:05:51.411084+0000","last_became_peered":"2024-05-08T06:05:51.411084+0000","last_unstale":"2024-05-08T06:08:01.045237+0000","last_undegraded":"2024-05-08T06:08:01.045237+0000","last_fullsized":"2024-05-08T06:08:01.045237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:08:11.522282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282383,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68601,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":817,"num_read_kb":481,"num_write":175,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":205803,"data_compressed":5508,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645362,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":888832,"data_stored":431859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841591,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":1088275,"data_compressed":37744,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005117,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":897024,"data_stored":432039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201348,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":892928,"data_stored":431937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397576,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":1024030,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561102,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":884736,"data_stored":431428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":495680,"data_compressed":1836,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738480,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":432699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411430,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":966656,"data_stored":496996,"data_compressed":1836,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117088,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":431739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920858,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":888832,"data_stored":432397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757332,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":1023182,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34281,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34281,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34281,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":64459,"data_compressed":1836,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":65775,"data_compressed":1836,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":66569,"data_compressed":1836,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T06:08:32.980 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T06:08:33.092 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:32 smithi123 ceph-mon[29787]: pgmap v402: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:08:33.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:32 smithi163 ceph-mon[34520]: pgmap v402: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:08:33.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:32 smithi192 ceph-mon[34664]: pgmap v402: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:08:33.451 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:08:33.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:33 smithi123 ceph-mon[29787]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T06:08:34.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:33 smithi163 ceph-mon[34520]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T06:08:34.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:33 smithi192 ceph-mon[34664]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T06:08:35.114 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:34 smithi123 ceph-mon[29787]: pgmap v403: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:08:35.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:34 smithi163 ceph-mon[34520]: pgmap v403: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:08:35.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:34 smithi192 ceph-mon[34664]: pgmap v403: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:08:35.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.933+0000 7fac98a00700 1 -- 172.21.15.123:0/3588305520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac94104590 msgr2=0x7fac94104970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:35.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.933+0000 7fac98a00700 1 --2- 172.21.15.123:0/3588305520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac94104590 0x7fac94104970 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fac84009990 tx=0x7fac84009ca0 comp rx=0 tx=0).stop 2024-05-08T06:08:35.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.933+0000 7fac98a00700 1 -- 172.21.15.123:0/3588305520 shutdown_connections 2024-05-08T06:08:35.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.933+0000 7fac98a00700 1 --2- 172.21.15.123:0/3588305520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac94104590 0x7fac94104970 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:35.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.933+0000 7fac98a00700 1 --2- 172.21.15.123:0/3588305520 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac9410d9c0 0x7fac9410fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:35.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.933+0000 7fac98a00700 1 --2- 172.21.15.123:0/3588305520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac94104eb0 0x7fac9410d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:35.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.933+0000 7fac98a00700 1 -- 172.21.15.123:0/3588305520 >> 172.21.15.123:0/3588305520 conn(0x7fac940fddc0 msgr2=0x7fac941001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:08:35.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.933+0000 7fac98a00700 1 -- 172.21.15.123:0/3588305520 shutdown_connections 2024-05-08T06:08:35.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.934+0000 7fac98a00700 1 -- 172.21.15.123:0/3588305520 wait complete. 2024-05-08T06:08:35.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.935+0000 7fac98a00700 1 Processor -- start 2024-05-08T06:08:35.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.935+0000 7fac98a00700 1 -- start start 2024-05-08T06:08:35.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.935+0000 7fac98a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac94104590 0x7fac9419bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:35.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.935+0000 7fac98a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac94104eb0 0x7fac9419c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:35.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.935+0000 7fac98a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac9410d9c0 0x7fac941a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:35.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.935+0000 7fac98a00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fac94113600 con 0x7fac94104590 2024-05-08T06:08:35.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.935+0000 7fac98a00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fac94113480 con 0x7fac94104eb0 2024-05-08T06:08:35.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.935+0000 7fac98a00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fac94113780 con 0x7fac9410d9c0 2024-05-08T06:08:35.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.935+0000 7fac93400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac94104590 0x7fac9419bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:35.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.936+0000 7fac93400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac94104590 0x7fac9419bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56406/0 (socket says 172.21.15.123:56406) 2024-05-08T06:08:35.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.936+0000 7fac93400700 1 -- 172.21.15.123:0/86295156 learned_addr learned my addr 172.21.15.123:0/86295156 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:08:35.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.936+0000 7fac92a00700 1 --2- 172.21.15.123:0/86295156 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac94104eb0 0x7fac9419c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:35.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.936+0000 7fac93e00700 1 --2- 172.21.15.123:0/86295156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac9410d9c0 0x7fac941a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:35.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.936+0000 7fac93400700 1 -- 172.21.15.123:0/86295156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac9410d9c0 msgr2=0x7fac941a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:35.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.936+0000 7fac93400700 1 --2- 172.21.15.123:0/86295156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac9410d9c0 0x7fac941a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:35.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.936+0000 7fac93400700 1 -- 172.21.15.123:0/86295156 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac94104eb0 msgr2=0x7fac9419c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:35.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.936+0000 7fac93400700 1 --2- 172.21.15.123:0/86295156 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac94104eb0 0x7fac9419c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:35.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.936+0000 7fac93400700 1 -- 172.21.15.123:0/86295156 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac941a0d60 con 0x7fac94104590 2024-05-08T06:08:35.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.936+0000 7fac93e00700 1 --2- 172.21.15.123:0/86295156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac9410d9c0 0x7fac941a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:08:35.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.936+0000 7fac93400700 1 --2- 172.21.15.123:0/86295156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac94104590 0x7fac9419bd50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fac84000c00 tx=0x7fac840057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:08:35.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.937+0000 7fac8be00700 1 -- 172.21.15.123:0/86295156 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac84027070 con 0x7fac94104590 2024-05-08T06:08:35.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.937+0000 7fac8be00700 1 -- 172.21.15.123:0/86295156 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac84022470 con 0x7fac94104590 2024-05-08T06:08:35.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.937+0000 7fac98a00700 1 -- 172.21.15.123:0/86295156 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac941a0ff0 con 0x7fac94104590 2024-05-08T06:08:35.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.937+0000 7fac8be00700 1 -- 172.21.15.123:0/86295156 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac84017650 con 0x7fac94104590 2024-05-08T06:08:35.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.937+0000 7fac98a00700 1 -- 172.21.15.123:0/86295156 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac941a88d0 con 0x7fac94104590 2024-05-08T06:08:35.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.938+0000 7fac98a00700 1 -- 172.21.15.123:0/86295156 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac9406a190 con 0x7fac94104590 2024-05-08T06:08:35.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.940+0000 7fac8be00700 1 -- 172.21.15.123:0/86295156 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fac8402a070 con 0x7fac94104590 2024-05-08T06:08:35.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.942+0000 7fac8be00700 1 --2- 172.21.15.123:0/86295156 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fac780b2900 0x7fac780b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:35.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.942+0000 7fac8be00700 1 -- 172.21.15.123:0/86295156 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fac840e2fc0 con 0x7fac94104590 2024-05-08T06:08:35.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.942+0000 7fac92a00700 1 --2- 172.21.15.123:0/86295156 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fac780b2900 0x7fac780b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:35.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.943+0000 7fac92a00700 1 --2- 172.21.15.123:0/86295156 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fac780b2900 0x7fac780b4dc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fac941102b0 tx=0x7fac80009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:08:35.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:35.949+0000 7fac8be00700 1 -- 172.21.15.123:0/86295156 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fac84013070 con 0x7fac94104590 2024-05-08T06:08:36.180 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:36.180+0000 7fac98a00700 1 -- 172.21.15.123:0/86295156 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fac941039a0 con 0x7fac780b2900 2024-05-08T06:08:36.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:36.189+0000 7fac8be00700 1 -- 172.21.15.123:0/86295156 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259047 (secure 0 0 0) 0x7fac941039a0 con 0x7fac780b2900 2024-05-08T06:08:36.194 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:08:36.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:36.195+0000 7fac98a00700 1 -- 172.21.15.123:0/86295156 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fac780b2900 msgr2=0x7fac780b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:36.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:36.195+0000 7fac98a00700 1 --2- 172.21.15.123:0/86295156 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fac780b2900 0x7fac780b4dc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fac941102b0 tx=0x7fac80009260 comp rx=0 tx=0).stop 2024-05-08T06:08:36.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:36.195+0000 7fac98a00700 1 -- 172.21.15.123:0/86295156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac94104590 msgr2=0x7fac9419bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:36.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:36.195+0000 7fac98a00700 1 --2- 172.21.15.123:0/86295156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac94104590 0x7fac9419bd50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fac84000c00 tx=0x7fac840057c0 comp rx=0 tx=0).stop 2024-05-08T06:08:36.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:36.195+0000 7fac98a00700 1 -- 172.21.15.123:0/86295156 shutdown_connections 2024-05-08T06:08:36.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:36.195+0000 7fac98a00700 1 --2- 172.21.15.123:0/86295156 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fac780b2900 0x7fac780b4dc0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:36.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:36.195+0000 7fac98a00700 1 --2- 172.21.15.123:0/86295156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac94104590 0x7fac9419bd50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:36.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:36.195+0000 7fac98a00700 1 --2- 172.21.15.123:0/86295156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac9410d9c0 0x7fac941a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:36.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:36.195+0000 7fac98a00700 1 --2- 172.21.15.123:0/86295156 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac94104eb0 0x7fac9419c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:36.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:36.195+0000 7fac98a00700 1 -- 172.21.15.123:0/86295156 >> 172.21.15.123:0/86295156 conn(0x7fac940fddc0 msgr2=0x7fac941001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:08:36.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:36.195+0000 7fac98a00700 1 -- 172.21.15.123:0/86295156 shutdown_connections 2024-05-08T06:08:36.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:36.195+0000 7fac98a00700 1 -- 172.21.15.123:0/86295156 wait complete. 2024-05-08T06:08:36.197 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-05-08T06:08:37.103 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:36 smithi192 ceph-mon[34664]: pgmap v404: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:08:37.113 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:36 smithi123 ceph-mon[29787]: pgmap v404: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:08:37.246 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":404,"stamp":"2024-05-08T06:08:35.107558+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659203,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":913,"num_read_kb":555,"num_write":300,"num_write_kb":2274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":2951896,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":743,"ondisk_log_size":743,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372480,"kb_used_data":10848,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569344,"statfs":{"total":1151940427776,"available":1151559008256,"internally_reserved":0,"allocated":11108352,"data_stored":7152261,"data_compressed":113232,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004213"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.064044+0000","last_change":"2024-05-08T06:05:51.431744+0000","last_active":"2024-05-08T06:08:01.064044+0000","last_peered":"2024-05-08T06:08:01.064044+0000","last_clean":"2024-05-08T06:08:01.064044+0000","last_became_active":"2024-05-08T06:05:51.431405+0000","last_became_peered":"2024-05-08T06:05:51.431405+0000","last_unstale":"2024-05-08T06:08:01.064044+0000","last_undegraded":"2024-05-08T06:08:01.064044+0000","last_fullsized":"2024-05-08T06:08:01.064044+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:56:40.503607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.056293+0000","last_change":"2024-05-08T06:05:51.426473+0000","last_active":"2024-05-08T06:08:01.056293+0000","last_peered":"2024-05-08T06:08:01.056293+0000","last_clean":"2024-05-08T06:08:01.056293+0000","last_became_active":"2024-05-08T06:05:51.424477+0000","last_became_peered":"2024-05-08T06:05:51.424477+0000","last_unstale":"2024-05-08T06:08:01.056293+0000","last_undegraded":"2024-05-08T06:08:01.056293+0000","last_fullsized":"2024-05-08T06:08:01.056293+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:26:55.747808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.060523+0000","last_change":"2024-05-08T06:05:51.423767+0000","last_active":"2024-05-08T06:08:01.060523+0000","last_peered":"2024-05-08T06:08:01.060523+0000","last_clean":"2024-05-08T06:08:01.060523+0000","last_became_active":"2024-05-08T06:05:51.423273+0000","last_became_peered":"2024-05-08T06:05:51.423273+0000","last_unstale":"2024-05-08T06:08:01.060523+0000","last_undegraded":"2024-05-08T06:08:01.060523+0000","last_fullsized":"2024-05-08T06:08:01.060523+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:24:44.099743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.047123+0000","last_change":"2024-05-08T06:05:51.416604+0000","last_active":"2024-05-08T06:08:01.047123+0000","last_peered":"2024-05-08T06:08:01.047123+0000","last_clean":"2024-05-08T06:08:01.047123+0000","last_became_active":"2024-05-08T06:05:51.416284+0000","last_became_peered":"2024-05-08T06:05:51.416284+0000","last_unstale":"2024-05-08T06:08:01.047123+0000","last_undegraded":"2024-05-08T06:08:01.047123+0000","last_fullsized":"2024-05-08T06:08:01.047123+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:05:26.014323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.059697+0000","last_change":"2024-05-08T06:05:51.423915+0000","last_active":"2024-05-08T06:08:01.059697+0000","last_peered":"2024-05-08T06:08:01.059697+0000","last_clean":"2024-05-08T06:08:01.059697+0000","last_became_active":"2024-05-08T06:05:51.423534+0000","last_became_peered":"2024-05-08T06:05:51.423534+0000","last_unstale":"2024-05-08T06:08:01.059697+0000","last_undegraded":"2024-05-08T06:08:01.059697+0000","last_fullsized":"2024-05-08T06:08:01.059697+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:06:46.197071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.041181+0000","last_change":"2024-05-08T06:05:51.411102+0000","last_active":"2024-05-08T06:08:01.041181+0000","last_peered":"2024-05-08T06:08:01.041181+0000","last_clean":"2024-05-08T06:08:01.041181+0000","last_became_active":"2024-05-08T06:05:51.410791+0000","last_became_peered":"2024-05-08T06:05:51.410791+0000","last_unstale":"2024-05-08T06:08:01.041181+0000","last_undegraded":"2024-05-08T06:08:01.041181+0000","last_fullsized":"2024-05-08T06:08:01.041181+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:31:47.861663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.041524+0000","last_change":"2024-05-08T06:05:51.417756+0000","last_active":"2024-05-08T06:08:01.041524+0000","last_peered":"2024-05-08T06:08:01.041524+0000","last_clean":"2024-05-08T06:08:01.041524+0000","last_became_active":"2024-05-08T06:05:51.417373+0000","last_became_peered":"2024-05-08T06:05:51.417373+0000","last_unstale":"2024-05-08T06:08:01.041524+0000","last_undegraded":"2024-05-08T06:08:01.041524+0000","last_fullsized":"2024-05-08T06:08:01.041524+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:07:48.330401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.063489+0000","last_change":"2024-05-08T06:05:51.413883+0000","last_active":"2024-05-08T06:08:01.063489+0000","last_peered":"2024-05-08T06:08:01.063489+0000","last_clean":"2024-05-08T06:08:01.063489+0000","last_became_active":"2024-05-08T06:05:51.413565+0000","last_became_peered":"2024-05-08T06:05:51.413565+0000","last_unstale":"2024-05-08T06:08:01.063489+0000","last_undegraded":"2024-05-08T06:08:01.063489+0000","last_fullsized":"2024-05-08T06:08:01.063489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:06:38.186225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.053054+0000","last_change":"2024-05-08T06:05:51.416300+0000","last_active":"2024-05-08T06:08:01.053054+0000","last_peered":"2024-05-08T06:08:01.053054+0000","last_clean":"2024-05-08T06:08:01.053054+0000","last_became_active":"2024-05-08T06:05:51.415970+0000","last_became_peered":"2024-05-08T06:05:51.415970+0000","last_unstale":"2024-05-08T06:08:01.053054+0000","last_undegraded":"2024-05-08T06:08:01.053054+0000","last_fullsized":"2024-05-08T06:08:01.053054+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:44:51.853339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.039789+0000","last_change":"2024-05-08T06:05:51.410396+0000","last_active":"2024-05-08T06:08:01.039789+0000","last_peered":"2024-05-08T06:08:01.039789+0000","last_clean":"2024-05-08T06:08:01.039789+0000","last_became_active":"2024-05-08T06:05:51.410067+0000","last_became_peered":"2024-05-08T06:05:51.410067+0000","last_unstale":"2024-05-08T06:08:01.039789+0000","last_undegraded":"2024-05-08T06:08:01.039789+0000","last_fullsized":"2024-05-08T06:08:01.039789+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:32:00.217217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.052539+0000","last_change":"2024-05-08T06:05:51.409276+0000","last_active":"2024-05-08T06:08:01.052539+0000","last_peered":"2024-05-08T06:08:01.052539+0000","last_clean":"2024-05-08T06:08:01.052539+0000","last_became_active":"2024-05-08T06:05:51.408768+0000","last_became_peered":"2024-05-08T06:05:51.408768+0000","last_unstale":"2024-05-08T06:08:01.052539+0000","last_undegraded":"2024-05-08T06:08:01.052539+0000","last_fullsized":"2024-05-08T06:08:01.052539+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:22:57.483944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.048049+0000","last_change":"2024-05-08T06:05:51.419083+0000","last_active":"2024-05-08T06:08:01.048049+0000","last_peered":"2024-05-08T06:08:01.048049+0000","last_clean":"2024-05-08T06:08:01.048049+0000","last_became_active":"2024-05-08T06:05:51.418685+0000","last_became_peered":"2024-05-08T06:05:51.418685+0000","last_unstale":"2024-05-08T06:08:01.048049+0000","last_undegraded":"2024-05-08T06:08:01.048049+0000","last_fullsized":"2024-05-08T06:08:01.048049+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:25:49.256242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.057514+0000","last_change":"2024-05-08T06:05:51.420399+0000","last_active":"2024-05-08T06:08:01.057514+0000","last_peered":"2024-05-08T06:08:01.057514+0000","last_clean":"2024-05-08T06:08:01.057514+0000","last_became_active":"2024-05-08T06:05:51.420076+0000","last_became_peered":"2024-05-08T06:05:51.420076+0000","last_unstale":"2024-05-08T06:08:01.057514+0000","last_undegraded":"2024-05-08T06:08:01.057514+0000","last_fullsized":"2024-05-08T06:08:01.057514+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:51:38.579280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.059465+0000","last_change":"2024-05-08T06:05:51.431518+0000","last_active":"2024-05-08T06:08:01.059465+0000","last_peered":"2024-05-08T06:08:01.059465+0000","last_clean":"2024-05-08T06:08:01.059465+0000","last_became_active":"2024-05-08T06:05:51.431029+0000","last_became_peered":"2024-05-08T06:05:51.431029+0000","last_unstale":"2024-05-08T06:08:01.059465+0000","last_undegraded":"2024-05-08T06:08:01.059465+0000","last_fullsized":"2024-05-08T06:08:01.059465+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:11:09.596221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.061333+0000","last_change":"2024-05-08T06:06:32.498002+0000","last_active":"2024-05-08T06:08:01.061333+0000","last_peered":"2024-05-08T06:08:01.061333+0000","last_clean":"2024-05-08T06:08:01.061333+0000","last_became_active":"2024-05-08T06:06:32.497367+0000","last_became_peered":"2024-05-08T06:06:32.497367+0000","last_unstale":"2024-05-08T06:08:01.061333+0000","last_undegraded":"2024-05-08T06:08:01.061333+0000","last_fullsized":"2024-05-08T06:08:01.061333+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:32:36.461777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.064064+0000","last_change":"2024-05-08T06:05:51.423857+0000","last_active":"2024-05-08T06:08:01.064064+0000","last_peered":"2024-05-08T06:08:01.064064+0000","last_clean":"2024-05-08T06:08:01.064064+0000","last_became_active":"2024-05-08T06:05:51.423487+0000","last_became_peered":"2024-05-08T06:05:51.423487+0000","last_unstale":"2024-05-08T06:08:01.064064+0000","last_undegraded":"2024-05-08T06:08:01.064064+0000","last_fullsized":"2024-05-08T06:08:01.064064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:17:33.269354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.064245+0000","last_change":"2024-05-08T06:05:51.431795+0000","last_active":"2024-05-08T06:08:01.064245+0000","last_peered":"2024-05-08T06:08:01.064245+0000","last_clean":"2024-05-08T06:08:01.064245+0000","last_became_active":"2024-05-08T06:05:51.431432+0000","last_became_peered":"2024-05-08T06:05:51.431432+0000","last_unstale":"2024-05-08T06:08:01.064245+0000","last_undegraded":"2024-05-08T06:08:01.064245+0000","last_fullsized":"2024-05-08T06:08:01.064245+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:11:16.561478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.048666+0000","last_change":"2024-05-08T06:05:51.414805+0000","last_active":"2024-05-08T06:08:01.048666+0000","last_peered":"2024-05-08T06:08:01.048666+0000","last_clean":"2024-05-08T06:08:01.048666+0000","last_became_active":"2024-05-08T06:05:51.414482+0000","last_became_peered":"2024-05-08T06:05:51.414482+0000","last_unstale":"2024-05-08T06:08:01.048666+0000","last_undegraded":"2024-05-08T06:08:01.048666+0000","last_fullsized":"2024-05-08T06:08:01.048666+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:08:13.444591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.064433+0000","last_change":"2024-05-08T06:05:51.418516+0000","last_active":"2024-05-08T06:08:01.064433+0000","last_peered":"2024-05-08T06:08:01.064433+0000","last_clean":"2024-05-08T06:08:01.064433+0000","last_became_active":"2024-05-08T06:05:51.418182+0000","last_became_peered":"2024-05-08T06:05:51.418182+0000","last_unstale":"2024-05-08T06:08:01.064433+0000","last_undegraded":"2024-05-08T06:08:01.064433+0000","last_fullsized":"2024-05-08T06:08:01.064433+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:32:33.338311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482461+0000","last_change":"2024-05-08T06:05:46.350509+0000","last_active":"2024-05-08T06:06:32.482461+0000","last_peered":"2024-05-08T06:06:32.482461+0000","last_clean":"2024-05-08T06:06:32.482461+0000","last_became_active":"2024-05-08T06:05:46.349986+0000","last_became_peered":"2024-05-08T06:05:46.349986+0000","last_unstale":"2024-05-08T06:06:32.482461+0000","last_undegraded":"2024-05-08T06:06:32.482461+0000","last_fullsized":"2024-05-08T06:06:32.482461+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:53:00.928971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.061004+0000","last_change":"2024-05-08T06:05:51.413621+0000","last_active":"2024-05-08T06:08:01.061004+0000","last_peered":"2024-05-08T06:08:01.061004+0000","last_clean":"2024-05-08T06:08:01.061004+0000","last_became_active":"2024-05-08T06:05:51.413188+0000","last_became_peered":"2024-05-08T06:05:51.413188+0000","last_unstale":"2024-05-08T06:08:01.061004+0000","last_undegraded":"2024-05-08T06:08:01.061004+0000","last_fullsized":"2024-05-08T06:08:01.061004+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:13:21.413696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.492763+0000","last_change":"2024-05-08T06:05:46.552444+0000","last_active":"2024-05-08T06:06:32.492763+0000","last_peered":"2024-05-08T06:06:32.492763+0000","last_clean":"2024-05-08T06:06:32.492763+0000","last_became_active":"2024-05-08T06:05:46.551310+0000","last_became_peered":"2024-05-08T06:05:46.551310+0000","last_unstale":"2024-05-08T06:06:32.492763+0000","last_undegraded":"2024-05-08T06:06:32.492763+0000","last_fullsized":"2024-05-08T06:06:32.492763+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:58:17.632832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.062343+0000","last_change":"2024-05-08T06:05:51.425649+0000","last_active":"2024-05-08T06:08:01.062343+0000","last_peered":"2024-05-08T06:08:01.062343+0000","last_clean":"2024-05-08T06:08:01.062343+0000","last_became_active":"2024-05-08T06:05:51.425288+0000","last_became_peered":"2024-05-08T06:05:51.425288+0000","last_unstale":"2024-05-08T06:08:01.062343+0000","last_undegraded":"2024-05-08T06:08:01.062343+0000","last_fullsized":"2024-05-08T06:08:01.062343+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:42:40.580641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480655+0000","last_change":"2024-05-08T06:05:46.350300+0000","last_active":"2024-05-08T06:06:32.480655+0000","last_peered":"2024-05-08T06:06:32.480655+0000","last_clean":"2024-05-08T06:06:32.480655+0000","last_became_active":"2024-05-08T06:05:46.349918+0000","last_became_peered":"2024-05-08T06:05:46.349918+0000","last_unstale":"2024-05-08T06:06:32.480655+0000","last_undegraded":"2024-05-08T06:06:32.480655+0000","last_fullsized":"2024-05-08T06:06:32.480655+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:32:41.952772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.057546+0000","last_change":"2024-05-08T06:05:51.430130+0000","last_active":"2024-05-08T06:08:01.057546+0000","last_peered":"2024-05-08T06:08:01.057546+0000","last_clean":"2024-05-08T06:08:01.057546+0000","last_became_active":"2024-05-08T06:05:51.429794+0000","last_became_peered":"2024-05-08T06:05:51.429794+0000","last_unstale":"2024-05-08T06:08:01.057546+0000","last_undegraded":"2024-05-08T06:08:01.057546+0000","last_fullsized":"2024-05-08T06:08:01.057546+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:21:55.773309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480539+0000","last_change":"2024-05-08T06:05:46.351664+0000","last_active":"2024-05-08T06:06:32.480539+0000","last_peered":"2024-05-08T06:06:32.480539+0000","last_clean":"2024-05-08T06:06:32.480539+0000","last_became_active":"2024-05-08T06:05:46.351305+0000","last_became_peered":"2024-05-08T06:05:46.351305+0000","last_unstale":"2024-05-08T06:06:32.480539+0000","last_undegraded":"2024-05-08T06:06:32.480539+0000","last_fullsized":"2024-05-08T06:06:32.480539+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:24:36.164315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.043418+0000","last_change":"2024-05-08T06:05:51.414793+0000","last_active":"2024-05-08T06:08:01.043418+0000","last_peered":"2024-05-08T06:08:01.043418+0000","last_clean":"2024-05-08T06:08:01.043418+0000","last_became_active":"2024-05-08T06:05:51.414317+0000","last_became_peered":"2024-05-08T06:05:51.414317+0000","last_unstale":"2024-05-08T06:08:01.043418+0000","last_undegraded":"2024-05-08T06:08:01.043418+0000","last_fullsized":"2024-05-08T06:08:01.043418+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:57:56.152836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":170,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:32.584506+0000","last_change":"2024-05-08T06:05:46.352683+0000","last_active":"2024-05-08T06:08:32.584506+0000","last_peered":"2024-05-08T06:08:32.584506+0000","last_clean":"2024-05-08T06:08:32.584506+0000","last_became_active":"2024-05-08T06:05:46.352278+0000","last_became_peered":"2024-05-08T06:05:46.352278+0000","last_unstale":"2024-05-08T06:08:32.584506+0000","last_undegraded":"2024-05-08T06:08:32.584506+0000","last_fullsized":"2024-05-08T06:08:32.584506+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:39:22.655516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":138,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.037458+0000","last_change":"2024-05-08T06:05:51.415822+0000","last_active":"2024-05-08T06:08:01.037458+0000","last_peered":"2024-05-08T06:08:01.037458+0000","last_clean":"2024-05-08T06:08:01.037458+0000","last_became_active":"2024-05-08T06:05:51.415449+0000","last_became_peered":"2024-05-08T06:05:51.415449+0000","last_unstale":"2024-05-08T06:08:01.037458+0000","last_undegraded":"2024-05-08T06:08:01.037458+0000","last_fullsized":"2024-05-08T06:08:01.037458+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:44:49.127335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.478073+0000","last_change":"2024-05-08T06:05:46.361288+0000","last_active":"2024-05-08T06:06:32.478073+0000","last_peered":"2024-05-08T06:06:32.478073+0000","last_clean":"2024-05-08T06:06:32.478073+0000","last_became_active":"2024-05-08T06:05:46.360761+0000","last_became_peered":"2024-05-08T06:05:46.360761+0000","last_unstale":"2024-05-08T06:06:32.478073+0000","last_undegraded":"2024-05-08T06:06:32.478073+0000","last_fullsized":"2024-05-08T06:06:32.478073+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:14:59.858935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.063490+0000","last_change":"2024-05-08T06:05:51.418547+0000","last_active":"2024-05-08T06:08:01.063490+0000","last_peered":"2024-05-08T06:08:01.063490+0000","last_clean":"2024-05-08T06:08:01.063490+0000","last_became_active":"2024-05-08T06:05:51.417588+0000","last_became_peered":"2024-05-08T06:05:51.417588+0000","last_unstale":"2024-05-08T06:08:01.063490+0000","last_undegraded":"2024-05-08T06:08:01.063490+0000","last_fullsized":"2024-05-08T06:08:01.063490+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:16:41.451351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.492080+0000","last_change":"2024-05-08T06:05:46.362077+0000","last_active":"2024-05-08T06:06:32.492080+0000","last_peered":"2024-05-08T06:06:32.492080+0000","last_clean":"2024-05-08T06:06:32.492080+0000","last_became_active":"2024-05-08T06:05:46.360823+0000","last_became_peered":"2024-05-08T06:05:46.360823+0000","last_unstale":"2024-05-08T06:06:32.492080+0000","last_undegraded":"2024-05-08T06:06:32.492080+0000","last_fullsized":"2024-05-08T06:06:32.492080+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:14.169822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.063350+0000","last_change":"2024-05-08T06:05:51.423687+0000","last_active":"2024-05-08T06:08:01.063350+0000","last_peered":"2024-05-08T06:08:01.063350+0000","last_clean":"2024-05-08T06:08:01.063350+0000","last_became_active":"2024-05-08T06:05:51.417656+0000","last_became_peered":"2024-05-08T06:05:51.417656+0000","last_unstale":"2024-05-08T06:08:01.063350+0000","last_undegraded":"2024-05-08T06:08:01.063350+0000","last_fullsized":"2024-05-08T06:08:01.063350+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:59:31.319915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.595803+0000","last_change":"2024-05-08T06:05:46.351892+0000","last_active":"2024-05-08T06:06:32.595803+0000","last_peered":"2024-05-08T06:06:32.595803+0000","last_clean":"2024-05-08T06:06:32.595803+0000","last_became_active":"2024-05-08T06:05:46.351523+0000","last_became_peered":"2024-05-08T06:05:46.351523+0000","last_unstale":"2024-05-08T06:06:32.595803+0000","last_undegraded":"2024-05-08T06:06:32.595803+0000","last_fullsized":"2024-05-08T06:06:32.595803+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:11:23.794313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.063933+0000","last_change":"2024-05-08T06:05:51.407681+0000","last_active":"2024-05-08T06:08:01.063933+0000","last_peered":"2024-05-08T06:08:01.063933+0000","last_clean":"2024-05-08T06:08:01.063933+0000","last_became_active":"2024-05-08T06:05:51.407243+0000","last_became_peered":"2024-05-08T06:05:51.407243+0000","last_unstale":"2024-05-08T06:08:01.063933+0000","last_undegraded":"2024-05-08T06:08:01.063933+0000","last_fullsized":"2024-05-08T06:08:01.063933+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:06:30.490268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'3","reported_seq":165,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:31.582605+0000","last_change":"2024-05-08T06:05:46.356675+0000","last_active":"2024-05-08T06:08:31.582605+0000","last_peered":"2024-05-08T06:08:31.582605+0000","last_clean":"2024-05-08T06:08:31.582605+0000","last_became_active":"2024-05-08T06:05:46.356360+0000","last_became_peered":"2024-05-08T06:05:46.356360+0000","last_unstale":"2024-05-08T06:08:31.582605+0000","last_undegraded":"2024-05-08T06:08:31.582605+0000","last_fullsized":"2024-05-08T06:08:31.582605+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:46:48.131551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":135,"num_read_kb":135,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.047752+0000","last_change":"2024-05-08T06:05:51.421325+0000","last_active":"2024-05-08T06:08:01.047752+0000","last_peered":"2024-05-08T06:08:01.047752+0000","last_clean":"2024-05-08T06:08:01.047752+0000","last_became_active":"2024-05-08T06:05:51.420905+0000","last_became_peered":"2024-05-08T06:05:51.420905+0000","last_unstale":"2024-05-08T06:08:01.047752+0000","last_undegraded":"2024-05-08T06:08:01.047752+0000","last_fullsized":"2024-05-08T06:08:01.047752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:37:27.324920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482723+0000","last_change":"2024-05-08T06:05:46.351958+0000","last_active":"2024-05-08T06:06:32.482723+0000","last_peered":"2024-05-08T06:06:32.482723+0000","last_clean":"2024-05-08T06:06:32.482723+0000","last_became_active":"2024-05-08T06:05:46.351561+0000","last_became_peered":"2024-05-08T06:05:46.351561+0000","last_unstale":"2024-05-08T06:06:32.482723+0000","last_undegraded":"2024-05-08T06:06:32.482723+0000","last_fullsized":"2024-05-08T06:06:32.482723+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:11:27.727029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.061468+0000","last_change":"2024-05-08T06:05:51.431809+0000","last_active":"2024-05-08T06:08:01.061468+0000","last_peered":"2024-05-08T06:08:01.061468+0000","last_clean":"2024-05-08T06:08:01.061468+0000","last_became_active":"2024-05-08T06:05:51.431416+0000","last_became_peered":"2024-05-08T06:05:51.431416+0000","last_unstale":"2024-05-08T06:08:01.061468+0000","last_undegraded":"2024-05-08T06:08:01.061468+0000","last_fullsized":"2024-05-08T06:08:01.061468+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:48:16.704134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.596352+0000","last_change":"2024-05-08T06:05:46.550763+0000","last_active":"2024-05-08T06:06:32.596352+0000","last_peered":"2024-05-08T06:06:32.596352+0000","last_clean":"2024-05-08T06:06:32.596352+0000","last_became_active":"2024-05-08T06:05:46.550297+0000","last_became_peered":"2024-05-08T06:05:46.550297+0000","last_unstale":"2024-05-08T06:06:32.596352+0000","last_undegraded":"2024-05-08T06:06:32.596352+0000","last_fullsized":"2024-05-08T06:06:32.596352+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:32:19.907220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.051960+0000","last_change":"2024-05-08T06:05:51.414738+0000","last_active":"2024-05-08T06:08:01.051960+0000","last_peered":"2024-05-08T06:08:01.051960+0000","last_clean":"2024-05-08T06:08:01.051960+0000","last_became_active":"2024-05-08T06:05:51.414369+0000","last_became_peered":"2024-05-08T06:05:51.414369+0000","last_unstale":"2024-05-08T06:08:01.051960+0000","last_undegraded":"2024-05-08T06:08:01.051960+0000","last_fullsized":"2024-05-08T06:08:01.051960+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:57:44.215343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'35","reported_seq":343,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:32.583853+0000","last_change":"2024-05-08T06:05:46.349825+0000","last_active":"2024-05-08T06:08:32.583853+0000","last_peered":"2024-05-08T06:08:32.583853+0000","last_clean":"2024-05-08T06:08:32.583853+0000","last_became_active":"2024-05-08T06:05:46.349425+0000","last_became_peered":"2024-05-08T06:05:46.349425+0000","last_unstale":"2024-05-08T06:08:32.583853+0000","last_undegraded":"2024-05-08T06:08:32.583853+0000","last_fullsized":"2024-05-08T06:08:32.583853+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:29:39.123817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64459,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":281,"num_read_kb":204,"num_write":36,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.022668+0000","last_change":"2024-05-08T06:05:51.411341+0000","last_active":"2024-05-08T06:08:01.022668+0000","last_peered":"2024-05-08T06:08:01.022668+0000","last_clean":"2024-05-08T06:08:01.022668+0000","last_became_active":"2024-05-08T06:05:51.411002+0000","last_became_peered":"2024-05-08T06:05:51.411002+0000","last_unstale":"2024-05-08T06:08:01.022668+0000","last_undegraded":"2024-05-08T06:08:01.022668+0000","last_fullsized":"2024-05-08T06:08:01.022668+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:27:07.013809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.509330+0000","last_change":"2024-05-08T06:06:32.508904+0000","last_active":"2024-05-08T06:06:32.509330+0000","last_peered":"2024-05-08T06:06:32.509330+0000","last_clean":"2024-05-08T06:06:32.509330+0000","last_became_active":"2024-05-08T06:06:32.496001+0000","last_became_peered":"2024-05-08T06:06:32.496001+0000","last_unstale":"2024-05-08T06:06:32.509330+0000","last_undegraded":"2024-05-08T06:06:32.509330+0000","last_fullsized":"2024-05-08T06:06:32.509330+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:34:32.417105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.065212+0000","last_change":"2024-05-08T06:05:51.418395+0000","last_active":"2024-05-08T06:08:01.065212+0000","last_peered":"2024-05-08T06:08:01.065212+0000","last_clean":"2024-05-08T06:08:01.065212+0000","last_became_active":"2024-05-08T06:05:51.418035+0000","last_became_peered":"2024-05-08T06:05:51.418035+0000","last_unstale":"2024-05-08T06:08:01.065212+0000","last_undegraded":"2024-05-08T06:08:01.065212+0000","last_fullsized":"2024-05-08T06:08:01.065212+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:58:16.529144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.595996+0000","last_change":"2024-05-08T06:05:46.363088+0000","last_active":"2024-05-08T06:06:32.595996+0000","last_peered":"2024-05-08T06:06:32.595996+0000","last_clean":"2024-05-08T06:06:32.595996+0000","last_became_active":"2024-05-08T06:05:46.362488+0000","last_became_peered":"2024-05-08T06:05:46.362488+0000","last_unstale":"2024-05-08T06:06:32.595996+0000","last_undegraded":"2024-05-08T06:06:32.595996+0000","last_fullsized":"2024-05-08T06:06:32.595996+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:28:10.039647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.059203+0000","last_change":"2024-05-08T06:05:51.415567+0000","last_active":"2024-05-08T06:08:01.059203+0000","last_peered":"2024-05-08T06:08:01.059203+0000","last_clean":"2024-05-08T06:08:01.059203+0000","last_became_active":"2024-05-08T06:05:51.415148+0000","last_became_peered":"2024-05-08T06:05:51.415148+0000","last_unstale":"2024-05-08T06:08:01.059203+0000","last_undegraded":"2024-05-08T06:08:01.059203+0000","last_fullsized":"2024-05-08T06:08:01.059203+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:51:48.871149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":169,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:29.677457+0000","last_change":"2024-05-08T06:05:46.348769+0000","last_active":"2024-05-08T06:08:29.677457+0000","last_peered":"2024-05-08T06:08:29.677457+0000","last_clean":"2024-05-08T06:08:29.677457+0000","last_became_active":"2024-05-08T06:05:46.348072+0000","last_became_peered":"2024-05-08T06:05:46.348072+0000","last_unstale":"2024-05-08T06:08:29.677457+0000","last_undegraded":"2024-05-08T06:08:29.677457+0000","last_fullsized":"2024-05-08T06:08:29.677457+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:45.292436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":138,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.049147+0000","last_change":"2024-05-08T06:05:51.414623+0000","last_active":"2024-05-08T06:08:01.049147+0000","last_peered":"2024-05-08T06:08:01.049147+0000","last_clean":"2024-05-08T06:08:01.049147+0000","last_became_active":"2024-05-08T06:05:51.414258+0000","last_became_peered":"2024-05-08T06:05:51.414258+0000","last_unstale":"2024-05-08T06:08:01.049147+0000","last_undegraded":"2024-05-08T06:08:01.049147+0000","last_fullsized":"2024-05-08T06:08:01.049147+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:38:48.297900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":43,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480030+0000","last_change":"2024-05-08T06:05:46.546936+0000","last_active":"2024-05-08T06:06:32.480030+0000","last_peered":"2024-05-08T06:06:32.480030+0000","last_clean":"2024-05-08T06:06:32.480030+0000","last_became_active":"2024-05-08T06:05:46.546574+0000","last_became_peered":"2024-05-08T06:05:46.546574+0000","last_unstale":"2024-05-08T06:06:32.480030+0000","last_undegraded":"2024-05-08T06:06:32.480030+0000","last_fullsized":"2024-05-08T06:06:32.480030+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:16:29.720304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.029792+0000","last_change":"2024-05-08T06:05:51.418012+0000","last_active":"2024-05-08T06:08:01.029792+0000","last_peered":"2024-05-08T06:08:01.029792+0000","last_clean":"2024-05-08T06:08:01.029792+0000","last_became_active":"2024-05-08T06:05:51.417646+0000","last_became_peered":"2024-05-08T06:05:51.417646+0000","last_unstale":"2024-05-08T06:08:01.029792+0000","last_undegraded":"2024-05-08T06:08:01.029792+0000","last_fullsized":"2024-05-08T06:08:01.029792+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:43:02.733967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480265+0000","last_change":"2024-05-08T06:05:46.347748+0000","last_active":"2024-05-08T06:06:32.480265+0000","last_peered":"2024-05-08T06:06:32.480265+0000","last_clean":"2024-05-08T06:06:32.480265+0000","last_became_active":"2024-05-08T06:05:46.347367+0000","last_became_peered":"2024-05-08T06:05:46.347367+0000","last_unstale":"2024-05-08T06:06:32.480265+0000","last_undegraded":"2024-05-08T06:06:32.480265+0000","last_fullsized":"2024-05-08T06:06:32.480265+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:51:50.973855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.038216+0000","last_change":"2024-05-08T06:06:32.597018+0000","last_active":"2024-05-08T06:08:01.038216+0000","last_peered":"2024-05-08T06:08:01.038216+0000","last_clean":"2024-05-08T06:08:01.038216+0000","last_became_active":"2024-05-08T06:06:32.596677+0000","last_became_peered":"2024-05-08T06:06:32.596677+0000","last_unstale":"2024-05-08T06:08:01.038216+0000","last_undegraded":"2024-05-08T06:08:01.038216+0000","last_fullsized":"2024-05-08T06:08:01.038216+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:02:46.223949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.481700+0000","last_change":"2024-05-08T06:05:46.554443+0000","last_active":"2024-05-08T06:06:32.481700+0000","last_peered":"2024-05-08T06:06:32.481700+0000","last_clean":"2024-05-08T06:06:32.481700+0000","last_became_active":"2024-05-08T06:05:46.553226+0000","last_became_peered":"2024-05-08T06:05:46.553226+0000","last_unstale":"2024-05-08T06:06:32.481700+0000","last_undegraded":"2024-05-08T06:06:32.481700+0000","last_fullsized":"2024-05-08T06:06:32.481700+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:56:39.784652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.060786+0000","last_change":"2024-05-08T06:05:51.413121+0000","last_active":"2024-05-08T06:08:01.060786+0000","last_peered":"2024-05-08T06:08:01.060786+0000","last_clean":"2024-05-08T06:08:01.060786+0000","last_became_active":"2024-05-08T06:05:51.412767+0000","last_became_peered":"2024-05-08T06:05:51.412767+0000","last_unstale":"2024-05-08T06:08:01.060786+0000","last_undegraded":"2024-05-08T06:08:01.060786+0000","last_fullsized":"2024-05-08T06:08:01.060786+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:59:13.416981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482457+0000","last_change":"2024-05-08T06:05:46.361349+0000","last_active":"2024-05-08T06:06:32.482457+0000","last_peered":"2024-05-08T06:06:32.482457+0000","last_clean":"2024-05-08T06:06:32.482457+0000","last_became_active":"2024-05-08T06:05:46.360654+0000","last_became_peered":"2024-05-08T06:05:46.360654+0000","last_unstale":"2024-05-08T06:06:32.482457+0000","last_undegraded":"2024-05-08T06:06:32.482457+0000","last_fullsized":"2024-05-08T06:06:32.482457+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:19:22.144874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.062066+0000","last_change":"2024-05-08T06:05:51.423836+0000","last_active":"2024-05-08T06:08:01.062066+0000","last_peered":"2024-05-08T06:08:01.062066+0000","last_clean":"2024-05-08T06:08:01.062066+0000","last_became_active":"2024-05-08T06:05:51.423424+0000","last_became_peered":"2024-05-08T06:05:51.423424+0000","last_unstale":"2024-05-08T06:08:01.062066+0000","last_undegraded":"2024-05-08T06:08:01.062066+0000","last_fullsized":"2024-05-08T06:08:01.062066+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:56:14.779765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.048549+0000","last_change":"2024-05-08T06:06:32.598978+0000","last_active":"2024-05-08T06:08:01.048549+0000","last_peered":"2024-05-08T06:08:01.048549+0000","last_clean":"2024-05-08T06:08:01.048549+0000","last_became_active":"2024-05-08T06:06:32.598158+0000","last_became_peered":"2024-05-08T06:06:32.598158+0000","last_unstale":"2024-05-08T06:08:01.048549+0000","last_undegraded":"2024-05-08T06:08:01.048549+0000","last_fullsized":"2024-05-08T06:08:01.048549+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:51:09.270066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":165,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:29.676599+0000","last_change":"2024-05-08T06:05:46.547450+0000","last_active":"2024-05-08T06:08:29.676599+0000","last_peered":"2024-05-08T06:08:29.676599+0000","last_clean":"2024-05-08T06:08:29.676599+0000","last_became_active":"2024-05-08T06:05:46.546444+0000","last_became_peered":"2024-05-08T06:05:46.546444+0000","last_unstale":"2024-05-08T06:08:29.676599+0000","last_undegraded":"2024-05-08T06:08:29.676599+0000","last_fullsized":"2024-05-08T06:08:29.676599+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:17:14.124224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":133,"num_read_kb":133,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.065170+0000","last_change":"2024-05-08T06:05:51.423191+0000","last_active":"2024-05-08T06:08:01.065170+0000","last_peered":"2024-05-08T06:08:01.065170+0000","last_clean":"2024-05-08T06:08:01.065170+0000","last_became_active":"2024-05-08T06:05:51.420912+0000","last_became_peered":"2024-05-08T06:05:51.420912+0000","last_unstale":"2024-05-08T06:08:01.065170+0000","last_undegraded":"2024-05-08T06:08:01.065170+0000","last_fullsized":"2024-05-08T06:08:01.065170+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:27:57.678527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.597388+0000","last_change":"2024-05-08T06:05:46.348650+0000","last_active":"2024-05-08T06:06:32.597388+0000","last_peered":"2024-05-08T06:06:32.597388+0000","last_clean":"2024-05-08T06:06:32.597388+0000","last_became_active":"2024-05-08T06:05:46.347266+0000","last_became_peered":"2024-05-08T06:05:46.347266+0000","last_unstale":"2024-05-08T06:06:32.597388+0000","last_undegraded":"2024-05-08T06:06:32.597388+0000","last_fullsized":"2024-05-08T06:06:32.597388+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:37:30.623580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":100,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482311+0000","last_change":"2024-05-08T06:04:09.404663+0000","last_active":"2024-05-08T06:06:32.482311+0000","last_peered":"2024-05-08T06:06:32.482311+0000","last_clean":"2024-05-08T06:06:32.482311+0000","last_became_active":"2024-05-08T06:04:09.400115+0000","last_became_peered":"2024-05-08T06:04:09.400115+0000","last_unstale":"2024-05-08T06:06:32.482311+0000","last_undegraded":"2024-05-08T06:06:32.482311+0000","last_fullsized":"2024-05-08T06:06:32.482311+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:00:17.419712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:00:17.419712+0000","last_clean_scrub_stamp":"2024-05-08T06:00:17.419712+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:27:39.154694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.053460+0000","last_change":"2024-05-08T06:05:51.421826+0000","last_active":"2024-05-08T06:08:01.053460+0000","last_peered":"2024-05-08T06:08:01.053460+0000","last_clean":"2024-05-08T06:08:01.053460+0000","last_became_active":"2024-05-08T06:05:51.420706+0000","last_became_peered":"2024-05-08T06:05:51.420706+0000","last_unstale":"2024-05-08T06:08:01.053460+0000","last_undegraded":"2024-05-08T06:08:01.053460+0000","last_fullsized":"2024-05-08T06:08:01.053460+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:06:16.242332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482383+0000","last_change":"2024-05-08T06:05:46.351365+0000","last_active":"2024-05-08T06:06:32.482383+0000","last_peered":"2024-05-08T06:06:32.482383+0000","last_clean":"2024-05-08T06:06:32.482383+0000","last_became_active":"2024-05-08T06:05:46.351022+0000","last_became_peered":"2024-05-08T06:05:46.351022+0000","last_unstale":"2024-05-08T06:06:32.482383+0000","last_undegraded":"2024-05-08T06:06:32.482383+0000","last_fullsized":"2024-05-08T06:06:32.482383+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:32:44.134994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.501668+0000","last_change":"2024-05-08T06:06:32.501263+0000","last_active":"2024-05-08T06:06:32.501668+0000","last_peered":"2024-05-08T06:06:32.501668+0000","last_clean":"2024-05-08T06:06:32.501668+0000","last_became_active":"2024-05-08T06:06:32.493795+0000","last_became_peered":"2024-05-08T06:06:32.493795+0000","last_unstale":"2024-05-08T06:06:32.501668+0000","last_undegraded":"2024-05-08T06:06:32.501668+0000","last_fullsized":"2024-05-08T06:06:32.501668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:49:03.898162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.049894+0000","last_change":"2024-05-08T06:05:51.428382+0000","last_active":"2024-05-08T06:08:01.049894+0000","last_peered":"2024-05-08T06:08:01.049894+0000","last_clean":"2024-05-08T06:08:01.049894+0000","last_became_active":"2024-05-08T06:05:51.421443+0000","last_became_peered":"2024-05-08T06:05:51.421443+0000","last_unstale":"2024-05-08T06:08:01.049894+0000","last_undegraded":"2024-05-08T06:08:01.049894+0000","last_fullsized":"2024-05-08T06:08:01.049894+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:26:10.215020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'13","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:07:56.949283+0000","last_change":"2024-05-08T06:05:46.364306+0000","last_active":"2024-05-08T06:07:56.949283+0000","last_peered":"2024-05-08T06:07:56.949283+0000","last_clean":"2024-05-08T06:07:56.949283+0000","last_became_active":"2024-05-08T06:05:46.363871+0000","last_became_peered":"2024-05-08T06:05:46.363871+0000","last_unstale":"2024-05-08T06:07:56.949283+0000","last_undegraded":"2024-05-08T06:07:56.949283+0000","last_fullsized":"2024-05-08T06:07:56.949283+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:38:34.531162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.061822+0000","last_change":"2024-05-08T06:05:51.415020+0000","last_active":"2024-05-08T06:08:01.061822+0000","last_peered":"2024-05-08T06:08:01.061822+0000","last_clean":"2024-05-08T06:08:01.061822+0000","last_became_active":"2024-05-08T06:05:51.413690+0000","last_became_peered":"2024-05-08T06:05:51.413690+0000","last_unstale":"2024-05-08T06:08:01.061822+0000","last_undegraded":"2024-05-08T06:08:01.061822+0000","last_fullsized":"2024-05-08T06:08:01.061822+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:55:38.121492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.597283+0000","last_change":"2024-05-08T06:05:46.359100+0000","last_active":"2024-05-08T06:06:32.597283+0000","last_peered":"2024-05-08T06:06:32.597283+0000","last_clean":"2024-05-08T06:06:32.597283+0000","last_became_active":"2024-05-08T06:05:46.358815+0000","last_became_peered":"2024-05-08T06:05:46.358815+0000","last_unstale":"2024-05-08T06:06:32.597283+0000","last_undegraded":"2024-05-08T06:06:32.597283+0000","last_fullsized":"2024-05-08T06:06:32.597283+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:22:58.532358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.064793+0000","last_change":"2024-05-08T06:05:51.429222+0000","last_active":"2024-05-08T06:08:01.064793+0000","last_peered":"2024-05-08T06:08:01.064793+0000","last_clean":"2024-05-08T06:08:01.064793+0000","last_became_active":"2024-05-08T06:05:51.428845+0000","last_became_peered":"2024-05-08T06:05:51.428845+0000","last_unstale":"2024-05-08T06:08:01.064793+0000","last_undegraded":"2024-05-08T06:08:01.064793+0000","last_fullsized":"2024-05-08T06:08:01.064793+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:36:41.435666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.596086+0000","last_change":"2024-05-08T06:05:46.547931+0000","last_active":"2024-05-08T06:06:32.596086+0000","last_peered":"2024-05-08T06:06:32.596086+0000","last_clean":"2024-05-08T06:06:32.596086+0000","last_became_active":"2024-05-08T06:05:46.547547+0000","last_became_peered":"2024-05-08T06:05:46.547547+0000","last_unstale":"2024-05-08T06:06:32.596086+0000","last_undegraded":"2024-05-08T06:06:32.596086+0000","last_fullsized":"2024-05-08T06:06:32.596086+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:27:03.502153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.015273+0000","last_change":"2024-05-08T06:05:51.413259+0000","last_active":"2024-05-08T06:08:01.015273+0000","last_peered":"2024-05-08T06:08:01.015273+0000","last_clean":"2024-05-08T06:08:01.015273+0000","last_became_active":"2024-05-08T06:05:51.412890+0000","last_became_peered":"2024-05-08T06:05:51.412890+0000","last_unstale":"2024-05-08T06:08:01.015273+0000","last_undegraded":"2024-05-08T06:08:01.015273+0000","last_fullsized":"2024-05-08T06:08:01.015273+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:04:56.690098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.489841+0000","last_change":"2024-05-08T06:05:46.365409+0000","last_active":"2024-05-08T06:06:32.489841+0000","last_peered":"2024-05-08T06:06:32.489841+0000","last_clean":"2024-05-08T06:06:32.489841+0000","last_became_active":"2024-05-08T06:05:46.361277+0000","last_became_peered":"2024-05-08T06:05:46.361277+0000","last_unstale":"2024-05-08T06:06:32.489841+0000","last_undegraded":"2024-05-08T06:06:32.489841+0000","last_fullsized":"2024-05-08T06:06:32.489841+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:06.927385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.063724+0000","last_change":"2024-05-08T06:05:51.423749+0000","last_active":"2024-05-08T06:08:01.063724+0000","last_peered":"2024-05-08T06:08:01.063724+0000","last_clean":"2024-05-08T06:08:01.063724+0000","last_became_active":"2024-05-08T06:05:51.423386+0000","last_became_peered":"2024-05-08T06:05:51.423386+0000","last_unstale":"2024-05-08T06:08:01.063724+0000","last_undegraded":"2024-05-08T06:08:01.063724+0000","last_fullsized":"2024-05-08T06:08:01.063724+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:41:26.261683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480440+0000","last_change":"2024-05-08T06:05:46.362497+0000","last_active":"2024-05-08T06:06:32.480440+0000","last_peered":"2024-05-08T06:06:32.480440+0000","last_clean":"2024-05-08T06:06:32.480440+0000","last_became_active":"2024-05-08T06:05:46.362110+0000","last_became_peered":"2024-05-08T06:05:46.362110+0000","last_unstale":"2024-05-08T06:06:32.480440+0000","last_undegraded":"2024-05-08T06:06:32.480440+0000","last_fullsized":"2024-05-08T06:06:32.480440+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:25:06.205380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.060664+0000","last_change":"2024-05-08T06:06:32.495078+0000","last_active":"2024-05-08T06:08:01.060664+0000","last_peered":"2024-05-08T06:08:01.060664+0000","last_clean":"2024-05-08T06:08:01.060664+0000","last_became_active":"2024-05-08T06:06:32.493855+0000","last_became_peered":"2024-05-08T06:06:32.493855+0000","last_unstale":"2024-05-08T06:08:01.060664+0000","last_undegraded":"2024-05-08T06:08:01.060664+0000","last_fullsized":"2024-05-08T06:08:01.060664+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:56:54.143080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.602199+0000","last_change":"2024-05-08T06:06:32.601826+0000","last_active":"2024-05-08T06:06:32.602199+0000","last_peered":"2024-05-08T06:06:32.602199+0000","last_clean":"2024-05-08T06:06:32.602199+0000","last_became_active":"2024-05-08T06:06:32.597823+0000","last_became_peered":"2024-05-08T06:06:32.597823+0000","last_unstale":"2024-05-08T06:06:32.602199+0000","last_undegraded":"2024-05-08T06:06:32.602199+0000","last_fullsized":"2024-05-08T06:06:32.602199+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:59:09.741096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.057017+0000","last_change":"2024-05-08T06:05:51.413744+0000","last_active":"2024-05-08T06:08:01.057017+0000","last_peered":"2024-05-08T06:08:01.057017+0000","last_clean":"2024-05-08T06:08:01.057017+0000","last_became_active":"2024-05-08T06:05:51.413346+0000","last_became_peered":"2024-05-08T06:05:51.413346+0000","last_unstale":"2024-05-08T06:08:01.057017+0000","last_undegraded":"2024-05-08T06:08:01.057017+0000","last_fullsized":"2024-05-08T06:08:01.057017+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:46:56.109755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480947+0000","last_change":"2024-05-08T06:05:46.552604+0000","last_active":"2024-05-08T06:06:32.480947+0000","last_peered":"2024-05-08T06:06:32.480947+0000","last_clean":"2024-05-08T06:06:32.480947+0000","last_became_active":"2024-05-08T06:05:46.552248+0000","last_became_peered":"2024-05-08T06:05:46.552248+0000","last_unstale":"2024-05-08T06:06:32.480947+0000","last_undegraded":"2024-05-08T06:06:32.480947+0000","last_fullsized":"2024-05-08T06:06:32.480947+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:39:10.306692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.054186+0000","last_change":"2024-05-08T06:05:51.404988+0000","last_active":"2024-05-08T06:08:01.054186+0000","last_peered":"2024-05-08T06:08:01.054186+0000","last_clean":"2024-05-08T06:08:01.054186+0000","last_became_active":"2024-05-08T06:05:51.404625+0000","last_became_peered":"2024-05-08T06:05:51.404625+0000","last_unstale":"2024-05-08T06:08:01.054186+0000","last_undegraded":"2024-05-08T06:08:01.054186+0000","last_fullsized":"2024-05-08T06:08:01.054186+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:33:27.919929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.596178+0000","last_change":"2024-05-08T06:05:46.361466+0000","last_active":"2024-05-08T06:06:32.596178+0000","last_peered":"2024-05-08T06:06:32.596178+0000","last_clean":"2024-05-08T06:06:32.596178+0000","last_became_active":"2024-05-08T06:05:46.361107+0000","last_became_peered":"2024-05-08T06:05:46.361107+0000","last_unstale":"2024-05-08T06:06:32.596178+0000","last_undegraded":"2024-05-08T06:06:32.596178+0000","last_fullsized":"2024-05-08T06:06:32.596178+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:15:31.314226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.062827+0000","last_change":"2024-05-08T06:05:51.416437+0000","last_active":"2024-05-08T06:08:01.062827+0000","last_peered":"2024-05-08T06:08:01.062827+0000","last_clean":"2024-05-08T06:08:01.062827+0000","last_became_active":"2024-05-08T06:05:51.416070+0000","last_became_peered":"2024-05-08T06:05:51.416070+0000","last_unstale":"2024-05-08T06:08:01.062827+0000","last_undegraded":"2024-05-08T06:08:01.062827+0000","last_fullsized":"2024-05-08T06:08:01.062827+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:35:23.652373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.493014+0000","last_change":"2024-05-08T06:05:46.362475+0000","last_active":"2024-05-08T06:06:32.493014+0000","last_peered":"2024-05-08T06:06:32.493014+0000","last_clean":"2024-05-08T06:06:32.493014+0000","last_became_active":"2024-05-08T06:05:46.361852+0000","last_became_peered":"2024-05-08T06:05:46.361852+0000","last_unstale":"2024-05-08T06:06:32.493014+0000","last_undegraded":"2024-05-08T06:06:32.493014+0000","last_fullsized":"2024-05-08T06:06:32.493014+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:16:54.536330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.049399+0000","last_change":"2024-05-08T06:05:51.410718+0000","last_active":"2024-05-08T06:08:01.049399+0000","last_peered":"2024-05-08T06:08:01.049399+0000","last_clean":"2024-05-08T06:08:01.049399+0000","last_became_active":"2024-05-08T06:05:51.408902+0000","last_became_peered":"2024-05-08T06:05:51.408902+0000","last_unstale":"2024-05-08T06:08:01.049399+0000","last_undegraded":"2024-05-08T06:08:01.049399+0000","last_fullsized":"2024-05-08T06:08:01.049399+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:23:33.857677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.038950+0000","last_change":"2024-05-08T06:05:51.414362+0000","last_active":"2024-05-08T06:08:01.038950+0000","last_peered":"2024-05-08T06:08:01.038950+0000","last_clean":"2024-05-08T06:08:01.038950+0000","last_became_active":"2024-05-08T06:05:51.413998+0000","last_became_peered":"2024-05-08T06:05:51.413998+0000","last_unstale":"2024-05-08T06:08:01.038950+0000","last_undegraded":"2024-05-08T06:08:01.038950+0000","last_fullsized":"2024-05-08T06:08:01.038950+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:38:22.198219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.032457+0000","last_change":"2024-05-08T06:05:51.415508+0000","last_active":"2024-05-08T06:08:01.032457+0000","last_peered":"2024-05-08T06:08:01.032457+0000","last_clean":"2024-05-08T06:08:01.032457+0000","last_became_active":"2024-05-08T06:05:51.415137+0000","last_became_peered":"2024-05-08T06:05:51.415137+0000","last_unstale":"2024-05-08T06:08:01.032457+0000","last_undegraded":"2024-05-08T06:08:01.032457+0000","last_fullsized":"2024-05-08T06:08:01.032457+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:15.007996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.064306+0000","last_change":"2024-05-08T06:05:51.422093+0000","last_active":"2024-05-08T06:08:01.064306+0000","last_peered":"2024-05-08T06:08:01.064306+0000","last_clean":"2024-05-08T06:08:01.064306+0000","last_became_active":"2024-05-08T06:05:51.421733+0000","last_became_peered":"2024-05-08T06:05:51.421733+0000","last_unstale":"2024-05-08T06:08:01.064306+0000","last_undegraded":"2024-05-08T06:08:01.064306+0000","last_fullsized":"2024-05-08T06:08:01.064306+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:35:55.950229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.060949+0000","last_change":"2024-05-08T06:05:51.434705+0000","last_active":"2024-05-08T06:08:01.060949+0000","last_peered":"2024-05-08T06:08:01.060949+0000","last_clean":"2024-05-08T06:08:01.060949+0000","last_became_active":"2024-05-08T06:05:51.434324+0000","last_became_peered":"2024-05-08T06:05:51.434324+0000","last_unstale":"2024-05-08T06:08:01.060949+0000","last_undegraded":"2024-05-08T06:08:01.060949+0000","last_fullsized":"2024-05-08T06:08:01.060949+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:38:23.101247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.063946+0000","last_change":"2024-05-08T06:06:32.499837+0000","last_active":"2024-05-08T06:08:01.063946+0000","last_peered":"2024-05-08T06:08:01.063946+0000","last_clean":"2024-05-08T06:08:01.063946+0000","last_became_active":"2024-05-08T06:06:32.499026+0000","last_became_peered":"2024-05-08T06:06:32.499026+0000","last_unstale":"2024-05-08T06:08:01.063946+0000","last_undegraded":"2024-05-08T06:08:01.063946+0000","last_fullsized":"2024-05-08T06:08:01.063946+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:48:51.910384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.038524+0000","last_change":"2024-05-08T06:05:51.412370+0000","last_active":"2024-05-08T06:08:01.038524+0000","last_peered":"2024-05-08T06:08:01.038524+0000","last_clean":"2024-05-08T06:08:01.038524+0000","last_became_active":"2024-05-08T06:05:51.412057+0000","last_became_peered":"2024-05-08T06:05:51.412057+0000","last_unstale":"2024-05-08T06:08:01.038524+0000","last_undegraded":"2024-05-08T06:08:01.038524+0000","last_fullsized":"2024-05-08T06:08:01.038524+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:20:51.549926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.048004+0000","last_change":"2024-05-08T06:05:51.408393+0000","last_active":"2024-05-08T06:08:01.048004+0000","last_peered":"2024-05-08T06:08:01.048004+0000","last_clean":"2024-05-08T06:08:01.048004+0000","last_became_active":"2024-05-08T06:05:51.408023+0000","last_became_peered":"2024-05-08T06:05:51.408023+0000","last_unstale":"2024-05-08T06:08:01.048004+0000","last_undegraded":"2024-05-08T06:08:01.048004+0000","last_fullsized":"2024-05-08T06:08:01.048004+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:53.898929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.059503+0000","last_change":"2024-05-08T06:05:51.418055+0000","last_active":"2024-05-08T06:08:01.059503+0000","last_peered":"2024-05-08T06:08:01.059503+0000","last_clean":"2024-05-08T06:08:01.059503+0000","last_became_active":"2024-05-08T06:05:51.417067+0000","last_became_peered":"2024-05-08T06:05:51.417067+0000","last_unstale":"2024-05-08T06:08:01.059503+0000","last_undegraded":"2024-05-08T06:08:01.059503+0000","last_fullsized":"2024-05-08T06:08:01.059503+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:07:35.413672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.063735+0000","last_change":"2024-05-08T06:05:51.421025+0000","last_active":"2024-05-08T06:08:01.063735+0000","last_peered":"2024-05-08T06:08:01.063735+0000","last_clean":"2024-05-08T06:08:01.063735+0000","last_became_active":"2024-05-08T06:05:51.420794+0000","last_became_peered":"2024-05-08T06:05:51.420794+0000","last_unstale":"2024-05-08T06:08:01.063735+0000","last_undegraded":"2024-05-08T06:08:01.063735+0000","last_fullsized":"2024-05-08T06:08:01.063735+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:45:01.852405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.060497+0000","last_change":"2024-05-08T06:05:51.417709+0000","last_active":"2024-05-08T06:08:01.060497+0000","last_peered":"2024-05-08T06:08:01.060497+0000","last_clean":"2024-05-08T06:08:01.060497+0000","last_became_active":"2024-05-08T06:05:51.417337+0000","last_became_peered":"2024-05-08T06:05:51.417337+0000","last_unstale":"2024-05-08T06:08:01.060497+0000","last_undegraded":"2024-05-08T06:08:01.060497+0000","last_fullsized":"2024-05-08T06:08:01.060497+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:32:24.551768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.065666+0000","last_change":"2024-05-08T06:05:51.425318+0000","last_active":"2024-05-08T06:08:01.065666+0000","last_peered":"2024-05-08T06:08:01.065666+0000","last_clean":"2024-05-08T06:08:01.065666+0000","last_became_active":"2024-05-08T06:05:51.424931+0000","last_became_peered":"2024-05-08T06:05:51.424931+0000","last_unstale":"2024-05-08T06:08:01.065666+0000","last_undegraded":"2024-05-08T06:08:01.065666+0000","last_fullsized":"2024-05-08T06:08:01.065666+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:59:45.110201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.064188+0000","last_change":"2024-05-08T06:05:51.433529+0000","last_active":"2024-05-08T06:08:01.064188+0000","last_peered":"2024-05-08T06:08:01.064188+0000","last_clean":"2024-05-08T06:08:01.064188+0000","last_became_active":"2024-05-08T06:05:51.433173+0000","last_became_peered":"2024-05-08T06:05:51.433173+0000","last_unstale":"2024-05-08T06:08:01.064188+0000","last_undegraded":"2024-05-08T06:08:01.064188+0000","last_fullsized":"2024-05-08T06:08:01.064188+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:00:29.042188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.045237+0000","last_change":"2024-05-08T06:05:51.411418+0000","last_active":"2024-05-08T06:08:01.045237+0000","last_peered":"2024-05-08T06:08:01.045237+0000","last_clean":"2024-05-08T06:08:01.045237+0000","last_became_active":"2024-05-08T06:05:51.411084+0000","last_became_peered":"2024-05-08T06:05:51.411084+0000","last_unstale":"2024-05-08T06:08:01.045237+0000","last_undegraded":"2024-05-08T06:08:01.045237+0000","last_fullsized":"2024-05-08T06:08:01.045237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:08:11.522282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282383,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68601,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":837,"num_read_kb":491,"num_write":175,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":205803,"data_compressed":5508,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645363,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":888832,"data_stored":431859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841592,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":1088275,"data_compressed":37744,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005118,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":897024,"data_stored":432039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201348,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":892928,"data_stored":431937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397577,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":1024030,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561102,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":884736,"data_stored":431428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574955,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":495680,"data_compressed":1836,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738481,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":432699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411430,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":966656,"data_stored":496996,"data_compressed":1836,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117089,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":431739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920858,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":888832,"data_stored":432397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757333,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":1023182,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34281,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34281,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34281,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":64459,"data_compressed":1836,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":65775,"data_compressed":1836,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":66569,"data_compressed":1836,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T06:08:37.251 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-08T06:08:37.251 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-08T06:08:37.251 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-08T06:08:37.251 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph health --format=json 2024-05-08T06:08:37.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:36 smithi163 ceph-mon[34520]: pgmap v404: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:08:37.722 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:08:38.100 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:37 smithi123 ceph-mon[29787]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T06:08:38.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:37 smithi163 ceph-mon[34520]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T06:08:38.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:37 smithi192 ceph-mon[34664]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T06:08:39.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:38 smithi123 ceph-mon[29787]: pgmap v405: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:08:39.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:38 smithi163 ceph-mon[34520]: pgmap v405: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:08:39.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:38 smithi192 ceph-mon[34664]: pgmap v405: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:08:40.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.197+0000 7f5d29600700 1 -- 172.21.15.123:0/3722094070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d24106080 msgr2=0x7f5d24112590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:40.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.197+0000 7f5d29600700 1 --2- 172.21.15.123:0/3722094070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d24106080 0x7f5d24112590 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c009f90 tx=0x7f5d1c00b330 comp rx=0 tx=0).stop 2024-05-08T06:08:40.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.197+0000 7f5d29600700 1 -- 172.21.15.123:0/3722094070 shutdown_connections 2024-05-08T06:08:40.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.197+0000 7f5d29600700 1 --2- 172.21.15.123:0/3722094070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d24106080 0x7f5d24112590 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:40.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.197+0000 7f5d29600700 1 --2- 172.21.15.123:0/3722094070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d2410b6c0 0x7f5d2410baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:40.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.197+0000 7f5d29600700 1 --2- 172.21.15.123:0/3722094070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d241056c0 0x7f5d24105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:40.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.197+0000 7f5d29600700 1 -- 172.21.15.123:0/3722094070 >> 172.21.15.123:0/3722094070 conn(0x7f5d241011b0 msgr2=0x7f5d241035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:08:40.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.197+0000 7f5d29600700 1 -- 172.21.15.123:0/3722094070 shutdown_connections 2024-05-08T06:08:40.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.197+0000 7f5d29600700 1 -- 172.21.15.123:0/3722094070 wait complete. 2024-05-08T06:08:40.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.198+0000 7f5d29600700 1 Processor -- start 2024-05-08T06:08:40.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.198+0000 7f5d29600700 1 -- start start 2024-05-08T06:08:40.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.198+0000 7f5d29600700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d241056c0 0x7f5d241a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:40.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.198+0000 7f5d29600700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d24106080 0x7f5d241a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:40.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.198+0000 7f5d29600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d2410b6c0 0x7f5d2419bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:40.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.198+0000 7f5d29600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5d24115900 con 0x7f5d2410b6c0 2024-05-08T06:08:40.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.198+0000 7f5d29600700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5d24115780 con 0x7f5d24106080 2024-05-08T06:08:40.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.198+0000 7f5d29600700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5d24115a80 con 0x7f5d241056c0 2024-05-08T06:08:40.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.199+0000 7f5d23400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d2410b6c0 0x7f5d2419bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:40.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.199+0000 7f5d1be00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d24106080 0x7f5d241a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:40.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.199+0000 7f5d23400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d2410b6c0 0x7f5d2419bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56436/0 (socket says 172.21.15.123:56436) 2024-05-08T06:08:40.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.199+0000 7f5d23400700 1 -- 172.21.15.123:0/3679359548 learned_addr learned my addr 172.21.15.123:0/3679359548 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:08:40.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.199+0000 7f5d22a00700 1 --2- 172.21.15.123:0/3679359548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d241056c0 0x7f5d241a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:40.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.199+0000 7f5d1be00700 1 -- 172.21.15.123:0/3679359548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d241056c0 msgr2=0x7f5d241a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:40.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.199+0000 7f5d1be00700 1 --2- 172.21.15.123:0/3679359548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d241056c0 0x7f5d241a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:40.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.199+0000 7f5d1be00700 1 -- 172.21.15.123:0/3679359548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d2410b6c0 msgr2=0x7f5d2419bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:40.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.199+0000 7f5d1be00700 1 --2- 172.21.15.123:0/3679359548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d2410b6c0 0x7f5d2419bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:40.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.199+0000 7f5d1be00700 1 -- 172.21.15.123:0/3679359548 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d2419c460 con 0x7f5d24106080 2024-05-08T06:08:40.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.199+0000 7f5d23400700 1 --2- 172.21.15.123:0/3679359548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d2410b6c0 0x7f5d2419bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:08:40.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.199+0000 7f5d1be00700 1 --2- 172.21.15.123:0/3679359548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d24106080 0x7f5d241a1fa0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c00eb90 tx=0x7f5d0c00eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:08:40.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.200+0000 7f5d1b400700 1 -- 172.21.15.123:0/3679359548 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c00cc90 con 0x7f5d24106080 2024-05-08T06:08:40.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.200+0000 7f5d29600700 1 -- 172.21.15.123:0/3679359548 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d2419c750 con 0x7f5d24106080 2024-05-08T06:08:40.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.200+0000 7f5d1b400700 1 -- 172.21.15.123:0/3679359548 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d0c00ce30 con 0x7f5d24106080 2024-05-08T06:08:40.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.200+0000 7f5d1b400700 1 -- 172.21.15.123:0/3679359548 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c010430 con 0x7f5d24106080 2024-05-08T06:08:40.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.200+0000 7f5d29600700 1 -- 172.21.15.123:0/3679359548 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d241a89e0 con 0x7f5d24106080 2024-05-08T06:08:40.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.201+0000 7f5d29600700 1 -- 172.21.15.123:0/3679359548 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d24106e00 con 0x7f5d24106080 2024-05-08T06:08:40.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.203+0000 7f5d1b400700 1 -- 172.21.15.123:0/3679359548 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f5d0c003ec0 con 0x7f5d24106080 2024-05-08T06:08:40.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.205+0000 7f5d1b400700 1 --2- 172.21.15.123:0/3679359548 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5d080b2900 0x7f5d080b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:40.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.205+0000 7f5d1b400700 1 -- 172.21.15.123:0/3679359548 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f5d0c0e2650 con 0x7f5d24106080 2024-05-08T06:08:40.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.205+0000 7f5d22a00700 1 --2- 172.21.15.123:0/3679359548 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5d080b2900 0x7f5d080b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:40.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.206+0000 7f5d22a00700 1 --2- 172.21.15.123:0/3679359548 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5d080b2900 0x7f5d080b4dc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5d14009be0 tx=0x7f5d14009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:08:40.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.207+0000 7f5d1b400700 1 -- 172.21.15.123:0/3679359548 <== mon.2 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5d0c0aa3b0 con 0x7f5d24106080 2024-05-08T06:08:40.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.487+0000 7f5d29600700 1 -- 172.21.15.123:0/3679359548 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f5d2402e510 con 0x7f5d24106080 2024-05-08T06:08:40.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.488+0000 7f5d1b400700 1 -- 172.21.15.123:0/3679359548 <== mon.2 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f5d0c0b23a0 con 0x7f5d24106080 2024-05-08T06:08:40.488 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:08:40.488 INFO:teuthology.orchestra.run.smithi123.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-08T06:08:40.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.491+0000 7f5d29600700 1 -- 172.21.15.123:0/3679359548 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5d080b2900 msgr2=0x7f5d080b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:40.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.491+0000 7f5d29600700 1 --2- 172.21.15.123:0/3679359548 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5d080b2900 0x7f5d080b4dc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5d14009be0 tx=0x7f5d14009260 comp rx=0 tx=0).stop 2024-05-08T06:08:40.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.491+0000 7f5d29600700 1 -- 172.21.15.123:0/3679359548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d24106080 msgr2=0x7f5d241a1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:40.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.491+0000 7f5d29600700 1 --2- 172.21.15.123:0/3679359548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d24106080 0x7f5d241a1fa0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c00eb90 tx=0x7f5d0c00eea0 comp rx=0 tx=0).stop 2024-05-08T06:08:40.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.491+0000 7f5d29600700 1 -- 172.21.15.123:0/3679359548 shutdown_connections 2024-05-08T06:08:40.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.491+0000 7f5d29600700 1 --2- 172.21.15.123:0/3679359548 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5d080b2900 0x7f5d080b4dc0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:40.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.491+0000 7f5d29600700 1 --2- 172.21.15.123:0/3679359548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d2410b6c0 0x7f5d2419bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:40.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.491+0000 7f5d29600700 1 --2- 172.21.15.123:0/3679359548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d241056c0 0x7f5d241a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:40.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.491+0000 7f5d29600700 1 --2- 172.21.15.123:0/3679359548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d24106080 0x7f5d241a1fa0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:40.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.491+0000 7f5d29600700 1 -- 172.21.15.123:0/3679359548 >> 172.21.15.123:0/3679359548 conn(0x7f5d241011b0 msgr2=0x7f5d241019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:08:40.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.491+0000 7f5d29600700 1 -- 172.21.15.123:0/3679359548 shutdown_connections 2024-05-08T06:08:40.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:40.492+0000 7f5d29600700 1 -- 172.21.15.123:0/3679359548 wait complete. 2024-05-08T06:08:41.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:40 smithi123 ceph-mon[29787]: pgmap v406: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:08:41.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:40 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/3679359548' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-08T06:08:41.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:40 smithi163 ceph-mon[34520]: pgmap v406: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:08:41.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:40 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/3679359548' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-08T06:08:41.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:40 smithi192 ceph-mon[34664]: pgmap v406: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:08:41.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:40 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/3679359548' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-08T06:08:41.465 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-08T06:08:41.465 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T06:08:41.898 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:08:43.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:42 smithi123 ceph-mon[29787]: pgmap v407: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:08:43.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:42 smithi163 ceph-mon[34520]: pgmap v407: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:08:43.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:42 smithi192 ceph-mon[34664]: pgmap v407: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:08:44.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.302+0000 7f90c3e00700 1 -- 172.21.15.123:0/4130492414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90c41055d0 msgr2=0x7f90c4105a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:44.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.302+0000 7f90c3e00700 1 --2- 172.21.15.123:0/4130492414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90c41055d0 0x7f90c4105a50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009990 tx=0x7f90b0009ca0 comp rx=0 tx=0).stop 2024-05-08T06:08:44.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.302+0000 7f90c3e00700 1 -- 172.21.15.123:0/4130492414 shutdown_connections 2024-05-08T06:08:44.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.302+0000 7f90c3e00700 1 --2- 172.21.15.123:0/4130492414 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90c4105f90 0x7f90c41124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:44.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.302+0000 7f90c3e00700 1 --2- 172.21.15.123:0/4130492414 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90c410b660 0x7f90c410ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:44.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.302+0000 7f90c3e00700 1 --2- 172.21.15.123:0/4130492414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90c41055d0 0x7f90c4105a50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:44.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.302+0000 7f90c3e00700 1 -- 172.21.15.123:0/4130492414 >> 172.21.15.123:0/4130492414 conn(0x7f90c41010c0 msgr2=0x7f90c41034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:08:44.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.302+0000 7f90c3e00700 1 -- 172.21.15.123:0/4130492414 shutdown_connections 2024-05-08T06:08:44.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.302+0000 7f90c3e00700 1 -- 172.21.15.123:0/4130492414 wait complete. 2024-05-08T06:08:44.304 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.303+0000 7f90c3e00700 1 Processor -- start 2024-05-08T06:08:44.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.304+0000 7f90c3e00700 1 -- start start 2024-05-08T06:08:44.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.304+0000 7f90c3e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90c41055d0 0x7f90c41a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:44.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.304+0000 7f90c3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90c4105f90 0x7f90c41a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:44.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.304+0000 7f90c3e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90c410b660 0x7f90c419bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:44.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.304+0000 7f90c3e00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f90c41157d0 con 0x7f90c4105f90 2024-05-08T06:08:44.305 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.304+0000 7f90c3e00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f90c4115650 con 0x7f90c410b660 2024-05-08T06:08:44.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.304+0000 7f90c3e00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f90c4115950 con 0x7f90c41055d0 2024-05-08T06:08:44.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.305+0000 7f90c2000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90c4105f90 0x7f90c41a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:44.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.305+0000 7f90c2000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90c4105f90 0x7f90c41a1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56462/0 (socket says 172.21.15.123:56462) 2024-05-08T06:08:44.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.305+0000 7f90c2000700 1 -- 172.21.15.123:0/2609032470 learned_addr learned my addr 172.21.15.123:0/2609032470 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:08:44.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.305+0000 7f90c2a00700 1 --2- 172.21.15.123:0/2609032470 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90c41055d0 0x7f90c41a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:44.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.305+0000 7f90c3400700 1 --2- 172.21.15.123:0/2609032470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90c410b660 0x7f90c419bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:44.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.305+0000 7f90c2000700 1 -- 172.21.15.123:0/2609032470 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90c41055d0 msgr2=0x7f90c41a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:44.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.305+0000 7f90c2000700 1 --2- 172.21.15.123:0/2609032470 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90c41055d0 0x7f90c41a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:44.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.305+0000 7f90c2000700 1 -- 172.21.15.123:0/2609032470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90c410b660 msgr2=0x7f90c419bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:44.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.305+0000 7f90c2000700 1 --2- 172.21.15.123:0/2609032470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90c410b660 0x7f90c419bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:44.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.305+0000 7f90c2000700 1 -- 172.21.15.123:0/2609032470 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90c419c210 con 0x7f90c4105f90 2024-05-08T06:08:44.306 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.305+0000 7f90c2000700 1 --2- 172.21.15.123:0/2609032470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90c4105f90 0x7f90c41a1de0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f90b0000c00 tx=0x7f90b000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:08:44.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.306+0000 7f90bb400700 1 -- 172.21.15.123:0/2609032470 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b0027070 con 0x7f90c4105f90 2024-05-08T06:08:44.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.306+0000 7f90bb400700 1 -- 172.21.15.123:0/2609032470 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90b0022470 con 0x7f90c4105f90 2024-05-08T06:08:44.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.306+0000 7f90bb400700 1 -- 172.21.15.123:0/2609032470 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b0017650 con 0x7f90c4105f90 2024-05-08T06:08:44.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.306+0000 7f90c3e00700 1 -- 172.21.15.123:0/2609032470 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90c419c4a0 con 0x7f90c4105f90 2024-05-08T06:08:44.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.306+0000 7f90c3e00700 1 -- 172.21.15.123:0/2609032470 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90c41a8770 con 0x7f90c4105f90 2024-05-08T06:08:44.307 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.307+0000 7f90c3e00700 1 -- 172.21.15.123:0/2609032470 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90c406a190 con 0x7f90c4105f90 2024-05-08T06:08:44.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.309+0000 7f90bb400700 1 -- 172.21.15.123:0/2609032470 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f90b002a070 con 0x7f90c4105f90 2024-05-08T06:08:44.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.312+0000 7f90bb400700 1 --2- 172.21.15.123:0/2609032470 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f90a80b2880 0x7f90a80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:44.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.312+0000 7f90bb400700 1 -- 172.21.15.123:0/2609032470 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f90b00e2eb0 con 0x7f90c4105f90 2024-05-08T06:08:44.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.312+0000 7f90c2a00700 1 --2- 172.21.15.123:0/2609032470 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f90a80b2880 0x7f90a80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:44.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.313+0000 7f90c2a00700 1 --2- 172.21.15.123:0/2609032470 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f90a80b2880 0x7f90a80b4d40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f90c410a790 tx=0x7f90b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:08:44.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.316+0000 7f90bb400700 1 -- 172.21.15.123:0/2609032470 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f90b0013070 con 0x7f90c4105f90 2024-05-08T06:08:44.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.597+0000 7f90c3e00700 1 -- 172.21.15.123:0/2609032470 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90c402e510 con 0x7f90c4105f90 2024-05-08T06:08:44.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.600+0000 7f90bb400700 1 -- 172.21.15.123:0/2609032470 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11885 (secure 0 0 0) 0x7f90b00aac10 con 0x7f90c4105f90 2024-05-08T06:08:44.600 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:08:44.601 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3646476522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3646476522},{"type":"v1","addr":"172.21.15.123:6835","nonce":3646476522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2763706497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2763706497},{"type":"v1","addr":"172.21.15.123:6837","nonce":2763706497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/274635265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":274635265},{"type":"v1","addr":"172.21.15.123:6839","nonce":274635265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/175423115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":175423115},{"type":"v1","addr":"172.21.15.192:6833","nonce":175423115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/3263009472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3263009472},{"type":"v1","addr":"172.21.15.163:6833","nonce":3263009472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3308612826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3308612826},{"type":"v1","addr":"172.21.15.163:6835","nonce":3308612826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T06:05:54.755550+0000","modified":"2024-05-08T06:06:22.594178+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24512,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/202513680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":202513680},{"type":"v1","addr":"172.21.15.192:6835","nonce":202513680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/2483545969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2483545969},{"type":"v1","addr":"172.21.15.192:6837","nonce":2483545969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/660511838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":660511838},{"type":"v1","addr":"172.21.15.163:6837","nonce":660511838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.192:6839/3598682360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":3598682360},{"type":"v1","addr":"172.21.15.192:6839","nonce":3598682360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6841/1571726231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":1571726231},{"type":"v1","addr":"172.21.15.123:6841","nonce":1571726231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.163:6839/1568161515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":1568161515},{"type":"v1","addr":"172.21.15.163:6839","nonce":1568161515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24512,24524,24496]},"id":1}]} 2024-05-08T06:08:44.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.603+0000 7f90c3e00700 1 -- 172.21.15.123:0/2609032470 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f90a80b2880 msgr2=0x7f90a80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:44.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.603+0000 7f90c3e00700 1 --2- 172.21.15.123:0/2609032470 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f90a80b2880 0x7f90a80b4d40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f90c410a790 tx=0x7f90b4009260 comp rx=0 tx=0).stop 2024-05-08T06:08:44.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.603+0000 7f90c3e00700 1 -- 172.21.15.123:0/2609032470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90c4105f90 msgr2=0x7f90c41a1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:44.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.603+0000 7f90c3e00700 1 --2- 172.21.15.123:0/2609032470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90c4105f90 0x7f90c41a1de0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f90b0000c00 tx=0x7f90b000bfd0 comp rx=0 tx=0).stop 2024-05-08T06:08:44.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.603+0000 7f90c3e00700 1 -- 172.21.15.123:0/2609032470 shutdown_connections 2024-05-08T06:08:44.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.603+0000 7f90c3e00700 1 --2- 172.21.15.123:0/2609032470 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f90a80b2880 0x7f90a80b4d40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:44.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.603+0000 7f90c3e00700 1 --2- 172.21.15.123:0/2609032470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90c410b660 0x7f90c419bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:44.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.603+0000 7f90c3e00700 1 --2- 172.21.15.123:0/2609032470 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90c41055d0 0x7f90c41a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:44.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.603+0000 7f90c3e00700 1 --2- 172.21.15.123:0/2609032470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90c4105f90 0x7f90c41a1de0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:44.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.603+0000 7f90c3e00700 1 -- 172.21.15.123:0/2609032470 >> 172.21.15.123:0/2609032470 conn(0x7f90c41010c0 msgr2=0x7f90c4102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:08:44.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.604+0000 7f90c3e00700 1 -- 172.21.15.123:0/2609032470 shutdown_connections 2024-05-08T06:08:44.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:44.604+0000 7f90c3e00700 1 -- 172.21.15.123:0/2609032470 wait complete. 2024-05-08T06:08:44.605 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 23 2024-05-08T06:08:45.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:44 smithi123 ceph-mon[29787]: pgmap v408: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:08:45.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:44 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/2609032470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:08:45.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:44 smithi163 ceph-mon[34520]: pgmap v408: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:08:45.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:44 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/2609032470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:08:45.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:44 smithi192 ceph-mon[34664]: pgmap v408: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:08:45.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:44 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/2609032470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:08:45.677 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T06:08:46.134 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:08:46.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:45 smithi123 ceph-mon[29787]: pgmap v409: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:08:46.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:45 smithi163 ceph-mon[34520]: pgmap v409: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:08:46.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:45 smithi192 ceph-mon[34664]: pgmap v409: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:08:48.242 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:48 smithi123 ceph-mon[29787]: pgmap v410: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:08:48.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:48 smithi163 ceph-mon[34520]: pgmap v410: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:08:48.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:48 smithi192 ceph-mon[34664]: pgmap v410: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:08:48.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.569+0000 7f3473200700 1 -- 172.21.15.123:0/1982366647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3474104590 msgr2=0x7f3474104970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:48.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.569+0000 7f3473200700 1 --2- 172.21.15.123:0/1982366647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3474104590 0x7f3474104970 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3464009990 tx=0x7f3464009ca0 comp rx=0 tx=0).stop 2024-05-08T06:08:48.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.569+0000 7f3473200700 1 -- 172.21.15.123:0/1982366647 shutdown_connections 2024-05-08T06:08:48.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.569+0000 7f3473200700 1 --2- 172.21.15.123:0/1982366647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f347410d9c0 0x7f347410fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:48.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.569+0000 7f3473200700 1 --2- 172.21.15.123:0/1982366647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3474104590 0x7f3474104970 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:48.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.569+0000 7f3473200700 1 --2- 172.21.15.123:0/1982366647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3474104eb0 0x7f347410d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:48.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.569+0000 7f3473200700 1 -- 172.21.15.123:0/1982366647 >> 172.21.15.123:0/1982366647 conn(0x7f34740fddc0 msgr2=0x7f34741001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:08:48.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.569+0000 7f3473200700 1 -- 172.21.15.123:0/1982366647 shutdown_connections 2024-05-08T06:08:48.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.569+0000 7f3473200700 1 -- 172.21.15.123:0/1982366647 wait complete. 2024-05-08T06:08:48.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.570+0000 7f3473200700 1 Processor -- start 2024-05-08T06:08:48.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.570+0000 7f3473200700 1 -- start start 2024-05-08T06:08:48.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.571+0000 7f3473200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3474104590 0x7f34741021f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:48.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.571+0000 7f3473200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3474104eb0 0x7f3474102730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:48.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.571+0000 7f3473200700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f347410d9c0 0x7f3474102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:48.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.571+0000 7f3473200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3474113600 con 0x7f3474104eb0 2024-05-08T06:08:48.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.571+0000 7f3473200700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f3474113480 con 0x7f3474104590 2024-05-08T06:08:48.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.571+0000 7f3473200700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3474113780 con 0x7f347410d9c0 2024-05-08T06:08:48.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.571+0000 7f3472800700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f347410d9c0 0x7f3474102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:48.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.571+0000 7f3469400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3474104eb0 0x7f3474102730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:48.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.571+0000 7f3471e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3474104590 0x7f34741021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:48.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.571+0000 7f3472800700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f347410d9c0 0x7f3474102c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:55466/0 (socket says 172.21.15.123:55466) 2024-05-08T06:08:48.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.571+0000 7f3472800700 1 -- 172.21.15.123:0/3181477207 learned_addr learned my addr 172.21.15.123:0/3181477207 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:08:48.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.572+0000 7f3472800700 1 -- 172.21.15.123:0/3181477207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3474104590 msgr2=0x7f34741021f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:48.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.572+0000 7f3472800700 1 --2- 172.21.15.123:0/3181477207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3474104590 0x7f34741021f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:48.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.572+0000 7f3472800700 1 -- 172.21.15.123:0/3181477207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3474104eb0 msgr2=0x7f3474102730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:48.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.572+0000 7f3472800700 1 --2- 172.21.15.123:0/3181477207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3474104eb0 0x7f3474102730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:48.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.572+0000 7f3472800700 1 -- 172.21.15.123:0/3181477207 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34741a8050 con 0x7f347410d9c0 2024-05-08T06:08:48.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.572+0000 7f3472800700 1 --2- 172.21.15.123:0/3181477207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f347410d9c0 0x7f3474102c70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f346c00ef30 tx=0x7f346c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:08:48.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.572+0000 7f346b400700 1 -- 172.21.15.123:0/3181477207 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c0092e0 con 0x7f347410d9c0 2024-05-08T06:08:48.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.572+0000 7f3473200700 1 -- 172.21.15.123:0/3181477207 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34741a82e0 con 0x7f347410d9c0 2024-05-08T06:08:48.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.572+0000 7f346b400700 1 -- 172.21.15.123:0/3181477207 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f346c004bb0 con 0x7f347410d9c0 2024-05-08T06:08:48.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.572+0000 7f346b400700 1 -- 172.21.15.123:0/3181477207 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c004020 con 0x7f347410d9c0 2024-05-08T06:08:48.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.572+0000 7f3473200700 1 -- 172.21.15.123:0/3181477207 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34741a87d0 con 0x7f347410d9c0 2024-05-08T06:08:48.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.573+0000 7f3473200700 1 -- 172.21.15.123:0/3181477207 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f347406a190 con 0x7f347410d9c0 2024-05-08T06:08:48.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.576+0000 7f346b400700 1 -- 172.21.15.123:0/3181477207 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f346c01b030 con 0x7f347410d9c0 2024-05-08T06:08:48.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.577+0000 7f346b400700 1 --2- 172.21.15.123:0/3181477207 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f34580b2880 0x7f34580b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:48.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.577+0000 7f346b400700 1 -- 172.21.15.123:0/3181477207 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f346c0e1ca0 con 0x7f347410d9c0 2024-05-08T06:08:48.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.577+0000 7f3471e00700 1 --2- 172.21.15.123:0/3181477207 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f34580b2880 0x7f34580b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:48.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.578+0000 7f3471e00700 1 --2- 172.21.15.123:0/3181477207 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f34580b2880 0x7f34580b4d40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f346400b2e0 tx=0x7f3464005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:08:48.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.581+0000 7f346b400700 1 -- 172.21.15.123:0/3181477207 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f346c022020 con 0x7f347410d9c0 2024-05-08T06:08:48.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.867+0000 7f3473200700 1 -- 172.21.15.123:0/3181477207 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f347402e510 con 0x7f347410d9c0 2024-05-08T06:08:48.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.868+0000 7f346b400700 1 -- 172.21.15.123:0/3181477207 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11885 (secure 0 0 0) 0x7f346c010040 con 0x7f347410d9c0 2024-05-08T06:08:48.868 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:08:48.869 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3646476522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3646476522},{"type":"v1","addr":"172.21.15.123:6835","nonce":3646476522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2763706497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2763706497},{"type":"v1","addr":"172.21.15.123:6837","nonce":2763706497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/274635265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":274635265},{"type":"v1","addr":"172.21.15.123:6839","nonce":274635265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/175423115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":175423115},{"type":"v1","addr":"172.21.15.192:6833","nonce":175423115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/3263009472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3263009472},{"type":"v1","addr":"172.21.15.163:6833","nonce":3263009472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3308612826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3308612826},{"type":"v1","addr":"172.21.15.163:6835","nonce":3308612826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T06:05:54.755550+0000","modified":"2024-05-08T06:06:22.594178+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24512,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/202513680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":202513680},{"type":"v1","addr":"172.21.15.192:6835","nonce":202513680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/2483545969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2483545969},{"type":"v1","addr":"172.21.15.192:6837","nonce":2483545969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/660511838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":660511838},{"type":"v1","addr":"172.21.15.163:6837","nonce":660511838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.192:6839/3598682360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":3598682360},{"type":"v1","addr":"172.21.15.192:6839","nonce":3598682360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6841/1571726231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":1571726231},{"type":"v1","addr":"172.21.15.123:6841","nonce":1571726231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.163:6839/1568161515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":1568161515},{"type":"v1","addr":"172.21.15.163:6839","nonce":1568161515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24512,24524,24496]},"id":1}]} 2024-05-08T06:08:48.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.871+0000 7f3473200700 1 -- 172.21.15.123:0/3181477207 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f34580b2880 msgr2=0x7f34580b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:48.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.871+0000 7f3473200700 1 --2- 172.21.15.123:0/3181477207 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f34580b2880 0x7f34580b4d40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f346400b2e0 tx=0x7f3464005fd0 comp rx=0 tx=0).stop 2024-05-08T06:08:48.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.871+0000 7f3473200700 1 -- 172.21.15.123:0/3181477207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f347410d9c0 msgr2=0x7f3474102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:48.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.871+0000 7f3473200700 1 --2- 172.21.15.123:0/3181477207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f347410d9c0 0x7f3474102c70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f346c00ef30 tx=0x7f346c00c3a0 comp rx=0 tx=0).stop 2024-05-08T06:08:48.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.871+0000 7f3473200700 1 -- 172.21.15.123:0/3181477207 shutdown_connections 2024-05-08T06:08:48.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.871+0000 7f3473200700 1 --2- 172.21.15.123:0/3181477207 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f34580b2880 0x7f34580b4d40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:48.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.871+0000 7f3473200700 1 --2- 172.21.15.123:0/3181477207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f347410d9c0 0x7f3474102c70 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:48.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.871+0000 7f3473200700 1 --2- 172.21.15.123:0/3181477207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3474104590 0x7f34741021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:48.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.871+0000 7f3473200700 1 --2- 172.21.15.123:0/3181477207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3474104eb0 0x7f3474102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:48.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.871+0000 7f3473200700 1 -- 172.21.15.123:0/3181477207 >> 172.21.15.123:0/3181477207 conn(0x7f34740fddc0 msgr2=0x7f34740feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:08:48.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.872+0000 7f3473200700 1 -- 172.21.15.123:0/3181477207 shutdown_connections 2024-05-08T06:08:48.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:48.872+0000 7f3473200700 1 -- 172.21.15.123:0/3181477207 wait complete. 2024-05-08T06:08:48.873 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 23 2024-05-08T06:08:49.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:49 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/3181477207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:08:49.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:49 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/3181477207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:08:49.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:49 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/3181477207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:08:49.776 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T06:08:50.225 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:08:50.439 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:50 smithi123 ceph-mon[29787]: pgmap v411: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:08:50.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:50 smithi163 ceph-mon[34520]: pgmap v411: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:08:50.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:50 smithi192 ceph-mon[34664]: pgmap v411: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:08:52.351 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:52 smithi123 ceph-mon[29787]: pgmap v412: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-05-08T06:08:52.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:52 smithi163 ceph-mon[34520]: pgmap v412: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-05-08T06:08:52.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:52 smithi192 ceph-mon[34664]: pgmap v412: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-05-08T06:08:52.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.665+0000 7fef66200700 1 -- 172.21.15.123:0/1123274162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef6010ff50 msgr2=0x7fef60112340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:52.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.665+0000 7fef66200700 1 --2- 172.21.15.123:0/1123274162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef6010ff50 0x7fef60112340 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fef5800b0a0 tx=0x7fef5800b3b0 comp rx=0 tx=0).stop 2024-05-08T06:08:52.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.666+0000 7fef66200700 1 -- 172.21.15.123:0/1123274162 shutdown_connections 2024-05-08T06:08:52.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.666+0000 7fef66200700 1 --2- 172.21.15.123:0/1123274162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef6010ff50 0x7fef60112340 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:52.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.666+0000 7fef66200700 1 --2- 172.21.15.123:0/1123274162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef6006b1b0 0x7fef6006b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:52.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.666+0000 7fef66200700 1 --2- 172.21.15.123:0/1123274162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef6006bad0 0x7fef6010fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:52.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.666+0000 7fef66200700 1 -- 172.21.15.123:0/1123274162 >> 172.21.15.123:0/1123274162 conn(0x7fef60079350 msgr2=0x7fef60079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:08:52.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.666+0000 7fef66200700 1 -- 172.21.15.123:0/1123274162 shutdown_connections 2024-05-08T06:08:52.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.666+0000 7fef66200700 1 -- 172.21.15.123:0/1123274162 wait complete. 2024-05-08T06:08:52.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.667+0000 7fef66200700 1 Processor -- start 2024-05-08T06:08:52.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.667+0000 7fef66200700 1 -- start start 2024-05-08T06:08:52.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.667+0000 7fef66200700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef6006b1b0 0x7fef6019be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:52.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.667+0000 7fef66200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef6006bad0 0x7fef6019c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:52.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.667+0000 7fef66200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef6010ff50 0x7fef601a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:52.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.668+0000 7fef66200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fef601158c0 con 0x7fef6006bad0 2024-05-08T06:08:52.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.668+0000 7fef66200700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fef60115740 con 0x7fef6010ff50 2024-05-08T06:08:52.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.668+0000 7fef66200700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fef60115a40 con 0x7fef6006b1b0 2024-05-08T06:08:52.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.668+0000 7fef5ea00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef6006bad0 0x7fef6019c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:52.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.668+0000 7fef5ea00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef6006bad0 0x7fef6019c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:37890/0 (socket says 172.21.15.123:37890) 2024-05-08T06:08:52.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.668+0000 7fef5ea00700 1 -- 172.21.15.123:0/3393357687 learned_addr learned my addr 172.21.15.123:0/3393357687 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:08:52.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.668+0000 7fef5f400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef6006b1b0 0x7fef6019be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:52.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.668+0000 7fef5fe00700 1 --2- 172.21.15.123:0/3393357687 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef6010ff50 0x7fef601a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:52.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.668+0000 7fef5f400700 1 -- 172.21.15.123:0/3393357687 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef6010ff50 msgr2=0x7fef601a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:52.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.668+0000 7fef5f400700 1 --2- 172.21.15.123:0/3393357687 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef6010ff50 0x7fef601a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:52.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.668+0000 7fef5f400700 1 -- 172.21.15.123:0/3393357687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef6006bad0 msgr2=0x7fef6019c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:52.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.668+0000 7fef5f400700 1 --2- 172.21.15.123:0/3393357687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef6006bad0 0x7fef6019c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:52.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.668+0000 7fef5f400700 1 -- 172.21.15.123:0/3393357687 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef601a0dd0 con 0x7fef6006b1b0 2024-05-08T06:08:52.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.668+0000 7fef5ea00700 1 --2- 172.21.15.123:0/3393357687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef6006bad0 0x7fef6019c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:08:52.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.669+0000 7fef5f400700 1 --2- 172.21.15.123:0/3393357687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef6006b1b0 0x7fef6019be10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fef5000e9f0 tx=0x7fef5000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:08:52.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.669+0000 7fef57e00700 1 -- 172.21.15.123:0/3393357687 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5000c9c0 con 0x7fef6006b1b0 2024-05-08T06:08:52.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.669+0000 7fef57e00700 1 -- 172.21.15.123:0/3393357687 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef5000cb60 con 0x7fef6006b1b0 2024-05-08T06:08:52.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.669+0000 7fef66200700 1 -- 172.21.15.123:0/3393357687 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef601a10c0 con 0x7fef6006b1b0 2024-05-08T06:08:52.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.669+0000 7fef57e00700 1 -- 172.21.15.123:0/3393357687 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef50018680 con 0x7fef6006b1b0 2024-05-08T06:08:52.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.669+0000 7fef66200700 1 -- 172.21.15.123:0/3393357687 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef601a8940 con 0x7fef6006b1b0 2024-05-08T06:08:52.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.670+0000 7fef66200700 1 -- 172.21.15.123:0/3393357687 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef6006a190 con 0x7fef6006b1b0 2024-05-08T06:08:52.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.673+0000 7fef57e00700 1 -- 172.21.15.123:0/3393357687 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fef50010430 con 0x7fef6006b1b0 2024-05-08T06:08:52.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.674+0000 7fef57e00700 1 --2- 172.21.15.123:0/3393357687 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fef440b2880 0x7fef440b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:52.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.674+0000 7fef57e00700 1 -- 172.21.15.123:0/3393357687 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fef500e2730 con 0x7fef6006b1b0 2024-05-08T06:08:52.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.675+0000 7fef5ea00700 1 --2- 172.21.15.123:0/3393357687 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fef440b2880 0x7fef440b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:52.676 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.675+0000 7fef5ea00700 1 --2- 172.21.15.123:0/3393357687 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fef440b2880 0x7fef440b4d40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fef60103960 tx=0x7fef4c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:08:52.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.676+0000 7fef57e00700 1 -- 172.21.15.123:0/3393357687 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fef500aa400 con 0x7fef6006b1b0 2024-05-08T06:08:52.910 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.909+0000 7fef66200700 1 -- 172.21.15.123:0/3393357687 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef6002e510 con 0x7fef6006b1b0 2024-05-08T06:08:52.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.911+0000 7fef57e00700 1 -- 172.21.15.123:0/3393357687 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22719 (secure 0 0 0) 0x7fef500a9d00 con 0x7fef6006b1b0 2024-05-08T06:08:52.913 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:08:52.913 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":82,"fsid":"85cfb2e6-0cff-11ef-bc97-c7b262605968","created":"2024-05-08T05:56:12.544750+0000","modified":"2024-05-08T06:06:32.471993+0000","last_up_change":"2024-05-08T06:04:33.654727+0000","last_in_change":"2024-05-08T06:04:13.809050+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T06:00:17.050518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T06:05:44.815334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T06:05:49.604457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a14fde9b-bfb1-40a7-a105-305e8981a778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6803","nonce":3608618058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6805","nonce":3608618058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6809","nonce":3608618058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6807","nonce":3608618058}]},"public_addr":"172.21.15.123:6803/3608618058","cluster_addr":"172.21.15.123:6805/3608618058","heartbeat_back_addr":"172.21.15.123:6809/3608618058","heartbeat_front_addr":"172.21.15.123:6807/3608618058","state":["exists","up"]},{"osd":1,"uuid":"e023ae3d-a82e-4bb6-8fef-889dd07fe183","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6801","nonce":1884753449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6803","nonce":1884753449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6807","nonce":1884753449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6805","nonce":1884753449}]},"public_addr":"172.21.15.163:6801/1884753449","cluster_addr":"172.21.15.163:6803/1884753449","heartbeat_back_addr":"172.21.15.163:6807/1884753449","heartbeat_front_addr":"172.21.15.163:6805/1884753449","state":["exists","up"]},{"osd":2,"uuid":"69f4cfca-63cb-4862-9480-d79891b0f338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6801","nonce":4161570404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6803","nonce":4161570404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6807","nonce":4161570404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6805","nonce":4161570404}]},"public_addr":"172.21.15.192:6801/4161570404","cluster_addr":"172.21.15.192:6803/4161570404","heartbeat_back_addr":"172.21.15.192:6807/4161570404","heartbeat_front_addr":"172.21.15.192:6805/4161570404","state":["exists","up"]},{"osd":3,"uuid":"71bb5ecd-c88c-45fb-8b51-0eefb53941c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6811","nonce":1243003258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6813","nonce":1243003258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6817","nonce":1243003258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6815","nonce":1243003258}]},"public_addr":"172.21.15.123:6811/1243003258","cluster_addr":"172.21.15.123:6813/1243003258","heartbeat_back_addr":"172.21.15.123:6817/1243003258","heartbeat_front_addr":"172.21.15.123:6815/1243003258","state":["exists","up"]},{"osd":4,"uuid":"5a7a3462-e74f-4d15-a85c-709b5d423745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6809","nonce":591680273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6811","nonce":591680273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6815","nonce":591680273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6813","nonce":591680273}]},"public_addr":"172.21.15.163:6809/591680273","cluster_addr":"172.21.15.163:6811/591680273","heartbeat_back_addr":"172.21.15.163:6815/591680273","heartbeat_front_addr":"172.21.15.163:6813/591680273","state":["exists","up"]},{"osd":5,"uuid":"85e3f1ab-f8c6-4025-bab9-9ddb6bd3e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6809","nonce":1734821180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6811","nonce":1734821180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6815","nonce":1734821180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6813","nonce":1734821180}]},"public_addr":"172.21.15.192:6809/1734821180","cluster_addr":"172.21.15.192:6811/1734821180","heartbeat_back_addr":"172.21.15.192:6815/1734821180","heartbeat_front_addr":"172.21.15.192:6813/1734821180","state":["exists","up"]},{"osd":6,"uuid":"f21a5624-6f8f-48be-bc6d-0a883d6b7b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6819","nonce":1002138878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6821","nonce":1002138878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6825","nonce":1002138878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6823","nonce":1002138878}]},"public_addr":"172.21.15.123:6819/1002138878","cluster_addr":"172.21.15.123:6821/1002138878","heartbeat_back_addr":"172.21.15.123:6825/1002138878","heartbeat_front_addr":"172.21.15.123:6823/1002138878","state":["exists","up"]},{"osd":7,"uuid":"6eb16729-d260-4041-9829-3e868795b111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6817","nonce":354411978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6819","nonce":354411978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6823","nonce":354411978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6821","nonce":354411978}]},"public_addr":"172.21.15.163:6817/354411978","cluster_addr":"172.21.15.163:6819/354411978","heartbeat_back_addr":"172.21.15.163:6823/354411978","heartbeat_front_addr":"172.21.15.163:6821/354411978","state":["exists","up"]},{"osd":8,"uuid":"122a281e-18b8-4728-97db-458e7432b2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6817","nonce":3669753689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6819","nonce":3669753689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6823","nonce":3669753689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6821","nonce":3669753689}]},"public_addr":"172.21.15.192:6817/3669753689","cluster_addr":"172.21.15.192:6819/3669753689","heartbeat_back_addr":"172.21.15.192:6823/3669753689","heartbeat_front_addr":"172.21.15.192:6821/3669753689","state":["exists","up"]},{"osd":9,"uuid":"9fa022a9-b873-4e07-806c-641a706a150f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6827","nonce":3104436473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6829","nonce":3104436473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6833","nonce":3104436473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6831","nonce":3104436473}]},"public_addr":"172.21.15.123:6827/3104436473","cluster_addr":"172.21.15.123:6829/3104436473","heartbeat_back_addr":"172.21.15.123:6833/3104436473","heartbeat_front_addr":"172.21.15.123:6831/3104436473","state":["exists","up"]},{"osd":10,"uuid":"fb59339b-e11b-4ffc-b202-7f67cb21da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6825","nonce":2628487883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6827","nonce":2628487883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6831","nonce":2628487883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6829","nonce":2628487883}]},"public_addr":"172.21.15.163:6825/2628487883","cluster_addr":"172.21.15.163:6827/2628487883","heartbeat_back_addr":"172.21.15.163:6831/2628487883","heartbeat_front_addr":"172.21.15.163:6829/2628487883","state":["exists","up"]},{"osd":11,"uuid":"be7a5f36-c1be-41d7-80ee-004d0546eb9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6825","nonce":1339159347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6827","nonce":1339159347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6831","nonce":1339159347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6829","nonce":1339159347}]},"public_addr":"172.21.15.192:6825/1339159347","cluster_addr":"172.21.15.192:6827/1339159347","heartbeat_back_addr":"172.21.15.192:6831/1339159347","heartbeat_front_addr":"172.21.15.192:6829/1339159347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:59:24.923234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:59:49.867071+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:00:13.454062+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:00:46.983594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:01:12.820452+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:01:37.451382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:02:13.325181+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:02:39.453436+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:03:03.837509+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:03:39.460579+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:04:06.314082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:04:31.723154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6801/3057688649":"2024-05-09T05:57:30.887936+0000","172.21.15.123:6800/3057688649":"2024-05-09T05:57:30.887936+0000","172.21.15.123:6800/2769724312":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/399884043":"2024-05-09T05:57:30.887936+0000","172.21.15.123:0/3406061897":"2024-05-09T05:57:30.887936+0000","172.21.15.123:0/644457268":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/3055092197":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/343390357":"2024-05-09T05:57:30.887936+0000","172.21.15.123:6801/2769724312":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/2263171562":"2024-05-09T05:56:55.849036+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T06:08:52.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.915+0000 7fef66200700 1 -- 172.21.15.123:0/3393357687 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fef440b2880 msgr2=0x7fef440b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:52.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.915+0000 7fef66200700 1 --2- 172.21.15.123:0/3393357687 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fef440b2880 0x7fef440b4d40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fef60103960 tx=0x7fef4c009260 comp rx=0 tx=0).stop 2024-05-08T06:08:52.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.915+0000 7fef66200700 1 -- 172.21.15.123:0/3393357687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef6006b1b0 msgr2=0x7fef6019be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:52.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.915+0000 7fef66200700 1 --2- 172.21.15.123:0/3393357687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef6006b1b0 0x7fef6019be10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fef5000e9f0 tx=0x7fef5000ed00 comp rx=0 tx=0).stop 2024-05-08T06:08:52.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.915+0000 7fef66200700 1 -- 172.21.15.123:0/3393357687 shutdown_connections 2024-05-08T06:08:52.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.915+0000 7fef66200700 1 --2- 172.21.15.123:0/3393357687 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fef440b2880 0x7fef440b4d40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:52.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.915+0000 7fef66200700 1 --2- 172.21.15.123:0/3393357687 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef6010ff50 0x7fef601a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:52.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.915+0000 7fef66200700 1 --2- 172.21.15.123:0/3393357687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef6006b1b0 0x7fef6019be10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:52.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.915+0000 7fef66200700 1 --2- 172.21.15.123:0/3393357687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef6006bad0 0x7fef6019c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:52.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.915+0000 7fef66200700 1 -- 172.21.15.123:0/3393357687 >> 172.21.15.123:0/3393357687 conn(0x7fef60079350 msgr2=0x7fef60111440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:08:52.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.915+0000 7fef66200700 1 -- 172.21.15.123:0/3393357687 shutdown_connections 2024-05-08T06:08:52.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:52.915+0000 7fef66200700 1 -- 172.21.15.123:0/3393357687 wait complete. 2024-05-08T06:08:53.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:53 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/3393357687' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:08:53.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:53 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/3393357687' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:08:53.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:53 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/3393357687' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:08:53.862 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T06:08:54.311 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:08:54.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:54 smithi123 ceph-mon[29787]: pgmap v413: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-08T06:08:54.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:54 smithi163 ceph-mon[34520]: pgmap v413: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-08T06:08:54.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:54 smithi192 ceph-mon[34664]: pgmap v413: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-08T06:08:56.437 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:56 smithi123 ceph-mon[29787]: pgmap v414: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-08T06:08:56.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:56 smithi163 ceph-mon[34520]: pgmap v414: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-08T06:08:56.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:56 smithi192 ceph-mon[34664]: pgmap v414: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-08T06:08:56.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.732+0000 7fc891200700 1 -- 172.21.15.123:0/4264031832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc88c075ff0 msgr2=0x7fc88c10f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:56.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.732+0000 7fc891200700 1 --2- 172.21.15.123:0/4264031832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc88c075ff0 0x7fc88c10f960 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc874009990 tx=0x7fc874009ca0 comp rx=0 tx=0).stop 2024-05-08T06:08:56.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.732+0000 7fc891200700 1 -- 172.21.15.123:0/4264031832 shutdown_connections 2024-05-08T06:08:56.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.732+0000 7fc891200700 1 --2- 172.21.15.123:0/4264031832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc88c10fea0 0x7fc88c112290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:56.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.732+0000 7fc891200700 1 --2- 172.21.15.123:0/4264031832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc88c0756d0 0x7fc88c075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:56.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.732+0000 7fc891200700 1 --2- 172.21.15.123:0/4264031832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc88c075ff0 0x7fc88c10f960 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:56.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.732+0000 7fc891200700 1 -- 172.21.15.123:0/4264031832 >> 172.21.15.123:0/4264031832 conn(0x7fc88c07a600 msgr2=0x7fc88c07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:08:56.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.733+0000 7fc891200700 1 -- 172.21.15.123:0/4264031832 shutdown_connections 2024-05-08T06:08:56.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.733+0000 7fc891200700 1 -- 172.21.15.123:0/4264031832 wait complete. 2024-05-08T06:08:56.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.734+0000 7fc891200700 1 Processor -- start 2024-05-08T06:08:56.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.734+0000 7fc891200700 1 -- start start 2024-05-08T06:08:56.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.734+0000 7fc891200700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc88c0756d0 0x7fc88c19bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:56.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.734+0000 7fc891200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc88c075ff0 0x7fc88c19c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:56.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.734+0000 7fc891200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc88c10fea0 0x7fc88c1a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:56.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.734+0000 7fc891200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc88c115870 con 0x7fc88c075ff0 2024-05-08T06:08:56.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.734+0000 7fc891200700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc88c1156f0 con 0x7fc88c10fea0 2024-05-08T06:08:56.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.734+0000 7fc891200700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc88c1159f0 con 0x7fc88c0756d0 2024-05-08T06:08:56.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.734+0000 7fc881400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc88c075ff0 0x7fc88c19c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:56.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.735+0000 7fc881400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc88c075ff0 0x7fc88c19c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47872/0 (socket says 172.21.15.123:47872) 2024-05-08T06:08:56.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.735+0000 7fc881400700 1 -- 172.21.15.123:0/4153572597 learned_addr learned my addr 172.21.15.123:0/4153572597 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:08:56.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.735+0000 7fc88a800700 1 --2- 172.21.15.123:0/4153572597 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc88c10fea0 0x7fc88c1a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:56.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.735+0000 7fc889e00700 1 --2- 172.21.15.123:0/4153572597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc88c0756d0 0x7fc88c19bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:56.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.735+0000 7fc881400700 1 -- 172.21.15.123:0/4153572597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc88c0756d0 msgr2=0x7fc88c19bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:56.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.735+0000 7fc881400700 1 --2- 172.21.15.123:0/4153572597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc88c0756d0 0x7fc88c19bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:56.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.735+0000 7fc881400700 1 -- 172.21.15.123:0/4153572597 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc88c10fea0 msgr2=0x7fc88c1a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:56.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.735+0000 7fc881400700 1 --2- 172.21.15.123:0/4153572597 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc88c10fea0 0x7fc88c1a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:56.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.735+0000 7fc881400700 1 -- 172.21.15.123:0/4153572597 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc88c1a0d90 con 0x7fc88c075ff0 2024-05-08T06:08:56.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.735+0000 7fc881400700 1 --2- 172.21.15.123:0/4153572597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc88c075ff0 0x7fc88c19c2c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc874000c00 tx=0x7fc87400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:08:56.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.735+0000 7fc883400700 1 -- 172.21.15.123:0/4153572597 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc874026070 con 0x7fc88c075ff0 2024-05-08T06:08:56.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.736+0000 7fc883400700 1 -- 172.21.15.123:0/4153572597 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc874021410 con 0x7fc88c075ff0 2024-05-08T06:08:56.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.736+0000 7fc891200700 1 -- 172.21.15.123:0/4153572597 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc88c1a1020 con 0x7fc88c075ff0 2024-05-08T06:08:56.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.736+0000 7fc883400700 1 -- 172.21.15.123:0/4153572597 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc874017650 con 0x7fc88c075ff0 2024-05-08T06:08:56.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.736+0000 7fc891200700 1 -- 172.21.15.123:0/4153572597 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc88c1a8900 con 0x7fc88c075ff0 2024-05-08T06:08:56.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.737+0000 7fc891200700 1 -- 172.21.15.123:0/4153572597 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc88c06a190 con 0x7fc88c075ff0 2024-05-08T06:08:56.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.740+0000 7fc883400700 1 -- 172.21.15.123:0/4153572597 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fc874029070 con 0x7fc88c075ff0 2024-05-08T06:08:56.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.741+0000 7fc883400700 1 --2- 172.21.15.123:0/4153572597 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fc8700b2900 0x7fc8700b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:08:56.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.742+0000 7fc883400700 1 -- 172.21.15.123:0/4153572597 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fc8740e2e70 con 0x7fc88c075ff0 2024-05-08T06:08:56.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.742+0000 7fc889e00700 1 --2- 172.21.15.123:0/4153572597 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fc8700b2900 0x7fc8700b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:08:56.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.743+0000 7fc889e00700 1 --2- 172.21.15.123:0/4153572597 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fc8700b2900 0x7fc8700b4dc0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc87c009bf0 tx=0x7fc87c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:08:56.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:56.747+0000 7fc883400700 1 -- 172.21.15.123:0/4153572597 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc87405edf0 con 0x7fc88c075ff0 2024-05-08T06:08:57.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:57.038+0000 7fc891200700 1 -- 172.21.15.123:0/4153572597 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc88c19d0e0 con 0x7fc88c075ff0 2024-05-08T06:08:57.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:57.039+0000 7fc883400700 1 -- 172.21.15.123:0/4153572597 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11885 (secure 0 0 0) 0x7fc874013070 con 0x7fc88c075ff0 2024-05-08T06:08:57.040 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:08:57.040 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3646476522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3646476522},{"type":"v1","addr":"172.21.15.123:6835","nonce":3646476522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2763706497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2763706497},{"type":"v1","addr":"172.21.15.123:6837","nonce":2763706497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/274635265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":274635265},{"type":"v1","addr":"172.21.15.123:6839","nonce":274635265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/175423115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":175423115},{"type":"v1","addr":"172.21.15.192:6833","nonce":175423115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/3263009472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3263009472},{"type":"v1","addr":"172.21.15.163:6833","nonce":3263009472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3308612826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3308612826},{"type":"v1","addr":"172.21.15.163:6835","nonce":3308612826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T06:05:54.755550+0000","modified":"2024-05-08T06:06:22.594178+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24512,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/202513680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":202513680},{"type":"v1","addr":"172.21.15.192:6835","nonce":202513680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/2483545969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2483545969},{"type":"v1","addr":"172.21.15.192:6837","nonce":2483545969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/660511838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":660511838},{"type":"v1","addr":"172.21.15.163:6837","nonce":660511838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.192:6839/3598682360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":3598682360},{"type":"v1","addr":"172.21.15.192:6839","nonce":3598682360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6841/1571726231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":1571726231},{"type":"v1","addr":"172.21.15.123:6841","nonce":1571726231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.163:6839/1568161515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":1568161515},{"type":"v1","addr":"172.21.15.163:6839","nonce":1568161515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24512,24524,24496]},"id":1}]} 2024-05-08T06:08:57.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:57.042+0000 7fc891200700 1 -- 172.21.15.123:0/4153572597 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fc8700b2900 msgr2=0x7fc8700b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:57.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:57.042+0000 7fc891200700 1 --2- 172.21.15.123:0/4153572597 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fc8700b2900 0x7fc8700b4dc0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc87c009bf0 tx=0x7fc87c009260 comp rx=0 tx=0).stop 2024-05-08T06:08:57.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:57.042+0000 7fc891200700 1 -- 172.21.15.123:0/4153572597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc88c075ff0 msgr2=0x7fc88c19c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:08:57.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:57.042+0000 7fc891200700 1 --2- 172.21.15.123:0/4153572597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc88c075ff0 0x7fc88c19c2c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc874000c00 tx=0x7fc87400bf60 comp rx=0 tx=0).stop 2024-05-08T06:08:57.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:57.042+0000 7fc891200700 1 -- 172.21.15.123:0/4153572597 shutdown_connections 2024-05-08T06:08:57.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:57.042+0000 7fc891200700 1 --2- 172.21.15.123:0/4153572597 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fc8700b2900 0x7fc8700b4dc0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:57.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:57.042+0000 7fc891200700 1 --2- 172.21.15.123:0/4153572597 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc88c10fea0 0x7fc88c1a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:57.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:57.042+0000 7fc891200700 1 --2- 172.21.15.123:0/4153572597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc88c0756d0 0x7fc88c19bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:57.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:57.042+0000 7fc891200700 1 --2- 172.21.15.123:0/4153572597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc88c075ff0 0x7fc88c19c2c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:08:57.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:57.042+0000 7fc891200700 1 -- 172.21.15.123:0/4153572597 >> 172.21.15.123:0/4153572597 conn(0x7fc88c07a600 msgr2=0x7fc88c1006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:08:57.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:57.043+0000 7fc891200700 1 -- 172.21.15.123:0/4153572597 shutdown_connections 2024-05-08T06:08:57.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:08:57.043+0000 7fc891200700 1 -- 172.21.15.123:0/4153572597 wait complete. 2024-05-08T06:08:57.044 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 23 2024-05-08T06:08:57.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:57 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/4153572597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:08:57.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:57 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/4153572597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:08:57.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:57 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/4153572597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:08:57.972 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-08T06:05:54.755550+0000', 'modified': '2024-05-08T06:06:22.594178+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24524, 'mds_1': 24512, 'mds_2': 24496}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24469': {'gid': 24469, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.192:6835/202513680', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6834', 'nonce': 202513680}, {'type': 'v1', 'addr': '172.21.15.192:6835', 'nonce': 202513680}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.192:6837/2483545969', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6836', 'nonce': 2483545969}, {'type': 'v1', 'addr': '172.21.15.192:6837', 'nonce': 2483545969}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.163:6837/660511838', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6836', 'nonce': 660511838}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 660511838}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'i', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.192:6839/3598682360', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6838', 'nonce': 3598682360}, {'type': 'v1', 'addr': '172.21.15.192:6839', 'nonce': 3598682360}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24512': {'gid': 24512, 'name': 'a', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.123:6841/1571726231', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6840', 'nonce': 1571726231}, {'type': 'v1', 'addr': '172.21.15.123:6841', 'nonce': 1571726231}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24524': {'gid': 24524, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.163:6839/1568161515', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6838', 'nonce': 1568161515}, {'type': 'v1', 'addr': '172.21.15.163:6839', 'nonce': 1568161515}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24524, 'qdb_cluster': [24512, 24524, 24496]} 2024-05-08T06:08:57.972 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-08T06:08:57.972 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-08T06:08:58.423 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:08:58.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:08:58 smithi163 ceph-mon[34520]: pgmap v415: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-08T06:08:58.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:08:58 smithi192 ceph-mon[34664]: pgmap v415: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-08T06:08:58.576 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:08:58 smithi123 ceph-mon[29787]: pgmap v415: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-08T06:09:00.281 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:00 smithi123 ceph-mon[29787]: pgmap v416: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-08T06:09:00.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:00 smithi163 ceph-mon[34520]: pgmap v416: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-08T06:09:00.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:00 smithi192 ceph-mon[34664]: pgmap v416: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-08T06:09:00.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.866+0000 7f8371e00700 1 -- 172.21.15.123:0/3265105378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f836c1055d0 msgr2=0x7f836c105a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:00.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.866+0000 7f8371e00700 1 --2- 172.21.15.123:0/3265105378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f836c1055d0 0x7f836c105a50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8358009990 tx=0x7f8358009ca0 comp rx=0 tx=0).stop 2024-05-08T06:09:00.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.866+0000 7f8371e00700 1 -- 172.21.15.123:0/3265105378 shutdown_connections 2024-05-08T06:09:00.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.866+0000 7f8371e00700 1 --2- 172.21.15.123:0/3265105378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f836c105f90 0x7f836c1124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:00.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.866+0000 7f8371e00700 1 --2- 172.21.15.123:0/3265105378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f836c10b660 0x7f836c10ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:00.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.866+0000 7f8371e00700 1 --2- 172.21.15.123:0/3265105378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f836c1055d0 0x7f836c105a50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:00.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.866+0000 7f8371e00700 1 -- 172.21.15.123:0/3265105378 >> 172.21.15.123:0/3265105378 conn(0x7f836c101080 msgr2=0x7f836c1034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:00.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.866+0000 7f8371e00700 1 -- 172.21.15.123:0/3265105378 shutdown_connections 2024-05-08T06:09:00.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.866+0000 7f8371e00700 1 -- 172.21.15.123:0/3265105378 wait complete. 2024-05-08T06:09:00.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.867+0000 7f8371e00700 1 Processor -- start 2024-05-08T06:09:00.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.867+0000 7f8371e00700 1 -- start start 2024-05-08T06:09:00.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.868+0000 7f8371e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f836c1055d0 0x7f836c1a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:00.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.868+0000 7f8371e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f836c105f90 0x7f836c1a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:00.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.868+0000 7f8371e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f836c10b660 0x7f836c19ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:00.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.868+0000 7f8371e00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f836c115820 con 0x7f836c105f90 2024-05-08T06:09:00.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.868+0000 7f8371e00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f836c1156a0 con 0x7f836c10b660 2024-05-08T06:09:00.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.868+0000 7f8371e00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f836c1159a0 con 0x7f836c1055d0 2024-05-08T06:09:00.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.868+0000 7f836be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f836c105f90 0x7f836c1a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:00.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.868+0000 7f836be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f836c105f90 0x7f836c1a1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47900/0 (socket says 172.21.15.123:47900) 2024-05-08T06:09:00.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.868+0000 7f836be00700 1 -- 172.21.15.123:0/1884652684 learned_addr learned my addr 172.21.15.123:0/1884652684 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:09:00.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.868+0000 7f8371400700 1 --2- 172.21.15.123:0/1884652684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f836c10b660 0x7f836c19ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:00.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.868+0000 7f8370a00700 1 --2- 172.21.15.123:0/1884652684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f836c1055d0 0x7f836c1a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:00.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.869+0000 7f836be00700 1 -- 172.21.15.123:0/1884652684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f836c1055d0 msgr2=0x7f836c1a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:00.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.869+0000 7f836be00700 1 --2- 172.21.15.123:0/1884652684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f836c1055d0 0x7f836c1a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:00.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.869+0000 7f836be00700 1 -- 172.21.15.123:0/1884652684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f836c10b660 msgr2=0x7f836c19ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:00.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.869+0000 7f836be00700 1 --2- 172.21.15.123:0/1884652684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f836c10b660 0x7f836c19ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:00.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.869+0000 7f836be00700 1 -- 172.21.15.123:0/1884652684 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f836c19c1a0 con 0x7f836c105f90 2024-05-08T06:09:00.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.869+0000 7f836be00700 1 --2- 172.21.15.123:0/1884652684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f836c105f90 0x7f836c1a1dc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8358000c00 tx=0x7f83580057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:00.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.869+0000 7f8369600700 1 -- 172.21.15.123:0/1884652684 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8358027070 con 0x7f836c105f90 2024-05-08T06:09:00.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.869+0000 7f8369600700 1 -- 172.21.15.123:0/1884652684 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8358022470 con 0x7f836c105f90 2024-05-08T06:09:00.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.869+0000 7f8369600700 1 -- 172.21.15.123:0/1884652684 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8358017650 con 0x7f836c105f90 2024-05-08T06:09:00.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.869+0000 7f8371e00700 1 -- 172.21.15.123:0/1884652684 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f836c19c430 con 0x7f836c105f90 2024-05-08T06:09:00.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.869+0000 7f8371e00700 1 -- 172.21.15.123:0/1884652684 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f836c1a8750 con 0x7f836c105f90 2024-05-08T06:09:00.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.873+0000 7f8369600700 1 -- 172.21.15.123:0/1884652684 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f835802a070 con 0x7f836c105f90 2024-05-08T06:09:00.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.874+0000 7f837dd85b80 1 Processor -- start 2024-05-08T06:09:00.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.874+0000 7f837dd85b80 1 -- start start 2024-05-08T06:09:00.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.875+0000 7f8369600700 1 --2- 172.21.15.123:0/1884652684 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f83540b2900 0x7f83540b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:00.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.875+0000 7f837dd85b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649d0099d50 0x5649d009a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:00.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.875+0000 7f837dd85b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d009a670 0x5649d00943e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:00.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.875+0000 7f837dd85b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5649d0094a00 0x5649d0094e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:00.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.875+0000 7f837dd85b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5649cffe69d0 con 0x5649d009a670 2024-05-08T06:09:00.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.875+0000 7f837dd85b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5649cff84710 con 0x5649d0094a00 2024-05-08T06:09:00.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.875+0000 7f837dd85b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5649cffe2990 con 0x5649d0099d50 2024-05-08T06:09:00.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.875+0000 7f8369600700 1 -- 172.21.15.123:0/1884652684 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f83580e2e40 con 0x7f836c105f90 2024-05-08T06:09:00.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.875+0000 7f8370a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649d0099d50 0x5649d009a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:00.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.875+0000 7f836be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d009a670 0x5649d00943e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:00.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.875+0000 7f8371400700 1 --2- 172.21.15.123:0/1884652684 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f83540b2900 0x7f83540b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:00.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.875+0000 7f8370a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649d0099d50 0x5649d009a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:40444/0 (socket says 172.21.15.123:40444) 2024-05-08T06:09:00.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.875+0000 7f8370a00700 1 -- 172.21.15.123:0/1373647484 learned_addr learned my addr 172.21.15.123:0/1373647484 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:09:00.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.875+0000 7f8371400700 1 --2- 172.21.15.123:0/1373647484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5649d0094a00 0x5649d0094e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:00.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.876+0000 7f836be00700 1 -- 172.21.15.123:0/1373647484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649d0099d50 msgr2=0x5649d009a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:00.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.876+0000 7f836be00700 1 --2- 172.21.15.123:0/1373647484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649d0099d50 0x5649d009a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:00.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.876+0000 7f836be00700 1 -- 172.21.15.123:0/1373647484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5649d0094a00 msgr2=0x5649d0094e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:00.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.876+0000 7f836be00700 1 --2- 172.21.15.123:0/1373647484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5649d0094a00 0x5649d0094e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:00.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.876+0000 7f836be00700 1 -- 172.21.15.123:0/1373647484 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649cff33880 con 0x5649d009a670 2024-05-08T06:09:00.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.876+0000 7f8371400700 1 --2- 172.21.15.123:0/1373647484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5649d0094a00 0x5649d0094e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:09:00.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.876+0000 7f836be00700 1 --2- 172.21.15.123:0/1373647484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d009a670 0x5649d00943e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f835800ffd0 tx=0x7f8358009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75c300f8d41926f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:00.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.876+0000 7f8371400700 1 --2- 172.21.15.123:0/1884652684 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f83540b2900 0x7f83540b4dc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x5649d00953c0 tx=0x7f8364009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:00.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.876+0000 7f835e000700 1 -- 172.21.15.123:0/1373647484 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83580920c0 con 0x5649d009a670 2024-05-08T06:09:00.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.877+0000 7f835e000700 1 -- 172.21.15.123:0/1373647484 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83580a9fb0 con 0x5649d009a670 2024-05-08T06:09:00.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.877+0000 7f835e000700 1 -- 172.21.15.123:0/1373647484 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83580b3270 con 0x5649d009a670 2024-05-08T06:09:00.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.877+0000 7f837dd85b80 1 -- 172.21.15.123:0/1373647484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d009a670 msgr2=0x5649d00943e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:00.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.877+0000 7f837dd85b80 1 --2- 172.21.15.123:0/1373647484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d009a670 0x5649d00943e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f835800ffd0 tx=0x7f8358009ca0 comp rx=0 tx=0).stop 2024-05-08T06:09:00.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.877+0000 7f837dd85b80 1 -- 172.21.15.123:0/1373647484 shutdown_connections 2024-05-08T06:09:00.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.877+0000 7f837dd85b80 1 --2- 172.21.15.123:0/1373647484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5649d0094a00 0x5649d0094e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:00.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.877+0000 7f837dd85b80 1 --2- 172.21.15.123:0/1373647484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649d0099d50 0x5649d009a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:00.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.877+0000 7f837dd85b80 1 --2- 172.21.15.123:0/1373647484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d009a670 0x5649d00943e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:00.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.878+0000 7f837dd85b80 1 -- 172.21.15.123:0/1373647484 >> 172.21.15.123:0/1373647484 conn(0x5649d000c680 msgr2=0x5649d008ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:00.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.878+0000 7f837dd85b80 1 -- 172.21.15.123:0/1373647484 shutdown_connections 2024-05-08T06:09:00.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.878+0000 7f837dd85b80 1 -- 172.21.15.123:0/1373647484 wait complete. 2024-05-08T06:09:00.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.878+0000 7f837dd85b80 1 -- start start 2024-05-08T06:09:00.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.878+0000 7f837dd85b80 1 Processor -- start 2024-05-08T06:09:00.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.879+0000 7f837dd85b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5649d0094a00 0x5649d00a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:00.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.879+0000 7f837dd85b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649d0099d50 0x5649d009d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:00.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.879+0000 7f837dd85b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d009a670 0x5649d009dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:00.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.879+0000 7f837dd85b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5649d00a0ec0 con 0x5649d009a670 2024-05-08T06:09:00.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.879+0000 7f837dd85b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5649cff4cad0 con 0x5649d0094a00 2024-05-08T06:09:00.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.879+0000 7f837dd85b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5649d00a1040 con 0x5649d0099d50 2024-05-08T06:09:00.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.879+0000 7f837dd85b80 10 client.0 _ll_register_callbacks cb 0x5649cff8dc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:09:00.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.879+0000 7f8371400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d009a670 0x5649d009dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:00.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.879+0000 7f836be00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649d0099d50 0x5649d009d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:00.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.879+0000 7f8370a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5649d0094a00 0x5649d00a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:00.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.879+0000 7f8371400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d009a670 0x5649d009dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47930/0 (socket says 172.21.15.123:47930) 2024-05-08T06:09:00.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.879+0000 7f8371400700 1 -- 172.21.15.123:0/4240445754 learned_addr learned my addr 172.21.15.123:0/4240445754 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:09:00.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.879+0000 7f836be00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649d0099d50 0x5649d009d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:40450/0 (socket says 172.21.15.123:40450) 2024-05-08T06:09:00.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.880+0000 7f836be00700 1 -- 172.21.15.123:0/4240445754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5649d0094a00 msgr2=0x5649d00a03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:00.882 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.880+0000 7f836be00700 1 --2- 172.21.15.123:0/4240445754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5649d0094a00 0x5649d00a03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:00.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.880+0000 7f836be00700 1 -- 172.21.15.123:0/4240445754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d009a670 msgr2=0x5649d009dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:00.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.880+0000 7f836be00700 1 --2- 172.21.15.123:0/4240445754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d009a670 0x5649d009dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:00.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.880+0000 7f836be00700 1 -- 172.21.15.123:0/4240445754 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649cffac3d0 con 0x5649d0099d50 2024-05-08T06:09:00.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.880+0000 7f836be00700 1 --2- 172.21.15.123:0/4240445754 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649d0099d50 0x5649d009d750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f83580aa1a0 tx=0x7f83580b3270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e01e00aaa30728cf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:00.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.880+0000 7f8352a00700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-05-08T06:09:00.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.880+0000 7f8352a00700 1 -- 172.21.15.123:0/4240445754 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83580aa4a0 con 0x5649d0099d50 2024-05-08T06:09:00.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.880+0000 7f837dd85b80 1 -- 172.21.15.123:0/4240445754 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83580a9fb0 con 0x5649d0099d50 2024-05-08T06:09:00.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.880+0000 7f8352a00700 1 -- 172.21.15.123:0/4240445754 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f8358011dd0 con 0x5649d0099d50 2024-05-08T06:09:00.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.880+0000 7f8352a00700 1 -- 172.21.15.123:0/4240445754 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f83580ae850 con 0x5649d0099d50 2024-05-08T06:09:00.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.881+0000 7f837dd85b80 10 client.24862 fetch_fsmap learned FSMap version 23 2024-05-08T06:09:00.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.881+0000 7f837dd85b80 1 -- 172.21.15.123:0/4240445754 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5649d00a0ec0 con 0x5649d0099d50 2024-05-08T06:09:00.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.881+0000 7f8352a00700 1 -- 172.21.15.123:0/4240445754 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83580a9fb0 con 0x5649d0099d50 2024-05-08T06:09:00.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.881+0000 7f8352a00700 1 -- 172.21.15.123:0/4240445754 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f83580aa800 con 0x5649d0099d50 2024-05-08T06:09:00.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.881+0000 7f837dd85b80 10 client.24862 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T06:09:00.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.881+0000 7f837dd85b80 10 client.24862 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T06:09:00.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.881+0000 7f837dd85b80 20 client.24862 populate_metadata read hostname 'smithi123' 2024-05-08T06:09:00.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.881+0000 7f837dd85b80 1 --2- 172.21.15.123:0/4240445754 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x5649d00ada30 0x5649d00aeea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:00.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.881+0000 7f837dd85b80 4 client.24862 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:09:00.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.881+0000 7f837dd85b80 1 -- 172.21.15.123:0/4240445754 --> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649d00610f0 con 0x5649d00ada30 2024-05-08T06:09:00.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.882+0000 7f8370a00700 1 --2- 172.21.15.123:0/4240445754 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x5649d00ada30 0x5649d00aeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:00.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.882+0000 7f8370a00700 1 --2- 172.21.15.123:0/4240445754 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x5649d00ada30 0x5649d00aeea0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c19978417ee99e82 server_cookie=744a725b360ba07b in_seq=0 out_seq=0 2024-05-08T06:09:00.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.882+0000 7f8352a00700 10 client.24862 ms_handle_connect on v2:172.21.15.192:6838/3598682360 2024-05-08T06:09:00.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.883+0000 7f8352a00700 1 -- 172.21.15.123:0/4240445754 <== mds.2 v2:172.21.15.192:6838/3598682360 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5649d00610f0 con 0x5649d00ada30 2024-05-08T06:09:00.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.883+0000 7f8352a00700 10 client.24862 handle_command_reply: tid=0 2024-05-08T06:09:00.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.883+0000 7f837dd85b80 1 client.24862 shutdown 2024-05-08T06:09:00.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.883+0000 7f837dd85b80 1 -- 172.21.15.123:0/4240445754 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649d0099d50 msgr2=0x5649d009d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:00.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.883+0000 7f837dd85b80 1 --2- 172.21.15.123:0/4240445754 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649d0099d50 0x5649d009d750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f83580aa1a0 tx=0x7f83580b3270 comp rx=0 tx=0).stop 2024-05-08T06:09:00.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.884+0000 7f837dd85b80 1 -- 172.21.15.123:0/4240445754 shutdown_connections 2024-05-08T06:09:00.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.884+0000 7f837dd85b80 1 --2- 172.21.15.123:0/4240445754 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x5649d00ada30 0x5649d00aeea0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:00.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.884+0000 7f837dd85b80 1 --2- 172.21.15.123:0/4240445754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d009a670 0x5649d009dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:00.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.884+0000 7f837dd85b80 1 --2- 172.21.15.123:0/4240445754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5649d0094a00 0x5649d00a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:00.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.884+0000 7f837dd85b80 1 --2- 172.21.15.123:0/4240445754 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649d0099d50 0x5649d009d750 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:00.886 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.884+0000 7f8352a00700 0 client.24862 ms_handle_reset on v2:172.21.15.192:6838/3598682360 2024-05-08T06:09:00.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.884+0000 7f837dd85b80 1 -- 172.21.15.123:0/4240445754 >> 172.21.15.123:0/4240445754 conn(0x5649d000c680 msgr2=0x5649d0089700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:00.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.884+0000 7f837dd85b80 1 -- 172.21.15.123:0/4240445754 shutdown_connections 2024-05-08T06:09:00.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.884+0000 7f837dd85b80 1 -- 172.21.15.123:0/4240445754 wait complete. 2024-05-08T06:09:00.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.884+0000 7f837dd85b80 20 client.24862 trim_cache size 0 max 16384 2024-05-08T06:09:00.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.906+0000 7f837dd85b80 1 Processor -- start 2024-05-08T06:09:00.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.907+0000 7f837dd85b80 1 -- start start 2024-05-08T06:09:00.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.907+0000 7f837dd85b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649d0094a00 0x5649d00943e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:00.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.907+0000 7f837dd85b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5649d0099d50 0x5649d009b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:00.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.907+0000 7f837dd85b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d009a670 0x5649d0089810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:00.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.907+0000 7f837dd85b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5649d00ad1e0 con 0x5649d009a670 2024-05-08T06:09:00.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.907+0000 7f837dd85b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5649d00ad8b0 con 0x5649d0099d50 2024-05-08T06:09:00.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.907+0000 7f837dd85b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5649d00adf80 con 0x5649d0094a00 2024-05-08T06:09:00.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.907+0000 7f8370a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649d0094a00 0x5649d00943e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:00.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.907+0000 7f8370a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649d0094a00 0x5649d00943e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:40460/0 (socket says 172.21.15.123:40460) 2024-05-08T06:09:00.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.907+0000 7f8370a00700 1 -- 172.21.15.123:0/3486813302 learned_addr learned my addr 172.21.15.123:0/3486813302 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:09:00.908 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.907+0000 7f8371400700 1 --2- 172.21.15.123:0/3486813302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d009a670 0x5649d0089810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:00.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.907+0000 7f836be00700 1 --2- 172.21.15.123:0/3486813302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5649d0099d50 0x5649d009b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:00.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.908+0000 7f8370a00700 1 -- 172.21.15.123:0/3486813302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5649d0099d50 msgr2=0x5649d009b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:00.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.908+0000 7f8370a00700 1 --2- 172.21.15.123:0/3486813302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5649d0099d50 0x5649d009b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:00.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.908+0000 7f8370a00700 1 -- 172.21.15.123:0/3486813302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d009a670 msgr2=0x5649d0089810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:00.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.908+0000 7f8370a00700 1 --2- 172.21.15.123:0/3486813302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d009a670 0x5649d0089810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:00.909 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.908+0000 7f8370a00700 1 -- 172.21.15.123:0/3486813302 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649d008b850 con 0x5649d0094a00 2024-05-08T06:09:00.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.908+0000 7f8370a00700 1 --2- 172.21.15.123:0/3486813302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649d0094a00 0x5649d00943e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x5649d00af640 tx=0x7f8360009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e25cf075fded9e70 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:00.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.908+0000 7f835d600700 1 -- 172.21.15.123:0/3486813302 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8360025070 con 0x5649d0094a00 2024-05-08T06:09:00.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.908+0000 7f835d600700 1 -- 172.21.15.123:0/3486813302 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8360010e30 con 0x5649d0094a00 2024-05-08T06:09:00.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.908+0000 7f835d600700 1 -- 172.21.15.123:0/3486813302 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8360020370 con 0x5649d0094a00 2024-05-08T06:09:00.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.909+0000 7f837dd85b80 1 -- 172.21.15.123:0/3486813302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649d0094a00 msgr2=0x5649d00943e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:00.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.909+0000 7f837dd85b80 1 --2- 172.21.15.123:0/3486813302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649d0094a00 0x5649d00943e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x5649d00af640 tx=0x7f8360009260 comp rx=0 tx=0).stop 2024-05-08T06:09:00.912 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.909+0000 7f837dd85b80 1 -- 172.21.15.123:0/3486813302 shutdown_connections 2024-05-08T06:09:00.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.909+0000 7f837dd85b80 1 --2- 172.21.15.123:0/3486813302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d009a670 0x5649d0089810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:00.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.909+0000 7f837dd85b80 1 --2- 172.21.15.123:0/3486813302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649d0094a00 0x5649d00943e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:00.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.909+0000 7f837dd85b80 1 --2- 172.21.15.123:0/3486813302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5649d0099d50 0x5649d009b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:00.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.909+0000 7f837dd85b80 1 -- 172.21.15.123:0/3486813302 >> 172.21.15.123:0/3486813302 conn(0x5649d000c680 msgr2=0x5649d009c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:00.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.909+0000 7f837dd85b80 1 -- 172.21.15.123:0/3486813302 shutdown_connections 2024-05-08T06:09:00.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.909+0000 7f837dd85b80 1 -- 172.21.15.123:0/3486813302 wait complete. 2024-05-08T06:09:00.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.910+0000 7f837dd85b80 1 -- start start 2024-05-08T06:09:00.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.910+0000 7f837dd85b80 1 Processor -- start 2024-05-08T06:09:00.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.910+0000 7f837dd85b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d0094a00 0x5649d00b8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:00.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.910+0000 7f837dd85b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649d0099d50 0x5649d00b8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:00.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.910+0000 7f837dd85b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5649d009a670 0x5649d00b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:00.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.910+0000 7f837dd85b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5649d00aed20 con 0x5649d0094a00 2024-05-08T06:09:00.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.910+0000 7f837dd85b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5649d00ae650 con 0x5649d009a670 2024-05-08T06:09:00.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.910+0000 7f837dd85b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5649d00af3f0 con 0x5649d0099d50 2024-05-08T06:09:00.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.910+0000 7f837dd85b80 10 client.0 _ll_register_callbacks cb 0x5649cff8dc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:09:00.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.911+0000 7f8370a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d0094a00 0x5649d00b8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:00.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.911+0000 7f8370a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d0094a00 0x5649d00b8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47942/0 (socket says 172.21.15.123:47942) 2024-05-08T06:09:00.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.911+0000 7f8370a00700 1 -- 172.21.15.123:0/3155118213 learned_addr learned my addr 172.21.15.123:0/3155118213 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:09:00.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.911+0000 7f836be00700 1 --2- 172.21.15.123:0/3155118213 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649d0099d50 0x5649d00b8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:00.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.911+0000 7f8371400700 1 --2- 172.21.15.123:0/3155118213 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5649d009a670 0x5649d00b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:00.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.911+0000 7f8370a00700 1 -- 172.21.15.123:0/3155118213 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649d0099d50 msgr2=0x5649d00b8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:00.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.911+0000 7f8370a00700 1 --2- 172.21.15.123:0/3155118213 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649d0099d50 0x5649d00b8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:00.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.911+0000 7f8370a00700 1 -- 172.21.15.123:0/3155118213 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5649d009a670 msgr2=0x5649d00b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:00.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.911+0000 7f8370a00700 1 --2- 172.21.15.123:0/3155118213 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5649d009a670 0x5649d00b32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:00.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.911+0000 7f8370a00700 1 -- 172.21.15.123:0/3155118213 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649d00afe00 con 0x5649d0094a00 2024-05-08T06:09:00.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.911+0000 7f8370a00700 1 --2- 172.21.15.123:0/3155118213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d0094a00 0x5649d00b8320 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f83600037c0 tx=0x7f83600037f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3da7f540c543414 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:00.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.911+0000 7f8352a00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-08T06:09:00.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.912+0000 7f8352a00700 1 -- 172.21.15.123:0/3155118213 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8360030060 con 0x5649d0094a00 2024-05-08T06:09:00.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.912+0000 7f8352a00700 1 -- 172.21.15.123:0/3155118213 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f8360004050 con 0x5649d0094a00 2024-05-08T06:09:00.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.912+0000 7f837dd85b80 1 -- 172.21.15.123:0/3155118213 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8360010e30 con 0x5649d0094a00 2024-05-08T06:09:00.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.912+0000 7f8352a00700 1 -- 172.21.15.123:0/3155118213 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8360025040 con 0x5649d0094a00 2024-05-08T06:09:00.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.912+0000 7f837dd85b80 10 client.14760 fetch_fsmap learned FSMap version 23 2024-05-08T06:09:00.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.912+0000 7f837dd85b80 1 -- 172.21.15.123:0/3155118213 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5649d0121dd0 con 0x5649d0094a00 2024-05-08T06:09:00.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.912+0000 7f8352a00700 1 -- 172.21.15.123:0/3155118213 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8360010e30 con 0x5649d0094a00 2024-05-08T06:09:00.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.913+0000 7f8352a00700 1 -- 172.21.15.123:0/3155118213 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f8360004850 con 0x5649d0094a00 2024-05-08T06:09:00.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.913+0000 7f837dd85b80 10 client.14760 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T06:09:00.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.913+0000 7f837dd85b80 10 client.14760 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T06:09:00.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.913+0000 7f837dd85b80 20 client.14760 populate_metadata read hostname 'smithi123' 2024-05-08T06:09:00.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.913+0000 7f837dd85b80 1 --2- 172.21.15.123:0/3155118213 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x5649d01265c0 0x5649d01252d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:00.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.913+0000 7f837dd85b80 4 client.14760 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T06:09:00.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.913+0000 7f837dd85b80 1 -- 172.21.15.123:0/3155118213 --> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] -- command(tid 0: {"prefix": "status"}) -- 0x5649cfff43b0 con 0x5649d01265c0 2024-05-08T06:09:00.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.913+0000 7f836be00700 1 --2- 172.21.15.123:0/3155118213 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x5649d01265c0 0x5649d01252d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:00.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.914+0000 7f836be00700 1 --2- 172.21.15.123:0/3155118213 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x5649d01265c0 0x5649d01252d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c1600e3d0d10ab7a server_cookie=b102cee89ab38b77 in_seq=0 out_seq=0 2024-05-08T06:09:00.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.914+0000 7f8352a00700 10 client.14760 ms_handle_connect on v2:172.21.15.192:6838/3598682360 2024-05-08T06:09:00.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.914+0000 7f8352a00700 1 -- 172.21.15.123:0/3155118213 <== mds.2 v2:172.21.15.192:6838/3598682360 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5649cfff43b0 con 0x5649d01265c0 2024-05-08T06:09:00.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.914+0000 7f8352a00700 10 client.14760 handle_command_reply: tid=0 2024-05-08T06:09:00.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.914+0000 7f837dd85b80 1 client.14760 shutdown 2024-05-08T06:09:00.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.914+0000 7f837dd85b80 1 -- 172.21.15.123:0/3155118213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d0094a00 msgr2=0x5649d00b8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:00.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.914+0000 7f837dd85b80 1 --2- 172.21.15.123:0/3155118213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d0094a00 0x5649d00b8320 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f83600037c0 tx=0x7f83600037f0 comp rx=0 tx=0).stop 2024-05-08T06:09:00.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.914+0000 7f837dd85b80 1 -- 172.21.15.123:0/3155118213 shutdown_connections 2024-05-08T06:09:00.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.914+0000 7f837dd85b80 1 --2- 172.21.15.123:0/3155118213 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x5649d01265c0 0x5649d01252d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:00.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.914+0000 7f837dd85b80 1 --2- 172.21.15.123:0/3155118213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d0094a00 0x5649d00b8320 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:00.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.914+0000 7f837dd85b80 1 --2- 172.21.15.123:0/3155118213 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5649d009a670 0x5649d00b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:00.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.914+0000 7f837dd85b80 1 --2- 172.21.15.123:0/3155118213 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5649d0099d50 0x5649d00b8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:00.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.914+0000 7f8352a00700 0 client.14760 ms_handle_reset on v2:172.21.15.192:6838/3598682360 2024-05-08T06:09:00.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.914+0000 7f837dd85b80 1 -- 172.21.15.123:0/3155118213 >> 172.21.15.123:0/3155118213 conn(0x5649d000c680 msgr2=0x5649d009b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:00.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.915+0000 7f837dd85b80 1 -- 172.21.15.123:0/3155118213 shutdown_connections 2024-05-08T06:09:00.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.915+0000 7f837dd85b80 1 -- 172.21.15.123:0/3155118213 wait complete. 2024-05-08T06:09:00.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.915+0000 7f837dd85b80 20 client.14760 trim_cache size 0 max 16384 2024-05-08T06:09:00.919 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-05-08T06:09:00.919 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_fsid": "85cfb2e6-0cff-11ef-bc97-c7b262605968", 2024-05-08T06:09:00.919 INFO:teuthology.orchestra.run.smithi123.stdout: "whoami": 2, 2024-05-08T06:09:00.919 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 24496, 2024-05-08T06:09:00.919 INFO:teuthology.orchestra.run.smithi123.stdout: "want_state": "up:active", 2024-05-08T06:09:00.919 INFO:teuthology.orchestra.run.smithi123.stdout: "state": "up:active", 2024-05-08T06:09:00.919 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs", 2024-05-08T06:09:00.919 INFO:teuthology.orchestra.run.smithi123.stdout: "rank_uptime": 166.520767519, 2024-05-08T06:09:00.919 INFO:teuthology.orchestra.run.smithi123.stdout: "mdsmap_epoch": 23, 2024-05-08T06:09:00.919 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch": 82, 2024-05-08T06:09:00.919 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch_barrier": 80, 2024-05-08T06:09:00.920 INFO:teuthology.orchestra.run.smithi123.stdout: "uptime": 224.223727981 2024-05-08T06:09:00.920 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-05-08T06:09:00.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.915+0000 7f8371e00700 1 -- 172.21.15.123:0/1884652684 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f83540b2900 msgr2=0x7f83540b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:00.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.915+0000 7f8371e00700 1 --2- 172.21.15.123:0/1884652684 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f83540b2900 0x7f83540b4dc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x5649d00953c0 tx=0x7f8364009920 comp rx=0 tx=0).stop 2024-05-08T06:09:00.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.915+0000 7f8371e00700 1 -- 172.21.15.123:0/1884652684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f836c105f90 msgr2=0x7f836c1a1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:00.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.916+0000 7f8371e00700 1 --2- 172.21.15.123:0/1884652684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f836c105f90 0x7f836c1a1dc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8358000c00 tx=0x7f83580057c0 comp rx=0 tx=0).stop 2024-05-08T06:09:00.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.916+0000 7f8371e00700 1 -- 172.21.15.123:0/1884652684 shutdown_connections 2024-05-08T06:09:00.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.916+0000 7f8371e00700 1 --2- 172.21.15.123:0/1884652684 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f83540b2900 0x7f83540b4dc0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:00.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.916+0000 7f8371e00700 1 --2- 172.21.15.123:0/1884652684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f836c10b660 0x7f836c19ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:00.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.916+0000 7f8371e00700 1 --2- 172.21.15.123:0/1884652684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f836c1055d0 0x7f836c1a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:00.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.916+0000 7f8371e00700 1 --2- 172.21.15.123:0/1884652684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f836c105f90 0x7f836c1a1dc0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:00.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.916+0000 7f8371e00700 1 -- 172.21.15.123:0/1884652684 >> 172.21.15.123:0/1884652684 conn(0x7f836c101080 msgr2=0x7f836c102780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:00.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.916+0000 7f8371e00700 1 -- 172.21.15.123:0/1884652684 shutdown_connections 2024-05-08T06:09:00.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:00.916+0000 7f8371e00700 1 -- 172.21.15.123:0/1884652684 wait complete. 2024-05-08T06:09:01.824 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell mds.a status 2024-05-08T06:09:02.264 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:09:02.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:02 smithi163 ceph-mon[34520]: pgmap v417: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-05-08T06:09:02.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:02 smithi192 ceph-mon[34664]: pgmap v417: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-05-08T06:09:02.595 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:02 smithi123 ceph-mon[29787]: pgmap v417: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-05-08T06:09:04.268 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:04 smithi123 ceph-mon[29787]: pgmap v418: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:09:04.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:04 smithi163 ceph-mon[34520]: pgmap v418: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:09:04.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:04 smithi192 ceph-mon[34664]: pgmap v418: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:09:04.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.616+0000 7f5f6bc00700 1 -- 172.21.15.123:0/2887770658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f64104100 msgr2=0x7f5f64108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:04.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.616+0000 7f5f6bc00700 1 --2- 172.21.15.123:0/2887770658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f64104100 0x7f5f64108120 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5f500099c0 tx=0x7f5f50009cd0 comp rx=0 tx=0).stop 2024-05-08T06:09:04.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.616+0000 7f5f6bc00700 1 -- 172.21.15.123:0/2887770658 shutdown_connections 2024-05-08T06:09:04.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.616+0000 7f5f6bc00700 1 --2- 172.21.15.123:0/2887770658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f64103750 0x7f5f64103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:04.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.616+0000 7f5f6bc00700 1 --2- 172.21.15.123:0/2887770658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f64108850 0x7f5f641103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:04.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.616+0000 7f5f6bc00700 1 --2- 172.21.15.123:0/2887770658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f64104100 0x7f5f64108120 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:04.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.616+0000 7f5f6bc00700 1 -- 172.21.15.123:0/2887770658 >> 172.21.15.123:0/2887770658 conn(0x7f5f6407a600 msgr2=0x7f5f6407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:04.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.616+0000 7f5f6bc00700 1 -- 172.21.15.123:0/2887770658 shutdown_connections 2024-05-08T06:09:04.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.616+0000 7f5f6bc00700 1 -- 172.21.15.123:0/2887770658 wait complete. 2024-05-08T06:09:04.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.617+0000 7f5f6bc00700 1 Processor -- start 2024-05-08T06:09:04.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.617+0000 7f5f6bc00700 1 -- start start 2024-05-08T06:09:04.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.618+0000 7f5f6bc00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f64103750 0x7f5f6419bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:04.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.618+0000 7f5f6bc00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f64104100 0x7f5f6419c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:04.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.618+0000 7f5f6bc00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f64108850 0x7f5f641a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:04.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.618+0000 7f5f6bc00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5f64113680 con 0x7f5f64108850 2024-05-08T06:09:04.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.618+0000 7f5f6bc00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5f64113500 con 0x7f5f64103750 2024-05-08T06:09:04.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.618+0000 7f5f6bc00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5f64113800 con 0x7f5f64104100 2024-05-08T06:09:04.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.618+0000 7f5f69200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f64103750 0x7f5f6419bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:04.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.618+0000 7f5f63e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f64104100 0x7f5f6419c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:04.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.618+0000 7f5f69200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f64103750 0x7f5f6419bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:54726/0 (socket says 172.21.15.123:54726) 2024-05-08T06:09:04.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.618+0000 7f5f69200700 1 -- 172.21.15.123:0/2950144218 learned_addr learned my addr 172.21.15.123:0/2950144218 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:09:04.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.618+0000 7f5f69c00700 1 --2- 172.21.15.123:0/2950144218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f64108850 0x7f5f641a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:04.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.619+0000 7f5f69200700 1 -- 172.21.15.123:0/2950144218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f64104100 msgr2=0x7f5f6419c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:04.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.619+0000 7f5f69200700 1 --2- 172.21.15.123:0/2950144218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f64104100 0x7f5f6419c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:04.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.619+0000 7f5f69200700 1 -- 172.21.15.123:0/2950144218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f64108850 msgr2=0x7f5f641a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:04.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.619+0000 7f5f69200700 1 --2- 172.21.15.123:0/2950144218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f64108850 0x7f5f641a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:04.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.619+0000 7f5f69200700 1 -- 172.21.15.123:0/2950144218 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f641a0de0 con 0x7f5f64103750 2024-05-08T06:09:04.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.619+0000 7f5f69c00700 1 --2- 172.21.15.123:0/2950144218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f64108850 0x7f5f641a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:09:04.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.619+0000 7f5f63e00700 1 --2- 172.21.15.123:0/2950144218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f64104100 0x7f5f6419c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:09:04.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.619+0000 7f5f69200700 1 --2- 172.21.15.123:0/2950144218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f64103750 0x7f5f6419bdd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800e9a0 tx=0x7f5f5800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:04.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.619+0000 7f5f61600700 1 -- 172.21.15.123:0/2950144218 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5800cbd0 con 0x7f5f64103750 2024-05-08T06:09:04.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.619+0000 7f5f6bc00700 1 -- 172.21.15.123:0/2950144218 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f641a10d0 con 0x7f5f64103750 2024-05-08T06:09:04.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.619+0000 7f5f61600700 1 -- 172.21.15.123:0/2950144218 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f5800cd70 con 0x7f5f64103750 2024-05-08T06:09:04.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.619+0000 7f5f61600700 1 -- 172.21.15.123:0/2950144218 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f58018680 con 0x7f5f64103750 2024-05-08T06:09:04.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.620+0000 7f5f6bc00700 1 -- 172.21.15.123:0/2950144218 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f641a8950 con 0x7f5f64103750 2024-05-08T06:09:04.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.623+0000 7f5f61600700 1 -- 172.21.15.123:0/2950144218 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f5f58010430 con 0x7f5f64103750 2024-05-08T06:09:04.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.625+0000 7f5f61600700 1 --2- 172.21.15.123:0/2950144218 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5f4c0b2880 0x7f5f4c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:04.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.625+0000 7f5f61600700 1 -- 172.21.15.123:0/2950144218 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f5f580e2500 con 0x7f5f64103750 2024-05-08T06:09:04.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.625+0000 7f5f63e00700 1 --2- 172.21.15.123:0/2950144218 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5f4c0b2880 0x7f5f4c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:04.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.626+0000 7f5f63e00700 1 --2- 172.21.15.123:0/2950144218 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5f4c0b2880 0x7f5f4c0b4d40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5f64103240 tx=0x7f5f50005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:04.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.627+0000 7f5f76660b80 1 Processor -- start 2024-05-08T06:09:04.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.627+0000 7f5f76660b80 1 -- start start 2024-05-08T06:09:04.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.628+0000 7f5f76660b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b1aff9b70 0x555b1aff9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:04.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.628+0000 7f5f76660b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1affa490 0x555b1aff4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:04.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.628+0000 7f5f76660b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b1aff48a0 0x555b1aff4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:04.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.628+0000 7f5f76660b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555b1af469d0 con 0x555b1affa490 2024-05-08T06:09:04.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.628+0000 7f5f76660b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x555b1aee4710 con 0x555b1aff9b70 2024-05-08T06:09:04.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.628+0000 7f5f76660b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555b1af42990 con 0x555b1aff48a0 2024-05-08T06:09:04.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.628+0000 7f5f69200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b1aff9b70 0x555b1aff9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:04.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.628+0000 7f5f69200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b1aff9b70 0x555b1aff9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:54728/0 (socket says 172.21.15.123:54728) 2024-05-08T06:09:04.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.628+0000 7f5f69200700 1 -- 172.21.15.123:0/1494017940 learned_addr learned my addr 172.21.15.123:0/1494017940 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:09:04.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.628+0000 7f5f63e00700 1 --2- 172.21.15.123:0/1494017940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1affa490 0x555b1aff4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:04.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.628+0000 7f5f69c00700 1 --2- 172.21.15.123:0/1494017940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b1aff48a0 0x555b1aff4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:04.629 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.628+0000 7f5f69200700 1 -- 172.21.15.123:0/1494017940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b1aff48a0 msgr2=0x555b1aff4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:04.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.629+0000 7f5f69200700 1 --2- 172.21.15.123:0/1494017940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b1aff48a0 0x555b1aff4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:04.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.629+0000 7f5f69200700 1 -- 172.21.15.123:0/1494017940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1affa490 msgr2=0x555b1aff4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:04.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.629+0000 7f5f69200700 1 --2- 172.21.15.123:0/1494017940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1affa490 0x555b1aff4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:04.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.629+0000 7f5f69200700 1 -- 172.21.15.123:0/1494017940 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b1ae93880 con 0x555b1aff9b70 2024-05-08T06:09:04.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.629+0000 7f5f63e00700 1 --2- 172.21.15.123:0/1494017940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1affa490 0x555b1aff4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:09:04.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.629+0000 7f5f69200700 1 --2- 172.21.15.123:0/1494017940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b1aff9b70 0x555b1aff9f50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5f580a9a60 tx=0x7f5f580a5990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d061a8e435b5a96 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:04.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.629+0000 7f5f56000700 1 -- 172.21.15.123:0/1494017940 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f580aa1d0 con 0x555b1aff9b70 2024-05-08T06:09:04.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.629+0000 7f5f56000700 1 -- 172.21.15.123:0/1494017940 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f580a9ad0 con 0x555b1aff9b70 2024-05-08T06:09:04.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.629+0000 7f5f56000700 1 -- 172.21.15.123:0/1494017940 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f580a9d30 con 0x555b1aff9b70 2024-05-08T06:09:04.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.630+0000 7f5f76660b80 1 -- 172.21.15.123:0/1494017940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b1aff9b70 msgr2=0x555b1aff9f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:04.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.630+0000 7f5f76660b80 1 --2- 172.21.15.123:0/1494017940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b1aff9b70 0x555b1aff9f50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5f580a9a60 tx=0x7f5f580a5990 comp rx=0 tx=0).stop 2024-05-08T06:09:04.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.630+0000 7f5f76660b80 1 -- 172.21.15.123:0/1494017940 shutdown_connections 2024-05-08T06:09:04.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.630+0000 7f5f76660b80 1 --2- 172.21.15.123:0/1494017940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b1aff48a0 0x555b1aff4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:04.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.630+0000 7f5f76660b80 1 --2- 172.21.15.123:0/1494017940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b1aff9b70 0x555b1aff9f50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:04.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.630+0000 7f5f76660b80 1 --2- 172.21.15.123:0/1494017940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1affa490 0x555b1aff4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:04.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.630+0000 7f5f76660b80 1 -- 172.21.15.123:0/1494017940 >> 172.21.15.123:0/1494017940 conn(0x555b1af6c680 msgr2=0x555b1afeeab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:04.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.631+0000 7f5f76660b80 1 -- 172.21.15.123:0/1494017940 shutdown_connections 2024-05-08T06:09:04.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.631+0000 7f5f76660b80 1 -- 172.21.15.123:0/1494017940 wait complete. 2024-05-08T06:09:04.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.631+0000 7f5f76660b80 1 -- start start 2024-05-08T06:09:04.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.632+0000 7f5f76660b80 1 Processor -- start 2024-05-08T06:09:04.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.632+0000 7f5f76660b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1aff48a0 0x555b1b000190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:04.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.632+0000 7f5f76660b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b1aff9b70 0x555b1affd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:04.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.632+0000 7f5f76660b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b1affa490 0x555b1affda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:04.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.632+0000 7f5f76660b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555b1b000ed0 con 0x555b1aff48a0 2024-05-08T06:09:04.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.632+0000 7f5f76660b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x555b1aeacad0 con 0x555b1affa490 2024-05-08T06:09:04.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.632+0000 7f5f76660b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555b1b001050 con 0x555b1aff9b70 2024-05-08T06:09:04.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.633+0000 7f5f76660b80 10 client.0 _ll_register_callbacks cb 0x555b1aeedc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:09:04.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.633+0000 7f5f69200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1aff48a0 0x555b1b000190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:04.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.633+0000 7f5f69200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1aff48a0 0x555b1b000190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47970/0 (socket says 172.21.15.123:47970) 2024-05-08T06:09:04.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.633+0000 7f5f69200700 1 -- 172.21.15.123:0/302437024 learned_addr learned my addr 172.21.15.123:0/302437024 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:09:04.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.633+0000 7f5f63e00700 1 --2- 172.21.15.123:0/302437024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b1aff9b70 0x555b1affd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:04.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.633+0000 7f5f69c00700 1 --2- 172.21.15.123:0/302437024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b1affa490 0x555b1affda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:04.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.633+0000 7f5f69200700 1 -- 172.21.15.123:0/302437024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b1aff9b70 msgr2=0x555b1affd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:04.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.633+0000 7f5f69200700 1 --2- 172.21.15.123:0/302437024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b1aff9b70 0x555b1affd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:04.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.633+0000 7f5f69200700 1 -- 172.21.15.123:0/302437024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b1affa490 msgr2=0x555b1affda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:04.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.633+0000 7f5f69200700 1 --2- 172.21.15.123:0/302437024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b1affa490 0x555b1affda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:04.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.633+0000 7f5f69200700 1 -- 172.21.15.123:0/302437024 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b1af0c3d0 con 0x555b1aff48a0 2024-05-08T06:09:04.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.634+0000 7f5f69200700 1 --2- 172.21.15.123:0/302437024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1aff48a0 0x555b1b000190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5f58010c30 tx=0x7f5f5800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1c8d7a90aaad59 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:04.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.634+0000 7f5f4aa00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-08T06:09:04.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.634+0000 7f5f4aa00700 1 -- 172.21.15.123:0/302437024 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5806e020 con 0x555b1aff48a0 2024-05-08T06:09:04.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.634+0000 7f5f4aa00700 1 -- 172.21.15.123:0/302437024 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f5f580a60c0 con 0x555b1aff48a0 2024-05-08T06:09:04.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.634+0000 7f5f76660b80 1 -- 172.21.15.123:0/302437024 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f580a9ad0 con 0x555b1aff48a0 2024-05-08T06:09:04.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.634+0000 7f5f4aa00700 1 -- 172.21.15.123:0/302437024 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f5f580c4050 con 0x555b1aff48a0 2024-05-08T06:09:04.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.634+0000 7f5f76660b80 10 client.14772 fetch_fsmap learned FSMap version 23 2024-05-08T06:09:04.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.634+0000 7f5f76660b80 1 -- 172.21.15.123:0/302437024 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x555b1b00b750 con 0x555b1aff48a0 2024-05-08T06:09:04.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.635+0000 7f5f4aa00700 1 -- 172.21.15.123:0/302437024 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f580a9ad0 con 0x555b1aff48a0 2024-05-08T06:09:04.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.635+0000 7f5f4aa00700 1 -- 172.21.15.123:0/302437024 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f5f58010720 con 0x555b1aff48a0 2024-05-08T06:09:04.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.636+0000 7f5f76660b80 10 client.14772 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T06:09:04.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.636+0000 7f5f76660b80 10 client.14772 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T06:09:04.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.636+0000 7f5f76660b80 20 client.14772 populate_metadata read hostname 'smithi123' 2024-05-08T06:09:04.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.636+0000 7f5f76660b80 1 --2- 172.21.15.123:0/302437024 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x555b1b00caa0 0x555b1b00ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:04.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.636+0000 7f5f76660b80 4 client.14772 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:09:04.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.636+0000 7f5f76660b80 1 -- 172.21.15.123:0/302437024 --> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b1afc10f0 con 0x555b1b00caa0 2024-05-08T06:09:04.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.636+0000 7f5f63e00700 1 --2- 172.21.15.123:0/302437024 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x555b1b00caa0 0x555b1b00ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:04.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.637+0000 7f5f4aa00700 10 client.14772 ms_handle_connect on v2:172.21.15.123:6840/1571726231 2024-05-08T06:09:04.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.637+0000 7f5f63e00700 1 --2- 172.21.15.123:0/302437024 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x555b1b00caa0 0x555b1b00ec20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=20a31bf766761f2c server_cookie=47cb22d3338b5a3b in_seq=0 out_seq=0 2024-05-08T06:09:04.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.638+0000 7f5f4aa00700 1 -- 172.21.15.123:0/302437024 <== mds.1 v2:172.21.15.123:6840/1571726231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555b1afc10f0 con 0x555b1b00caa0 2024-05-08T06:09:04.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.638+0000 7f5f4aa00700 10 client.14772 handle_command_reply: tid=0 2024-05-08T06:09:04.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.638+0000 7f5f76660b80 1 client.14772 shutdown 2024-05-08T06:09:04.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.638+0000 7f5f76660b80 1 -- 172.21.15.123:0/302437024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1aff48a0 msgr2=0x555b1b000190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:04.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.638+0000 7f5f76660b80 1 --2- 172.21.15.123:0/302437024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1aff48a0 0x555b1b000190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5f58010c30 tx=0x7f5f5800ef60 comp rx=0 tx=0).stop 2024-05-08T06:09:04.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.638+0000 7f5f76660b80 1 -- 172.21.15.123:0/302437024 shutdown_connections 2024-05-08T06:09:04.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.638+0000 7f5f76660b80 1 --2- 172.21.15.123:0/302437024 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x555b1b00caa0 0x555b1b00ec20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:04.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.638+0000 7f5f76660b80 1 --2- 172.21.15.123:0/302437024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1aff48a0 0x555b1b000190 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:04.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.638+0000 7f5f76660b80 1 --2- 172.21.15.123:0/302437024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b1affa490 0x555b1affda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:04.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.638+0000 7f5f76660b80 1 --2- 172.21.15.123:0/302437024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b1aff9b70 0x555b1affd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:04.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.638+0000 7f5f4aa00700 0 client.14772 ms_handle_reset on v2:172.21.15.123:6840/1571726231 2024-05-08T06:09:04.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.638+0000 7f5f76660b80 1 -- 172.21.15.123:0/302437024 >> 172.21.15.123:0/302437024 conn(0x555b1af6c680 msgr2=0x555b1afee140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:04.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.639+0000 7f5f76660b80 1 -- 172.21.15.123:0/302437024 shutdown_connections 2024-05-08T06:09:04.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.639+0000 7f5f76660b80 1 -- 172.21.15.123:0/302437024 wait complete. 2024-05-08T06:09:04.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.639+0000 7f5f76660b80 20 client.14772 trim_cache size 0 max 16384 2024-05-08T06:09:04.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.661+0000 7f5f76660b80 1 Processor -- start 2024-05-08T06:09:04.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.662+0000 7f5f76660b80 1 -- start start 2024-05-08T06:09:04.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.662+0000 7f5f76660b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1aff48a0 0x555b1aff4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:04.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.662+0000 7f5f76660b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b1aff9b70 0x555b1b00e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:04.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.662+0000 7f5f76660b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b1affa490 0x555b1affaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:04.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.662+0000 7f5f76660b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555b1b00cf60 con 0x555b1aff48a0 2024-05-08T06:09:04.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.662+0000 7f5f76660b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x555b1b00d630 con 0x555b1affa490 2024-05-08T06:09:04.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.662+0000 7f5f76660b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555b1b00dd00 con 0x555b1aff9b70 2024-05-08T06:09:04.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.662+0000 7f5f69200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1aff48a0 0x555b1aff4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:04.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.662+0000 7f5f69c00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b1affa490 0x555b1affaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:04.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.662+0000 7f5f69c00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b1affa490 0x555b1affaf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:54746/0 (socket says 172.21.15.123:54746) 2024-05-08T06:09:04.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.662+0000 7f5f69c00700 1 -- 172.21.15.123:0/874399029 learned_addr learned my addr 172.21.15.123:0/874399029 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:09:04.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.662+0000 7f5f63e00700 1 --2- 172.21.15.123:0/874399029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b1aff9b70 0x555b1b00e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:04.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.663+0000 7f5f69c00700 1 -- 172.21.15.123:0/874399029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b1aff9b70 msgr2=0x555b1b00e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:04.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.663+0000 7f5f69c00700 1 --2- 172.21.15.123:0/874399029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b1aff9b70 0x555b1b00e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:04.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.663+0000 7f5f69c00700 1 -- 172.21.15.123:0/874399029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1aff48a0 msgr2=0x555b1aff4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:04.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.663+0000 7f5f69c00700 1 --2- 172.21.15.123:0/874399029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1aff48a0 0x555b1aff4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:04.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.663+0000 7f5f69c00700 1 -- 172.21.15.123:0/874399029 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b1aff0550 con 0x555b1affa490 2024-05-08T06:09:04.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.663+0000 7f5f69c00700 1 --2- 172.21.15.123:0/874399029 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b1affa490 0x555b1affaf80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c007550 tx=0x7f5f5c00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25f56c2a8f4bae05 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:04.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.663+0000 7f5f55600700 1 -- 172.21.15.123:0/874399029 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c01b070 con 0x555b1affa490 2024-05-08T06:09:04.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.664+0000 7f5f55600700 1 -- 172.21.15.123:0/874399029 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f5c017af0 con 0x555b1affa490 2024-05-08T06:09:04.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.664+0000 7f5f55600700 1 -- 172.21.15.123:0/874399029 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c016c40 con 0x555b1affa490 2024-05-08T06:09:04.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.665+0000 7f5f76660b80 1 -- 172.21.15.123:0/874399029 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b1affa490 msgr2=0x555b1affaf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:04.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.665+0000 7f5f76660b80 1 --2- 172.21.15.123:0/874399029 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b1affa490 0x555b1affaf80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c007550 tx=0x7f5f5c00b040 comp rx=0 tx=0).stop 2024-05-08T06:09:04.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.665+0000 7f5f76660b80 1 -- 172.21.15.123:0/874399029 shutdown_connections 2024-05-08T06:09:04.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.665+0000 7f5f76660b80 1 --2- 172.21.15.123:0/874399029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1aff48a0 0x555b1aff4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:04.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.665+0000 7f5f76660b80 1 --2- 172.21.15.123:0/874399029 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b1affa490 0x555b1affaf80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:04.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.665+0000 7f5f76660b80 1 --2- 172.21.15.123:0/874399029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b1aff9b70 0x555b1b00e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:04.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.665+0000 7f5f76660b80 1 -- 172.21.15.123:0/874399029 >> 172.21.15.123:0/874399029 conn(0x555b1af6c680 msgr2=0x555b1affa870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:04.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.665+0000 7f5f76660b80 1 -- 172.21.15.123:0/874399029 shutdown_connections 2024-05-08T06:09:04.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.665+0000 7f5f76660b80 1 -- 172.21.15.123:0/874399029 wait complete. 2024-05-08T06:09:04.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.665+0000 7f5f76660b80 1 -- start start 2024-05-08T06:09:04.666 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.666+0000 7f5f76660b80 1 Processor -- start 2024-05-08T06:09:04.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.666+0000 7f5f76660b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b1aff48a0 0x555b1b0180a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:04.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.666+0000 7f5f76660b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b1aff9b70 0x555b1b0185e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:04.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.666+0000 7f5f76660b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1affa490 0x555b1b013050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:04.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.666+0000 7f5f76660b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555b1b00eaa0 con 0x555b1affa490 2024-05-08T06:09:04.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.666+0000 7f5f76660b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x555b1b00e3d0 con 0x555b1aff48a0 2024-05-08T06:09:04.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.666+0000 7f5f76660b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555b1b00f170 con 0x555b1aff9b70 2024-05-08T06:09:04.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.666+0000 7f5f76660b80 10 client.0 _ll_register_callbacks cb 0x555b1aeedc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:09:04.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.667+0000 7f5f63e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b1aff9b70 0x555b1b0185e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:04.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.667+0000 7f5f69200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b1aff48a0 0x555b1b0180a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:04.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.667+0000 7f5f63e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b1aff9b70 0x555b1b0185e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:40530/0 (socket says 172.21.15.123:40530) 2024-05-08T06:09:04.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.667+0000 7f5f63e00700 1 -- 172.21.15.123:0/3223238245 learned_addr learned my addr 172.21.15.123:0/3223238245 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:09:04.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.667+0000 7f5f69c00700 1 --2- 172.21.15.123:0/3223238245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1affa490 0x555b1b013050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:04.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.667+0000 7f5f63e00700 1 -- 172.21.15.123:0/3223238245 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b1aff48a0 msgr2=0x555b1b0180a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:04.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.667+0000 7f5f63e00700 1 --2- 172.21.15.123:0/3223238245 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b1aff48a0 0x555b1b0180a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:04.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.667+0000 7f5f63e00700 1 -- 172.21.15.123:0/3223238245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1affa490 msgr2=0x555b1b013050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:04.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.667+0000 7f5f63e00700 1 --2- 172.21.15.123:0/3223238245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1affa490 0x555b1b013050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:04.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.667+0000 7f5f63e00700 1 -- 172.21.15.123:0/3223238245 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b1aff4ed0 con 0x555b1aff9b70 2024-05-08T06:09:04.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.667+0000 7f5f69200700 1 --2- 172.21.15.123:0/3223238245 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b1aff48a0 0x555b1b0180a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:09:04.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.667+0000 7f5f63e00700 1 --2- 172.21.15.123:0/3223238245 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b1aff9b70 0x555b1b0185e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5f500099c0 tx=0x7f5f50009cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5480edf267426c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:04.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.667+0000 7f5f4aa00700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-05-08T06:09:04.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.668+0000 7f5f4aa00700 1 -- 172.21.15.123:0/3223238245 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f50003d10 con 0x555b1aff9b70 2024-05-08T06:09:04.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.668+0000 7f5f76660b80 1 -- 172.21.15.123:0/3223238245 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f5c017af0 con 0x555b1aff9b70 2024-05-08T06:09:04.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.668+0000 7f5f4aa00700 1 -- 172.21.15.123:0/3223238245 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f5f50034110 con 0x555b1aff9b70 2024-05-08T06:09:04.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.668+0000 7f5f4aa00700 1 -- 172.21.15.123:0/3223238245 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f5f50034310 con 0x555b1aff9b70 2024-05-08T06:09:04.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.668+0000 7f5f76660b80 10 client.24886 fetch_fsmap learned FSMap version 23 2024-05-08T06:09:04.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.668+0000 7f5f76660b80 1 -- 172.21.15.123:0/3223238245 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x555b1b079310 con 0x555b1aff9b70 2024-05-08T06:09:04.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.669+0000 7f5f4aa00700 1 -- 172.21.15.123:0/3223238245 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f5c017af0 con 0x555b1aff9b70 2024-05-08T06:09:04.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.669+0000 7f5f4aa00700 1 -- 172.21.15.123:0/3223238245 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f5f500327f0 con 0x555b1aff9b70 2024-05-08T06:09:04.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.669+0000 7f5f76660b80 10 client.24886 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T06:09:04.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.669+0000 7f5f76660b80 10 client.24886 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T06:09:04.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.669+0000 7f5f76660b80 20 client.24886 populate_metadata read hostname 'smithi123' 2024-05-08T06:09:04.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.669+0000 7f5f76660b80 1 --2- 172.21.15.123:0/3223238245 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x555b1b07df80 0x555b1b07cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:04.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.669+0000 7f5f76660b80 4 client.24886 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T06:09:04.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.669+0000 7f5f76660b80 1 -- 172.21.15.123:0/3223238245 --> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] -- command(tid 0: {"prefix": "status"}) -- 0x555b1af543b0 con 0x555b1b07df80 2024-05-08T06:09:04.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.669+0000 7f5f69200700 1 --2- 172.21.15.123:0/3223238245 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x555b1b07df80 0x555b1b07cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:04.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.670+0000 7f5f69200700 1 --2- 172.21.15.123:0/3223238245 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x555b1b07df80 0x555b1b07cc90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=280b173c401917b7 server_cookie=531a0208d4908d71 in_seq=0 out_seq=0 2024-05-08T06:09:04.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.670+0000 7f5f4aa00700 10 client.24886 ms_handle_connect on v2:172.21.15.123:6840/1571726231 2024-05-08T06:09:04.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.670+0000 7f5f4aa00700 1 -- 172.21.15.123:0/3223238245 <== mds.1 v2:172.21.15.123:6840/1571726231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x555b1af543b0 con 0x555b1b07df80 2024-05-08T06:09:04.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.670+0000 7f5f4aa00700 10 client.24886 handle_command_reply: tid=0 2024-05-08T06:09:04.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.670+0000 7f5f76660b80 1 client.24886 shutdown 2024-05-08T06:09:04.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.670+0000 7f5f76660b80 1 -- 172.21.15.123:0/3223238245 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b1aff9b70 msgr2=0x555b1b0185e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:04.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.670+0000 7f5f76660b80 1 --2- 172.21.15.123:0/3223238245 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b1aff9b70 0x555b1b0185e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5f500099c0 tx=0x7f5f50009cd0 comp rx=0 tx=0).stop 2024-05-08T06:09:04.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.671+0000 7f5f76660b80 1 -- 172.21.15.123:0/3223238245 shutdown_connections 2024-05-08T06:09:04.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.671+0000 7f5f76660b80 1 --2- 172.21.15.123:0/3223238245 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x555b1b07df80 0x555b1b07cc90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:04.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.671+0000 7f5f76660b80 1 --2- 172.21.15.123:0/3223238245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1affa490 0x555b1b013050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:04.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.671+0000 7f5f76660b80 1 --2- 172.21.15.123:0/3223238245 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555b1aff48a0 0x555b1b0180a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:04.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.671+0000 7f5f76660b80 1 --2- 172.21.15.123:0/3223238245 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b1aff9b70 0x555b1b0185e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:04.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.671+0000 7f5f76660b80 1 -- 172.21.15.123:0/3223238245 >> 172.21.15.123:0/3223238245 conn(0x555b1af6c680 msgr2=0x555b1aff8710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:04.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.671+0000 7f5f4aa00700 0 client.24886 ms_handle_reset on v2:172.21.15.123:6840/1571726231 2024-05-08T06:09:04.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.671+0000 7f5f76660b80 1 -- 172.21.15.123:0/3223238245 shutdown_connections 2024-05-08T06:09:04.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.671+0000 7f5f76660b80 1 -- 172.21.15.123:0/3223238245 wait complete. 2024-05-08T06:09:04.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.671+0000 7f5f76660b80 20 client.24886 trim_cache size 0 max 16384 2024-05-08T06:09:04.673 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-05-08T06:09:04.673 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_fsid": "85cfb2e6-0cff-11ef-bc97-c7b262605968", 2024-05-08T06:09:04.673 INFO:teuthology.orchestra.run.smithi123.stdout: "whoami": 1, 2024-05-08T06:09:04.673 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 24512, 2024-05-08T06:09:04.673 INFO:teuthology.orchestra.run.smithi123.stdout: "want_state": "up:active", 2024-05-08T06:09:04.673 INFO:teuthology.orchestra.run.smithi123.stdout: "state": "up:active", 2024-05-08T06:09:04.673 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs", 2024-05-08T06:09:04.673 INFO:teuthology.orchestra.run.smithi123.stdout: "rank_uptime": 171.285080034, 2024-05-08T06:09:04.674 INFO:teuthology.orchestra.run.smithi123.stdout: "mdsmap_epoch": 23, 2024-05-08T06:09:04.674 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch": 82, 2024-05-08T06:09:04.674 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch_barrier": 80, 2024-05-08T06:09:04.674 INFO:teuthology.orchestra.run.smithi123.stdout: "uptime": 217.767082346 2024-05-08T06:09:04.674 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-05-08T06:09:04.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.672+0000 7f5f6bc00700 1 -- 172.21.15.123:0/2950144218 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5f4c0b2880 msgr2=0x7f5f4c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:04.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.672+0000 7f5f6bc00700 1 --2- 172.21.15.123:0/2950144218 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5f4c0b2880 0x7f5f4c0b4d40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5f64103240 tx=0x7f5f50005fb0 comp rx=0 tx=0).stop 2024-05-08T06:09:04.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.672+0000 7f5f6bc00700 1 -- 172.21.15.123:0/2950144218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f64103750 msgr2=0x7f5f6419bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:04.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.672+0000 7f5f6bc00700 1 --2- 172.21.15.123:0/2950144218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f64103750 0x7f5f6419bdd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800e9a0 tx=0x7f5f5800ecb0 comp rx=0 tx=0).stop 2024-05-08T06:09:04.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.672+0000 7f5f6bc00700 1 -- 172.21.15.123:0/2950144218 shutdown_connections 2024-05-08T06:09:04.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.672+0000 7f5f6bc00700 1 --2- 172.21.15.123:0/2950144218 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5f4c0b2880 0x7f5f4c0b4d40 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:04.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.672+0000 7f5f6bc00700 1 --2- 172.21.15.123:0/2950144218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f64108850 0x7f5f641a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:04.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.672+0000 7f5f6bc00700 1 --2- 172.21.15.123:0/2950144218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f64103750 0x7f5f6419bdd0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:04.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.672+0000 7f5f6bc00700 1 --2- 172.21.15.123:0/2950144218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f64104100 0x7f5f6419c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:04.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.672+0000 7f5f6bc00700 1 -- 172.21.15.123:0/2950144218 >> 172.21.15.123:0/2950144218 conn(0x7f5f6407a600 msgr2=0x7f5f641018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:04.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.672+0000 7f5f6bc00700 1 -- 172.21.15.123:0/2950144218 shutdown_connections 2024-05-08T06:09:04.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:04.672+0000 7f5f6bc00700 1 -- 172.21.15.123:0/2950144218 wait complete. 2024-05-08T06:09:05.771 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-08T06:09:06.207 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:09:06.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:06 smithi163 ceph-mon[34520]: pgmap v419: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:09:06.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:06 smithi192 ceph-mon[34664]: pgmap v419: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:09:06.586 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:06 smithi123 ceph-mon[29787]: pgmap v419: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:09:08.297 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:08 smithi123 ceph-mon[29787]: pgmap v420: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:09:08.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:08 smithi163 ceph-mon[34520]: pgmap v420: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:09:08.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:08 smithi192 ceph-mon[34664]: pgmap v420: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:09:08.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.612+0000 7fcabe800700 1 -- 172.21.15.123:0/3370421827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcab8106080 msgr2=0x7fcab8112590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:08.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.612+0000 7fcabe800700 1 --2- 172.21.15.123:0/3370421827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcab8106080 0x7fcab8112590 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcab000b0a0 tx=0x7fcab000b3b0 comp rx=0 tx=0).stop 2024-05-08T06:09:08.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.612+0000 7fcabe800700 1 -- 172.21.15.123:0/3370421827 shutdown_connections 2024-05-08T06:09:08.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.612+0000 7fcabe800700 1 --2- 172.21.15.123:0/3370421827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcab8106080 0x7fcab8112590 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:08.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.612+0000 7fcabe800700 1 --2- 172.21.15.123:0/3370421827 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcab810b6c0 0x7fcab810baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:08.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.612+0000 7fcabe800700 1 --2- 172.21.15.123:0/3370421827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcab81056c0 0x7fcab8105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:08.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.612+0000 7fcabe800700 1 -- 172.21.15.123:0/3370421827 >> 172.21.15.123:0/3370421827 conn(0x7fcab81011b0 msgr2=0x7fcab81035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:08.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.612+0000 7fcabe800700 1 -- 172.21.15.123:0/3370421827 shutdown_connections 2024-05-08T06:09:08.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.612+0000 7fcabe800700 1 -- 172.21.15.123:0/3370421827 wait complete. 2024-05-08T06:09:08.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.613+0000 7fcabe800700 1 Processor -- start 2024-05-08T06:09:08.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.614+0000 7fcabe800700 1 -- start start 2024-05-08T06:09:08.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.614+0000 7fcabe800700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcab81056c0 0x7fcab819bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:08.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.614+0000 7fcabe800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcab8106080 0x7fcab819c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:08.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.614+0000 7fcabe800700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcab810b6c0 0x7fcab81a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:08.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.614+0000 7fcabe800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcab8115900 con 0x7fcab8106080 2024-05-08T06:09:08.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.614+0000 7fcabe800700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fcab8115780 con 0x7fcab81056c0 2024-05-08T06:09:08.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.614+0000 7fcabe800700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcab8115a80 con 0x7fcab810b6c0 2024-05-08T06:09:08.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.614+0000 7fcab6a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcab8106080 0x7fcab819c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:08.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.614+0000 7fcab6a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcab8106080 0x7fcab819c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50802/0 (socket says 172.21.15.123:50802) 2024-05-08T06:09:08.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.614+0000 7fcab6a00700 1 -- 172.21.15.123:0/2602425915 learned_addr learned my addr 172.21.15.123:0/2602425915 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:09:08.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.614+0000 7fcab7400700 1 --2- 172.21.15.123:0/2602425915 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcab81056c0 0x7fcab819bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:08.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.614+0000 7fcab7e00700 1 --2- 172.21.15.123:0/2602425915 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcab810b6c0 0x7fcab81a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:08.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.615+0000 7fcab6a00700 1 -- 172.21.15.123:0/2602425915 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcab810b6c0 msgr2=0x7fcab81a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:08.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.615+0000 7fcab6a00700 1 --2- 172.21.15.123:0/2602425915 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcab810b6c0 0x7fcab81a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:08.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.615+0000 7fcab6a00700 1 -- 172.21.15.123:0/2602425915 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcab81056c0 msgr2=0x7fcab819bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:08.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.615+0000 7fcab6a00700 1 --2- 172.21.15.123:0/2602425915 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcab81056c0 0x7fcab819bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:08.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.615+0000 7fcab6a00700 1 -- 172.21.15.123:0/2602425915 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcab81a0d70 con 0x7fcab8106080 2024-05-08T06:09:08.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.615+0000 7fcab6a00700 1 --2- 172.21.15.123:0/2602425915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcab8106080 0x7fcab819c2f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcaa400e9a0 tx=0x7fcaa400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:08.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.615+0000 7fcaafe00700 1 -- 172.21.15.123:0/2602425915 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa400ca10 con 0x7fcab8106080 2024-05-08T06:09:08.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.615+0000 7fcaafe00700 1 -- 172.21.15.123:0/2602425915 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaa400cbb0 con 0x7fcab8106080 2024-05-08T06:09:08.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.615+0000 7fcabe800700 1 -- 172.21.15.123:0/2602425915 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcab81a1060 con 0x7fcab8106080 2024-05-08T06:09:08.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.615+0000 7fcaafe00700 1 -- 172.21.15.123:0/2602425915 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4018680 con 0x7fcab8106080 2024-05-08T06:09:08.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.616+0000 7fcabe800700 1 -- 172.21.15.123:0/2602425915 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcab81a88e0 con 0x7fcab8106080 2024-05-08T06:09:08.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.620+0000 7fcaafe00700 1 -- 172.21.15.123:0/2602425915 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fcaa4010430 con 0x7fcab8106080 2024-05-08T06:09:08.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.620+0000 7fcac90e9b80 1 Processor -- start 2024-05-08T06:09:08.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.620+0000 7fcac90e9b80 1 -- start start 2024-05-08T06:09:08.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.621+0000 7fcac90e9b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ef26d7eb70 0x55ef26d7ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:08.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.621+0000 7fcac90e9b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef26d7f490 0x55ef26d79280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:08.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.621+0000 7fcac90e9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef26d798a0 0x55ef26d79d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:08.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.621+0000 7fcac90e9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ef26ccb9d0 con 0x55ef26d798a0 2024-05-08T06:09:08.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.621+0000 7fcac90e9b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55ef26c69710 con 0x55ef26d7eb70 2024-05-08T06:09:08.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.621+0000 7fcac90e9b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55ef26cc7990 con 0x55ef26d7f490 2024-05-08T06:09:08.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.621+0000 7fcab7400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef26d7f490 0x55ef26d79280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:08.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.621+0000 7fcab6a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef26d798a0 0x55ef26d79d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:08.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.621+0000 7fcab7400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef26d7f490 0x55ef26d79280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:35394/0 (socket says 172.21.15.123:35394) 2024-05-08T06:09:08.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.621+0000 7fcab7400700 1 -- 172.21.15.123:0/3974813001 learned_addr learned my addr 172.21.15.123:0/3974813001 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:09:08.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.622+0000 7fcab7e00700 1 --2- 172.21.15.123:0/3974813001 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ef26d7eb70 0x55ef26d7ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:08.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.622+0000 7fcab7400700 1 -- 172.21.15.123:0/3974813001 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ef26d7eb70 msgr2=0x55ef26d7ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:08.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.622+0000 7fcab7400700 1 --2- 172.21.15.123:0/3974813001 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ef26d7eb70 0x55ef26d7ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:08.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.622+0000 7fcab7400700 1 -- 172.21.15.123:0/3974813001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef26d798a0 msgr2=0x55ef26d79d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:08.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.622+0000 7fcab7400700 1 --2- 172.21.15.123:0/3974813001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef26d798a0 0x55ef26d79d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:08.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.622+0000 7fcaafe00700 1 --2- 172.21.15.123:0/2602425915 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fca9c0b2900 0x7fca9c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:08.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.622+0000 7fcab7400700 1 -- 172.21.15.123:0/3974813001 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef26c18880 con 0x55ef26d7f490 2024-05-08T06:09:08.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.622+0000 7fcaafe00700 1 -- 172.21.15.123:0/2602425915 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fcaa40e2370 con 0x7fcab8106080 2024-05-08T06:09:08.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.622+0000 7fcab7e00700 1 --2- 172.21.15.123:0/2602425915 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fca9c0b2900 0x7fca9c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:08.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.622+0000 7fcab7400700 1 --2- 172.21.15.123:0/3974813001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef26d7f490 0x55ef26d79280 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8009d00 tx=0x7fcaa8011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78a91651e74e2db7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:08.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.622+0000 7fcaacc00700 1 -- 172.21.15.123:0/3974813001 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8004230 con 0x55ef26d7f490 2024-05-08T06:09:08.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.623+0000 7fcaacc00700 1 -- 172.21.15.123:0/3974813001 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaa80043d0 con 0x55ef26d7f490 2024-05-08T06:09:08.624 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.623+0000 7fcaacc00700 1 -- 172.21.15.123:0/3974813001 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8019920 con 0x55ef26d7f490 2024-05-08T06:09:08.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.623+0000 7fcab7e00700 1 --2- 172.21.15.123:0/2602425915 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fca9c0b2900 0x7fca9c0b4dc0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcab0009fd0 tx=0x7fcab000b720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:08.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.623+0000 7fcac90e9b80 1 -- 172.21.15.123:0/3974813001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef26d7f490 msgr2=0x55ef26d79280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:08.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.623+0000 7fcac90e9b80 1 --2- 172.21.15.123:0/3974813001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef26d7f490 0x55ef26d79280 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8009d00 tx=0x7fcaa8011720 comp rx=0 tx=0).stop 2024-05-08T06:09:08.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.623+0000 7fcac90e9b80 1 -- 172.21.15.123:0/3974813001 shutdown_connections 2024-05-08T06:09:08.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.623+0000 7fcac90e9b80 1 --2- 172.21.15.123:0/3974813001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef26d798a0 0x55ef26d79d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:08.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.624+0000 7fcac90e9b80 1 --2- 172.21.15.123:0/3974813001 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ef26d7eb70 0x55ef26d7ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:08.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.624+0000 7fcac90e9b80 1 --2- 172.21.15.123:0/3974813001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef26d7f490 0x55ef26d79280 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:08.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.624+0000 7fcac90e9b80 1 -- 172.21.15.123:0/3974813001 >> 172.21.15.123:0/3974813001 conn(0x55ef26cf1680 msgr2=0x55ef26d73ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:08.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.624+0000 7fcac90e9b80 1 -- 172.21.15.123:0/3974813001 shutdown_connections 2024-05-08T06:09:08.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.624+0000 7fcac90e9b80 1 -- 172.21.15.123:0/3974813001 wait complete. 2024-05-08T06:09:08.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.624+0000 7fcac90e9b80 1 -- start start 2024-05-08T06:09:08.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.624+0000 7fcac90e9b80 1 Processor -- start 2024-05-08T06:09:08.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.625+0000 7fcac90e9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef26d798a0 0x55ef26d85180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:08.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.625+0000 7fcac90e9b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ef26d7eb70 0x55ef26d82520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:08.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.625+0000 7fcac90e9b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef26d7f490 0x55ef26d82a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:08.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.625+0000 7fcac90e9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ef26d85ed0 con 0x55ef26d798a0 2024-05-08T06:09:08.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.625+0000 7fcac90e9b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55ef26c31ad0 con 0x55ef26d7eb70 2024-05-08T06:09:08.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.625+0000 7fcac90e9b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55ef26d86050 con 0x55ef26d7f490 2024-05-08T06:09:08.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.625+0000 7fcac90e9b80 10 client.0 _ll_register_callbacks cb 0x55ef26c72c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:09:08.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.625+0000 7fcab7400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef26d798a0 0x55ef26d85180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:08.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.625+0000 7fcab7400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef26d798a0 0x55ef26d85180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50818/0 (socket says 172.21.15.123:50818) 2024-05-08T06:09:08.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.625+0000 7fcab6a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ef26d7eb70 0x55ef26d82520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:08.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.625+0000 7fcab7400700 1 -- 172.21.15.123:0/468898900 learned_addr learned my addr 172.21.15.123:0/468898900 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:09:08.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.625+0000 7fcab7e00700 1 --2- 172.21.15.123:0/468898900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef26d7f490 0x55ef26d82a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:08.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.626+0000 7fcab6a00700 1 -- 172.21.15.123:0/468898900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef26d7f490 msgr2=0x55ef26d82a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:08.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.626+0000 7fcab6a00700 1 --2- 172.21.15.123:0/468898900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef26d7f490 0x55ef26d82a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:08.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.626+0000 7fcab6a00700 1 -- 172.21.15.123:0/468898900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef26d798a0 msgr2=0x55ef26d85180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:08.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.626+0000 7fcab6a00700 1 --2- 172.21.15.123:0/468898900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef26d798a0 0x55ef26d85180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:08.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.626+0000 7fcab6a00700 1 -- 172.21.15.123:0/468898900 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef26c913d0 con 0x55ef26d7eb70 2024-05-08T06:09:08.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.626+0000 7fcab6a00700 1 --2- 172.21.15.123:0/468898900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ef26d7eb70 0x55ef26d82520 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x55ef26d7a260 tx=0x7fcaa401f5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78b5e3f2f83b014e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:08.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.626+0000 7fcaa1600700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-05-08T06:09:08.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.626+0000 7fcaa1600700 1 -- 172.21.15.123:0/468898900 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa40a9290 con 0x55ef26d7eb70 2024-05-08T06:09:08.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.626+0000 7fcac90e9b80 1 -- 172.21.15.123:0/468898900 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcaa80043d0 con 0x55ef26d7eb70 2024-05-08T06:09:08.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.627+0000 7fcaa1600700 1 -- 172.21.15.123:0/468898900 <== mon.2 v2:172.21.15.163:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fcaa40b2310 con 0x55ef26d7eb70 2024-05-08T06:09:08.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.627+0000 7fcaa1600700 1 -- 172.21.15.123:0/468898900 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fcaa40a90b0 con 0x55ef26d7eb70 2024-05-08T06:09:08.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.627+0000 7fcac90e9b80 10 client.24905 fetch_fsmap learned FSMap version 23 2024-05-08T06:09:08.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.627+0000 7fcac90e9b80 1 -- 172.21.15.123:0/468898900 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55ef26d90850 con 0x55ef26d7eb70 2024-05-08T06:09:08.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.627+0000 7fcaa1600700 1 -- 172.21.15.123:0/468898900 <== mon.2 v2:172.21.15.163:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaa80043d0 con 0x55ef26d7eb70 2024-05-08T06:09:08.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.627+0000 7fcaa1600700 1 -- 172.21.15.123:0/468898900 <== mon.2 v2:172.21.15.163:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fcaa4010720 con 0x55ef26d7eb70 2024-05-08T06:09:08.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.627+0000 7fcac90e9b80 10 client.24905 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T06:09:08.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.627+0000 7fcac90e9b80 10 client.24905 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T06:09:08.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.627+0000 7fcac90e9b80 20 client.24905 populate_metadata read hostname 'smithi123' 2024-05-08T06:09:08.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.627+0000 7fcac90e9b80 1 --2- 172.21.15.123:0/468898900 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55ef26d92e00 0x55ef26d94270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:08.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.627+0000 7fcac90e9b80 4 client.24905 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:09:08.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.627+0000 7fcac90e9b80 1 -- 172.21.15.123:0/468898900 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef26d460f0 con 0x55ef26d92e00 2024-05-08T06:09:08.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.628+0000 7fcab7400700 1 --2- 172.21.15.123:0/468898900 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55ef26d92e00 0x55ef26d94270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:08.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.628+0000 7fcab7400700 1 --2- 172.21.15.123:0/468898900 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55ef26d92e00 0x55ef26d94270 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f83da0b14b358fe server_cookie=1620cb23ad6e7cc2 in_seq=0 out_seq=0 2024-05-08T06:09:08.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.628+0000 7fcaa1600700 10 client.24905 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:09:08.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.629+0000 7fcaa1600700 1 -- 172.21.15.123:0/468898900 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ef26d460f0 con 0x55ef26d92e00 2024-05-08T06:09:08.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.629+0000 7fcaa1600700 10 client.24905 handle_command_reply: tid=0 2024-05-08T06:09:08.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.629+0000 7fcac90e9b80 1 client.24905 shutdown 2024-05-08T06:09:08.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.630+0000 7fcac90e9b80 1 -- 172.21.15.123:0/468898900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ef26d7eb70 msgr2=0x55ef26d82520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:08.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.630+0000 7fcac90e9b80 1 --2- 172.21.15.123:0/468898900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ef26d7eb70 0x55ef26d82520 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x55ef26d7a260 tx=0x7fcaa401f5f0 comp rx=0 tx=0).stop 2024-05-08T06:09:08.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.630+0000 7fcac90e9b80 1 -- 172.21.15.123:0/468898900 shutdown_connections 2024-05-08T06:09:08.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.630+0000 7fcac90e9b80 1 --2- 172.21.15.123:0/468898900 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55ef26d92e00 0x55ef26d94270 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:08.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.630+0000 7fcac90e9b80 1 --2- 172.21.15.123:0/468898900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef26d798a0 0x55ef26d85180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:08.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.630+0000 7fcac90e9b80 1 --2- 172.21.15.123:0/468898900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef26d7f490 0x55ef26d82a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:08.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.630+0000 7fcac90e9b80 1 --2- 172.21.15.123:0/468898900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ef26d7eb70 0x55ef26d82520 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:08.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.630+0000 7fcaa1600700 0 client.24905 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:09:08.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.630+0000 7fcac90e9b80 1 -- 172.21.15.123:0/468898900 >> 172.21.15.123:0/468898900 conn(0x55ef26cf1680 msgr2=0x55ef26d73140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:08.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.630+0000 7fcac90e9b80 1 -- 172.21.15.123:0/468898900 shutdown_connections 2024-05-08T06:09:08.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.630+0000 7fcac90e9b80 1 -- 172.21.15.123:0/468898900 wait complete. 2024-05-08T06:09:08.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.630+0000 7fcac90e9b80 20 client.24905 trim_cache size 0 max 16384 2024-05-08T06:09:08.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.653+0000 7fcac90e9b80 1 Processor -- start 2024-05-08T06:09:08.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.653+0000 7fcac90e9b80 1 -- start start 2024-05-08T06:09:08.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.653+0000 7fcac90e9b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef26d798a0 0x55ef26d738c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:08.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.653+0000 7fcac90e9b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ef26d7eb70 0x55ef26d7c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:08.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.654+0000 7fcac90e9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef26d7f490 0x55ef26d7cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:08.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.654+0000 7fcac90e9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ef26d925b0 con 0x55ef26d7f490 2024-05-08T06:09:08.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.654+0000 7fcac90e9b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55ef26d92c80 con 0x55ef26d7eb70 2024-05-08T06:09:08.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.654+0000 7fcac90e9b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55ef26d93350 con 0x55ef26d798a0 2024-05-08T06:09:08.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.654+0000 7fcab6a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ef26d7eb70 0x55ef26d7c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:08.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.654+0000 7fcab6a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ef26d7eb70 0x55ef26d7c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:49556/0 (socket says 172.21.15.123:49556) 2024-05-08T06:09:08.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.654+0000 7fcab6a00700 1 -- 172.21.15.123:0/4032469072 learned_addr learned my addr 172.21.15.123:0/4032469072 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:09:08.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.654+0000 7fcab7e00700 1 --2- 172.21.15.123:0/4032469072 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef26d7f490 0x55ef26d7cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:08.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.654+0000 7fcab7400700 1 --2- 172.21.15.123:0/4032469072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef26d798a0 0x55ef26d738c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:08.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.654+0000 7fcab6a00700 1 -- 172.21.15.123:0/4032469072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef26d798a0 msgr2=0x55ef26d738c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:08.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.654+0000 7fcab6a00700 1 --2- 172.21.15.123:0/4032469072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef26d798a0 0x55ef26d738c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:08.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.654+0000 7fcab6a00700 1 -- 172.21.15.123:0/4032469072 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef26d7f490 msgr2=0x55ef26d7cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:08.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.654+0000 7fcab6a00700 1 --2- 172.21.15.123:0/4032469072 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef26d7f490 0x55ef26d7cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:08.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.654+0000 7fcab6a00700 1 -- 172.21.15.123:0/4032469072 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef26d80ab0 con 0x55ef26d7eb70 2024-05-08T06:09:08.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.654+0000 7fcab7400700 1 --2- 172.21.15.123:0/4032469072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef26d798a0 0x55ef26d738c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:09:08.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.655+0000 7fcab6a00700 1 --2- 172.21.15.123:0/4032469072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ef26d7eb70 0x55ef26d7c6c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcaa40a6bb0 tx=0x7fcaa400e8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe68677b7725f34b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:08.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.655+0000 7fcaa3e00700 1 -- 172.21.15.123:0/4032469072 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa40a9290 con 0x55ef26d7eb70 2024-05-08T06:09:08.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.655+0000 7fcaa3e00700 1 -- 172.21.15.123:0/4032469072 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaa40b20e0 con 0x55ef26d7eb70 2024-05-08T06:09:08.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.655+0000 7fcaa3e00700 1 -- 172.21.15.123:0/4032469072 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa40a9940 con 0x55ef26d7eb70 2024-05-08T06:09:08.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.656+0000 7fcac90e9b80 1 -- 172.21.15.123:0/4032469072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ef26d7eb70 msgr2=0x55ef26d7c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:08.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.656+0000 7fcac90e9b80 1 --2- 172.21.15.123:0/4032469072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ef26d7eb70 0x55ef26d7c6c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcaa40a6bb0 tx=0x7fcaa400e8f0 comp rx=0 tx=0).stop 2024-05-08T06:09:08.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.656+0000 7fcac90e9b80 1 -- 172.21.15.123:0/4032469072 shutdown_connections 2024-05-08T06:09:08.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.656+0000 7fcac90e9b80 1 --2- 172.21.15.123:0/4032469072 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef26d7f490 0x55ef26d7cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:08.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.656+0000 7fcac90e9b80 1 --2- 172.21.15.123:0/4032469072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef26d798a0 0x55ef26d738c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:08.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.656+0000 7fcac90e9b80 1 --2- 172.21.15.123:0/4032469072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ef26d7eb70 0x55ef26d7c6c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:08.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.656+0000 7fcac90e9b80 1 -- 172.21.15.123:0/4032469072 >> 172.21.15.123:0/4032469072 conn(0x55ef26cf1680 msgr2=0x55ef26d75590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:08.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.656+0000 7fcac90e9b80 1 -- 172.21.15.123:0/4032469072 shutdown_connections 2024-05-08T06:09:08.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.656+0000 7fcac90e9b80 1 -- 172.21.15.123:0/4032469072 wait complete. 2024-05-08T06:09:08.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.656+0000 7fcac90e9b80 1 -- start start 2024-05-08T06:09:08.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.657+0000 7fcac90e9b80 1 Processor -- start 2024-05-08T06:09:08.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.657+0000 7fcac90e9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef26d798a0 0x55ef26d9d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:08.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.657+0000 7fcac90e9b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef26d7eb70 0x55ef26d9dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:08.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.657+0000 7fcac90e9b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ef26d7f490 0x55ef26d986a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:08.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.657+0000 7fcac90e9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ef26d940f0 con 0x55ef26d798a0 2024-05-08T06:09:08.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.657+0000 7fcac90e9b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55ef26d93a20 con 0x55ef26d7f490 2024-05-08T06:09:08.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.657+0000 7fcac90e9b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55ef26d947c0 con 0x55ef26d7eb70 2024-05-08T06:09:08.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.657+0000 7fcac90e9b80 10 client.0 _ll_register_callbacks cb 0x55ef26c72c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:09:08.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.658+0000 7fcab7400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef26d798a0 0x55ef26d9d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:08.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.658+0000 7fcab7400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef26d798a0 0x55ef26d9d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50842/0 (socket says 172.21.15.123:50842) 2024-05-08T06:09:08.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.658+0000 7fcab7400700 1 -- 172.21.15.123:0/2792547983 learned_addr learned my addr 172.21.15.123:0/2792547983 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:09:08.659 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.658+0000 7fcab6a00700 1 --2- 172.21.15.123:0/2792547983 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef26d7eb70 0x55ef26d9dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:08.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.658+0000 7fcab7e00700 1 --2- 172.21.15.123:0/2792547983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ef26d7f490 0x55ef26d986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:08.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.658+0000 7fcab6a00700 1 -- 172.21.15.123:0/2792547983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ef26d7f490 msgr2=0x55ef26d986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:08.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.658+0000 7fcab6a00700 1 --2- 172.21.15.123:0/2792547983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ef26d7f490 0x55ef26d986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:08.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.658+0000 7fcab6a00700 1 -- 172.21.15.123:0/2792547983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef26d798a0 msgr2=0x55ef26d9d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:08.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.658+0000 7fcab6a00700 1 --2- 172.21.15.123:0/2792547983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef26d798a0 0x55ef26d9d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:08.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.658+0000 7fcab6a00700 1 -- 172.21.15.123:0/2792547983 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef26d98ce0 con 0x55ef26d7eb70 2024-05-08T06:09:08.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.658+0000 7fcab7400700 1 --2- 172.21.15.123:0/2792547983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef26d798a0 0x55ef26d9d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:09:08.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.658+0000 7fcab6a00700 1 --2- 172.21.15.123:0/2792547983 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef26d7eb70 0x55ef26d9dc30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x55ef26d90720 tx=0x7fcaa4010950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea6f2d21a83ef2ca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:08.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.658+0000 7fcaa1600700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-05-08T06:09:08.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.659+0000 7fcaa1600700 1 -- 172.21.15.123:0/2792547983 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa40a9290 con 0x55ef26d7eb70 2024-05-08T06:09:08.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.659+0000 7fcaa1600700 1 -- 172.21.15.123:0/2792547983 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fcaa40b2520 con 0x55ef26d7eb70 2024-05-08T06:09:08.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.659+0000 7fcac90e9b80 1 -- 172.21.15.123:0/2792547983 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ef26cd93b0 con 0x55ef26d7eb70 2024-05-08T06:09:08.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.659+0000 7fcaa1600700 1 -- 172.21.15.123:0/2792547983 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fcaa40adcf0 con 0x55ef26d7eb70 2024-05-08T06:09:08.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.659+0000 7fcac90e9b80 10 client.24904 fetch_fsmap learned FSMap version 23 2024-05-08T06:09:08.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.659+0000 7fcac90e9b80 1 -- 172.21.15.123:0/2792547983 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55ef26dfebb0 con 0x55ef26d7eb70 2024-05-08T06:09:08.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.659+0000 7fcaa1600700 1 -- 172.21.15.123:0/2792547983 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ef26cd93b0 con 0x55ef26d7eb70 2024-05-08T06:09:08.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.659+0000 7fcaa1600700 1 -- 172.21.15.123:0/2792547983 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fcaa40a52b0 con 0x55ef26d7eb70 2024-05-08T06:09:08.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.659+0000 7fcac90e9b80 10 client.24904 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T06:09:08.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.659+0000 7fcac90e9b80 10 client.24904 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T06:09:08.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.659+0000 7fcac90e9b80 20 client.24904 populate_metadata read hostname 'smithi123' 2024-05-08T06:09:08.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.659+0000 7fcac90e9b80 1 --2- 172.21.15.123:0/2792547983 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55ef26e03140 0x55ef26e01e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:08.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.659+0000 7fcac90e9b80 4 client.24904 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T06:09:08.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.659+0000 7fcac90e9b80 1 -- 172.21.15.123:0/2792547983 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "status"}) -- 0x55ef26cd3450 con 0x55ef26e03140 2024-05-08T06:09:08.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.660+0000 7fcab7400700 1 --2- 172.21.15.123:0/2792547983 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55ef26e03140 0x55ef26e01e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:08.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.660+0000 7fcab7400700 1 --2- 172.21.15.123:0/2792547983 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55ef26e03140 0x55ef26e01e50 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49fa843d1416c31d server_cookie=8218910bad5520d0 in_seq=0 out_seq=0 2024-05-08T06:09:08.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.660+0000 7fcaa1600700 10 client.24904 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:09:08.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.661+0000 7fcaa1600700 1 -- 172.21.15.123:0/2792547983 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55ef26cd3450 con 0x55ef26e03140 2024-05-08T06:09:08.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.661+0000 7fcaa1600700 10 client.24904 handle_command_reply: tid=0 2024-05-08T06:09:08.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.661+0000 7fcac90e9b80 1 client.24904 shutdown 2024-05-08T06:09:08.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.661+0000 7fcac90e9b80 1 -- 172.21.15.123:0/2792547983 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef26d7eb70 msgr2=0x55ef26d9dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:08.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.661+0000 7fcac90e9b80 1 --2- 172.21.15.123:0/2792547983 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef26d7eb70 0x55ef26d9dc30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x55ef26d90720 tx=0x7fcaa4010950 comp rx=0 tx=0).stop 2024-05-08T06:09:08.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.661+0000 7fcac90e9b80 1 -- 172.21.15.123:0/2792547983 shutdown_connections 2024-05-08T06:09:08.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.661+0000 7fcac90e9b80 1 --2- 172.21.15.123:0/2792547983 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55ef26e03140 0x55ef26e01e50 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:08.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.661+0000 7fcac90e9b80 1 --2- 172.21.15.123:0/2792547983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef26d798a0 0x55ef26d9d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:08.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.661+0000 7fcac90e9b80 1 --2- 172.21.15.123:0/2792547983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ef26d7f490 0x55ef26d986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:08.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.661+0000 7fcaa1600700 0 client.24904 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:09:08.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.661+0000 7fcac90e9b80 1 --2- 172.21.15.123:0/2792547983 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef26d7eb70 0x55ef26d9dc30 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:08.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.661+0000 7fcac90e9b80 1 -- 172.21.15.123:0/2792547983 >> 172.21.15.123:0/2792547983 conn(0x55ef26cf1680 msgr2=0x55ef26d7c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:08.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.661+0000 7fcac90e9b80 1 -- 172.21.15.123:0/2792547983 shutdown_connections 2024-05-08T06:09:08.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.661+0000 7fcac90e9b80 1 -- 172.21.15.123:0/2792547983 wait complete. 2024-05-08T06:09:08.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.661+0000 7fcac90e9b80 20 client.24904 trim_cache size 0 max 16384 2024-05-08T06:09:08.666 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-05-08T06:09:08.666 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_fsid": "85cfb2e6-0cff-11ef-bc97-c7b262605968", 2024-05-08T06:09:08.666 INFO:teuthology.orchestra.run.smithi123.stdout: "whoami": 0, 2024-05-08T06:09:08.666 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 24524, 2024-05-08T06:09:08.666 INFO:teuthology.orchestra.run.smithi123.stdout: "want_state": "up:active", 2024-05-08T06:09:08.666 INFO:teuthology.orchestra.run.smithi123.stdout: "state": "up:active", 2024-05-08T06:09:08.666 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs", 2024-05-08T06:09:08.666 INFO:teuthology.orchestra.run.smithi123.stdout: "rank_uptime": 193.229532756, 2024-05-08T06:09:08.667 INFO:teuthology.orchestra.run.smithi123.stdout: "mdsmap_epoch": 23, 2024-05-08T06:09:08.667 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch": 82, 2024-05-08T06:09:08.667 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch_barrier": 80, 2024-05-08T06:09:08.667 INFO:teuthology.orchestra.run.smithi123.stdout: "uptime": 218.860646218 2024-05-08T06:09:08.667 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-05-08T06:09:08.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.662+0000 7fcabe800700 1 -- 172.21.15.123:0/2602425915 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fca9c0b2900 msgr2=0x7fca9c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:08.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.662+0000 7fcabe800700 1 --2- 172.21.15.123:0/2602425915 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fca9c0b2900 0x7fca9c0b4dc0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcab0009fd0 tx=0x7fcab000b720 comp rx=0 tx=0).stop 2024-05-08T06:09:08.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.662+0000 7fcabe800700 1 -- 172.21.15.123:0/2602425915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcab8106080 msgr2=0x7fcab819c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:08.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.662+0000 7fcabe800700 1 --2- 172.21.15.123:0/2602425915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcab8106080 0x7fcab819c2f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcaa400e9a0 tx=0x7fcaa400ecb0 comp rx=0 tx=0).stop 2024-05-08T06:09:08.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.662+0000 7fcabe800700 1 -- 172.21.15.123:0/2602425915 shutdown_connections 2024-05-08T06:09:08.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.662+0000 7fcabe800700 1 --2- 172.21.15.123:0/2602425915 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fca9c0b2900 0x7fca9c0b4dc0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:08.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.662+0000 7fcabe800700 1 --2- 172.21.15.123:0/2602425915 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcab810b6c0 0x7fcab81a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:08.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.662+0000 7fcabe800700 1 --2- 172.21.15.123:0/2602425915 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcab81056c0 0x7fcab819bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:08.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.662+0000 7fcabe800700 1 --2- 172.21.15.123:0/2602425915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcab8106080 0x7fcab819c2f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:08.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.662+0000 7fcabe800700 1 -- 172.21.15.123:0/2602425915 >> 172.21.15.123:0/2602425915 conn(0x7fcab81011b0 msgr2=0x7fcab81017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:08.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.662+0000 7fcabe800700 1 -- 172.21.15.123:0/2602425915 shutdown_connections 2024-05-08T06:09:08.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:08.662+0000 7fcabe800700 1 -- 172.21.15.123:0/2602425915 wait complete. 2024-05-08T06:09:09.655 INFO:tasks.cephadm:Setup complete, yielding 2024-05-08T06:09:09.655 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-08T06:09:09.669 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi123.front.sepia.ceph.com 2024-05-08T06:09:09.669 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- bash -c 'ceph orch status' 2024-05-08T06:09:10.125 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:09:10.501 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:10 smithi123 ceph-mon[29787]: pgmap v421: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:09:10.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:10 smithi163 ceph-mon[34520]: pgmap v421: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:09:10.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:10 smithi192 ceph-mon[34664]: pgmap v421: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:09:12.513 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:12 smithi123 ceph-mon[29787]: pgmap v422: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:09:12.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:12 smithi163 ceph-mon[34520]: pgmap v422: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:09:12.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.542+0000 7f0d8e400700 1 -- 172.21.15.123:0/3430808543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d88077470 msgr2=0x7f0d880758d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:12.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.542+0000 7f0d8e400700 1 --2- 172.21.15.123:0/3430808543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d88077470 0x7f0d880758d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0d78009990 tx=0x7f0d78009ca0 comp rx=0 tx=0).stop 2024-05-08T06:09:12.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.542+0000 7f0d8e400700 1 -- 172.21.15.123:0/3430808543 shutdown_connections 2024-05-08T06:09:12.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.542+0000 7f0d8e400700 1 --2- 172.21.15.123:0/3430808543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d88077470 0x7f0d880758d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:12.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.542+0000 7f0d8e400700 1 --2- 172.21.15.123:0/3430808543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d8810b3e0 0x7f0d8810d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:12.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.542+0000 7f0d8e400700 1 --2- 172.21.15.123:0/3430808543 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d88075ea0 0x7f0d88076320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:12.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.542+0000 7f0d8e400700 1 -- 172.21.15.123:0/3430808543 >> 172.21.15.123:0/3430808543 conn(0x7f0d880feaa0 msgr2=0x7f0d88100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:12.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.543+0000 7f0d8e400700 1 -- 172.21.15.123:0/3430808543 shutdown_connections 2024-05-08T06:09:12.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.543+0000 7f0d8e400700 1 -- 172.21.15.123:0/3430808543 wait complete. 2024-05-08T06:09:12.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.544+0000 7f0d8e400700 1 Processor -- start 2024-05-08T06:09:12.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.544+0000 7f0d8e400700 1 -- start start 2024-05-08T06:09:12.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.544+0000 7f0d8e400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d88075ea0 0x7f0d8819b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:12.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.544+0000 7f0d8e400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d88077470 0x7f0d8819be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:12.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.544+0000 7f0d8e400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d8810b3e0 0x7f0d881a2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:12.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.544+0000 7f0d8e400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0d881111c0 con 0x7f0d8810b3e0 2024-05-08T06:09:12.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.544+0000 7f0d8e400700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0d88111040 con 0x7f0d88077470 2024-05-08T06:09:12.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.544+0000 7f0d8e400700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0d88111340 con 0x7f0d88075ea0 2024-05-08T06:09:12.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.544+0000 7f0d87e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d8810b3e0 0x7f0d881a2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:12.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.544+0000 7f0d87400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d88075ea0 0x7f0d8819b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:12.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.544+0000 7f0d86a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d88077470 0x7f0d8819be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:12.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.544+0000 7f0d86a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d88077470 0x7f0d8819be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:49584/0 (socket says 172.21.15.123:49584) 2024-05-08T06:09:12.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.544+0000 7f0d86a00700 1 -- 172.21.15.123:0/1026808433 learned_addr learned my addr 172.21.15.123:0/1026808433 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:09:12.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.545+0000 7f0d86a00700 1 -- 172.21.15.123:0/1026808433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d88075ea0 msgr2=0x7f0d8819b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:12.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.545+0000 7f0d86a00700 1 --2- 172.21.15.123:0/1026808433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d88075ea0 0x7f0d8819b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:12.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.545+0000 7f0d86a00700 1 -- 172.21.15.123:0/1026808433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d8810b3e0 msgr2=0x7f0d881a2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:12.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.545+0000 7f0d86a00700 1 --2- 172.21.15.123:0/1026808433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d8810b3e0 0x7f0d881a2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:12.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.545+0000 7f0d86a00700 1 -- 172.21.15.123:0/1026808433 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d881a34f0 con 0x7f0d88077470 2024-05-08T06:09:12.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.545+0000 7f0d87e00700 1 --2- 172.21.15.123:0/1026808433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d8810b3e0 0x7f0d881a2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:09:12.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.545+0000 7f0d86a00700 1 --2- 172.21.15.123:0/1026808433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d88077470 0x7f0d8819be20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0d7400e9a0 tx=0x7f0d7400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:12.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.545+0000 7f0d7fe00700 1 -- 172.21.15.123:0/1026808433 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7400ca10 con 0x7f0d88077470 2024-05-08T06:09:12.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.545+0000 7f0d8e400700 1 -- 172.21.15.123:0/1026808433 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d881a37e0 con 0x7f0d88077470 2024-05-08T06:09:12.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.545+0000 7f0d7fe00700 1 -- 172.21.15.123:0/1026808433 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d7400cbb0 con 0x7f0d88077470 2024-05-08T06:09:12.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.545+0000 7f0d7fe00700 1 -- 172.21.15.123:0/1026808433 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d74018680 con 0x7f0d88077470 2024-05-08T06:09:12.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.546+0000 7f0d8e400700 1 -- 172.21.15.123:0/1026808433 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d881a3d20 con 0x7f0d88077470 2024-05-08T06:09:12.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.547+0000 7f0d8e400700 1 -- 172.21.15.123:0/1026808433 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d8806a190 con 0x7f0d88077470 2024-05-08T06:09:12.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.550+0000 7f0d7fe00700 1 -- 172.21.15.123:0/1026808433 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f0d74010430 con 0x7f0d88077470 2024-05-08T06:09:12.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:12 smithi192 ceph-mon[34664]: pgmap v422: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:09:12.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.551+0000 7f0d7fe00700 1 --2- 172.21.15.123:0/1026808433 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f0d6c0b2880 0x7f0d6c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:12.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.551+0000 7f0d7fe00700 1 -- 172.21.15.123:0/1026808433 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f0d740e23d0 con 0x7f0d88077470 2024-05-08T06:09:12.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.551+0000 7f0d87400700 1 --2- 172.21.15.123:0/1026808433 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f0d6c0b2880 0x7f0d6c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:12.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.552+0000 7f0d87400700 1 --2- 172.21.15.123:0/1026808433 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f0d6c0b2880 0x7f0d6c0b4d40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800b2e0 tx=0x7f0d78005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:12.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.556+0000 7f0d7fe00700 1 -- 172.21.15.123:0/1026808433 <== mon.2 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0d740aa0a0 con 0x7f0d88077470 2024-05-08T06:09:12.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.805+0000 7f0d8e400700 1 -- 172.21.15.123:0/1026808433 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f0d880638e0 con 0x7f0d6c0b2880 2024-05-08T06:09:12.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.807+0000 7f0d7fe00700 1 -- 172.21.15.123:0/1026808433 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f0d880638e0 con 0x7f0d6c0b2880 2024-05-08T06:09:12.808 INFO:teuthology.orchestra.run.smithi123.stdout:Backend: cephadm 2024-05-08T06:09:12.808 INFO:teuthology.orchestra.run.smithi123.stdout:Available: Yes 2024-05-08T06:09:12.808 INFO:teuthology.orchestra.run.smithi123.stdout:Paused: No 2024-05-08T06:09:12.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.810+0000 7f0d8e400700 1 -- 172.21.15.123:0/1026808433 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f0d6c0b2880 msgr2=0x7f0d6c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:12.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.810+0000 7f0d8e400700 1 --2- 172.21.15.123:0/1026808433 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f0d6c0b2880 0x7f0d6c0b4d40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800b2e0 tx=0x7f0d78005fb0 comp rx=0 tx=0).stop 2024-05-08T06:09:12.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.810+0000 7f0d8e400700 1 -- 172.21.15.123:0/1026808433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d88077470 msgr2=0x7f0d8819be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:12.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.810+0000 7f0d8e400700 1 --2- 172.21.15.123:0/1026808433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d88077470 0x7f0d8819be20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0d7400e9a0 tx=0x7f0d7400ecb0 comp rx=0 tx=0).stop 2024-05-08T06:09:12.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.811+0000 7f0d8e400700 1 -- 172.21.15.123:0/1026808433 shutdown_connections 2024-05-08T06:09:12.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.811+0000 7f0d8e400700 1 --2- 172.21.15.123:0/1026808433 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f0d6c0b2880 0x7f0d6c0b4d40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:12.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.811+0000 7f0d8e400700 1 --2- 172.21.15.123:0/1026808433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d8810b3e0 0x7f0d881a2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:12.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.811+0000 7f0d8e400700 1 --2- 172.21.15.123:0/1026808433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d88075ea0 0x7f0d8819b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:12.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.811+0000 7f0d8e400700 1 --2- 172.21.15.123:0/1026808433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d88077470 0x7f0d8819be20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:12.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.811+0000 7f0d8e400700 1 -- 172.21.15.123:0/1026808433 >> 172.21.15.123:0/1026808433 conn(0x7f0d880feaa0 msgr2=0x7f0d88107330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:12.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.811+0000 7f0d8e400700 1 -- 172.21.15.123:0/1026808433 shutdown_connections 2024-05-08T06:09:12.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:12.811+0000 7f0d8e400700 1 -- 172.21.15.123:0/1026808433 wait complete. 2024-05-08T06:09:13.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:13 smithi163 ceph-mon[34520]: from='client.24917 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:09:14.167 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:13 smithi123 ceph-mon[29787]: from='client.24917 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:09:14.167 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:13 smithi192 ceph-mon[34664]: from='client.24917 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:09:14.168 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-08T06:09:14.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:14 smithi163 ceph-mon[34520]: pgmap v423: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:09:14.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:14 smithi192 ceph-mon[34664]: pgmap v423: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:09:14.615 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:09:14.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:14 smithi123 ceph-mon[29787]: pgmap v423: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:09:16.433 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:16 smithi123 ceph-mon[29787]: pgmap v424: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:09:16.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:16 smithi163 ceph-mon[34520]: pgmap v424: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:09:16.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:16 smithi192 ceph-mon[34664]: pgmap v424: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:09:16.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:16.997+0000 7fcf66a00700 1 -- 172.21.15.123:0/2497255675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf60104730 msgr2=0x7fcf60104b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:16.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:16.997+0000 7fcf66a00700 1 --2- 172.21.15.123:0/2497255675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf60104730 0x7fcf60104b90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c0099c0 tx=0x7fcf4c009cd0 comp rx=0 tx=0).stop 2024-05-08T06:09:16.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:16.997+0000 7fcf66a00700 1 -- 172.21.15.123:0/2497255675 shutdown_connections 2024-05-08T06:09:16.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:16.997+0000 7fcf66a00700 1 --2- 172.21.15.123:0/2497255675 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf601050d0 0x7fcf6010b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:16.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:16.997+0000 7fcf66a00700 1 --2- 172.21.15.123:0/2497255675 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf60103530 0x7fcf60103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:16.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:16.997+0000 7fcf66a00700 1 --2- 172.21.15.123:0/2497255675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf60104730 0x7fcf60104b90 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:16.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:16.997+0000 7fcf66a00700 1 -- 172.21.15.123:0/2497255675 >> 172.21.15.123:0/2497255675 conn(0x7fcf600feaf0 msgr2=0x7fcf60100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:16.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:16.997+0000 7fcf66a00700 1 -- 172.21.15.123:0/2497255675 shutdown_connections 2024-05-08T06:09:16.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:16.997+0000 7fcf66a00700 1 -- 172.21.15.123:0/2497255675 wait complete. 2024-05-08T06:09:17.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:16.998+0000 7fcf66a00700 1 Processor -- start 2024-05-08T06:09:17.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:16.998+0000 7fcf66a00700 1 -- start start 2024-05-08T06:09:17.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:16.998+0000 7fcf66a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf60103530 0x7fcf6019b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:17.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:16.998+0000 7fcf66a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf60104730 0x7fcf6019bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:17.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:16.998+0000 7fcf66a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf601050d0 0x7fcf601a2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:17.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:16.998+0000 7fcf66a00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6010efc0 con 0x7fcf60104730 2024-05-08T06:09:17.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:16.998+0000 7fcf66a00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6010ee40 con 0x7fcf601050d0 2024-05-08T06:09:17.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:16.998+0000 7fcf66a00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6010f140 con 0x7fcf60103530 2024-05-08T06:09:17.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:16.999+0000 7fcf5f400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf60104730 0x7fcf6019bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:17.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:16.999+0000 7fcf5f400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf60104730 0x7fcf6019bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55886/0 (socket says 172.21.15.123:55886) 2024-05-08T06:09:17.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:16.999+0000 7fcf5f400700 1 -- 172.21.15.123:0/860152115 learned_addr learned my addr 172.21.15.123:0/860152115 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:09:17.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:16.999+0000 7fcf5fe00700 1 --2- 172.21.15.123:0/860152115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf60103530 0x7fcf6019b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:17.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:16.999+0000 7fcf64a00700 1 --2- 172.21.15.123:0/860152115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf601050d0 0x7fcf601a2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:17.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:16.999+0000 7fcf5fe00700 1 -- 172.21.15.123:0/860152115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf601050d0 msgr2=0x7fcf601a2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:17.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:16.999+0000 7fcf5fe00700 1 --2- 172.21.15.123:0/860152115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf601050d0 0x7fcf601a2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:17.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:16.999+0000 7fcf5fe00700 1 -- 172.21.15.123:0/860152115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf60104730 msgr2=0x7fcf6019bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:17.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:16.999+0000 7fcf5fe00700 1 --2- 172.21.15.123:0/860152115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf60104730 0x7fcf6019bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:17.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:16.999+0000 7fcf5fe00700 1 -- 172.21.15.123:0/860152115 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf601a34d0 con 0x7fcf60103530 2024-05-08T06:09:17.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:16.999+0000 7fcf5f400700 1 --2- 172.21.15.123:0/860152115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf60104730 0x7fcf6019bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:09:17.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:17.000+0000 7fcf5fe00700 1 --2- 172.21.15.123:0/860152115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf60103530 0x7fcf6019b870 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000e9f0 tx=0x7fcf5000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:17.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:17.000+0000 7fcf5cc00700 1 -- 172.21.15.123:0/860152115 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf50004230 con 0x7fcf60103530 2024-05-08T06:09:17.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:17.000+0000 7fcf66a00700 1 -- 172.21.15.123:0/860152115 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf601a37c0 con 0x7fcf60103530 2024-05-08T06:09:17.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:17.000+0000 7fcf5cc00700 1 -- 172.21.15.123:0/860152115 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf500043d0 con 0x7fcf60103530 2024-05-08T06:09:17.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:17.000+0000 7fcf5cc00700 1 -- 172.21.15.123:0/860152115 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf50003d30 con 0x7fcf60103530 2024-05-08T06:09:17.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:17.000+0000 7fcf66a00700 1 -- 172.21.15.123:0/860152115 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf601a3cb0 con 0x7fcf60103530 2024-05-08T06:09:17.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:17.001+0000 7fcf66a00700 1 -- 172.21.15.123:0/860152115 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf6006a190 con 0x7fcf60103530 2024-05-08T06:09:17.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:17.003+0000 7fcf5cc00700 1 -- 172.21.15.123:0/860152115 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fcf5000c7c0 con 0x7fcf60103530 2024-05-08T06:09:17.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:17.005+0000 7fcf5cc00700 1 --2- 172.21.15.123:0/860152115 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fcf440b2880 0x7fcf440b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:17.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:17.005+0000 7fcf5cc00700 1 -- 172.21.15.123:0/860152115 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fcf500e2570 con 0x7fcf60103530 2024-05-08T06:09:17.005 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:17.005+0000 7fcf5f400700 1 --2- 172.21.15.123:0/860152115 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fcf440b2880 0x7fcf440b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:17.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:17.006+0000 7fcf5f400700 1 --2- 172.21.15.123:0/860152115 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fcf440b2880 0x7fcf440b4d40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c009990 tx=0x7fcf4c0037c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:17.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:17.009+0000 7fcf5cc00700 1 -- 172.21.15.123:0/860152115 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcf500aa240 con 0x7fcf60103530 2024-05-08T06:09:17.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:17.249+0000 7fcf66a00700 1 -- 172.21.15.123:0/860152115 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fcf60076df0 con 0x7fcf440b2880 2024-05-08T06:09:17.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:17.277+0000 7fcf5cc00700 1 -- 172.21.15.123:0/860152115 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7fcf60076df0 con 0x7fcf440b2880 2024-05-08T06:09:17.278 INFO:teuthology.orchestra.run.smithi123.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-08T06:09:17.278 INFO:teuthology.orchestra.run.smithi123.stdout:mds.a smithi123 running (3m) 53s ago 3m 18.0M - 19.0.0-3538-gbee299f4 352576a2df49 daf390136276 2024-05-08T06:09:17.278 INFO:teuthology.orchestra.run.smithi123.stdout:mds.b smithi163 running (3m) 89s ago 3m 26.3M - 19.0.0-3538-gbee299f4 352576a2df49 222c13906a21 2024-05-08T06:09:17.278 INFO:teuthology.orchestra.run.smithi123.stdout:mds.c smithi192 running (4m) 89s ago 4m 21.0M - 19.0.0-3538-gbee299f4 352576a2df49 61baa468f6d0 2024-05-08T06:09:17.278 INFO:teuthology.orchestra.run.smithi123.stdout:mds.d smithi123 running (4m) 53s ago 4m 13.0M - 19.0.0-3538-gbee299f4 352576a2df49 781cf0787d16 2024-05-08T06:09:17.278 INFO:teuthology.orchestra.run.smithi123.stdout:mds.e smithi163 running (4m) 89s ago 4m 16.9M - 19.0.0-3538-gbee299f4 352576a2df49 7203fdc45ce1 2024-05-08T06:09:17.278 INFO:teuthology.orchestra.run.smithi123.stdout:mds.f smithi192 running (4m) 89s ago 4m 18.8M - 19.0.0-3538-gbee299f4 352576a2df49 67904393a6f4 2024-05-08T06:09:17.278 INFO:teuthology.orchestra.run.smithi123.stdout:mds.g smithi123 running (3m) 53s ago 3m 12.9M - 19.0.0-3538-gbee299f4 352576a2df49 18d9e4aac5d9 2024-05-08T06:09:17.279 INFO:teuthology.orchestra.run.smithi123.stdout:mds.h smithi163 running (4m) 89s ago 4m 22.3M - 19.0.0-3538-gbee299f4 352576a2df49 ae165ce0973e 2024-05-08T06:09:17.279 INFO:teuthology.orchestra.run.smithi123.stdout:mds.i smithi192 running (4m) 89s ago 4m 19.4M - 19.0.0-3538-gbee299f4 352576a2df49 58e62652450f 2024-05-08T06:09:17.279 INFO:teuthology.orchestra.run.smithi123.stdout:mds.j smithi123 running (4m) 53s ago 4m 13.1M - 19.0.0-3538-gbee299f4 352576a2df49 4b34666c1f36 2024-05-08T06:09:17.279 INFO:teuthology.orchestra.run.smithi123.stdout:mds.k smithi163 running (4m) 89s ago 4m 17.0M - 19.0.0-3538-gbee299f4 352576a2df49 aafe0eeceb55 2024-05-08T06:09:17.279 INFO:teuthology.orchestra.run.smithi123.stdout:mds.l smithi192 running (4m) 89s ago 4m 16.8M - 19.0.0-3538-gbee299f4 352576a2df49 9f56e97b86fe 2024-05-08T06:09:17.279 INFO:teuthology.orchestra.run.smithi123.stdout:mgr.x smithi123 *:9283,8765 running (12m) 53s ago 12m 530M - 19.0.0-3538-gbee299f4 352576a2df49 1df64aa6e04a 2024-05-08T06:09:17.279 INFO:teuthology.orchestra.run.smithi123.stdout:mgr.y smithi163 *:8443,8765 running (10m) 89s ago 10m 470M - 19.0.0-3538-gbee299f4 352576a2df49 de9650aaecc9 2024-05-08T06:09:17.279 INFO:teuthology.orchestra.run.smithi123.stdout:mgr.z smithi192 *:8443,8765 running (10m) 89s ago 10m 470M - 19.0.0-3538-gbee299f4 352576a2df49 85c10a5c884b 2024-05-08T06:09:17.279 INFO:teuthology.orchestra.run.smithi123.stdout:mon.a smithi123 running (12m) 53s ago 13m 48.9M 2048M 19.0.0-3538-gbee299f4 352576a2df49 f0e5e4ca8080 2024-05-08T06:09:17.279 INFO:teuthology.orchestra.run.smithi123.stdout:mon.b smithi163 running (10m) 89s ago 10m 44.6M 2048M 19.0.0-3538-gbee299f4 352576a2df49 72a80a2675fe 2024-05-08T06:09:17.279 INFO:teuthology.orchestra.run.smithi123.stdout:mon.c smithi192 running (10m) 89s ago 10m 45.4M 2048M 19.0.0-3538-gbee299f4 352576a2df49 adfc8c6ace0c 2024-05-08T06:09:17.279 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0 smithi123 running (9m) 53s ago 9m 45.2M 4272M 19.0.0-3538-gbee299f4 352576a2df49 fa983cfb7497 2024-05-08T06:09:17.279 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1 smithi163 running (9m) 89s ago 9m 57.7M 4272M 19.0.0-3538-gbee299f4 352576a2df49 0279cbe930da 2024-05-08T06:09:17.279 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2 smithi192 running (9m) 89s ago 9m 59.8M 4272M 19.0.0-3538-gbee299f4 352576a2df49 df36c3c198f8 2024-05-08T06:09:17.280 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3 smithi123 running (8m) 53s ago 8m 55.0M 4272M 19.0.0-3538-gbee299f4 352576a2df49 2bdbe3117231 2024-05-08T06:09:17.280 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4 smithi163 running (8m) 89s ago 8m 42.9M 4272M 19.0.0-3538-gbee299f4 352576a2df49 6e390b3ce4b1 2024-05-08T06:09:17.280 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5 smithi192 running (7m) 89s ago 7m 43.3M 4272M 19.0.0-3538-gbee299f4 352576a2df49 8b3bd3b46b98 2024-05-08T06:09:17.280 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6 smithi123 running (7m) 53s ago 7m 50.9M 4272M 19.0.0-3538-gbee299f4 352576a2df49 2f0d57bdb785 2024-05-08T06:09:17.280 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7 smithi163 running (6m) 89s ago 6m 43.5M 4272M 19.0.0-3538-gbee299f4 352576a2df49 0e5766329585 2024-05-08T06:09:17.280 INFO:teuthology.orchestra.run.smithi123.stdout:osd.8 smithi192 running (6m) 89s ago 6m 55.8M 4272M 19.0.0-3538-gbee299f4 352576a2df49 eb91d693cf9a 2024-05-08T06:09:17.280 INFO:teuthology.orchestra.run.smithi123.stdout:osd.9 smithi123 running (5m) 53s ago 5m 50.0M 4272M 19.0.0-3538-gbee299f4 352576a2df49 023b70f352c7 2024-05-08T06:09:17.280 INFO:teuthology.orchestra.run.smithi123.stdout:osd.10 smithi163 running (5m) 89s ago 5m 53.0M 4272M 19.0.0-3538-gbee299f4 352576a2df49 4bdbdd3fe5d1 2024-05-08T06:09:17.280 INFO:teuthology.orchestra.run.smithi123.stdout:osd.11 smithi192 running (4m) 89s ago 4m 56.1M 4272M 19.0.0-3538-gbee299f4 352576a2df49 e1982ccf0aa3 2024-05-08T06:09:17.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:17.280+0000 7fcf66a00700 1 -- 172.21.15.123:0/860152115 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fcf440b2880 msgr2=0x7fcf440b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:17.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:17.280+0000 7fcf66a00700 1 --2- 172.21.15.123:0/860152115 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fcf440b2880 0x7fcf440b4d40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c009990 tx=0x7fcf4c0037c0 comp rx=0 tx=0).stop 2024-05-08T06:09:17.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:17.280+0000 7fcf66a00700 1 -- 172.21.15.123:0/860152115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf60103530 msgr2=0x7fcf6019b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:17.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:17.280+0000 7fcf66a00700 1 --2- 172.21.15.123:0/860152115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf60103530 0x7fcf6019b870 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000e9f0 tx=0x7fcf5000ed00 comp rx=0 tx=0).stop 2024-05-08T06:09:17.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:17.280+0000 7fcf66a00700 1 -- 172.21.15.123:0/860152115 shutdown_connections 2024-05-08T06:09:17.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:17.280+0000 7fcf66a00700 1 --2- 172.21.15.123:0/860152115 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fcf440b2880 0x7fcf440b4d40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:17.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:17.280+0000 7fcf66a00700 1 --2- 172.21.15.123:0/860152115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf601050d0 0x7fcf601a2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:17.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:17.280+0000 7fcf66a00700 1 --2- 172.21.15.123:0/860152115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf60103530 0x7fcf6019b870 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:17.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:17.280+0000 7fcf66a00700 1 --2- 172.21.15.123:0/860152115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf60104730 0x7fcf6019bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:17.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:17.280+0000 7fcf66a00700 1 -- 172.21.15.123:0/860152115 >> 172.21.15.123:0/860152115 conn(0x7fcf600feaf0 msgr2=0x7fcf600756d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:17.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:17.280+0000 7fcf66a00700 1 -- 172.21.15.123:0/860152115 shutdown_connections 2024-05-08T06:09:17.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:17.281+0000 7fcf66a00700 1 -- 172.21.15.123:0/860152115 wait complete. 2024-05-08T06:09:18.121 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-08T06:09:18.374 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:18 smithi123 ceph-mon[29787]: pgmap v425: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:09:18.374 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:18 smithi123 ceph-mon[29787]: from='client.24916 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:09:18.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:18 smithi163 ceph-mon[34520]: pgmap v425: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:09:18.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:18 smithi163 ceph-mon[34520]: from='client.24916 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:09:18.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:18 smithi192 ceph-mon[34664]: pgmap v425: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:09:18.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:18 smithi192 ceph-mon[34664]: from='client.24916 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:09:18.578 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:09:20.367 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:20 smithi123 ceph-mon[29787]: pgmap v426: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:09:20.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:20 smithi163 ceph-mon[34520]: pgmap v426: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:09:20.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:20 smithi192 ceph-mon[34664]: pgmap v426: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:09:20.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.917+0000 7f46af200700 1 -- 172.21.15.123:0/912754574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46b0104fc0 msgr2=0x7f46b010b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:20.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.917+0000 7f46af200700 1 --2- 172.21.15.123:0/912754574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46b0104fc0 0x7f46b010b870 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f46a800b0d0 tx=0x7f46a800b3e0 comp rx=0 tx=0).stop 2024-05-08T06:09:20.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.917+0000 7f46af200700 1 -- 172.21.15.123:0/912754574 shutdown_connections 2024-05-08T06:09:20.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.917+0000 7f46af200700 1 --2- 172.21.15.123:0/912754574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46b0104fc0 0x7f46b010b870 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:20.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.917+0000 7f46af200700 1 --2- 172.21.15.123:0/912754574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46b0103420 0x7f46b0103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:20.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.917+0000 7f46af200700 1 --2- 172.21.15.123:0/912754574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46b0104620 0x7f46b0104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:20.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.917+0000 7f46af200700 1 -- 172.21.15.123:0/912754574 >> 172.21.15.123:0/912754574 conn(0x7f46b00fe9e0 msgr2=0x7f46b0100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:20.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.918+0000 7f46af200700 1 -- 172.21.15.123:0/912754574 shutdown_connections 2024-05-08T06:09:20.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.918+0000 7f46af200700 1 -- 172.21.15.123:0/912754574 wait complete. 2024-05-08T06:09:20.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.919+0000 7f46af200700 1 Processor -- start 2024-05-08T06:09:20.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.919+0000 7f46af200700 1 -- start start 2024-05-08T06:09:20.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.919+0000 7f46af200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46b0103420 0x7f46b0199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:20.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.919+0000 7f46af200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46b0104620 0x7f46b0199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:20.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.919+0000 7f46af200700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46b0104fc0 0x7f46b01a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:20.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.919+0000 7f46af200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f46b010ef10 con 0x7f46b0104620 2024-05-08T06:09:20.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.919+0000 7f46af200700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f46b010ed90 con 0x7f46b0103420 2024-05-08T06:09:20.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.919+0000 7f46af200700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f46b010f090 con 0x7f46b0104fc0 2024-05-08T06:09:20.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.919+0000 7f46ad400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46b0104620 0x7f46b0199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:20.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.919+0000 7f46ad400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46b0104620 0x7f46b0199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55906/0 (socket says 172.21.15.123:55906) 2024-05-08T06:09:20.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.920+0000 7f46ad400700 1 -- 172.21.15.123:0/1350118584 learned_addr learned my addr 172.21.15.123:0/1350118584 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:09:20.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.920+0000 7f46ade00700 1 --2- 172.21.15.123:0/1350118584 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46b0103420 0x7f46b0199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:20.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.920+0000 7f46ae800700 1 --2- 172.21.15.123:0/1350118584 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46b0104fc0 0x7f46b01a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:20.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.920+0000 7f46ad400700 1 -- 172.21.15.123:0/1350118584 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46b0104fc0 msgr2=0x7f46b01a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:20.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.920+0000 7f46ad400700 1 --2- 172.21.15.123:0/1350118584 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46b0104fc0 0x7f46b01a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:20.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.920+0000 7f46ad400700 1 -- 172.21.15.123:0/1350118584 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46b0103420 msgr2=0x7f46b0199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:20.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.920+0000 7f46ad400700 1 --2- 172.21.15.123:0/1350118584 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46b0103420 0x7f46b0199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:20.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.920+0000 7f46ad400700 1 -- 172.21.15.123:0/1350118584 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b01a1160 con 0x7f46b0104620 2024-05-08T06:09:20.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.920+0000 7f46ade00700 1 --2- 172.21.15.123:0/1350118584 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46b0103420 0x7f46b0199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:09:20.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.920+0000 7f46ad400700 1 --2- 172.21.15.123:0/1350118584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46b0104620 0x7f46b0199a40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f469c00e9f0 tx=0x7f469c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:20.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.920+0000 7f46a6a00700 1 -- 172.21.15.123:0/1350118584 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c00cbd0 con 0x7f46b0104620 2024-05-08T06:09:20.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.921+0000 7f46af200700 1 -- 172.21.15.123:0/1350118584 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46b01a1450 con 0x7f46b0104620 2024-05-08T06:09:20.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.921+0000 7f46a6a00700 1 -- 172.21.15.123:0/1350118584 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f469c00ee90 con 0x7f46b0104620 2024-05-08T06:09:20.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.921+0000 7f46a6a00700 1 -- 172.21.15.123:0/1350118584 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c0055b0 con 0x7f46b0104620 2024-05-08T06:09:20.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.921+0000 7f46af200700 1 -- 172.21.15.123:0/1350118584 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46b01a1940 con 0x7f46b0104620 2024-05-08T06:09:20.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.922+0000 7f46af200700 1 -- 172.21.15.123:0/1350118584 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46b006a190 con 0x7f46b0104620 2024-05-08T06:09:20.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.925+0000 7f46a6a00700 1 -- 172.21.15.123:0/1350118584 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f469c004750 con 0x7f46b0104620 2024-05-08T06:09:20.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.926+0000 7f46a6a00700 1 --2- 172.21.15.123:0/1350118584 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f46940b2880 0x7f46940b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:20.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.926+0000 7f46a6a00700 1 -- 172.21.15.123:0/1350118584 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f469c0e2bf0 con 0x7f46b0104620 2024-05-08T06:09:20.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.927+0000 7f46ade00700 1 --2- 172.21.15.123:0/1350118584 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f46940b2880 0x7f46940b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:20.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.927+0000 7f46ade00700 1 --2- 172.21.15.123:0/1350118584 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f46940b2880 0x7f46940b4d40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f46b0104480 tx=0x7f46a0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:20.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:20.928+0000 7f46a6a00700 1 -- 172.21.15.123:0/1350118584 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f469c0aa8c0 con 0x7f46b0104620 2024-05-08T06:09:21.173 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:21.173+0000 7f46af200700 1 -- 172.21.15.123:0/1350118584 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f46b00638e0 con 0x7f46940b2880 2024-05-08T06:09:21.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:21.182+0000 7f46a6a00700 1 -- 172.21.15.123:0/1350118584 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f46b00638e0 con 0x7f46940b2880 2024-05-08T06:09:21.182 INFO:teuthology.orchestra.run.smithi123.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-08T06:09:21.183 INFO:teuthology.orchestra.run.smithi123.stdout:mds.all 12/12 92s ago 4m smithi123=a;smithi123=d;smithi123=g;smithi123=j;smithi163=b;smithi163=e;smithi163=h;smithi163=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l;count:12 2024-05-08T06:09:21.183 INFO:teuthology.orchestra.run.smithi123.stdout:mgr 3/3 92s ago 10m smithi123=x;smithi163=y;smithi192=z;count:3 2024-05-08T06:09:21.183 INFO:teuthology.orchestra.run.smithi123.stdout:mon 3/3 92s ago 10m smithi123:172.21.15.123=a;smithi163:172.21.15.163=b;smithi192:172.21.15.192=c;count:3 2024-05-08T06:09:21.183 INFO:teuthology.orchestra.run.smithi123.stdout:osd 12 92s ago - 2024-05-08T06:09:21.185 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:21.185+0000 7f46af200700 1 -- 172.21.15.123:0/1350118584 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f46940b2880 msgr2=0x7f46940b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:21.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:21.185+0000 7f46af200700 1 --2- 172.21.15.123:0/1350118584 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f46940b2880 0x7f46940b4d40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f46b0104480 tx=0x7f46a0009260 comp rx=0 tx=0).stop 2024-05-08T06:09:21.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:21.185+0000 7f46af200700 1 -- 172.21.15.123:0/1350118584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46b0104620 msgr2=0x7f46b0199a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:21.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:21.185+0000 7f46af200700 1 --2- 172.21.15.123:0/1350118584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46b0104620 0x7f46b0199a40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f469c00e9f0 tx=0x7f469c00ed00 comp rx=0 tx=0).stop 2024-05-08T06:09:21.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:21.185+0000 7f46af200700 1 -- 172.21.15.123:0/1350118584 shutdown_connections 2024-05-08T06:09:21.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:21.185+0000 7f46af200700 1 --2- 172.21.15.123:0/1350118584 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f46940b2880 0x7f46940b4d40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:21.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:21.185+0000 7f46af200700 1 --2- 172.21.15.123:0/1350118584 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46b0104fc0 0x7f46b01a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:21.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:21.185+0000 7f46af200700 1 --2- 172.21.15.123:0/1350118584 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46b0103420 0x7f46b0199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:21.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:21.185+0000 7f46af200700 1 --2- 172.21.15.123:0/1350118584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46b0104620 0x7f46b0199a40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:21.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:21.185+0000 7f46af200700 1 -- 172.21.15.123:0/1350118584 >> 172.21.15.123:0/1350118584 conn(0x7f46b00fe9e0 msgr2=0x7f46b0107920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:21.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:21.186+0000 7f46af200700 1 -- 172.21.15.123:0/1350118584 shutdown_connections 2024-05-08T06:09:21.186 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:21.186+0000 7f46af200700 1 -- 172.21.15.123:0/1350118584 wait complete. 2024-05-08T06:09:22.078 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-08T06:09:22.532 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:09:22.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:22 smithi192 ceph-mon[34664]: pgmap v427: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:09:22.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:22 smithi192 ceph-mon[34664]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:09:22.680 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:22 smithi123 ceph-mon[29787]: pgmap v427: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:09:22.680 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:22 smithi123 ceph-mon[29787]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:09:22.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:22 smithi163 ceph-mon[34520]: pgmap v427: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:09:22.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:22 smithi163 ceph-mon[34520]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:09:24.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:24 smithi192 ceph-mon[34664]: pgmap v428: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:09:24.592 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:24 smithi123 ceph-mon[29787]: pgmap v428: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:09:24.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:24 smithi163 ceph-mon[34520]: pgmap v428: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:09:24.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.919+0000 7f5591400700 1 -- 172.21.15.123:0/3061527119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f558c10ad40 msgr2=0x7f558c10b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:24.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.919+0000 7f5591400700 1 --2- 172.21.15.123:0/3061527119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f558c10ad40 0x7f558c10b1a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f55740099c0 tx=0x7f5574009cd0 comp rx=0 tx=0).stop 2024-05-08T06:09:24.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.920+0000 7f5591400700 1 -- 172.21.15.123:0/3061527119 shutdown_connections 2024-05-08T06:09:24.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.920+0000 7f5591400700 1 --2- 172.21.15.123:0/3061527119 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f558c10b6e0 0x7f558c111f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:24.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.920+0000 7f5591400700 1 --2- 172.21.15.123:0/3061527119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f558c109b40 0x7f558c109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:24.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.920+0000 7f5591400700 1 --2- 172.21.15.123:0/3061527119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f558c10ad40 0x7f558c10b1a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:24.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.920+0000 7f5591400700 1 -- 172.21.15.123:0/3061527119 >> 172.21.15.123:0/3061527119 conn(0x7f558c078b10 msgr2=0x7f558c07af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:24.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.920+0000 7f5591400700 1 -- 172.21.15.123:0/3061527119 shutdown_connections 2024-05-08T06:09:24.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.920+0000 7f5591400700 1 -- 172.21.15.123:0/3061527119 wait complete. 2024-05-08T06:09:24.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.921+0000 7f5591400700 1 Processor -- start 2024-05-08T06:09:24.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.921+0000 7f5591400700 1 -- start start 2024-05-08T06:09:24.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.921+0000 7f5591400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f558c109b40 0x7f558c19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:24.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.921+0000 7f5591400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f558c10ad40 0x7f558c1a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:24.922 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.921+0000 7f5591400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f558c10b6e0 0x7f558c1a7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:24.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.921+0000 7f5591400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f558c115620 con 0x7f558c10ad40 2024-05-08T06:09:24.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.921+0000 7f5591400700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f558c1154a0 con 0x7f558c10b6e0 2024-05-08T06:09:24.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.921+0000 7f5591400700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f558c1157a0 con 0x7f558c109b40 2024-05-08T06:09:24.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.922+0000 7f5581e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f558c10ad40 0x7f558c1a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:24.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.922+0000 7f558a800700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f558c109b40 0x7f558c19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:24.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.922+0000 7f558b200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f558c10b6e0 0x7f558c1a7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:24.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.922+0000 7f5581e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f558c10ad40 0x7f558c1a01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51316/0 (socket says 172.21.15.123:51316) 2024-05-08T06:09:24.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.922+0000 7f5581e00700 1 -- 172.21.15.123:0/3542523798 learned_addr learned my addr 172.21.15.123:0/3542523798 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:09:24.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.922+0000 7f558a800700 1 -- 172.21.15.123:0/3542523798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f558c10b6e0 msgr2=0x7f558c1a7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:24.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.922+0000 7f558a800700 1 --2- 172.21.15.123:0/3542523798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f558c10b6e0 0x7f558c1a7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:24.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.922+0000 7f558a800700 1 -- 172.21.15.123:0/3542523798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f558c10ad40 msgr2=0x7f558c1a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:24.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.922+0000 7f558a800700 1 --2- 172.21.15.123:0/3542523798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f558c10ad40 0x7f558c1a01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:24.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.922+0000 7f558a800700 1 -- 172.21.15.123:0/3542523798 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558c1a7910 con 0x7f558c109b40 2024-05-08T06:09:24.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.922+0000 7f558b200700 1 --2- 172.21.15.123:0/3542523798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f558c10b6e0 0x7f558c1a7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:09:24.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.922+0000 7f558a800700 1 --2- 172.21.15.123:0/3542523798 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f558c109b40 0x7f558c19fc90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f557c00ef90 tx=0x7f557c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:24.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.923+0000 7f5583e00700 1 -- 172.21.15.123:0/3542523798 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c003a00 con 0x7f558c109b40 2024-05-08T06:09:24.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.923+0000 7f5583e00700 1 -- 172.21.15.123:0/3542523798 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f557c003ba0 con 0x7f558c109b40 2024-05-08T06:09:24.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.923+0000 7f5583e00700 1 -- 172.21.15.123:0/3542523798 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c0047a0 con 0x7f558c109b40 2024-05-08T06:09:24.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.923+0000 7f5591400700 1 -- 172.21.15.123:0/3542523798 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f558c1a7c00 con 0x7f558c109b40 2024-05-08T06:09:24.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.923+0000 7f5591400700 1 -- 172.21.15.123:0/3542523798 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f558c1a8140 con 0x7f558c109b40 2024-05-08T06:09:24.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.924+0000 7f5591400700 1 -- 172.21.15.123:0/3542523798 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f558c06a190 con 0x7f558c109b40 2024-05-08T06:09:24.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.926+0000 7f5583e00700 1 -- 172.21.15.123:0/3542523798 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f557c012050 con 0x7f558c109b40 2024-05-08T06:09:24.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.928+0000 7f5583e00700 1 --2- 172.21.15.123:0/3542523798 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f55700b2900 0x7f55700b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:24.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.928+0000 7f5583e00700 1 -- 172.21.15.123:0/3542523798 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f557c0e22e0 con 0x7f558c109b40 2024-05-08T06:09:24.928 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.928+0000 7f5581e00700 1 --2- 172.21.15.123:0/3542523798 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f55700b2900 0x7f55700b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:24.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.929+0000 7f5581e00700 1 --2- 172.21.15.123:0/3542523798 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f55700b2900 0x7f55700b4dc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f558c1a11a0 tx=0x7f5574005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:24.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:24.930+0000 7f5583e00700 1 -- 172.21.15.123:0/3542523798 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f557c010040 con 0x7f558c109b40 2024-05-08T06:09:25.176 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:25.176+0000 7f5591400700 1 -- 172.21.15.123:0/3542523798 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f558c0638e0 con 0x7f55700b2900 2024-05-08T06:09:25.178 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:25.178+0000 7f5583e00700 1 -- 172.21.15.123:0/3542523798 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f558c0638e0 con 0x7f55700b2900 2024-05-08T06:09:25.178 INFO:teuthology.orchestra.run.smithi123.stdout:HOST ADDR LABELS STATUS 2024-05-08T06:09:25.178 INFO:teuthology.orchestra.run.smithi123.stdout:smithi123 172.21.15.123 2024-05-08T06:09:25.178 INFO:teuthology.orchestra.run.smithi123.stdout:smithi163 172.21.15.163 2024-05-08T06:09:25.178 INFO:teuthology.orchestra.run.smithi123.stdout:smithi192 172.21.15.192 2024-05-08T06:09:25.178 INFO:teuthology.orchestra.run.smithi123.stdout:3 hosts in cluster 2024-05-08T06:09:25.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:25.181+0000 7f5591400700 1 -- 172.21.15.123:0/3542523798 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f55700b2900 msgr2=0x7f55700b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:25.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:25.181+0000 7f5591400700 1 --2- 172.21.15.123:0/3542523798 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f55700b2900 0x7f55700b4dc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f558c1a11a0 tx=0x7f5574005fd0 comp rx=0 tx=0).stop 2024-05-08T06:09:25.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:25.181+0000 7f5591400700 1 -- 172.21.15.123:0/3542523798 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f558c109b40 msgr2=0x7f558c19fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:25.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:25.181+0000 7f5591400700 1 --2- 172.21.15.123:0/3542523798 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f558c109b40 0x7f558c19fc90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f557c00ef90 tx=0x7f557c00c3a0 comp rx=0 tx=0).stop 2024-05-08T06:09:25.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:25.181+0000 7f5591400700 1 -- 172.21.15.123:0/3542523798 shutdown_connections 2024-05-08T06:09:25.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:25.181+0000 7f5591400700 1 --2- 172.21.15.123:0/3542523798 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f55700b2900 0x7f55700b4dc0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:25.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:25.181+0000 7f5591400700 1 --2- 172.21.15.123:0/3542523798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f558c10b6e0 0x7f558c1a7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:25.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:25.181+0000 7f5591400700 1 --2- 172.21.15.123:0/3542523798 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f558c109b40 0x7f558c19fc90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:25.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:25.181+0000 7f5591400700 1 --2- 172.21.15.123:0/3542523798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f558c10ad40 0x7f558c1a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:25.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:25.181+0000 7f5591400700 1 -- 172.21.15.123:0/3542523798 >> 172.21.15.123:0/3542523798 conn(0x7f558c078b10 msgr2=0x7f558c10e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:25.182 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:25.182+0000 7f5591400700 1 -- 172.21.15.123:0/3542523798 shutdown_connections 2024-05-08T06:09:25.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:25.182+0000 7f5591400700 1 -- 172.21.15.123:0/3542523798 wait complete. 2024-05-08T06:09:25.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:25 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:09:25.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:25 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:09:25.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:25 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:09:26.106 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-08T06:09:26.531 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:09:26.916 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:26 smithi123 ceph-mon[29787]: pgmap v429: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:09:26.916 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:26 smithi123 ceph-mon[29787]: from='client.24928 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:09:26.916 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:26 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:09:26.916 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:26 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:09:26.917 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:26 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:09:26.917 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:26 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:09:27.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:26 smithi163 ceph-mon[34520]: pgmap v429: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:09:27.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:26 smithi163 ceph-mon[34520]: from='client.24928 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:09:27.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:26 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:09:27.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:26 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:09:27.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:26 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:09:27.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:26 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:09:27.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:26 smithi192 ceph-mon[34664]: pgmap v429: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:09:27.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:26 smithi192 ceph-mon[34664]: from='client.24928 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:09:27.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:26 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:09:27.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:26 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:09:27.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:26 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:09:27.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:26 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:09:27.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:27 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:09:27.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:27 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:09:27.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:27 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:09:28.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:27 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:09:28.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:27 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:09:28.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:27 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:09:28.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:09:28.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:09:28.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:09:28.867 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:28 smithi123 ceph-mon[29787]: pgmap v430: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:09:28.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.929+0000 7f136b800700 1 -- 172.21.15.123:0/2248408722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13641091f0 msgr2=0x7f136410b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:28.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.929+0000 7f136b800700 1 --2- 172.21.15.123:0/2248408722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13641091f0 0x7f136410b660 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f135c00b0a0 tx=0x7f135c00b3b0 comp rx=0 tx=0).stop 2024-05-08T06:09:28.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.929+0000 7f136b800700 1 -- 172.21.15.123:0/2248408722 shutdown_connections 2024-05-08T06:09:28.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.929+0000 7f136b800700 1 --2- 172.21.15.123:0/2248408722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13641091f0 0x7f136410b660 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:28.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.929+0000 7f136b800700 1 --2- 172.21.15.123:0/2248408722 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1364103f90 0x7f1364106380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:28.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.929+0000 7f136b800700 1 --2- 172.21.15.123:0/2248408722 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13641068c0 0x7f1364108cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:28.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.929+0000 7f136b800700 1 -- 172.21.15.123:0/2248408722 >> 172.21.15.123:0/2248408722 conn(0x7f13640fd9a0 msgr2=0x7f13640ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:28.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.929+0000 7f136b800700 1 -- 172.21.15.123:0/2248408722 shutdown_connections 2024-05-08T06:09:28.930 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.929+0000 7f136b800700 1 -- 172.21.15.123:0/2248408722 wait complete. 2024-05-08T06:09:28.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.930+0000 7f136b800700 1 Processor -- start 2024-05-08T06:09:28.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.931+0000 7f136b800700 1 -- start start 2024-05-08T06:09:28.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.931+0000 7f136b800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1364103f90 0x7f13641038f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:28.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.931+0000 7f136b800700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13641068c0 0x7f1364101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:28.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.931+0000 7f136b800700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13641091f0 0x7f1364102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:28.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.931+0000 7f136b800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f136410ef80 con 0x7f1364103f90 2024-05-08T06:09:28.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.931+0000 7f136b800700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f136410ee00 con 0x7f13641068c0 2024-05-08T06:09:28.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.931+0000 7f136b800700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f136410f100 con 0x7f13641091f0 2024-05-08T06:09:28.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.931+0000 7f1368e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1364103f90 0x7f13641038f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:28.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.931+0000 7f1363e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13641068c0 0x7f1364101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:28.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.931+0000 7f1363e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13641068c0 0x7f1364101f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:59014/0 (socket says 172.21.15.123:59014) 2024-05-08T06:09:28.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.931+0000 7f1368e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1364103f90 0x7f13641038f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51332/0 (socket says 172.21.15.123:51332) 2024-05-08T06:09:28.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.931+0000 7f1363e00700 1 -- 172.21.15.123:0/551644894 learned_addr learned my addr 172.21.15.123:0/551644894 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:09:28.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.931+0000 7f1369800700 1 --2- 172.21.15.123:0/551644894 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13641091f0 0x7f1364102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:28.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.932+0000 7f1363e00700 1 -- 172.21.15.123:0/551644894 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13641091f0 msgr2=0x7f1364102480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:28.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.932+0000 7f1363e00700 1 --2- 172.21.15.123:0/551644894 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13641091f0 0x7f1364102480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:28.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.932+0000 7f1363e00700 1 -- 172.21.15.123:0/551644894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1364103f90 msgr2=0x7f13641038f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:28.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.932+0000 7f1363e00700 1 --2- 172.21.15.123:0/551644894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1364103f90 0x7f13641038f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:28.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.932+0000 7f1363e00700 1 -- 172.21.15.123:0/551644894 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1364102d30 con 0x7f13641068c0 2024-05-08T06:09:28.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.932+0000 7f1363e00700 1 --2- 172.21.15.123:0/551644894 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13641068c0 0x7f1364101f40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f135000e9a0 tx=0x7f135000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:28.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.932+0000 7f1361600700 1 -- 172.21.15.123:0/551644894 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135000ca10 con 0x7f13641068c0 2024-05-08T06:09:28.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.932+0000 7f1361600700 1 -- 172.21.15.123:0/551644894 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f135000cbb0 con 0x7f13641068c0 2024-05-08T06:09:28.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.932+0000 7f136b800700 1 -- 172.21.15.123:0/551644894 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13641a3810 con 0x7f13641068c0 2024-05-08T06:09:28.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.932+0000 7f1361600700 1 -- 172.21.15.123:0/551644894 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1350018680 con 0x7f13641068c0 2024-05-08T06:09:28.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.933+0000 7f136b800700 1 -- 172.21.15.123:0/551644894 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13641a3d20 con 0x7f13641068c0 2024-05-08T06:09:28.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.934+0000 7f136b800700 1 -- 172.21.15.123:0/551644894 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f136406a190 con 0x7f13641068c0 2024-05-08T06:09:28.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.936+0000 7f1361600700 1 -- 172.21.15.123:0/551644894 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f1350010430 con 0x7f13641068c0 2024-05-08T06:09:28.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.937+0000 7f1361600700 1 --2- 172.21.15.123:0/551644894 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f134c0b2880 0x7f134c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:28.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.938+0000 7f1361600700 1 -- 172.21.15.123:0/551644894 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f13500e2710 con 0x7f13641068c0 2024-05-08T06:09:28.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.938+0000 7f1368e00700 1 --2- 172.21.15.123:0/551644894 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f134c0b2880 0x7f134c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:28.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.938+0000 7f1368e00700 1 --2- 172.21.15.123:0/551644894 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f134c0b2880 0x7f134c0b4d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1358009c60 tx=0x7f1358009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:28.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:28.939+0000 7f1361600700 1 -- 172.21.15.123:0/551644894 <== mon.2 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f13500aa3e0 con 0x7f13641068c0 2024-05-08T06:09:29.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:28 smithi163 ceph-mon[34520]: pgmap v430: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:09:29.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:28 smithi192 ceph-mon[34664]: pgmap v430: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:09:29.183 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:29.183+0000 7f136b800700 1 -- 172.21.15.123:0/551644894 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f13640638e0 con 0x7f134c0b2880 2024-05-08T06:09:29.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:29.189+0000 7f1361600700 1 -- 172.21.15.123:0/551644894 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f13640638e0 con 0x7f134c0b2880 2024-05-08T06:09:29.189 INFO:teuthology.orchestra.run.smithi123.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-08T06:09:29.189 INFO:teuthology.orchestra.run.smithi123.stdout:smithi123 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400YK400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-05-08T06:09:29.189 INFO:teuthology.orchestra.run.smithi123.stdout:smithi163 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GU400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-05-08T06:09:29.189 INFO:teuthology.orchestra.run.smithi123.stdout:smithi192 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500NN400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-05-08T06:09:29.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:29.192+0000 7f136b800700 1 -- 172.21.15.123:0/551644894 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f134c0b2880 msgr2=0x7f134c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:29.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:29.192+0000 7f136b800700 1 --2- 172.21.15.123:0/551644894 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f134c0b2880 0x7f134c0b4d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1358009c60 tx=0x7f1358009260 comp rx=0 tx=0).stop 2024-05-08T06:09:29.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:29.192+0000 7f136b800700 1 -- 172.21.15.123:0/551644894 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13641068c0 msgr2=0x7f1364101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:29.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:29.192+0000 7f136b800700 1 --2- 172.21.15.123:0/551644894 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13641068c0 0x7f1364101f40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f135000e9a0 tx=0x7f135000ecb0 comp rx=0 tx=0).stop 2024-05-08T06:09:29.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:29.192+0000 7f136b800700 1 -- 172.21.15.123:0/551644894 shutdown_connections 2024-05-08T06:09:29.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:29.192+0000 7f136b800700 1 --2- 172.21.15.123:0/551644894 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f134c0b2880 0x7f134c0b4d40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:29.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:29.192+0000 7f136b800700 1 --2- 172.21.15.123:0/551644894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1364103f90 0x7f13641038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:29.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:29.192+0000 7f136b800700 1 --2- 172.21.15.123:0/551644894 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13641091f0 0x7f1364102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:29.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:29.192+0000 7f136b800700 1 --2- 172.21.15.123:0/551644894 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13641068c0 0x7f1364101f40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:29.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:29.192+0000 7f136b800700 1 -- 172.21.15.123:0/551644894 >> 172.21.15.123:0/551644894 conn(0x7f13640fd9a0 msgr2=0x7f13641048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:29.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:29.192+0000 7f136b800700 1 -- 172.21.15.123:0/551644894 shutdown_connections 2024-05-08T06:09:29.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:29.192+0000 7f136b800700 1 -- 172.21.15.123:0/551644894 wait complete. 2024-05-08T06:09:30.150 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-08T06:09:30.164 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi123.front.sepia.ceph.com 2024-05-08T06:09:30.164 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-08T06:09:30.602 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:09:30.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:30 smithi123 ceph-mon[29787]: pgmap v431: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:09:30.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:30 smithi123 ceph-mon[29787]: from='client.24941 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:09:31.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:30 smithi163 ceph-mon[34520]: pgmap v431: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:09:31.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:30 smithi163 ceph-mon[34520]: from='client.24941 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:09:31.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:30 smithi192 ceph-mon[34664]: pgmap v431: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:09:31.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:30 smithi192 ceph-mon[34664]: from='client.24941 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:09:32.933 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:32 smithi123 ceph-mon[29787]: pgmap v432: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:09:32.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.952+0000 7f2248c00700 1 -- 172.21.15.123:0/2889008228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22441050f0 msgr2=0x7f224410b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:32.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.952+0000 7f2248c00700 1 --2- 172.21.15.123:0/2889008228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22441050f0 0x7f224410b9a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f223c00b0d0 tx=0x7f223c00b3e0 comp rx=0 tx=0).stop 2024-05-08T06:09:32.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.952+0000 7f2248c00700 1 -- 172.21.15.123:0/2889008228 shutdown_connections 2024-05-08T06:09:32.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.952+0000 7f2248c00700 1 --2- 172.21.15.123:0/2889008228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22441050f0 0x7f224410b9a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:32.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.952+0000 7f2248c00700 1 --2- 172.21.15.123:0/2889008228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2244103550 0x7f2244103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:32.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.952+0000 7f2248c00700 1 --2- 172.21.15.123:0/2889008228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2244104750 0x7f2244104bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:32.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.952+0000 7f2248c00700 1 -- 172.21.15.123:0/2889008228 >> 172.21.15.123:0/2889008228 conn(0x7f22440feaf0 msgr2=0x7f2244100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:32.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.952+0000 7f2248c00700 1 -- 172.21.15.123:0/2889008228 shutdown_connections 2024-05-08T06:09:32.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.952+0000 7f2248c00700 1 -- 172.21.15.123:0/2889008228 wait complete. 2024-05-08T06:09:32.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.953+0000 7f2248c00700 1 Processor -- start 2024-05-08T06:09:32.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.954+0000 7f2248c00700 1 -- start start 2024-05-08T06:09:32.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.954+0000 7f2248c00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2244103550 0x7f224419b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:32.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.954+0000 7f2248c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2244104750 0x7f224419bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:32.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.954+0000 7f2248c00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22441050f0 0x7f22441a2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:32.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.954+0000 7f2248c00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f224410eff0 con 0x7f2244104750 2024-05-08T06:09:32.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.954+0000 7f2248c00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f224410ee70 con 0x7f22441050f0 2024-05-08T06:09:32.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.954+0000 7f2248c00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f224410f170 con 0x7f2244103550 2024-05-08T06:09:32.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.954+0000 7f2241e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2244103550 0x7f224419b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:32.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.954+0000 7f2241400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2244104750 0x7f224419bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:32.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.954+0000 7f2241e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2244103550 0x7f224419b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:57998/0 (socket says 172.21.15.123:57998) 2024-05-08T06:09:32.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.954+0000 7f2241e00700 1 -- 172.21.15.123:0/3453072492 learned_addr learned my addr 172.21.15.123:0/3453072492 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:09:32.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.954+0000 7f2242800700 1 --2- 172.21.15.123:0/3453072492 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22441050f0 0x7f22441a2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:32.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.955+0000 7f2242800700 1 -- 172.21.15.123:0/3453072492 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2244103550 msgr2=0x7f224419b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:32.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.955+0000 7f2242800700 1 --2- 172.21.15.123:0/3453072492 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2244103550 0x7f224419b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:32.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.955+0000 7f2242800700 1 -- 172.21.15.123:0/3453072492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2244104750 msgr2=0x7f224419bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:32.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.955+0000 7f2242800700 1 --2- 172.21.15.123:0/3453072492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2244104750 0x7f224419bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:32.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.955+0000 7f2242800700 1 -- 172.21.15.123:0/3453072492 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22441a3440 con 0x7f22441050f0 2024-05-08T06:09:32.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.955+0000 7f2241e00700 1 --2- 172.21.15.123:0/3453072492 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2244103550 0x7f224419b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:09:32.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.955+0000 7f2242800700 1 --2- 172.21.15.123:0/3453072492 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22441050f0 0x7f22441a2da0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f223c00b770 tx=0x7f223c004ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:32.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.955+0000 7f223aa00700 1 -- 172.21.15.123:0/3453072492 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223c021070 con 0x7f22441050f0 2024-05-08T06:09:32.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.955+0000 7f2248c00700 1 -- 172.21.15.123:0/3453072492 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22441a36d0 con 0x7f22441050f0 2024-05-08T06:09:32.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.955+0000 7f223aa00700 1 -- 172.21.15.123:0/3453072492 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f223c007b80 con 0x7f22441050f0 2024-05-08T06:09:32.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.955+0000 7f223aa00700 1 -- 172.21.15.123:0/3453072492 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223c01b6b0 con 0x7f22441050f0 2024-05-08T06:09:32.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.956+0000 7f2248c00700 1 -- 172.21.15.123:0/3453072492 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22441a3b20 con 0x7f22441050f0 2024-05-08T06:09:32.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.957+0000 7f2248c00700 1 -- 172.21.15.123:0/3453072492 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f224406a190 con 0x7f22441050f0 2024-05-08T06:09:32.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.959+0000 7f223aa00700 1 -- 172.21.15.123:0/3453072492 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f223c024070 con 0x7f22441050f0 2024-05-08T06:09:32.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.960+0000 7f223aa00700 1 --2- 172.21.15.123:0/3453072492 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f22280b2900 0x7f22280b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:32.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.960+0000 7f223aa00700 1 -- 172.21.15.123:0/3453072492 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f223c0e3260 con 0x7f22441050f0 2024-05-08T06:09:32.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.960+0000 7f2241e00700 1 --2- 172.21.15.123:0/3453072492 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f22280b2900 0x7f22280b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:32.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.961+0000 7f2241e00700 1 --2- 172.21.15.123:0/3453072492 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f22280b2900 0x7f22280b4dc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2234005950 tx=0x7f2234009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:32.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:32.963+0000 7f223aa00700 1 -- 172.21.15.123:0/3453072492 <== mon.2 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f223c0aafc0 con 0x7f22441050f0 2024-05-08T06:09:33.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:32 smithi163 ceph-mon[34520]: pgmap v432: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:09:33.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:32 smithi192 ceph-mon[34664]: pgmap v432: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:09:33.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:33.241+0000 7f2248c00700 1 -- 172.21.15.123:0/3453072492 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f22441a4020 con 0x7f22441050f0 2024-05-08T06:09:33.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:33.242+0000 7f223aa00700 1 -- 172.21.15.123:0/3453072492 <== mon.2 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) ==== 76+0+3060 (secure 0 0 0) 0x7f223c0aa8c0 con 0x7f22441050f0 2024-05-08T06:09:33.243 INFO:teuthology.orchestra.run.smithi123.stdout:e23 2024-05-08T06:09:33.243 INFO:teuthology.orchestra.run.smithi123.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T06:09:33.243 INFO:teuthology.orchestra.run.smithi123.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-08T06:09:33.243 INFO:teuthology.orchestra.run.smithi123.stdout:legacy client fscid: 1 2024-05-08T06:09:33.244 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:09:33.244 INFO:teuthology.orchestra.run.smithi123.stdout:Filesystem 'cephfs' (1) 2024-05-08T06:09:33.244 INFO:teuthology.orchestra.run.smithi123.stdout:fs_name cephfs 2024-05-08T06:09:33.244 INFO:teuthology.orchestra.run.smithi123.stdout:epoch 23 2024-05-08T06:09:33.244 INFO:teuthology.orchestra.run.smithi123.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-08T06:09:33.244 INFO:teuthology.orchestra.run.smithi123.stdout:created 2024-05-08T06:05:54.755550+0000 2024-05-08T06:09:33.244 INFO:teuthology.orchestra.run.smithi123.stdout:modified 2024-05-08T06:06:22.594178+0000 2024-05-08T06:09:33.244 INFO:teuthology.orchestra.run.smithi123.stdout:tableserver 0 2024-05-08T06:09:33.244 INFO:teuthology.orchestra.run.smithi123.stdout:root 0 2024-05-08T06:09:33.244 INFO:teuthology.orchestra.run.smithi123.stdout:session_timeout 300 2024-05-08T06:09:33.244 INFO:teuthology.orchestra.run.smithi123.stdout:session_autoclose 300 2024-05-08T06:09:33.244 INFO:teuthology.orchestra.run.smithi123.stdout:max_file_size 1099511627776 2024-05-08T06:09:33.244 INFO:teuthology.orchestra.run.smithi123.stdout:max_xattr_size 65536 2024-05-08T06:09:33.244 INFO:teuthology.orchestra.run.smithi123.stdout:required_client_features {} 2024-05-08T06:09:33.244 INFO:teuthology.orchestra.run.smithi123.stdout:last_failure 0 2024-05-08T06:09:33.245 INFO:teuthology.orchestra.run.smithi123.stdout:last_failure_osd_epoch 0 2024-05-08T06:09:33.245 INFO:teuthology.orchestra.run.smithi123.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-08T06:09:33.245 INFO:teuthology.orchestra.run.smithi123.stdout:max_mds 3 2024-05-08T06:09:33.245 INFO:teuthology.orchestra.run.smithi123.stdout:in 0,1,2 2024-05-08T06:09:33.245 INFO:teuthology.orchestra.run.smithi123.stdout:up {0=24524,1=24512,2=24496} 2024-05-08T06:09:33.245 INFO:teuthology.orchestra.run.smithi123.stdout:failed 2024-05-08T06:09:33.245 INFO:teuthology.orchestra.run.smithi123.stdout:damaged 2024-05-08T06:09:33.245 INFO:teuthology.orchestra.run.smithi123.stdout:stopped 2024-05-08T06:09:33.245 INFO:teuthology.orchestra.run.smithi123.stdout:data_pools [3] 2024-05-08T06:09:33.245 INFO:teuthology.orchestra.run.smithi123.stdout:metadata_pool 2 2024-05-08T06:09:33.245 INFO:teuthology.orchestra.run.smithi123.stdout:inline_data disabled 2024-05-08T06:09:33.245 INFO:teuthology.orchestra.run.smithi123.stdout:balancer 2024-05-08T06:09:33.245 INFO:teuthology.orchestra.run.smithi123.stdout:bal_rank_mask -1 2024-05-08T06:09:33.246 INFO:teuthology.orchestra.run.smithi123.stdout:standby_count_wanted 0 2024-05-08T06:09:33.246 INFO:teuthology.orchestra.run.smithi123.stdout:qdb_cluster leader: 24524 members: 24512,24524,24496 2024-05-08T06:09:33.246 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.b{0:24524} state up:active seq 8 addr [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T06:09:33.246 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.h{0:24485} state up:standby-replay seq 1 addr [v2:172.21.15.163:6836/660511838,v1:172.21.15.163:6837/660511838] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T06:09:33.246 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.a{1:24512} state up:active seq 13 addr [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T06:09:33.246 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.c{1:24481} state up:standby-replay seq 1 addr [v2:172.21.15.192:6836/2483545969,v1:172.21.15.192:6837/2483545969] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T06:09:33.246 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.i{2:24496} state up:active seq 16 addr [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T06:09:33.246 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.f{2:24469} state up:standby-replay seq 1 addr [v2:172.21.15.192:6834/202513680,v1:172.21.15.192:6835/202513680] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T06:09:33.246 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:09:33.246 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:09:33.246 INFO:teuthology.orchestra.run.smithi123.stdout:Standby daemons: 2024-05-08T06:09:33.246 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:09:33.246 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.j{-1:14358} state up:standby seq 1 addr [v2:172.21.15.123:6834/3646476522,v1:172.21.15.123:6835/3646476522] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T06:09:33.247 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.d{-1:14364} state up:standby seq 1 addr [v2:172.21.15.123:6836/2763706497,v1:172.21.15.123:6837/2763706497] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T06:09:33.247 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.g{-1:14388} state up:standby seq 1 addr [v2:172.21.15.123:6838/274635265,v1:172.21.15.123:6839/274635265] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T06:09:33.247 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.l{-1:24430} state up:standby seq 1 addr [v2:172.21.15.192:6832/175423115,v1:172.21.15.192:6833/175423115] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T06:09:33.247 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.k{-1:24437} state up:standby seq 1 addr [v2:172.21.15.163:6832/3263009472,v1:172.21.15.163:6833/3263009472] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T06:09:33.247 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.e{-1:24467} state up:standby seq 1 addr [v2:172.21.15.163:6834/3308612826,v1:172.21.15.163:6835/3308612826] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T06:09:33.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:33.246+0000 7f2248c00700 1 -- 172.21.15.123:0/3453072492 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f22280b2900 msgr2=0x7f22280b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:33.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:33.246+0000 7f2248c00700 1 --2- 172.21.15.123:0/3453072492 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f22280b2900 0x7f22280b4dc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2234005950 tx=0x7f2234009260 comp rx=0 tx=0).stop 2024-05-08T06:09:33.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:33.246+0000 7f2248c00700 1 -- 172.21.15.123:0/3453072492 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22441050f0 msgr2=0x7f22441a2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:33.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:33.246+0000 7f2248c00700 1 --2- 172.21.15.123:0/3453072492 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22441050f0 0x7f22441a2da0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f223c00b770 tx=0x7f223c004ce0 comp rx=0 tx=0).stop 2024-05-08T06:09:33.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:33.246+0000 7f2248c00700 1 -- 172.21.15.123:0/3453072492 shutdown_connections 2024-05-08T06:09:33.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:33.246+0000 7f2248c00700 1 --2- 172.21.15.123:0/3453072492 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f22280b2900 0x7f22280b4dc0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:33.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:33.246+0000 7f2248c00700 1 --2- 172.21.15.123:0/3453072492 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22441050f0 0x7f22441a2da0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:33.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:33.246+0000 7f2248c00700 1 --2- 172.21.15.123:0/3453072492 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2244103550 0x7f224419b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:33.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:33.246+0000 7f2248c00700 1 --2- 172.21.15.123:0/3453072492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2244104750 0x7f224419bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:33.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:33.246+0000 7f2248c00700 1 -- 172.21.15.123:0/3453072492 >> 172.21.15.123:0/3453072492 conn(0x7f22440feaf0 msgr2=0x7f22440756d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:33.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:33.246+0000 7f2248c00700 1 -- 172.21.15.123:0/3453072492 shutdown_connections 2024-05-08T06:09:33.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:33.246+0000 7f2248c00700 1 -- 172.21.15.123:0/3453072492 wait complete. 2024-05-08T06:09:33.248 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 23 2024-05-08T06:09:33.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:33 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/3453072492' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-08T06:09:34.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:33 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/3453072492' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-08T06:09:34.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:33 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/3453072492' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-08T06:09:34.242 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-08T06:09:34.691 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:09:34.836 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:34 smithi123 ceph-mon[29787]: pgmap v433: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:09:35.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:34 smithi163 ceph-mon[34520]: pgmap v433: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:09:35.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:34 smithi192 ceph-mon[34664]: pgmap v433: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:09:36.844 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:36 smithi123 ceph-mon[29787]: pgmap v434: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:09:37.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:36 smithi163 ceph-mon[34520]: pgmap v434: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:09:37.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:36 smithi192 ceph-mon[34664]: pgmap v434: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:09:37.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.428+0000 7fbe29400700 1 -- 172.21.15.123:0/3870812866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe2410b5d0 msgr2=0x7fbe24111e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:37.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.428+0000 7fbe29400700 1 --2- 172.21.15.123:0/3870812866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe2410b5d0 0x7fbe24111e80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c00b0d0 tx=0x7fbe1c00b3e0 comp rx=0 tx=0).stop 2024-05-08T06:09:37.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.429+0000 7fbe29400700 1 -- 172.21.15.123:0/3870812866 shutdown_connections 2024-05-08T06:09:37.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.429+0000 7fbe29400700 1 --2- 172.21.15.123:0/3870812866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe2410b5d0 0x7fbe24111e80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:37.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.429+0000 7fbe29400700 1 --2- 172.21.15.123:0/3870812866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe24109a30 0x7fbe24109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:37.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.429+0000 7fbe29400700 1 --2- 172.21.15.123:0/3870812866 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe2410ac30 0x7fbe2410b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:37.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.429+0000 7fbe29400700 1 -- 172.21.15.123:0/3870812866 >> 172.21.15.123:0/3870812866 conn(0x7fbe24078a00 msgr2=0x7fbe2407ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:37.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.429+0000 7fbe29400700 1 -- 172.21.15.123:0/3870812866 shutdown_connections 2024-05-08T06:09:37.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.429+0000 7fbe29400700 1 -- 172.21.15.123:0/3870812866 wait complete. 2024-05-08T06:09:37.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.430+0000 7fbe29400700 1 Processor -- start 2024-05-08T06:09:37.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.430+0000 7fbe29400700 1 -- start start 2024-05-08T06:09:37.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.430+0000 7fbe29400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe24109a30 0x7fbe2419fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:37.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.430+0000 7fbe29400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe2410ac30 0x7fbe241a0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:37.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.430+0000 7fbe29400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe2410b5d0 0x7fbe241a70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:37.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.430+0000 7fbe29400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbe24115500 con 0x7fbe24109a30 2024-05-08T06:09:37.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.430+0000 7fbe29400700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbe24115380 con 0x7fbe2410ac30 2024-05-08T06:09:37.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.430+0000 7fbe29400700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbe24115680 con 0x7fbe2410b5d0 2024-05-08T06:09:37.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.431+0000 7fbe23e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe24109a30 0x7fbe2419fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:37.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.431+0000 7fbe23400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe2410ac30 0x7fbe241a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:37.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.431+0000 7fbe23400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe2410ac30 0x7fbe241a0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:45472/0 (socket says 172.21.15.123:45472) 2024-05-08T06:09:37.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.431+0000 7fbe23400700 1 -- 172.21.15.123:0/3509042766 learned_addr learned my addr 172.21.15.123:0/3509042766 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:09:37.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.431+0000 7fbe28a00700 1 --2- 172.21.15.123:0/3509042766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe2410b5d0 0x7fbe241a70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:37.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.431+0000 7fbe23400700 1 -- 172.21.15.123:0/3509042766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe2410b5d0 msgr2=0x7fbe241a70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:37.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.431+0000 7fbe23400700 1 --2- 172.21.15.123:0/3509042766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe2410b5d0 0x7fbe241a70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:37.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.431+0000 7fbe23400700 1 -- 172.21.15.123:0/3509042766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe24109a30 msgr2=0x7fbe2419fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:37.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.431+0000 7fbe23400700 1 --2- 172.21.15.123:0/3509042766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe24109a30 0x7fbe2419fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:37.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.431+0000 7fbe23400700 1 -- 172.21.15.123:0/3509042766 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe241a7770 con 0x7fbe2410ac30 2024-05-08T06:09:37.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.431+0000 7fbe23e00700 1 --2- 172.21.15.123:0/3509042766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe24109a30 0x7fbe2419fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:09:37.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.431+0000 7fbe23400700 1 --2- 172.21.15.123:0/3509042766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe2410ac30 0x7fbe241a0050 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbe1000e9a0 tx=0x7fbe1000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:37.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.432+0000 7fbe20c00700 1 -- 172.21.15.123:0/3509042766 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1000ca10 con 0x7fbe2410ac30 2024-05-08T06:09:37.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.432+0000 7fbe29400700 1 -- 172.21.15.123:0/3509042766 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe241a7a60 con 0x7fbe2410ac30 2024-05-08T06:09:37.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.432+0000 7fbe20c00700 1 -- 172.21.15.123:0/3509042766 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe1000cbb0 con 0x7fbe2410ac30 2024-05-08T06:09:37.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.432+0000 7fbe20c00700 1 -- 172.21.15.123:0/3509042766 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe10018680 con 0x7fbe2410ac30 2024-05-08T06:09:37.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.432+0000 7fbe29400700 1 -- 172.21.15.123:0/3509042766 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe241a7ff0 con 0x7fbe2410ac30 2024-05-08T06:09:37.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.433+0000 7fbe29400700 1 -- 172.21.15.123:0/3509042766 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe2406a190 con 0x7fbe2410ac30 2024-05-08T06:09:37.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.436+0000 7fbe20c00700 1 -- 172.21.15.123:0/3509042766 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fbe10010430 con 0x7fbe2410ac30 2024-05-08T06:09:37.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.437+0000 7fbe20c00700 1 --2- 172.21.15.123:0/3509042766 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fbe080b2880 0x7fbe080b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:37.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.437+0000 7fbe20c00700 1 -- 172.21.15.123:0/3509042766 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fbe100e2530 con 0x7fbe2410ac30 2024-05-08T06:09:37.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.438+0000 7fbe23e00700 1 --2- 172.21.15.123:0/3509042766 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fbe080b2880 0x7fbe080b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:37.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.438+0000 7fbe23e00700 1 --2- 172.21.15.123:0/3509042766 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fbe080b2880 0x7fbe080b4d40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbe2410aa90 tx=0x7fbe14007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:37.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.439+0000 7fbe20c00700 1 -- 172.21.15.123:0/3509042766 <== mon.2 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbe100aa200 con 0x7fbe2410ac30 2024-05-08T06:09:37.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.655+0000 7fbe29400700 1 -- 172.21.15.123:0/3509042766 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fbe2402e510 con 0x7fbe2410ac30 2024-05-08T06:09:37.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.656+0000 7fbe20c00700 1 -- 172.21.15.123:0/3509042766 <== mon.2 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v82) ==== 56+0+5310 (secure 0 0 0) 0x7fbe100a9b00 con 0x7fbe2410ac30 2024-05-08T06:09:37.659 INFO:teuthology.orchestra.run.smithi123.stdout:epoch 82 2024-05-08T06:09:37.659 INFO:teuthology.orchestra.run.smithi123.stdout:fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 2024-05-08T06:09:37.659 INFO:teuthology.orchestra.run.smithi123.stdout:created 2024-05-08T05:56:12.544750+0000 2024-05-08T06:09:37.659 INFO:teuthology.orchestra.run.smithi123.stdout:modified 2024-05-08T06:06:32.471993+0000 2024-05-08T06:09:37.659 INFO:teuthology.orchestra.run.smithi123.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-08T06:09:37.659 INFO:teuthology.orchestra.run.smithi123.stdout:crush_version 26 2024-05-08T06:09:37.659 INFO:teuthology.orchestra.run.smithi123.stdout:full_ratio 0.95 2024-05-08T06:09:37.659 INFO:teuthology.orchestra.run.smithi123.stdout:backfillfull_ratio 0.9 2024-05-08T06:09:37.659 INFO:teuthology.orchestra.run.smithi123.stdout:nearfull_ratio 0.85 2024-05-08T06:09:37.659 INFO:teuthology.orchestra.run.smithi123.stdout:require_min_compat_client luminous 2024-05-08T06:09:37.659 INFO:teuthology.orchestra.run.smithi123.stdout:min_compat_client luminous 2024-05-08T06:09:37.659 INFO:teuthology.orchestra.run.smithi123.stdout:require_osd_release squid 2024-05-08T06:09:37.659 INFO:teuthology.orchestra.run.smithi123.stdout:stretch_mode_enabled false 2024-05-08T06:09:37.659 INFO:teuthology.orchestra.run.smithi123.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-08T06:09:37.660 INFO:teuthology.orchestra.run.smithi123.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-08T06:09:37.660 INFO:teuthology.orchestra.run.smithi123.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-05-08T06:09:37.660 INFO:teuthology.orchestra.run.smithi123.stdout:max_osd 12 2024-05-08T06:09:37.660 INFO:teuthology.orchestra.run.smithi123.stdout:osd.0 up in weight 1 up_from 8 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6802/3608618058,v1:172.21.15.123:6803/3608618058] [v2:172.21.15.123:6804/3608618058,v1:172.21.15.123:6805/3608618058] exists,up a14fde9b-bfb1-40a7-a105-305e8981a778 2024-05-08T06:09:37.660 INFO:teuthology.orchestra.run.smithi123.stdout:osd.1 up in weight 1 up_from 13 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.163:6800/1884753449,v1:172.21.15.163:6801/1884753449] [v2:172.21.15.163:6802/1884753449,v1:172.21.15.163:6803/1884753449] exists,up e023ae3d-a82e-4bb6-8fef-889dd07fe183 2024-05-08T06:09:37.660 INFO:teuthology.orchestra.run.smithi123.stdout:osd.2 up in weight 1 up_from 18 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6800/4161570404,v1:172.21.15.192:6801/4161570404] [v2:172.21.15.192:6802/4161570404,v1:172.21.15.192:6803/4161570404] exists,up 69f4cfca-63cb-4862-9480-d79891b0f338 2024-05-08T06:09:37.660 INFO:teuthology.orchestra.run.smithi123.stdout:osd.3 up in weight 1 up_from 27 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6810/1243003258,v1:172.21.15.123:6811/1243003258] [v2:172.21.15.123:6812/1243003258,v1:172.21.15.123:6813/1243003258] exists,up 71bb5ecd-c88c-45fb-8b51-0eefb53941c2 2024-05-08T06:09:37.660 INFO:teuthology.orchestra.run.smithi123.stdout:osd.4 up in weight 1 up_from 33 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.163:6808/591680273,v1:172.21.15.163:6809/591680273] [v2:172.21.15.163:6810/591680273,v1:172.21.15.163:6811/591680273] exists,up 5a7a3462-e74f-4d15-a85c-709b5d423745 2024-05-08T06:09:37.660 INFO:teuthology.orchestra.run.smithi123.stdout:osd.5 up in weight 1 up_from 38 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6808/1734821180,v1:172.21.15.192:6809/1734821180] [v2:172.21.15.192:6810/1734821180,v1:172.21.15.192:6811/1734821180] exists,up 85e3f1ab-f8c6-4025-bab9-9ddb6bd3e2dd 2024-05-08T06:09:37.660 INFO:teuthology.orchestra.run.smithi123.stdout:osd.6 up in weight 1 up_from 44 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6818/1002138878,v1:172.21.15.123:6819/1002138878] [v2:172.21.15.123:6820/1002138878,v1:172.21.15.123:6821/1002138878] exists,up f21a5624-6f8f-48be-bc6d-0a883d6b7b32 2024-05-08T06:09:37.660 INFO:teuthology.orchestra.run.smithi123.stdout:osd.7 up in weight 1 up_from 49 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.163:6816/354411978,v1:172.21.15.163:6817/354411978] [v2:172.21.15.163:6818/354411978,v1:172.21.15.163:6819/354411978] exists,up 6eb16729-d260-4041-9829-3e868795b111 2024-05-08T06:09:37.660 INFO:teuthology.orchestra.run.smithi123.stdout:osd.8 up in weight 1 up_from 55 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6816/3669753689,v1:172.21.15.192:6817/3669753689] [v2:172.21.15.192:6818/3669753689,v1:172.21.15.192:6819/3669753689] exists,up 122a281e-18b8-4728-97db-458e7432b2b2 2024-05-08T06:09:37.660 INFO:teuthology.orchestra.run.smithi123.stdout:osd.9 up in weight 1 up_from 61 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6826/3104436473,v1:172.21.15.123:6827/3104436473] [v2:172.21.15.123:6828/3104436473,v1:172.21.15.123:6829/3104436473] exists,up 9fa022a9-b873-4e07-806c-641a706a150f 2024-05-08T06:09:37.661 INFO:teuthology.orchestra.run.smithi123.stdout:osd.10 up in weight 1 up_from 66 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.163:6824/2628487883,v1:172.21.15.163:6825/2628487883] [v2:172.21.15.163:6826/2628487883,v1:172.21.15.163:6827/2628487883] exists,up fb59339b-e11b-4ffc-b202-7f67cb21da23 2024-05-08T06:09:37.661 INFO:teuthology.orchestra.run.smithi123.stdout:osd.11 up in weight 1 up_from 72 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6824/1339159347,v1:172.21.15.192:6825/1339159347] [v2:172.21.15.192:6826/1339159347,v1:172.21.15.192:6827/1339159347] exists,up be7a5f36-c1be-41d7-80ee-004d0546eb9e 2024-05-08T06:09:37.661 INFO:teuthology.orchestra.run.smithi123.stdout:pg_upmap_items 2.5 [8,11] 2024-05-08T06:09:37.661 INFO:teuthology.orchestra.run.smithi123.stdout:pg_upmap_items 2.b [2,4] 2024-05-08T06:09:37.661 INFO:teuthology.orchestra.run.smithi123.stdout:pg_upmap_items 2.13 [2,11] 2024-05-08T06:09:37.661 INFO:teuthology.orchestra.run.smithi123.stdout:pg_upmap_items 3.0 [5,9] 2024-05-08T06:09:37.661 INFO:teuthology.orchestra.run.smithi123.stdout:pg_upmap_items 3.b [3,9] 2024-05-08T06:09:37.661 INFO:teuthology.orchestra.run.smithi123.stdout:pg_upmap_items 3.e [4,6] 2024-05-08T06:09:37.661 INFO:teuthology.orchestra.run.smithi123.stdout:pg_upmap_items 3.24 [4,9] 2024-05-08T06:09:37.661 INFO:teuthology.orchestra.run.smithi123.stdout:pg_upmap_items 3.31 [4,8] 2024-05-08T06:09:37.661 INFO:teuthology.orchestra.run.smithi123.stdout:blocklist 172.21.15.123:6801/3057688649 expires 2024-05-09T05:57:30.887936+0000 2024-05-08T06:09:37.661 INFO:teuthology.orchestra.run.smithi123.stdout:blocklist 172.21.15.123:6800/3057688649 expires 2024-05-09T05:57:30.887936+0000 2024-05-08T06:09:37.661 INFO:teuthology.orchestra.run.smithi123.stdout:blocklist 172.21.15.123:6800/2769724312 expires 2024-05-09T05:56:55.849036+0000 2024-05-08T06:09:37.661 INFO:teuthology.orchestra.run.smithi123.stdout:blocklist 172.21.15.123:0/399884043 expires 2024-05-09T05:57:30.887936+0000 2024-05-08T06:09:37.662 INFO:teuthology.orchestra.run.smithi123.stdout:blocklist 172.21.15.123:0/3406061897 expires 2024-05-09T05:57:30.887936+0000 2024-05-08T06:09:37.662 INFO:teuthology.orchestra.run.smithi123.stdout:blocklist 172.21.15.123:0/644457268 expires 2024-05-09T05:56:55.849036+0000 2024-05-08T06:09:37.662 INFO:teuthology.orchestra.run.smithi123.stdout:blocklist 172.21.15.123:0/3055092197 expires 2024-05-09T05:56:55.849036+0000 2024-05-08T06:09:37.662 INFO:teuthology.orchestra.run.smithi123.stdout:blocklist 172.21.15.123:0/343390357 expires 2024-05-09T05:57:30.887936+0000 2024-05-08T06:09:37.662 INFO:teuthology.orchestra.run.smithi123.stdout:blocklist 172.21.15.123:6801/2769724312 expires 2024-05-09T05:56:55.849036+0000 2024-05-08T06:09:37.662 INFO:teuthology.orchestra.run.smithi123.stdout:blocklist 172.21.15.123:0/2263171562 expires 2024-05-09T05:56:55.849036+0000 2024-05-08T06:09:37.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.660+0000 7fbe29400700 1 -- 172.21.15.123:0/3509042766 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fbe080b2880 msgr2=0x7fbe080b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:37.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.660+0000 7fbe29400700 1 --2- 172.21.15.123:0/3509042766 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fbe080b2880 0x7fbe080b4d40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbe2410aa90 tx=0x7fbe14007400 comp rx=0 tx=0).stop 2024-05-08T06:09:37.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.660+0000 7fbe29400700 1 -- 172.21.15.123:0/3509042766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe2410ac30 msgr2=0x7fbe241a0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:37.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.660+0000 7fbe29400700 1 --2- 172.21.15.123:0/3509042766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe2410ac30 0x7fbe241a0050 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbe1000e9a0 tx=0x7fbe1000ecb0 comp rx=0 tx=0).stop 2024-05-08T06:09:37.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.660+0000 7fbe29400700 1 -- 172.21.15.123:0/3509042766 shutdown_connections 2024-05-08T06:09:37.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.660+0000 7fbe29400700 1 --2- 172.21.15.123:0/3509042766 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fbe080b2880 0x7fbe080b4d40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:37.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.660+0000 7fbe29400700 1 --2- 172.21.15.123:0/3509042766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe24109a30 0x7fbe2419fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:37.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.660+0000 7fbe29400700 1 --2- 172.21.15.123:0/3509042766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe2410b5d0 0x7fbe241a70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:37.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.660+0000 7fbe29400700 1 --2- 172.21.15.123:0/3509042766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe2410ac30 0x7fbe241a0050 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:37.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.660+0000 7fbe29400700 1 -- 172.21.15.123:0/3509042766 >> 172.21.15.123:0/3509042766 conn(0x7fbe24078a00 msgr2=0x7fbe2410df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:37.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.660+0000 7fbe29400700 1 -- 172.21.15.123:0/3509042766 shutdown_connections 2024-05-08T06:09:37.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:37.660+0000 7fbe29400700 1 -- 172.21.15.123:0/3509042766 wait complete. 2024-05-08T06:09:37.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:37 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/3509042766' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-08T06:09:38.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:37 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/3509042766' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-08T06:09:38.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:37 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/3509042766' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-08T06:09:38.730 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:38 smithi123 ceph-mon[29787]: pgmap v435: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:09:38.734 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-08T06:09:38.746 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T06:09:38.951 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:38 smithi192 ceph-mon[34664]: pgmap v435: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:09:39.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:38 smithi163 ceph-mon[34520]: pgmap v435: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:09:39.190 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:09:40.954 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:40 smithi123 ceph-mon[29787]: pgmap v436: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:09:41.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:40 smithi163 ceph-mon[34520]: pgmap v436: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:09:41.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:40 smithi192 ceph-mon[34664]: pgmap v436: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:09:41.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.870+0000 7f941fe00700 1 -- 172.21.15.123:0/3520116688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94181043b0 msgr2=0x7f9418104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:41.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.870+0000 7f941fe00700 1 --2- 172.21.15.123:0/3520116688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94181043b0 0x7f9418104790 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f940c009990 tx=0x7f940c009ca0 comp rx=0 tx=0).stop 2024-05-08T06:09:41.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.870+0000 7f941fe00700 1 -- 172.21.15.123:0/3520116688 shutdown_connections 2024-05-08T06:09:41.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.870+0000 7f941fe00700 1 --2- 172.21.15.123:0/3520116688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94181043b0 0x7f9418104790 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:41.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.870+0000 7f941fe00700 1 --2- 172.21.15.123:0/3520116688 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9418102530 0x7f9418112220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:41.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.870+0000 7f941fe00700 1 --2- 172.21.15.123:0/3520116688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9418101a90 0x7f9418101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:41.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.870+0000 7f941fe00700 1 -- 172.21.15.123:0/3520116688 >> 172.21.15.123:0/3520116688 conn(0x7f9418078a10 msgr2=0x7f94180ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:41.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.870+0000 7f941fe00700 1 -- 172.21.15.123:0/3520116688 shutdown_connections 2024-05-08T06:09:41.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.871+0000 7f941fe00700 1 -- 172.21.15.123:0/3520116688 wait complete. 2024-05-08T06:09:41.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.872+0000 7f941fe00700 1 Processor -- start 2024-05-08T06:09:41.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.872+0000 7f941fe00700 1 -- start start 2024-05-08T06:09:41.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.872+0000 7f941fe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9418101a90 0x7f941819bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:41.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.872+0000 7f941fe00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9418102530 0x7f941819c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:41.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.872+0000 7f941fe00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94181043b0 0x7f94181a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:41.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.872+0000 7f941fe00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9418115890 con 0x7f9418101a90 2024-05-08T06:09:41.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.872+0000 7f941fe00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9418115710 con 0x7f94181043b0 2024-05-08T06:09:41.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.872+0000 7f941fe00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9418115a10 con 0x7f9418102530 2024-05-08T06:09:41.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.873+0000 7f941d400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9418101a90 0x7f941819bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:41.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.873+0000 7f941d400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9418101a90 0x7f941819bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:46366/0 (socket says 172.21.15.123:46366) 2024-05-08T06:09:41.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.873+0000 7f941d400700 1 -- 172.21.15.123:0/2433290936 learned_addr learned my addr 172.21.15.123:0/2433290936 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:09:41.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.873+0000 7f941de00700 1 --2- 172.21.15.123:0/2433290936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94181043b0 0x7f94181a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:41.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.873+0000 7f941ca00700 1 --2- 172.21.15.123:0/2433290936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9418102530 0x7f941819c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:41.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.873+0000 7f941de00700 1 -- 172.21.15.123:0/2433290936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9418102530 msgr2=0x7f941819c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:41.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.873+0000 7f941de00700 1 --2- 172.21.15.123:0/2433290936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9418102530 0x7f941819c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:41.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.873+0000 7f941de00700 1 -- 172.21.15.123:0/2433290936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9418101a90 msgr2=0x7f941819bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:41.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.873+0000 7f941de00700 1 --2- 172.21.15.123:0/2433290936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9418101a90 0x7f941819bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:41.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.873+0000 7f941de00700 1 -- 172.21.15.123:0/2433290936 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94181a0d70 con 0x7f94181043b0 2024-05-08T06:09:41.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.873+0000 7f941ca00700 1 --2- 172.21.15.123:0/2433290936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9418102530 0x7f941819c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:09:41.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.873+0000 7f941d400700 1 --2- 172.21.15.123:0/2433290936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9418101a90 0x7f941819bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:09:41.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.874+0000 7f941de00700 1 --2- 172.21.15.123:0/2433290936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94181043b0 0x7f94181a0630 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9410007f10 tx=0x7f941000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:41.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.874+0000 7f9416000700 1 -- 172.21.15.123:0/2433290936 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410004db0 con 0x7f94181043b0 2024-05-08T06:09:41.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.874+0000 7f941fe00700 1 -- 172.21.15.123:0/2433290936 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94181a1060 con 0x7f94181043b0 2024-05-08T06:09:41.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.874+0000 7f9416000700 1 -- 172.21.15.123:0/2433290936 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9410009e20 con 0x7f94181043b0 2024-05-08T06:09:41.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.874+0000 7f9416000700 1 -- 172.21.15.123:0/2433290936 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410015400 con 0x7f94181043b0 2024-05-08T06:09:41.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.874+0000 7f941fe00700 1 -- 172.21.15.123:0/2433290936 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94181a88e0 con 0x7f94181043b0 2024-05-08T06:09:41.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.875+0000 7f941fe00700 1 -- 172.21.15.123:0/2433290936 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f941806a190 con 0x7f94181043b0 2024-05-08T06:09:41.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.877+0000 7f9416000700 1 -- 172.21.15.123:0/2433290936 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f9410005260 con 0x7f94181043b0 2024-05-08T06:09:41.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.879+0000 7f9416000700 1 --2- 172.21.15.123:0/2433290936 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f94000b2900 0x7f94000b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:41.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.879+0000 7f9416000700 1 -- 172.21.15.123:0/2433290936 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f94100e1ad0 con 0x7f94181043b0 2024-05-08T06:09:41.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.879+0000 7f941d400700 1 --2- 172.21.15.123:0/2433290936 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f94000b2900 0x7f94000b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:41.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.880+0000 7f941d400700 1 --2- 172.21.15.123:0/2433290936 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f94000b2900 0x7f94000b4dc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f940c006010 tx=0x7f940c00b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:41.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:41.883+0000 7f9416000700 1 -- 172.21.15.123:0/2433290936 <== mon.2 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9410010070 con 0x7f94181043b0 2024-05-08T06:09:42.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:42.162+0000 7f941fe00700 1 -- 172.21.15.123:0/2433290936 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f941802e510 con 0x7f94181043b0 2024-05-08T06:09:42.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:42.163+0000 7f9416000700 1 -- 172.21.15.123:0/2433290936 <== mon.2 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11885 (secure 0 0 0) 0x7f94100a97a0 con 0x7f94181043b0 2024-05-08T06:09:42.163 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:09:42.164 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3646476522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3646476522},{"type":"v1","addr":"172.21.15.123:6835","nonce":3646476522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2763706497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2763706497},{"type":"v1","addr":"172.21.15.123:6837","nonce":2763706497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/274635265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":274635265},{"type":"v1","addr":"172.21.15.123:6839","nonce":274635265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/175423115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":175423115},{"type":"v1","addr":"172.21.15.192:6833","nonce":175423115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/3263009472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3263009472},{"type":"v1","addr":"172.21.15.163:6833","nonce":3263009472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3308612826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3308612826},{"type":"v1","addr":"172.21.15.163:6835","nonce":3308612826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T06:05:54.755550+0000","modified":"2024-05-08T06:06:22.594178+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24512,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/202513680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":202513680},{"type":"v1","addr":"172.21.15.192:6835","nonce":202513680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/2483545969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2483545969},{"type":"v1","addr":"172.21.15.192:6837","nonce":2483545969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/660511838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":660511838},{"type":"v1","addr":"172.21.15.163:6837","nonce":660511838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.192:6839/3598682360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":3598682360},{"type":"v1","addr":"172.21.15.192:6839","nonce":3598682360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6841/1571726231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":1571726231},{"type":"v1","addr":"172.21.15.123:6841","nonce":1571726231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.163:6839/1568161515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":1568161515},{"type":"v1","addr":"172.21.15.163:6839","nonce":1568161515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24512,24524,24496]},"id":1}]} 2024-05-08T06:09:42.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:42.166+0000 7f941fe00700 1 -- 172.21.15.123:0/2433290936 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f94000b2900 msgr2=0x7f94000b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:42.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:42.166+0000 7f941fe00700 1 --2- 172.21.15.123:0/2433290936 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f94000b2900 0x7f94000b4dc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f940c006010 tx=0x7f940c00b280 comp rx=0 tx=0).stop 2024-05-08T06:09:42.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:42.166+0000 7f941fe00700 1 -- 172.21.15.123:0/2433290936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94181043b0 msgr2=0x7f94181a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:42.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:42.166+0000 7f941fe00700 1 --2- 172.21.15.123:0/2433290936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94181043b0 0x7f94181a0630 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9410007f10 tx=0x7f941000d3b0 comp rx=0 tx=0).stop 2024-05-08T06:09:42.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:42.166+0000 7f941fe00700 1 -- 172.21.15.123:0/2433290936 shutdown_connections 2024-05-08T06:09:42.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:42.167+0000 7f941fe00700 1 --2- 172.21.15.123:0/2433290936 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f94000b2900 0x7f94000b4dc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:42.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:42.167+0000 7f941fe00700 1 --2- 172.21.15.123:0/2433290936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9418101a90 0x7f941819bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:42.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:42.167+0000 7f941fe00700 1 --2- 172.21.15.123:0/2433290936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94181043b0 0x7f94181a0630 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:42.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:42.167+0000 7f941fe00700 1 --2- 172.21.15.123:0/2433290936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9418102530 0x7f941819c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:42.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:42.167+0000 7f941fe00700 1 -- 172.21.15.123:0/2433290936 >> 172.21.15.123:0/2433290936 conn(0x7f9418078a10 msgr2=0x7f94180ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:42.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:42.167+0000 7f941fe00700 1 -- 172.21.15.123:0/2433290936 shutdown_connections 2024-05-08T06:09:42.168 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:42.167+0000 7f941fe00700 1 -- 172.21.15.123:0/2433290936 wait complete. 2024-05-08T06:09:42.168 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 23 2024-05-08T06:09:42.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:42 smithi123 ceph-mon[29787]: pgmap v437: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:09:42.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:42 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/2433290936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:09:43.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:42 smithi163 ceph-mon[34520]: pgmap v437: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:09:43.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:42 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/2433290936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:09:43.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:42 smithi192 ceph-mon[34664]: pgmap v437: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:09:43.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:42 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/2433290936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:09:43.217 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T06:09:43.690 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:09:45.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:44 smithi163 ceph-mon[34520]: pgmap v438: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:09:45.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:44 smithi192 ceph-mon[34664]: pgmap v438: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:09:45.123 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:44 smithi123 ceph-mon[29787]: pgmap v438: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:09:46.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.062+0000 7ff363000700 1 -- 172.21.15.123:0/3596484099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff35c109b10 msgr2=0x7ff35c109f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:46.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.062+0000 7ff363000700 1 --2- 172.21.15.123:0/3596484099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff35c109b10 0x7ff35c109f90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff34400b0a0 tx=0x7ff34400b3b0 comp rx=0 tx=0).stop 2024-05-08T06:09:46.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.062+0000 7ff363000700 1 -- 172.21.15.123:0/3596484099 shutdown_connections 2024-05-08T06:09:46.062 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.062+0000 7ff363000700 1 --2- 172.21.15.123:0/3596484099 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff35c10a4d0 0x7ff35c116990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:46.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.062+0000 7ff363000700 1 --2- 172.21.15.123:0/3596484099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff35c10fac0 0x7ff35c10fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:46.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.062+0000 7ff363000700 1 --2- 172.21.15.123:0/3596484099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff35c109b10 0x7ff35c109f90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:46.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.062+0000 7ff363000700 1 -- 172.21.15.123:0/3596484099 >> 172.21.15.123:0/3596484099 conn(0x7ff35c076c70 msgr2=0x7ff35c079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:46.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.062+0000 7ff363000700 1 -- 172.21.15.123:0/3596484099 shutdown_connections 2024-05-08T06:09:46.063 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.062+0000 7ff363000700 1 -- 172.21.15.123:0/3596484099 wait complete. 2024-05-08T06:09:46.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.063+0000 7ff363000700 1 Processor -- start 2024-05-08T06:09:46.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.063+0000 7ff363000700 1 -- start start 2024-05-08T06:09:46.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.063+0000 7ff363000700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff35c109b10 0x7ff35c1a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:46.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.063+0000 7ff363000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff35c10a4d0 0x7ff35c1a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:46.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.064+0000 7ff363000700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff35c10fac0 0x7ff35c1a4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:46.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.064+0000 7ff363000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff35c119d00 con 0x7ff35c10a4d0 2024-05-08T06:09:46.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.064+0000 7ff363000700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff35c119b80 con 0x7ff35c109b10 2024-05-08T06:09:46.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.064+0000 7ff363000700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff35c119e80 con 0x7ff35c10fac0 2024-05-08T06:09:46.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.064+0000 7ff353400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff35c10a4d0 0x7ff35c1a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:46.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.064+0000 7ff353400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff35c10a4d0 0x7ff35c1a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39648/0 (socket says 172.21.15.123:39648) 2024-05-08T06:09:46.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.064+0000 7ff353400700 1 -- 172.21.15.123:0/177275288 learned_addr learned my addr 172.21.15.123:0/177275288 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:09:46.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.064+0000 7ff35be00700 1 --2- 172.21.15.123:0/177275288 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff35c109b10 0x7ff35c1a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:46.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.064+0000 7ff361000700 1 --2- 172.21.15.123:0/177275288 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff35c10fac0 0x7ff35c1a4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:46.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.064+0000 7ff353400700 1 -- 172.21.15.123:0/177275288 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff35c10fac0 msgr2=0x7ff35c1a4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:46.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.064+0000 7ff353400700 1 --2- 172.21.15.123:0/177275288 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff35c10fac0 0x7ff35c1a4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:46.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.064+0000 7ff353400700 1 -- 172.21.15.123:0/177275288 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff35c109b10 msgr2=0x7ff35c1a01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:46.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.064+0000 7ff353400700 1 --2- 172.21.15.123:0/177275288 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff35c109b10 0x7ff35c1a01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:46.066 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.064+0000 7ff353400700 1 -- 172.21.15.123:0/177275288 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff35c1a5170 con 0x7ff35c10a4d0 2024-05-08T06:09:46.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.065+0000 7ff353400700 1 --2- 172.21.15.123:0/177275288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff35c10a4d0 0x7ff35c1a06f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff34400b070 tx=0x7ff344004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:46.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.065+0000 7ff359600700 1 -- 172.21.15.123:0/177275288 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff344021070 con 0x7ff35c10a4d0 2024-05-08T06:09:46.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.065+0000 7ff359600700 1 -- 172.21.15.123:0/177275288 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff344007e30 con 0x7ff35c10a4d0 2024-05-08T06:09:46.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.065+0000 7ff363000700 1 -- 172.21.15.123:0/177275288 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff35c1a5400 con 0x7ff35c10a4d0 2024-05-08T06:09:46.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.065+0000 7ff359600700 1 -- 172.21.15.123:0/177275288 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff344004760 con 0x7ff35c10a4d0 2024-05-08T06:09:46.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.065+0000 7ff363000700 1 -- 172.21.15.123:0/177275288 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff35c1acce0 con 0x7ff35c10a4d0 2024-05-08T06:09:46.067 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.066+0000 7ff363000700 1 -- 172.21.15.123:0/177275288 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff35c112050 con 0x7ff35c10a4d0 2024-05-08T06:09:46.068 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.069+0000 7ff359600700 1 -- 172.21.15.123:0/177275288 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7ff344003de0 con 0x7ff35c10a4d0 2024-05-08T06:09:46.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.070+0000 7ff359600700 1 --2- 172.21.15.123:0/177275288 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7ff3400b2900 0x7ff3400b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:46.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.070+0000 7ff359600700 1 -- 172.21.15.123:0/177275288 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7ff34400f020 con 0x7ff35c10a4d0 2024-05-08T06:09:46.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.070+0000 7ff35be00700 1 --2- 172.21.15.123:0/177275288 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7ff3400b2900 0x7ff3400b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:46.071 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.071+0000 7ff35be00700 1 --2- 172.21.15.123:0/177275288 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7ff3400b2900 0x7ff3400b4dc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff34c0045b0 tx=0x7ff34c00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:46.072 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.072+0000 7ff359600700 1 -- 172.21.15.123:0/177275288 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff34402a020 con 0x7ff35c10a4d0 2024-05-08T06:09:46.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.348+0000 7ff363000700 1 -- 172.21.15.123:0/177275288 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff35c1a14c0 con 0x7ff35c10a4d0 2024-05-08T06:09:46.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.350+0000 7ff359600700 1 -- 172.21.15.123:0/177275288 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11885 (secure 0 0 0) 0x7ff3440ab7c0 con 0x7ff35c10a4d0 2024-05-08T06:09:46.350 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:09:46.351 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3646476522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3646476522},{"type":"v1","addr":"172.21.15.123:6835","nonce":3646476522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2763706497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2763706497},{"type":"v1","addr":"172.21.15.123:6837","nonce":2763706497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/274635265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":274635265},{"type":"v1","addr":"172.21.15.123:6839","nonce":274635265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/175423115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":175423115},{"type":"v1","addr":"172.21.15.192:6833","nonce":175423115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/3263009472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3263009472},{"type":"v1","addr":"172.21.15.163:6833","nonce":3263009472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3308612826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3308612826},{"type":"v1","addr":"172.21.15.163:6835","nonce":3308612826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T06:05:54.755550+0000","modified":"2024-05-08T06:06:22.594178+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24512,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/202513680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":202513680},{"type":"v1","addr":"172.21.15.192:6835","nonce":202513680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/2483545969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2483545969},{"type":"v1","addr":"172.21.15.192:6837","nonce":2483545969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/660511838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":660511838},{"type":"v1","addr":"172.21.15.163:6837","nonce":660511838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.192:6839/3598682360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":3598682360},{"type":"v1","addr":"172.21.15.192:6839","nonce":3598682360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6841/1571726231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":1571726231},{"type":"v1","addr":"172.21.15.123:6841","nonce":1571726231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.163:6839/1568161515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":1568161515},{"type":"v1","addr":"172.21.15.163:6839","nonce":1568161515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24512,24524,24496]},"id":1}]} 2024-05-08T06:09:46.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.353+0000 7ff363000700 1 -- 172.21.15.123:0/177275288 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7ff3400b2900 msgr2=0x7ff3400b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:46.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.353+0000 7ff363000700 1 --2- 172.21.15.123:0/177275288 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7ff3400b2900 0x7ff3400b4dc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff34c0045b0 tx=0x7ff34c00b3c0 comp rx=0 tx=0).stop 2024-05-08T06:09:46.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.353+0000 7ff363000700 1 -- 172.21.15.123:0/177275288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff35c10a4d0 msgr2=0x7ff35c1a06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:46.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.353+0000 7ff363000700 1 --2- 172.21.15.123:0/177275288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff35c10a4d0 0x7ff35c1a06f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff34400b070 tx=0x7ff344004d00 comp rx=0 tx=0).stop 2024-05-08T06:09:46.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.354+0000 7ff363000700 1 -- 172.21.15.123:0/177275288 shutdown_connections 2024-05-08T06:09:46.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.354+0000 7ff363000700 1 --2- 172.21.15.123:0/177275288 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7ff3400b2900 0x7ff3400b4dc0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:46.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.354+0000 7ff363000700 1 --2- 172.21.15.123:0/177275288 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff35c10fac0 0x7ff35c1a4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:46.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.354+0000 7ff363000700 1 --2- 172.21.15.123:0/177275288 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff35c109b10 0x7ff35c1a01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:46.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.354+0000 7ff363000700 1 --2- 172.21.15.123:0/177275288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff35c10a4d0 0x7ff35c1a06f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:46.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.354+0000 7ff363000700 1 -- 172.21.15.123:0/177275288 >> 172.21.15.123:0/177275288 conn(0x7ff35c076c70 msgr2=0x7ff35c0774e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:46.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.354+0000 7ff363000700 1 -- 172.21.15.123:0/177275288 shutdown_connections 2024-05-08T06:09:46.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:46.354+0000 7ff363000700 1 -- 172.21.15.123:0/177275288 wait complete. 2024-05-08T06:09:46.355 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 23 2024-05-08T06:09:47.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:46 smithi163 ceph-mon[34520]: pgmap v439: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:09:47.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:46 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/177275288' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:09:47.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:46 smithi192 ceph-mon[34664]: pgmap v439: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:09:47.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:46 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/177275288' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:09:47.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:46 smithi123 ceph-mon[29787]: pgmap v439: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:09:47.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:46 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/177275288' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:09:47.367 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T06:09:47.817 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:09:49.021 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:48 smithi123 ceph-mon[29787]: pgmap v440: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:09:49.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:48 smithi163 ceph-mon[34520]: pgmap v440: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:09:49.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:48 smithi192 ceph-mon[34664]: pgmap v440: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:09:50.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.319+0000 7f02ffe00700 1 -- 172.21.15.123:0/2379239649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0300107800 msgr2=0x7f0300107cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:50.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.319+0000 7f02ffe00700 1 --2- 172.21.15.123:0/2379239649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0300107800 0x7f0300107cb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f02f800b0a0 tx=0x7f02f800b3b0 comp rx=0 tx=0).stop 2024-05-08T06:09:50.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.319+0000 7f02ffe00700 1 -- 172.21.15.123:0/2379239649 shutdown_connections 2024-05-08T06:09:50.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.319+0000 7f02ffe00700 1 --2- 172.21.15.123:0/2379239649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0300107800 0x7f0300107cb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:50.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.319+0000 7f02ffe00700 1 --2- 172.21.15.123:0/2379239649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03001023f0 0x7f03001027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:50.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.319+0000 7f02ffe00700 1 --2- 172.21.15.123:0/2379239649 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0300102d10 0x7f03001071e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:50.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.319+0000 7f02ffe00700 1 -- 172.21.15.123:0/2379239649 >> 172.21.15.123:0/2379239649 conn(0x7f03000fde50 msgr2=0x7f0300100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:50.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.319+0000 7f02ffe00700 1 -- 172.21.15.123:0/2379239649 shutdown_connections 2024-05-08T06:09:50.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.319+0000 7f02ffe00700 1 -- 172.21.15.123:0/2379239649 wait complete. 2024-05-08T06:09:50.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.320+0000 7f02ffe00700 1 Processor -- start 2024-05-08T06:09:50.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.321+0000 7f02ffe00700 1 -- start start 2024-05-08T06:09:50.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.321+0000 7f02ffe00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03001023f0 0x7f03001a00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:50.321 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.321+0000 7f02ffe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0300102d10 0x7f03001a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:50.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.321+0000 7f02ffe00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0300107800 0x7f03001a49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:50.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.321+0000 7f02ffe00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0300119c40 con 0x7f0300102d10 2024-05-08T06:09:50.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.321+0000 7f02ffe00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0300119ac0 con 0x7f03001023f0 2024-05-08T06:09:50.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.321+0000 7f02ffe00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0300119dc0 con 0x7f0300107800 2024-05-08T06:09:50.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.321+0000 7f02ff400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0300107800 0x7f03001a49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:50.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.321+0000 7f02ff400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0300107800 0x7f03001a49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:47618/0 (socket says 172.21.15.123:47618) 2024-05-08T06:09:50.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.322+0000 7f02ff400700 1 -- 172.21.15.123:0/450667713 learned_addr learned my addr 172.21.15.123:0/450667713 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:09:50.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.321+0000 7f02fe000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0300102d10 0x7f03001a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:50.322 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.322+0000 7f02fea00700 1 --2- 172.21.15.123:0/450667713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03001023f0 0x7f03001a00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:50.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.322+0000 7f02ff400700 1 -- 172.21.15.123:0/450667713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03001023f0 msgr2=0x7f03001a00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:50.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.322+0000 7f02ff400700 1 --2- 172.21.15.123:0/450667713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03001023f0 0x7f03001a00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:50.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.322+0000 7f02ff400700 1 -- 172.21.15.123:0/450667713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0300102d10 msgr2=0x7f03001a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:50.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.322+0000 7f02ff400700 1 --2- 172.21.15.123:0/450667713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0300102d10 0x7f03001a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:50.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.322+0000 7f02ff400700 1 -- 172.21.15.123:0/450667713 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03001a50e0 con 0x7f0300107800 2024-05-08T06:09:50.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.322+0000 7f02ff400700 1 --2- 172.21.15.123:0/450667713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0300107800 0x7f03001a49a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f02f800b780 tx=0x7f02f8004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:50.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.323+0000 7f02f7400700 1 -- 172.21.15.123:0/450667713 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f8021070 con 0x7f0300107800 2024-05-08T06:09:50.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.323+0000 7f02ffe00700 1 -- 172.21.15.123:0/450667713 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03001a5370 con 0x7f0300107800 2024-05-08T06:09:50.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.323+0000 7f02f7400700 1 -- 172.21.15.123:0/450667713 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02f8007e30 con 0x7f0300107800 2024-05-08T06:09:50.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.323+0000 7f02f7400700 1 -- 172.21.15.123:0/450667713 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f8004700 con 0x7f0300107800 2024-05-08T06:09:50.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.323+0000 7f02ffe00700 1 -- 172.21.15.123:0/450667713 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03001acc50 con 0x7f0300107800 2024-05-08T06:09:50.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.324+0000 7f02ffe00700 1 -- 172.21.15.123:0/450667713 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f030006a190 con 0x7f0300107800 2024-05-08T06:09:50.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.326+0000 7f02f7400700 1 -- 172.21.15.123:0/450667713 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f02f8007980 con 0x7f0300107800 2024-05-08T06:09:50.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.327+0000 7f02f7400700 1 --2- 172.21.15.123:0/450667713 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f02e40b2880 0x7f02e40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:50.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.328+0000 7f02f7400700 1 -- 172.21.15.123:0/450667713 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f02f800f020 con 0x7f0300107800 2024-05-08T06:09:50.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.328+0000 7f02fea00700 1 --2- 172.21.15.123:0/450667713 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f02e40b2880 0x7f02e40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:50.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.328+0000 7f02fea00700 1 --2- 172.21.15.123:0/450667713 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f02e40b2880 0x7f02e40b4d40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f02f0009c60 tx=0x7f02f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:50.330 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.330+0000 7f02f7400700 1 -- 172.21.15.123:0/450667713 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f02f802a020 con 0x7f0300107800 2024-05-08T06:09:50.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.566+0000 7f02ffe00700 1 -- 172.21.15.123:0/450667713 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f03001acee0 con 0x7f0300107800 2024-05-08T06:09:50.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.568+0000 7f02f7400700 1 -- 172.21.15.123:0/450667713 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22719 (secure 0 0 0) 0x7f02f80cd090 con 0x7f0300107800 2024-05-08T06:09:50.569 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:09:50.569 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":82,"fsid":"85cfb2e6-0cff-11ef-bc97-c7b262605968","created":"2024-05-08T05:56:12.544750+0000","modified":"2024-05-08T06:06:32.471993+0000","last_up_change":"2024-05-08T06:04:33.654727+0000","last_in_change":"2024-05-08T06:04:13.809050+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T06:00:17.050518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T06:05:44.815334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T06:05:49.604457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a14fde9b-bfb1-40a7-a105-305e8981a778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6803","nonce":3608618058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6805","nonce":3608618058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6809","nonce":3608618058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6807","nonce":3608618058}]},"public_addr":"172.21.15.123:6803/3608618058","cluster_addr":"172.21.15.123:6805/3608618058","heartbeat_back_addr":"172.21.15.123:6809/3608618058","heartbeat_front_addr":"172.21.15.123:6807/3608618058","state":["exists","up"]},{"osd":1,"uuid":"e023ae3d-a82e-4bb6-8fef-889dd07fe183","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6801","nonce":1884753449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6803","nonce":1884753449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6807","nonce":1884753449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6805","nonce":1884753449}]},"public_addr":"172.21.15.163:6801/1884753449","cluster_addr":"172.21.15.163:6803/1884753449","heartbeat_back_addr":"172.21.15.163:6807/1884753449","heartbeat_front_addr":"172.21.15.163:6805/1884753449","state":["exists","up"]},{"osd":2,"uuid":"69f4cfca-63cb-4862-9480-d79891b0f338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6801","nonce":4161570404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6803","nonce":4161570404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6807","nonce":4161570404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6805","nonce":4161570404}]},"public_addr":"172.21.15.192:6801/4161570404","cluster_addr":"172.21.15.192:6803/4161570404","heartbeat_back_addr":"172.21.15.192:6807/4161570404","heartbeat_front_addr":"172.21.15.192:6805/4161570404","state":["exists","up"]},{"osd":3,"uuid":"71bb5ecd-c88c-45fb-8b51-0eefb53941c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6811","nonce":1243003258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6813","nonce":1243003258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6817","nonce":1243003258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6815","nonce":1243003258}]},"public_addr":"172.21.15.123:6811/1243003258","cluster_addr":"172.21.15.123:6813/1243003258","heartbeat_back_addr":"172.21.15.123:6817/1243003258","heartbeat_front_addr":"172.21.15.123:6815/1243003258","state":["exists","up"]},{"osd":4,"uuid":"5a7a3462-e74f-4d15-a85c-709b5d423745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6809","nonce":591680273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6811","nonce":591680273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6815","nonce":591680273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6813","nonce":591680273}]},"public_addr":"172.21.15.163:6809/591680273","cluster_addr":"172.21.15.163:6811/591680273","heartbeat_back_addr":"172.21.15.163:6815/591680273","heartbeat_front_addr":"172.21.15.163:6813/591680273","state":["exists","up"]},{"osd":5,"uuid":"85e3f1ab-f8c6-4025-bab9-9ddb6bd3e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6809","nonce":1734821180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6811","nonce":1734821180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6815","nonce":1734821180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6813","nonce":1734821180}]},"public_addr":"172.21.15.192:6809/1734821180","cluster_addr":"172.21.15.192:6811/1734821180","heartbeat_back_addr":"172.21.15.192:6815/1734821180","heartbeat_front_addr":"172.21.15.192:6813/1734821180","state":["exists","up"]},{"osd":6,"uuid":"f21a5624-6f8f-48be-bc6d-0a883d6b7b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6819","nonce":1002138878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6821","nonce":1002138878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6825","nonce":1002138878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6823","nonce":1002138878}]},"public_addr":"172.21.15.123:6819/1002138878","cluster_addr":"172.21.15.123:6821/1002138878","heartbeat_back_addr":"172.21.15.123:6825/1002138878","heartbeat_front_addr":"172.21.15.123:6823/1002138878","state":["exists","up"]},{"osd":7,"uuid":"6eb16729-d260-4041-9829-3e868795b111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6817","nonce":354411978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6819","nonce":354411978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6823","nonce":354411978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6821","nonce":354411978}]},"public_addr":"172.21.15.163:6817/354411978","cluster_addr":"172.21.15.163:6819/354411978","heartbeat_back_addr":"172.21.15.163:6823/354411978","heartbeat_front_addr":"172.21.15.163:6821/354411978","state":["exists","up"]},{"osd":8,"uuid":"122a281e-18b8-4728-97db-458e7432b2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6817","nonce":3669753689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6819","nonce":3669753689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6823","nonce":3669753689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6821","nonce":3669753689}]},"public_addr":"172.21.15.192:6817/3669753689","cluster_addr":"172.21.15.192:6819/3669753689","heartbeat_back_addr":"172.21.15.192:6823/3669753689","heartbeat_front_addr":"172.21.15.192:6821/3669753689","state":["exists","up"]},{"osd":9,"uuid":"9fa022a9-b873-4e07-806c-641a706a150f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6827","nonce":3104436473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6829","nonce":3104436473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6833","nonce":3104436473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6831","nonce":3104436473}]},"public_addr":"172.21.15.123:6827/3104436473","cluster_addr":"172.21.15.123:6829/3104436473","heartbeat_back_addr":"172.21.15.123:6833/3104436473","heartbeat_front_addr":"172.21.15.123:6831/3104436473","state":["exists","up"]},{"osd":10,"uuid":"fb59339b-e11b-4ffc-b202-7f67cb21da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6825","nonce":2628487883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6827","nonce":2628487883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6831","nonce":2628487883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6829","nonce":2628487883}]},"public_addr":"172.21.15.163:6825/2628487883","cluster_addr":"172.21.15.163:6827/2628487883","heartbeat_back_addr":"172.21.15.163:6831/2628487883","heartbeat_front_addr":"172.21.15.163:6829/2628487883","state":["exists","up"]},{"osd":11,"uuid":"be7a5f36-c1be-41d7-80ee-004d0546eb9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6825","nonce":1339159347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6827","nonce":1339159347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6831","nonce":1339159347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6829","nonce":1339159347}]},"public_addr":"172.21.15.192:6825/1339159347","cluster_addr":"172.21.15.192:6827/1339159347","heartbeat_back_addr":"172.21.15.192:6831/1339159347","heartbeat_front_addr":"172.21.15.192:6829/1339159347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:59:24.923234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:59:49.867071+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:00:13.454062+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:00:46.983594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:01:12.820452+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:01:37.451382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:02:13.325181+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:02:39.453436+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:03:03.837509+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:03:39.460579+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:04:06.314082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:04:31.723154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6801/3057688649":"2024-05-09T05:57:30.887936+0000","172.21.15.123:6800/3057688649":"2024-05-09T05:57:30.887936+0000","172.21.15.123:6800/2769724312":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/399884043":"2024-05-09T05:57:30.887936+0000","172.21.15.123:0/3406061897":"2024-05-09T05:57:30.887936+0000","172.21.15.123:0/644457268":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/3055092197":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/343390357":"2024-05-09T05:57:30.887936+0000","172.21.15.123:6801/2769724312":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/2263171562":"2024-05-09T05:56:55.849036+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T06:09:50.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.572+0000 7f02ffe00700 1 -- 172.21.15.123:0/450667713 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f02e40b2880 msgr2=0x7f02e40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:50.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.572+0000 7f02ffe00700 1 --2- 172.21.15.123:0/450667713 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f02e40b2880 0x7f02e40b4d40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f02f0009c60 tx=0x7f02f0009260 comp rx=0 tx=0).stop 2024-05-08T06:09:50.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.572+0000 7f02ffe00700 1 -- 172.21.15.123:0/450667713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0300107800 msgr2=0x7f03001a49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:50.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.572+0000 7f02ffe00700 1 --2- 172.21.15.123:0/450667713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0300107800 0x7f03001a49a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f02f800b780 tx=0x7f02f8004d00 comp rx=0 tx=0).stop 2024-05-08T06:09:50.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.573+0000 7f02ffe00700 1 -- 172.21.15.123:0/450667713 shutdown_connections 2024-05-08T06:09:50.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.573+0000 7f02ffe00700 1 --2- 172.21.15.123:0/450667713 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f02e40b2880 0x7f02e40b4d40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:50.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.573+0000 7f02ffe00700 1 --2- 172.21.15.123:0/450667713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0300107800 0x7f03001a49a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:50.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.573+0000 7f02ffe00700 1 --2- 172.21.15.123:0/450667713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03001023f0 0x7f03001a00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:50.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.573+0000 7f02ffe00700 1 --2- 172.21.15.123:0/450667713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0300102d10 0x7f03001a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:50.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.573+0000 7f02ffe00700 1 -- 172.21.15.123:0/450667713 >> 172.21.15.123:0/450667713 conn(0x7f03000fde50 msgr2=0x7f03001051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:50.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.573+0000 7f02ffe00700 1 -- 172.21.15.123:0/450667713 shutdown_connections 2024-05-08T06:09:50.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:50.573+0000 7f02ffe00700 1 -- 172.21.15.123:0/450667713 wait complete. 2024-05-08T06:09:50.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:50 smithi123 ceph-mon[29787]: pgmap v441: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:09:50.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:50 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/450667713' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:09:51.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:50 smithi163 ceph-mon[34520]: pgmap v441: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:09:51.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:50 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/450667713' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:09:51.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:50 smithi192 ceph-mon[34664]: pgmap v441: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:09:51.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:50 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/450667713' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:09:51.467 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-08T06:05:54.755550+0000', 'modified': '2024-05-08T06:06:22.594178+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24524, 'mds_1': 24512, 'mds_2': 24496}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24469': {'gid': 24469, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.192:6835/202513680', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6834', 'nonce': 202513680}, {'type': 'v1', 'addr': '172.21.15.192:6835', 'nonce': 202513680}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.192:6837/2483545969', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6836', 'nonce': 2483545969}, {'type': 'v1', 'addr': '172.21.15.192:6837', 'nonce': 2483545969}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.163:6837/660511838', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6836', 'nonce': 660511838}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 660511838}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'i', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.192:6839/3598682360', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6838', 'nonce': 3598682360}, {'type': 'v1', 'addr': '172.21.15.192:6839', 'nonce': 3598682360}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24512': {'gid': 24512, 'name': 'a', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.123:6841/1571726231', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6840', 'nonce': 1571726231}, {'type': 'v1', 'addr': '172.21.15.123:6841', 'nonce': 1571726231}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24524': {'gid': 24524, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.163:6839/1568161515', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6838', 'nonce': 1568161515}, {'type': 'v1', 'addr': '172.21.15.163:6839', 'nonce': 1568161515}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24524, 'qdb_cluster': [24512, 24524, 24496]} 2024-05-08T06:09:51.467 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-08T06:09:51.467 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-08T06:09:51.913 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:09:52.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:52 smithi123 ceph-mon[29787]: pgmap v442: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:09:53.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:52 smithi163 ceph-mon[34520]: pgmap v442: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:09:53.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:52 smithi192 ceph-mon[34664]: pgmap v442: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:09:54.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.461+0000 7f67eaa00700 1 -- 172.21.15.123:0/3899399205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67e4104100 msgr2=0x7f67e4108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:54.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.461+0000 7f67eaa00700 1 --2- 172.21.15.123:0/3899399205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67e4104100 0x7f67e4108120 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f67cc0099c0 tx=0x7f67cc009cd0 comp rx=0 tx=0).stop 2024-05-08T06:09:54.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.461+0000 7f67eaa00700 1 -- 172.21.15.123:0/3899399205 shutdown_connections 2024-05-08T06:09:54.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.461+0000 7f67eaa00700 1 --2- 172.21.15.123:0/3899399205 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67e4108850 0x7f67e41103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:54.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.461+0000 7f67eaa00700 1 --2- 172.21.15.123:0/3899399205 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67e4103750 0x7f67e4103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:54.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.461+0000 7f67eaa00700 1 --2- 172.21.15.123:0/3899399205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67e4104100 0x7f67e4108120 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:54.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.461+0000 7f67eaa00700 1 -- 172.21.15.123:0/3899399205 >> 172.21.15.123:0/3899399205 conn(0x7f67e407a600 msgr2=0x7f67e407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:54.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.462+0000 7f67eaa00700 1 -- 172.21.15.123:0/3899399205 shutdown_connections 2024-05-08T06:09:54.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.462+0000 7f67eaa00700 1 -- 172.21.15.123:0/3899399205 wait complete. 2024-05-08T06:09:54.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.463+0000 7f67eaa00700 1 Processor -- start 2024-05-08T06:09:54.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.463+0000 7f67eaa00700 1 -- start start 2024-05-08T06:09:54.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.464+0000 7f67eaa00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67e4103750 0x7f67e419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:54.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.464+0000 7f67eaa00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67e4104100 0x7f67e419fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:54.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.464+0000 7f67eaa00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67e4108850 0x7f67e4199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:54.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.464+0000 7f67eaa00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f67e4113610 con 0x7f67e4108850 2024-05-08T06:09:54.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.464+0000 7f67eaa00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f67e4113490 con 0x7f67e4104100 2024-05-08T06:09:54.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.464+0000 7f67eaa00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f67e4113790 con 0x7f67e4103750 2024-05-08T06:09:54.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.464+0000 7f67e3e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67e4103750 0x7f67e419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:54.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.464+0000 7f67e3e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67e4103750 0x7f67e419f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:47632/0 (socket says 172.21.15.123:47632) 2024-05-08T06:09:54.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.464+0000 7f67e3e00700 1 -- 172.21.15.123:0/2994162265 learned_addr learned my addr 172.21.15.123:0/2994162265 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:09:54.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.464+0000 7f67e8a00700 1 --2- 172.21.15.123:0/2994162265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67e4108850 0x7f67e4199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:54.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.464+0000 7f67db400700 1 --2- 172.21.15.123:0/2994162265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67e4104100 0x7f67e419fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:54.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.464+0000 7f67e3e00700 1 -- 172.21.15.123:0/2994162265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67e4104100 msgr2=0x7f67e419fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:54.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.464+0000 7f67e3e00700 1 --2- 172.21.15.123:0/2994162265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67e4104100 0x7f67e419fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:54.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.464+0000 7f67e3e00700 1 -- 172.21.15.123:0/2994162265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67e4108850 msgr2=0x7f67e4199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:54.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.464+0000 7f67e3e00700 1 --2- 172.21.15.123:0/2994162265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67e4108850 0x7f67e4199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:54.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.464+0000 7f67e3e00700 1 -- 172.21.15.123:0/2994162265 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67e419a070 con 0x7f67e4103750 2024-05-08T06:09:54.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.465+0000 7f67e3e00700 1 --2- 172.21.15.123:0/2994162265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67e4103750 0x7f67e419f700 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f67d400ef10 tx=0x7f67d400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:54.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.465+0000 7f67e1600700 1 -- 172.21.15.123:0/2994162265 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d400c9c0 con 0x7f67e4103750 2024-05-08T06:09:54.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.465+0000 7f67eaa00700 1 -- 172.21.15.123:0/2994162265 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67e419a300 con 0x7f67e4103750 2024-05-08T06:09:54.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.465+0000 7f67e1600700 1 -- 172.21.15.123:0/2994162265 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67d4019070 con 0x7f67e4103750 2024-05-08T06:09:54.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.465+0000 7f67e1600700 1 -- 172.21.15.123:0/2994162265 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d4014480 con 0x7f67e4103750 2024-05-08T06:09:54.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.465+0000 7f67eaa00700 1 -- 172.21.15.123:0/2994162265 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67e41a65d0 con 0x7f67e4103750 2024-05-08T06:09:54.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.469+0000 7f67e1600700 1 -- 172.21.15.123:0/2994162265 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f67d4012050 con 0x7f67e4103750 2024-05-08T06:09:54.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.470+0000 7f67f53ccb80 1 Processor -- start 2024-05-08T06:09:54.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.470+0000 7f67f53ccb80 1 -- start start 2024-05-08T06:09:54.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.470+0000 7f67f53ccb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f2379420 0x5583f2379800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:54.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.470+0000 7f67f53ccb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583f2379d40 0x5583f237d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:54.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.470+0000 7f67f53ccb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5583f237ddd0 0x5583f237e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:54.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.470+0000 7f67f53ccb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5583f22c79d0 con 0x5583f2379420 2024-05-08T06:09:54.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.470+0000 7f67f53ccb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5583f2265710 con 0x5583f237ddd0 2024-05-08T06:09:54.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.470+0000 7f67f53ccb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5583f22c3990 con 0x5583f2379d40 2024-05-08T06:09:54.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.471+0000 7f67e1600700 1 --2- 172.21.15.123:0/2994162265 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f67c80b2900 0x7f67c80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:54.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.471+0000 7f67e8a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f2379420 0x5583f2379800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:54.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.471+0000 7f67e8a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f2379420 0x5583f2379800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39688/0 (socket says 172.21.15.123:39688) 2024-05-08T06:09:54.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.471+0000 7f67e8a00700 1 -- 172.21.15.123:0/3796408879 learned_addr learned my addr 172.21.15.123:0/3796408879 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:09:54.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.471+0000 7f67e3e00700 1 --2- 172.21.15.123:0/3796408879 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583f2379d40 0x5583f237d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:54.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.471+0000 7f67db400700 1 --2- 172.21.15.123:0/3796408879 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5583f237ddd0 0x5583f237e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:54.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.471+0000 7f67e1600700 1 -- 172.21.15.123:0/2994162265 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f67d40e2110 con 0x7f67e4103750 2024-05-08T06:09:54.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.471+0000 7f67e8a00700 1 --2- 172.21.15.123:0/2994162265 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f67c80b2900 0x7f67c80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:54.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.471+0000 7f67e3e00700 1 -- 172.21.15.123:0/3796408879 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5583f237ddd0 msgr2=0x5583f237e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:54.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.471+0000 7f67e3e00700 1 --2- 172.21.15.123:0/3796408879 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5583f237ddd0 0x5583f237e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:54.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.471+0000 7f67e3e00700 1 -- 172.21.15.123:0/3796408879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f2379420 msgr2=0x5583f2379800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:54.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.471+0000 7f67e3e00700 1 --2- 172.21.15.123:0/3796408879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f2379420 0x5583f2379800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:54.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.471+0000 7f67e3e00700 1 -- 172.21.15.123:0/3796408879 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583f2214880 con 0x5583f2379d40 2024-05-08T06:09:54.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.472+0000 7f67e3e00700 1 --2- 172.21.15.123:0/3796408879 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583f2379d40 0x5583f237d890 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f67d400eee0 tx=0x7f67d4077300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e1b58a39245a712 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:54.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.472+0000 7f67d9600700 1 -- 172.21.15.123:0/3796408879 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d4077140 con 0x5583f2379d40 2024-05-08T06:09:54.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.472+0000 7f67d9600700 1 -- 172.21.15.123:0/3796408879 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67d40a91f0 con 0x5583f2379d40 2024-05-08T06:09:54.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.472+0000 7f67d9600700 1 -- 172.21.15.123:0/3796408879 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d40b2540 con 0x5583f2379d40 2024-05-08T06:09:54.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.472+0000 7f67f53ccb80 1 -- 172.21.15.123:0/3796408879 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583f2379d40 msgr2=0x5583f237d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:54.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.473+0000 7f67f53ccb80 1 --2- 172.21.15.123:0/3796408879 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583f2379d40 0x5583f237d890 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f67d400eee0 tx=0x7f67d4077300 comp rx=0 tx=0).stop 2024-05-08T06:09:54.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.473+0000 7f67e8a00700 1 --2- 172.21.15.123:0/2994162265 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f67c80b2900 0x7f67c80b4dc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x5583f23704d0 tx=0x7f67dc0047c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:54.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.473+0000 7f67f53ccb80 1 -- 172.21.15.123:0/3796408879 shutdown_connections 2024-05-08T06:09:54.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.473+0000 7f67f53ccb80 1 --2- 172.21.15.123:0/3796408879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f2379420 0x5583f2379800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:54.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.473+0000 7f67f53ccb80 1 --2- 172.21.15.123:0/3796408879 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5583f237ddd0 0x5583f237e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:54.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.473+0000 7f67f53ccb80 1 --2- 172.21.15.123:0/3796408879 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583f2379d40 0x5583f237d890 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:54.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.473+0000 7f67f53ccb80 1 -- 172.21.15.123:0/3796408879 >> 172.21.15.123:0/3796408879 conn(0x5583f22a3b10 msgr2=0x5583f236f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:54.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.473+0000 7f67f53ccb80 1 -- 172.21.15.123:0/3796408879 shutdown_connections 2024-05-08T06:09:54.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.473+0000 7f67f53ccb80 1 -- 172.21.15.123:0/3796408879 wait complete. 2024-05-08T06:09:54.476 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.473+0000 7f67f53ccb80 1 -- start start 2024-05-08T06:09:54.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.473+0000 7f67f53ccb80 1 Processor -- start 2024-05-08T06:09:54.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.474+0000 7f67f53ccb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583f2379420 0x5583f23852f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:54.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.474+0000 7f67f53ccb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f2379d40 0x5583f23826e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:54.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.474+0000 7f67f53ccb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5583f237ddd0 0x5583f2382c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:54.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.474+0000 7f67f53ccb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5583f2385f60 con 0x5583f2379d40 2024-05-08T06:09:54.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.474+0000 7f67f53ccb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5583f222dad0 con 0x5583f237ddd0 2024-05-08T06:09:54.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.474+0000 7f67f53ccb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5583f23860e0 con 0x5583f2379420 2024-05-08T06:09:54.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.474+0000 7f67f53ccb80 10 client.0 _ll_register_callbacks cb 0x5583f226ec20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:09:54.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.474+0000 7f67e3e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583f2379420 0x5583f23852f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:54.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.474+0000 7f67db400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f2379d40 0x5583f23826e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:54.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.474+0000 7f67e3e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583f2379420 0x5583f23852f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:47652/0 (socket says 172.21.15.123:47652) 2024-05-08T06:09:54.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.474+0000 7f67e3e00700 1 -- 172.21.15.123:0/2386272578 learned_addr learned my addr 172.21.15.123:0/2386272578 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:09:54.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.474+0000 7f67e8a00700 1 --2- 172.21.15.123:0/2386272578 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5583f237ddd0 0x5583f2382c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:54.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.475+0000 7f67e3e00700 1 -- 172.21.15.123:0/2386272578 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5583f237ddd0 msgr2=0x5583f2382c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:54.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.475+0000 7f67e3e00700 1 --2- 172.21.15.123:0/2386272578 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5583f237ddd0 0x5583f2382c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:54.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.475+0000 7f67e3e00700 1 -- 172.21.15.123:0/2386272578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f2379d40 msgr2=0x5583f23826e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:54.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.475+0000 7f67e3e00700 1 --2- 172.21.15.123:0/2386272578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f2379d40 0x5583f23826e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:54.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.475+0000 7f67e3e00700 1 -- 172.21.15.123:0/2386272578 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583f228d3d0 con 0x5583f2379420 2024-05-08T06:09:54.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.475+0000 7f67db400700 1 --2- 172.21.15.123:0/2386272578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f2379d40 0x5583f23826e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:09:54.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.475+0000 7f67e3e00700 1 --2- 172.21.15.123:0/2386272578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583f2379420 0x5583f23852f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f67d40b2010 tx=0x7f67d400c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45e3100764d7854e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:54.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.475+0000 7f67d2000700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-05-08T06:09:54.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.475+0000 7f67d2000700 1 -- 172.21.15.123:0/2386272578 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d40a9de0 con 0x5583f2379420 2024-05-08T06:09:54.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.475+0000 7f67d2000700 1 -- 172.21.15.123:0/2386272578 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f67d40a5cd0 con 0x5583f2379420 2024-05-08T06:09:54.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.475+0000 7f67f53ccb80 1 -- 172.21.15.123:0/2386272578 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67d40a91f0 con 0x5583f2379420 2024-05-08T06:09:54.478 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.475+0000 7f67d2000700 1 -- 172.21.15.123:0/2386272578 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f67d40b2540 con 0x5583f2379420 2024-05-08T06:09:54.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.476+0000 7f67f53ccb80 10 client.24976 fetch_fsmap learned FSMap version 23 2024-05-08T06:09:54.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.476+0000 7f67f53ccb80 1 -- 172.21.15.123:0/2386272578 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5583f2385f60 con 0x5583f2379420 2024-05-08T06:09:54.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.476+0000 7f67d2000700 1 -- 172.21.15.123:0/2386272578 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67d40a91f0 con 0x5583f2379420 2024-05-08T06:09:54.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.476+0000 7f67d2000700 1 -- 172.21.15.123:0/2386272578 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f67d40a9a40 con 0x5583f2379420 2024-05-08T06:09:54.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.476+0000 7f67f53ccb80 10 client.24976 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T06:09:54.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.476+0000 7f67f53ccb80 10 client.24976 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T06:09:54.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.476+0000 7f67f53ccb80 20 client.24976 populate_metadata read hostname 'smithi123' 2024-05-08T06:09:54.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.476+0000 7f67f53ccb80 1 --2- 172.21.15.123:0/2386272578 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x5583f23928b0 0x5583f2393d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:54.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.476+0000 7f67f53ccb80 4 client.24976 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:09:54.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.476+0000 7f67f53ccb80 1 -- 172.21.15.123:0/2386272578 --> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583f2342050 con 0x5583f23928b0 2024-05-08T06:09:54.479 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.477+0000 7f67db400700 1 --2- 172.21.15.123:0/2386272578 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x5583f23928b0 0x5583f2393d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:54.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.477+0000 7f67db400700 1 --2- 172.21.15.123:0/2386272578 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x5583f23928b0 0x5583f2393d20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=346ad181d0195f3 server_cookie=8f37e8b2023eb7e7 in_seq=0 out_seq=0 2024-05-08T06:09:54.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.477+0000 7f67d2000700 10 client.24976 ms_handle_connect on v2:172.21.15.192:6838/3598682360 2024-05-08T06:09:54.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.478+0000 7f67d2000700 1 -- 172.21.15.123:0/2386272578 <== mds.2 v2:172.21.15.192:6838/3598682360 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5583f2342050 con 0x5583f23928b0 2024-05-08T06:09:54.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.478+0000 7f67d2000700 10 client.24976 handle_command_reply: tid=0 2024-05-08T06:09:54.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.478+0000 7f67f53ccb80 1 client.24976 shutdown 2024-05-08T06:09:54.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.478+0000 7f67f53ccb80 1 -- 172.21.15.123:0/2386272578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583f2379420 msgr2=0x5583f23852f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:54.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.478+0000 7f67f53ccb80 1 --2- 172.21.15.123:0/2386272578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583f2379420 0x5583f23852f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f67d40b2010 tx=0x7f67d400c820 comp rx=0 tx=0).stop 2024-05-08T06:09:54.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.478+0000 7f67f53ccb80 1 -- 172.21.15.123:0/2386272578 shutdown_connections 2024-05-08T06:09:54.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.478+0000 7f67f53ccb80 1 --2- 172.21.15.123:0/2386272578 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x5583f23928b0 0x5583f2393d20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:54.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.478+0000 7f67f53ccb80 1 --2- 172.21.15.123:0/2386272578 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5583f237ddd0 0x5583f2382c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:54.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.478+0000 7f67f53ccb80 1 --2- 172.21.15.123:0/2386272578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583f2379420 0x5583f23852f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:54.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.478+0000 7f67d2000700 0 client.24976 ms_handle_reset on v2:172.21.15.192:6838/3598682360 2024-05-08T06:09:54.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.478+0000 7f67f53ccb80 1 --2- 172.21.15.123:0/2386272578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f2379d40 0x5583f23826e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:54.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.479+0000 7f67f53ccb80 1 -- 172.21.15.123:0/2386272578 >> 172.21.15.123:0/2386272578 conn(0x5583f22a3b10 msgr2=0x5583f237e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:54.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.479+0000 7f67f53ccb80 1 -- 172.21.15.123:0/2386272578 shutdown_connections 2024-05-08T06:09:54.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.479+0000 7f67f53ccb80 1 -- 172.21.15.123:0/2386272578 wait complete. 2024-05-08T06:09:54.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.479+0000 7f67f53ccb80 20 client.24976 trim_cache size 0 max 16384 2024-05-08T06:09:54.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.501+0000 7f67f53ccb80 1 Processor -- start 2024-05-08T06:09:54.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.501+0000 7f67f53ccb80 1 -- start start 2024-05-08T06:09:54.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.501+0000 7f67f53ccb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f2379420 0x5583f2380310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:54.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.501+0000 7f67f53ccb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583f2379d40 0x5583f2382db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:54.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.502+0000 7f67f53ccb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5583f237ddd0 0x5583f23699d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:54.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.502+0000 7f67f53ccb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5583f2392060 con 0x5583f2379420 2024-05-08T06:09:54.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.502+0000 7f67f53ccb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5583f2392730 con 0x5583f237ddd0 2024-05-08T06:09:54.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.502+0000 7f67f53ccb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5583f2392e00 con 0x5583f2379d40 2024-05-08T06:09:54.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.502+0000 7f67e3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f2379420 0x5583f2380310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:54.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.502+0000 7f67e3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f2379420 0x5583f2380310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39702/0 (socket says 172.21.15.123:39702) 2024-05-08T06:09:54.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.502+0000 7f67e3e00700 1 -- 172.21.15.123:0/3460091420 learned_addr learned my addr 172.21.15.123:0/3460091420 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:09:54.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.502+0000 7f67db400700 1 --2- 172.21.15.123:0/3460091420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583f2379d40 0x5583f2382db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:54.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.502+0000 7f67e8a00700 1 --2- 172.21.15.123:0/3460091420 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5583f237ddd0 0x5583f23699d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:54.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.502+0000 7f67e3e00700 1 -- 172.21.15.123:0/3460091420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583f2379d40 msgr2=0x5583f2382db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:54.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.502+0000 7f67e3e00700 1 --2- 172.21.15.123:0/3460091420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583f2379d40 0x5583f2382db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:54.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.502+0000 7f67e3e00700 1 -- 172.21.15.123:0/3460091420 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5583f237ddd0 msgr2=0x5583f23699d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:54.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.502+0000 7f67e3e00700 1 --2- 172.21.15.123:0/3460091420 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5583f237ddd0 0x5583f23699d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:54.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.502+0000 7f67e3e00700 1 -- 172.21.15.123:0/3460091420 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583f236c3a0 con 0x5583f2379420 2024-05-08T06:09:54.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.502+0000 7f67e8a00700 1 --2- 172.21.15.123:0/3460091420 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5583f237ddd0 0x5583f23699d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:09:54.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.502+0000 7f67db400700 1 --2- 172.21.15.123:0/3460091420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583f2379d40 0x5583f2382db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:09:54.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.503+0000 7f67e3e00700 1 --2- 172.21.15.123:0/3460091420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f2379420 0x5583f2380310 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f67d4000f80 tx=0x7f67d40b2430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a29cdceb49cc26 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:54.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.503+0000 7f67d8c00700 1 -- 172.21.15.123:0/3460091420 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d40a91f0 con 0x5583f2379420 2024-05-08T06:09:54.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.503+0000 7f67d8c00700 1 -- 172.21.15.123:0/3460091420 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67d40a9de0 con 0x5583f2379420 2024-05-08T06:09:54.504 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.503+0000 7f67d8c00700 1 -- 172.21.15.123:0/3460091420 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d406f170 con 0x5583f2379420 2024-05-08T06:09:54.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.504+0000 7f67f53ccb80 1 -- 172.21.15.123:0/3460091420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f2379420 msgr2=0x5583f2380310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:54.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.504+0000 7f67f53ccb80 1 --2- 172.21.15.123:0/3460091420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f2379420 0x5583f2380310 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f67d4000f80 tx=0x7f67d40b2430 comp rx=0 tx=0).stop 2024-05-08T06:09:54.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.504+0000 7f67f53ccb80 1 -- 172.21.15.123:0/3460091420 shutdown_connections 2024-05-08T06:09:54.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.504+0000 7f67f53ccb80 1 --2- 172.21.15.123:0/3460091420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f2379420 0x5583f2380310 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:54.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.504+0000 7f67f53ccb80 1 --2- 172.21.15.123:0/3460091420 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5583f237ddd0 0x5583f23699d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:54.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.504+0000 7f67f53ccb80 1 --2- 172.21.15.123:0/3460091420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583f2379d40 0x5583f2382db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:54.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.504+0000 7f67f53ccb80 1 -- 172.21.15.123:0/3460091420 >> 172.21.15.123:0/3460091420 conn(0x5583f22a3b10 msgr2=0x5583f23826a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:54.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.504+0000 7f67f53ccb80 1 -- 172.21.15.123:0/3460091420 shutdown_connections 2024-05-08T06:09:54.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.504+0000 7f67f53ccb80 1 -- 172.21.15.123:0/3460091420 wait complete. 2024-05-08T06:09:54.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.504+0000 7f67f53ccb80 1 -- start start 2024-05-08T06:09:54.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.505+0000 7f67f53ccb80 1 Processor -- start 2024-05-08T06:09:54.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.505+0000 7f67f53ccb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5583f2379420 0x5583f2378b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:54.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.505+0000 7f67f53ccb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583f2379d40 0x5583f239d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:54.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.505+0000 7f67f53ccb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f237ddd0 0x5583f239d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:54.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.505+0000 7f67f53ccb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5583f2393ba0 con 0x5583f237ddd0 2024-05-08T06:09:54.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.505+0000 7f67f53ccb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5583f23934d0 con 0x5583f2379420 2024-05-08T06:09:54.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.505+0000 7f67f53ccb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5583f2394270 con 0x5583f2379d40 2024-05-08T06:09:54.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.505+0000 7f67f53ccb80 10 client.0 _ll_register_callbacks cb 0x5583f23825d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:09:54.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.506+0000 7f67e3e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5583f2379420 0x5583f2378b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:54.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.506+0000 7f67e3e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5583f2379420 0x5583f2378b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:33610/0 (socket says 172.21.15.123:33610) 2024-05-08T06:09:54.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.506+0000 7f67e3e00700 1 -- 172.21.15.123:0/3698309143 learned_addr learned my addr 172.21.15.123:0/3698309143 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:09:54.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.506+0000 7f67e8a00700 1 --2- 172.21.15.123:0/3698309143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f237ddd0 0x5583f239d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:54.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.506+0000 7f67db400700 1 --2- 172.21.15.123:0/3698309143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583f2379d40 0x5583f239d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:54.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.506+0000 7f67e3e00700 1 -- 172.21.15.123:0/3698309143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583f2379d40 msgr2=0x5583f239d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:54.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.506+0000 7f67e3e00700 1 --2- 172.21.15.123:0/3698309143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583f2379d40 0x5583f239d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:54.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.506+0000 7f67e3e00700 1 -- 172.21.15.123:0/3698309143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f237ddd0 msgr2=0x5583f239d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:54.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.506+0000 7f67e3e00700 1 --2- 172.21.15.123:0/3698309143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f237ddd0 0x5583f239d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:54.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.506+0000 7f67e3e00700 1 -- 172.21.15.123:0/3698309143 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583f2376000 con 0x5583f2379420 2024-05-08T06:09:54.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.506+0000 7f67e3e00700 1 --2- 172.21.15.123:0/3698309143 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5583f2379420 0x5583f2378b30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f67d40a93e0 tx=0x7f67d40b23b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41a547b78d7aeb7a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:54.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.506+0000 7f67d2000700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-05-08T06:09:54.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.506+0000 7f67d2000700 1 -- 172.21.15.123:0/3698309143 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d4074070 con 0x5583f2379420 2024-05-08T06:09:54.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.506+0000 7f67f53ccb80 1 -- 172.21.15.123:0/3698309143 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67d40a9de0 con 0x5583f2379420 2024-05-08T06:09:54.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.506+0000 7f67d2000700 1 -- 172.21.15.123:0/3698309143 <== mon.2 v2:172.21.15.163:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f67d40a3e00 con 0x5583f2379420 2024-05-08T06:09:54.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.507+0000 7f67d2000700 1 -- 172.21.15.123:0/3698309143 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f67d40a91f0 con 0x5583f2379420 2024-05-08T06:09:54.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.507+0000 7f67f53ccb80 10 client.24989 fetch_fsmap learned FSMap version 23 2024-05-08T06:09:54.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.507+0000 7f67f53ccb80 1 -- 172.21.15.123:0/3698309143 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5583f2376000 con 0x5583f2379420 2024-05-08T06:09:54.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.507+0000 7f67d2000700 1 -- 172.21.15.123:0/3698309143 <== mon.2 v2:172.21.15.163:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67d40a9de0 con 0x5583f2379420 2024-05-08T06:09:54.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.507+0000 7f67d2000700 1 -- 172.21.15.123:0/3698309143 <== mon.2 v2:172.21.15.163:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f67d40a8940 con 0x5583f2379420 2024-05-08T06:09:54.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.507+0000 7f67f53ccb80 10 client.24989 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T06:09:54.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.507+0000 7f67f53ccb80 10 client.24989 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T06:09:54.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.507+0000 7f67f53ccb80 20 client.24989 populate_metadata read hostname 'smithi123' 2024-05-08T06:09:54.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.507+0000 7f67f53ccb80 1 --2- 172.21.15.123:0/3698309143 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x5583f2406d10 0x5583f2405a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:54.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.508+0000 7f67f53ccb80 4 client.24989 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T06:09:54.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.508+0000 7f67f53ccb80 1 -- 172.21.15.123:0/3698309143 --> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] -- command(tid 0: {"prefix": "status"}) -- 0x5583f22d53b0 con 0x5583f2406d10 2024-05-08T06:09:54.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.508+0000 7f67db400700 1 --2- 172.21.15.123:0/3698309143 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x5583f2406d10 0x5583f2405a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:54.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.508+0000 7f67db400700 1 --2- 172.21.15.123:0/3698309143 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x5583f2406d10 0x5583f2405a20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=60c202953f68a990 server_cookie=ccb425b05b5f98fc in_seq=0 out_seq=0 2024-05-08T06:09:54.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.508+0000 7f67d2000700 10 client.24989 ms_handle_connect on v2:172.21.15.192:6838/3598682360 2024-05-08T06:09:54.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.509+0000 7f67d2000700 1 -- 172.21.15.123:0/3698309143 <== mds.2 v2:172.21.15.192:6838/3598682360 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5583f22d53b0 con 0x5583f2406d10 2024-05-08T06:09:54.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.509+0000 7f67d2000700 10 client.24989 handle_command_reply: tid=0 2024-05-08T06:09:54.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.509+0000 7f67f53ccb80 1 client.24989 shutdown 2024-05-08T06:09:54.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.509+0000 7f67f53ccb80 1 -- 172.21.15.123:0/3698309143 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5583f2379420 msgr2=0x5583f2378b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:54.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.509+0000 7f67f53ccb80 1 --2- 172.21.15.123:0/3698309143 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5583f2379420 0x5583f2378b30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f67d40a93e0 tx=0x7f67d40b23b0 comp rx=0 tx=0).stop 2024-05-08T06:09:54.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.509+0000 7f67f53ccb80 1 -- 172.21.15.123:0/3698309143 shutdown_connections 2024-05-08T06:09:54.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.509+0000 7f67f53ccb80 1 --2- 172.21.15.123:0/3698309143 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x5583f2406d10 0x5583f2405a20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:54.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.509+0000 7f67f53ccb80 1 --2- 172.21.15.123:0/3698309143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583f237ddd0 0x5583f239d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:54.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.509+0000 7f67f53ccb80 1 --2- 172.21.15.123:0/3698309143 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5583f2379420 0x5583f2378b30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:54.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.509+0000 7f67f53ccb80 1 --2- 172.21.15.123:0/3698309143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583f2379d40 0x5583f239d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:54.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.509+0000 7f67d2000700 0 client.24989 ms_handle_reset on v2:172.21.15.192:6838/3598682360 2024-05-08T06:09:54.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.509+0000 7f67f53ccb80 1 -- 172.21.15.123:0/3698309143 >> 172.21.15.123:0/3698309143 conn(0x5583f22a3b10 msgr2=0x5583f236a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:54.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.509+0000 7f67f53ccb80 1 -- 172.21.15.123:0/3698309143 shutdown_connections 2024-05-08T06:09:54.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.509+0000 7f67f53ccb80 1 -- 172.21.15.123:0/3698309143 wait complete. 2024-05-08T06:09:54.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.509+0000 7f67f53ccb80 20 client.24989 trim_cache size 0 max 16384 2024-05-08T06:09:54.513 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-05-08T06:09:54.513 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_fsid": "85cfb2e6-0cff-11ef-bc97-c7b262605968", 2024-05-08T06:09:54.513 INFO:teuthology.orchestra.run.smithi123.stdout: "whoami": 2, 2024-05-08T06:09:54.513 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 24496, 2024-05-08T06:09:54.513 INFO:teuthology.orchestra.run.smithi123.stdout: "want_state": "up:active", 2024-05-08T06:09:54.513 INFO:teuthology.orchestra.run.smithi123.stdout: "state": "up:active", 2024-05-08T06:09:54.513 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs", 2024-05-08T06:09:54.513 INFO:teuthology.orchestra.run.smithi123.stdout: "rank_uptime": 220.11540963799999, 2024-05-08T06:09:54.513 INFO:teuthology.orchestra.run.smithi123.stdout: "mdsmap_epoch": 23, 2024-05-08T06:09:54.513 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch": 82, 2024-05-08T06:09:54.513 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch_barrier": 80, 2024-05-08T06:09:54.514 INFO:teuthology.orchestra.run.smithi123.stdout: "uptime": 277.81837027300003 2024-05-08T06:09:54.514 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-05-08T06:09:54.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.510+0000 7f67eaa00700 1 -- 172.21.15.123:0/2994162265 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f67c80b2900 msgr2=0x7f67c80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:54.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.510+0000 7f67eaa00700 1 --2- 172.21.15.123:0/2994162265 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f67c80b2900 0x7f67c80b4dc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x5583f23704d0 tx=0x7f67dc0047c0 comp rx=0 tx=0).stop 2024-05-08T06:09:54.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.510+0000 7f67eaa00700 1 -- 172.21.15.123:0/2994162265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67e4103750 msgr2=0x7f67e419f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:54.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.510+0000 7f67eaa00700 1 --2- 172.21.15.123:0/2994162265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67e4103750 0x7f67e419f700 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f67d400ef10 tx=0x7f67d400c3a0 comp rx=0 tx=0).stop 2024-05-08T06:09:54.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.510+0000 7f67eaa00700 1 -- 172.21.15.123:0/2994162265 shutdown_connections 2024-05-08T06:09:54.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.510+0000 7f67eaa00700 1 --2- 172.21.15.123:0/2994162265 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f67c80b2900 0x7f67c80b4dc0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:54.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.510+0000 7f67eaa00700 1 --2- 172.21.15.123:0/2994162265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67e4108850 0x7f67e4199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:54.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.510+0000 7f67eaa00700 1 --2- 172.21.15.123:0/2994162265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67e4103750 0x7f67e419f700 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:54.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.510+0000 7f67eaa00700 1 --2- 172.21.15.123:0/2994162265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67e4104100 0x7f67e419fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:54.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.510+0000 7f67eaa00700 1 -- 172.21.15.123:0/2994162265 >> 172.21.15.123:0/2994162265 conn(0x7f67e407a600 msgr2=0x7f67e41017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:54.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.510+0000 7f67eaa00700 1 -- 172.21.15.123:0/2994162265 shutdown_connections 2024-05-08T06:09:54.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:54.510+0000 7f67eaa00700 1 -- 172.21.15.123:0/2994162265 wait complete. 2024-05-08T06:09:55.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:54 smithi163 ceph-mon[34520]: pgmap v443: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:09:55.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:54 smithi192 ceph-mon[34664]: pgmap v443: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:09:55.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:54 smithi123 ceph-mon[29787]: pgmap v443: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:09:55.521 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell mds.a status 2024-05-08T06:09:55.974 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:09:56.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:56 smithi123 ceph-mon[29787]: pgmap v444: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:09:57.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:56 smithi163 ceph-mon[34520]: pgmap v444: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:09:57.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:56 smithi192 ceph-mon[34664]: pgmap v444: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:09:58.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.338+0000 7feee1000700 1 -- 172.21.15.123:0/693616637 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feedc10fb60 msgr2=0x7feedc10ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:58.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.338+0000 7feee1000700 1 --2- 172.21.15.123:0/693616637 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feedc10fb60 0x7feedc10ff40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7feecc009990 tx=0x7feecc009ca0 comp rx=0 tx=0).stop 2024-05-08T06:09:58.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.338+0000 7feee1000700 1 -- 172.21.15.123:0/693616637 shutdown_connections 2024-05-08T06:09:58.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.338+0000 7feee1000700 1 --2- 172.21.15.123:0/693616637 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feedc10fb60 0x7feedc10ff40 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:58.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.338+0000 7feee1000700 1 --2- 172.21.15.123:0/693616637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feedc10a490 0x7feedc1169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:58.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.338+0000 7feee1000700 1 --2- 172.21.15.123:0/693616637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feedc109ad0 0x7feedc109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:58.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.338+0000 7feee1000700 1 -- 172.21.15.123:0/693616637 >> 172.21.15.123:0/693616637 conn(0x7feedc076c70 msgr2=0x7feedc1082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:58.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.339+0000 7feee1000700 1 -- 172.21.15.123:0/693616637 shutdown_connections 2024-05-08T06:09:58.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.339+0000 7feee1000700 1 -- 172.21.15.123:0/693616637 wait complete. 2024-05-08T06:09:58.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.340+0000 7feee1000700 1 Processor -- start 2024-05-08T06:09:58.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.340+0000 7feee1000700 1 -- start start 2024-05-08T06:09:58.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.340+0000 7feee1000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feedc109ad0 0x7feedc1a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:58.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.340+0000 7feee1000700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feedc10a490 0x7feedc1a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:58.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.340+0000 7feee1000700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feedc10fb60 0x7feedc1a4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:58.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.340+0000 7feee1000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feedc119d20 con 0x7feedc109ad0 2024-05-08T06:09:58.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.340+0000 7feee1000700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7feedc119ba0 con 0x7feedc10a490 2024-05-08T06:09:58.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.340+0000 7feee1000700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7feedc119ea0 con 0x7feedc10fb60 2024-05-08T06:09:58.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.340+0000 7feed9e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feedc109ad0 0x7feedc1a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:58.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.340+0000 7feed9400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feedc10a490 0x7feedc1a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:58.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.341+0000 7feed9400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feedc10a490 0x7feedc1a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:34790/0 (socket says 172.21.15.123:34790) 2024-05-08T06:09:58.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.341+0000 7feed9400700 1 -- 172.21.15.123:0/4132117562 learned_addr learned my addr 172.21.15.123:0/4132117562 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:09:58.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.341+0000 7feeda800700 1 --2- 172.21.15.123:0/4132117562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feedc10fb60 0x7feedc1a4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:58.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.341+0000 7feed9400700 1 -- 172.21.15.123:0/4132117562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feedc10fb60 msgr2=0x7feedc1a4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:58.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.341+0000 7feed9400700 1 --2- 172.21.15.123:0/4132117562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feedc10fb60 0x7feedc1a4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:58.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.341+0000 7feed9400700 1 -- 172.21.15.123:0/4132117562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feedc109ad0 msgr2=0x7feedc1a01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:58.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.341+0000 7feed9400700 1 --2- 172.21.15.123:0/4132117562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feedc109ad0 0x7feedc1a01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:58.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.341+0000 7feed9400700 1 -- 172.21.15.123:0/4132117562 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feedc1a51c0 con 0x7feedc10a490 2024-05-08T06:09:58.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.341+0000 7feed9400700 1 --2- 172.21.15.123:0/4132117562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feedc10a490 0x7feedc1a06f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7feec800d980 tx=0x7feec800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:58.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.342+0000 7feed2a00700 1 -- 172.21.15.123:0/4132117562 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec8017070 con 0x7feedc10a490 2024-05-08T06:09:58.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.342+0000 7feee1000700 1 -- 172.21.15.123:0/4132117562 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feedc1a54b0 con 0x7feedc10a490 2024-05-08T06:09:58.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.342+0000 7feed2a00700 1 -- 172.21.15.123:0/4132117562 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feec8005020 con 0x7feedc10a490 2024-05-08T06:09:58.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.342+0000 7feed2a00700 1 -- 172.21.15.123:0/4132117562 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec8009a20 con 0x7feedc10a490 2024-05-08T06:09:58.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.342+0000 7feee1000700 1 -- 172.21.15.123:0/4132117562 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feedc1acd30 con 0x7feedc10a490 2024-05-08T06:09:58.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.346+0000 7feed2a00700 1 -- 172.21.15.123:0/4132117562 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7feec80051d0 con 0x7feedc10a490 2024-05-08T06:09:58.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.346+0000 7feeeba32b80 1 Processor -- start 2024-05-08T06:09:58.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.346+0000 7feeeba32b80 1 -- start start 2024-05-08T06:09:58.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.347+0000 7feeeba32b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ef37fe010 0x558ef37fe3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:58.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.347+0000 7feeeba32b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558ef37fe930 0x558ef3802440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:58.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.347+0000 7feeeba32b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ef3802980 0x558ef3802de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:58.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.347+0000 7feeeba32b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558ef37509d0 con 0x558ef37fe010 2024-05-08T06:09:58.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.347+0000 7feeeba32b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x558ef36ee710 con 0x558ef37fe930 2024-05-08T06:09:58.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.347+0000 7feeeba32b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558ef374c990 con 0x558ef3802980 2024-05-08T06:09:58.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.347+0000 7feeda800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ef37fe010 0x558ef37fe3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:58.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.347+0000 7feed9400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ef3802980 0x558ef3802de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:58.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.347+0000 7feeda800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ef37fe010 0x558ef37fe3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55176/0 (socket says 172.21.15.123:55176) 2024-05-08T06:09:58.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.347+0000 7feeda800700 1 -- 172.21.15.123:0/3662593720 learned_addr learned my addr 172.21.15.123:0/3662593720 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:09:58.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.347+0000 7feed9e00700 1 --2- 172.21.15.123:0/3662593720 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558ef37fe930 0x558ef3802440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:58.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.348+0000 7feeda800700 1 -- 172.21.15.123:0/3662593720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ef3802980 msgr2=0x558ef3802de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:58.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.348+0000 7feeda800700 1 --2- 172.21.15.123:0/3662593720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ef3802980 0x558ef3802de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:58.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.348+0000 7feeda800700 1 -- 172.21.15.123:0/3662593720 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558ef37fe930 msgr2=0x558ef3802440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:58.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.348+0000 7feeda800700 1 --2- 172.21.15.123:0/3662593720 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558ef37fe930 0x558ef3802440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:58.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.348+0000 7feeda800700 1 -- 172.21.15.123:0/3662593720 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ef369d880 con 0x558ef37fe010 2024-05-08T06:09:58.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.348+0000 7feed2a00700 1 --2- 172.21.15.123:0/4132117562 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7feec00b2900 0x7feec00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:58.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.348+0000 7feeda800700 1 --2- 172.21.15.123:0/3662593720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ef37fe010 0x558ef37fe3f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7feed4011630 tx=0x7feed4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76495bb5bb246630 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:58.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.348+0000 7feed2a00700 1 -- 172.21.15.123:0/4132117562 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7feec80e1fe0 con 0x7feedc10a490 2024-05-08T06:09:58.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.348+0000 7feed9e00700 1 --2- 172.21.15.123:0/4132117562 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7feec00b2900 0x7feec00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:58.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.348+0000 7feec7400700 1 -- 172.21.15.123:0/3662593720 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed4003a00 con 0x558ef37fe010 2024-05-08T06:09:58.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.349+0000 7feec7400700 1 -- 172.21.15.123:0/3662593720 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feed401a070 con 0x558ef37fe010 2024-05-08T06:09:58.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.349+0000 7feec7400700 1 -- 172.21.15.123:0/3662593720 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed4007760 con 0x558ef37fe010 2024-05-08T06:09:58.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.349+0000 7feeeba32b80 1 -- 172.21.15.123:0/3662593720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ef37fe010 msgr2=0x558ef37fe3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:58.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.349+0000 7feeeba32b80 1 --2- 172.21.15.123:0/3662593720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ef37fe010 0x558ef37fe3f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7feed4011630 tx=0x7feed4009920 comp rx=0 tx=0).stop 2024-05-08T06:09:58.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.349+0000 7feed9e00700 1 --2- 172.21.15.123:0/4132117562 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7feec00b2900 0x7feec00b4dc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7feecc00b2e0 tx=0x7feecc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:58.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.349+0000 7feeeba32b80 1 -- 172.21.15.123:0/3662593720 shutdown_connections 2024-05-08T06:09:58.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.349+0000 7feeeba32b80 1 --2- 172.21.15.123:0/3662593720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ef37fe010 0x558ef37fe3f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:58.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.349+0000 7feeeba32b80 1 --2- 172.21.15.123:0/3662593720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ef3802980 0x558ef3802de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:58.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.349+0000 7feeeba32b80 1 --2- 172.21.15.123:0/3662593720 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558ef37fe930 0x558ef3802440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:58.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.349+0000 7feeeba32b80 1 -- 172.21.15.123:0/3662593720 >> 172.21.15.123:0/3662593720 conn(0x558ef372cb10 msgr2=0x558ef37f8750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:58.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.350+0000 7feeeba32b80 1 -- 172.21.15.123:0/3662593720 shutdown_connections 2024-05-08T06:09:58.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.350+0000 7feeeba32b80 1 -- 172.21.15.123:0/3662593720 wait complete. 2024-05-08T06:09:58.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.350+0000 7feeeba32b80 1 -- start start 2024-05-08T06:09:58.352 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.350+0000 7feeeba32b80 1 Processor -- start 2024-05-08T06:09:58.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.351+0000 7feeeba32b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ef37fe010 0x558ef3809d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:58.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.351+0000 7feeeba32b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ef37fe930 0x558ef38070f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:58.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.351+0000 7feeeba32b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558ef3802980 0x558ef3807630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:58.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.351+0000 7feeeba32b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558ef380ab30 con 0x558ef37fe010 2024-05-08T06:09:58.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.351+0000 7feeeba32b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x558ef36b6ad0 con 0x558ef3802980 2024-05-08T06:09:58.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.351+0000 7feeeba32b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558ef380acb0 con 0x558ef37fe930 2024-05-08T06:09:58.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.351+0000 7feeeba32b80 10 client.0 _ll_register_callbacks cb 0x558ef36f7c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:09:58.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.351+0000 7feed9e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ef37fe010 0x558ef3809d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:58.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.351+0000 7feed9400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ef37fe930 0x558ef38070f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:58.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.351+0000 7feed9400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ef37fe930 0x558ef38070f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:33972/0 (socket says 172.21.15.123:33972) 2024-05-08T06:09:58.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.351+0000 7feed9400700 1 -- 172.21.15.123:0/1796350942 learned_addr learned my addr 172.21.15.123:0/1796350942 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:09:58.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.351+0000 7feed9e00700 1 --2- 172.21.15.123:0/1796350942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ef37fe010 0x558ef3809d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55186/0 (socket says 172.21.15.123:55186) 2024-05-08T06:09:58.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.351+0000 7feeda800700 1 --2- 172.21.15.123:0/1796350942 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558ef3802980 0x558ef3807630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:58.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.352+0000 7feed9400700 1 -- 172.21.15.123:0/1796350942 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558ef3802980 msgr2=0x558ef3807630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:58.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.352+0000 7feed9400700 1 --2- 172.21.15.123:0/1796350942 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558ef3802980 0x558ef3807630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:58.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.352+0000 7feed9400700 1 -- 172.21.15.123:0/1796350942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ef37fe010 msgr2=0x558ef3809d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:58.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.352+0000 7feed9400700 1 --2- 172.21.15.123:0/1796350942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ef37fe010 0x558ef3809d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:58.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.352+0000 7feed9400700 1 -- 172.21.15.123:0/1796350942 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ef37163d0 con 0x558ef37fe930 2024-05-08T06:09:58.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.352+0000 7feed9400700 1 --2- 172.21.15.123:0/1796350942 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ef37fe930 0x558ef38070f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7feec800d950 tx=0x7feec8005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9047942655b7368 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:58.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.352+0000 7feebbe00700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-05-08T06:09:58.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.352+0000 7feebbe00700 1 -- 172.21.15.123:0/1796350942 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec80a8d80 con 0x558ef37fe930 2024-05-08T06:09:58.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.352+0000 7feebbe00700 1 -- 172.21.15.123:0/1796350942 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7feec80e40d0 con 0x558ef37fe930 2024-05-08T06:09:58.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.352+0000 7feeeba32b80 1 -- 172.21.15.123:0/1796350942 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feed401a070 con 0x558ef37fe930 2024-05-08T06:09:58.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.352+0000 7feebbe00700 1 -- 172.21.15.123:0/1796350942 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7feec80a9120 con 0x558ef37fe930 2024-05-08T06:09:58.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.352+0000 7feeeba32b80 10 client.24994 fetch_fsmap learned FSMap version 23 2024-05-08T06:09:58.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.353+0000 7feeeba32b80 1 -- 172.21.15.123:0/1796350942 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x558ef3815370 con 0x558ef37fe930 2024-05-08T06:09:58.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.353+0000 7feebbe00700 1 -- 172.21.15.123:0/1796350942 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feed401a070 con 0x558ef37fe930 2024-05-08T06:09:58.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.353+0000 7feebbe00700 1 -- 172.21.15.123:0/1796350942 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7feec80a9870 con 0x558ef37fe930 2024-05-08T06:09:58.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.353+0000 7feeeba32b80 10 client.24994 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T06:09:58.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.353+0000 7feeeba32b80 10 client.24994 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T06:09:58.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.353+0000 7feeeba32b80 20 client.24994 populate_metadata read hostname 'smithi123' 2024-05-08T06:09:58.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.353+0000 7feeeba32b80 1 --2- 172.21.15.123:0/1796350942 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x558ef38178a0 0x558ef3818d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:58.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.353+0000 7feeeba32b80 4 client.24994 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:09:58.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.353+0000 7feeeba32b80 1 -- 172.21.15.123:0/1796350942 --> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ef37cb050 con 0x558ef38178a0 2024-05-08T06:09:58.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.354+0000 7feed9e00700 1 --2- 172.21.15.123:0/1796350942 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x558ef38178a0 0x558ef3818d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:58.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.354+0000 7feed9e00700 1 --2- 172.21.15.123:0/1796350942 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x558ef38178a0 0x558ef3818d10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f8f5017e5699fb92 server_cookie=7f12e03ee436d9cd in_seq=0 out_seq=0 2024-05-08T06:09:58.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.354+0000 7feebbe00700 10 client.24994 ms_handle_connect on v2:172.21.15.123:6840/1571726231 2024-05-08T06:09:58.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.355+0000 7feebbe00700 1 -- 172.21.15.123:0/1796350942 <== mds.1 v2:172.21.15.123:6840/1571726231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558ef37cb050 con 0x558ef38178a0 2024-05-08T06:09:58.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.355+0000 7feebbe00700 10 client.24994 handle_command_reply: tid=0 2024-05-08T06:09:58.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.355+0000 7feeeba32b80 1 client.24994 shutdown 2024-05-08T06:09:58.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.355+0000 7feeeba32b80 1 -- 172.21.15.123:0/1796350942 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ef37fe930 msgr2=0x558ef38070f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:58.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.355+0000 7feeeba32b80 1 --2- 172.21.15.123:0/1796350942 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ef37fe930 0x558ef38070f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7feec800d950 tx=0x7feec8005720 comp rx=0 tx=0).stop 2024-05-08T06:09:58.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.355+0000 7feeeba32b80 1 -- 172.21.15.123:0/1796350942 shutdown_connections 2024-05-08T06:09:58.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.355+0000 7feeeba32b80 1 --2- 172.21.15.123:0/1796350942 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x558ef38178a0 0x558ef3818d10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:58.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.355+0000 7feeeba32b80 1 --2- 172.21.15.123:0/1796350942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ef37fe010 0x558ef3809d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:58.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.355+0000 7feeeba32b80 1 --2- 172.21.15.123:0/1796350942 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558ef3802980 0x558ef3807630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:58.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.355+0000 7feebbe00700 0 client.24994 ms_handle_reset on v2:172.21.15.123:6840/1571726231 2024-05-08T06:09:58.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.355+0000 7feeeba32b80 1 --2- 172.21.15.123:0/1796350942 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ef37fe930 0x558ef38070f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:58.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.355+0000 7feeeba32b80 1 -- 172.21.15.123:0/1796350942 >> 172.21.15.123:0/1796350942 conn(0x558ef372cb10 msgr2=0x558ef37f3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:58.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.356+0000 7feeeba32b80 1 -- 172.21.15.123:0/1796350942 shutdown_connections 2024-05-08T06:09:58.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.356+0000 7feeeba32b80 1 -- 172.21.15.123:0/1796350942 wait complete. 2024-05-08T06:09:58.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.356+0000 7feeeba32b80 20 client.24994 trim_cache size 0 max 16384 2024-05-08T06:09:58.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.378+0000 7feeeba32b80 1 Processor -- start 2024-05-08T06:09:58.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.379+0000 7feeeba32b80 1 -- start start 2024-05-08T06:09:58.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.379+0000 7feeeba32b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558ef37fe010 0x558ef3805a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:58.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.379+0000 7feeeba32b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ef37fe930 0x558ef3805f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:58.379 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.379+0000 7feeeba32b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ef3802980 0x558ef3800340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:58.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.379+0000 7feeeba32b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558ef3817050 con 0x558ef3802980 2024-05-08T06:09:58.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.379+0000 7feeeba32b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x558ef3817720 con 0x558ef37fe010 2024-05-08T06:09:58.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.379+0000 7feeeba32b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558ef3817df0 con 0x558ef37fe930 2024-05-08T06:09:58.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.379+0000 7feeda800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ef3802980 0x558ef3800340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:58.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.379+0000 7feed9e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558ef37fe010 0x558ef3805a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:58.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.379+0000 7feed9e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558ef37fe010 0x558ef3805a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:34804/0 (socket says 172.21.15.123:34804) 2024-05-08T06:09:58.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.379+0000 7feed9400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ef37fe930 0x558ef3805f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:58.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.379+0000 7feed9e00700 1 -- 172.21.15.123:0/34411700 learned_addr learned my addr 172.21.15.123:0/34411700 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:09:58.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.380+0000 7feed9400700 1 -- 172.21.15.123:0/34411700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558ef37fe010 msgr2=0x558ef3805a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:58.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.380+0000 7feed9400700 1 --2- 172.21.15.123:0/34411700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558ef37fe010 0x558ef3805a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:58.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.380+0000 7feed9400700 1 -- 172.21.15.123:0/34411700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ef3802980 msgr2=0x558ef3800340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:58.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.380+0000 7feed9400700 1 --2- 172.21.15.123:0/34411700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ef3802980 0x558ef3800340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:58.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.380+0000 7feed9400700 1 -- 172.21.15.123:0/34411700 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ef37f53a0 con 0x558ef37fe930 2024-05-08T06:09:58.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.380+0000 7feed9e00700 1 --2- 172.21.15.123:0/34411700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558ef37fe010 0x558ef3805a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:09:58.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.380+0000 7feed9400700 1 --2- 172.21.15.123:0/34411700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ef37fe930 0x558ef3805f40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7feec80dfe60 tx=0x7feec80057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b26f55ba8d0ffba1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:58.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.380+0000 7feec6a00700 1 -- 172.21.15.123:0/34411700 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec80a9610 con 0x558ef37fe930 2024-05-08T06:09:58.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.380+0000 7feec6a00700 1 -- 172.21.15.123:0/34411700 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feec8015070 con 0x558ef37fe930 2024-05-08T06:09:58.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.380+0000 7feec6a00700 1 -- 172.21.15.123:0/34411700 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec80e52b0 con 0x558ef37fe930 2024-05-08T06:09:58.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.381+0000 7feeeba32b80 1 -- 172.21.15.123:0/34411700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ef37fe930 msgr2=0x558ef3805f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:58.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.381+0000 7feeeba32b80 1 --2- 172.21.15.123:0/34411700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ef37fe930 0x558ef3805f40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7feec80dfe60 tx=0x7feec80057c0 comp rx=0 tx=0).stop 2024-05-08T06:09:58.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.381+0000 7feeeba32b80 1 -- 172.21.15.123:0/34411700 shutdown_connections 2024-05-08T06:09:58.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.381+0000 7feeeba32b80 1 --2- 172.21.15.123:0/34411700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ef3802980 0x558ef3800340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:58.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.381+0000 7feeeba32b80 1 --2- 172.21.15.123:0/34411700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558ef37fe010 0x558ef3805a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:58.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.381+0000 7feeeba32b80 1 --2- 172.21.15.123:0/34411700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ef37fe930 0x558ef3805f40 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:58.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.381+0000 7feeeba32b80 1 -- 172.21.15.123:0/34411700 >> 172.21.15.123:0/34411700 conn(0x558ef372cb10 msgr2=0x558ef38070b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:58.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.381+0000 7feeeba32b80 1 -- 172.21.15.123:0/34411700 shutdown_connections 2024-05-08T06:09:58.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.381+0000 7feeeba32b80 1 -- 172.21.15.123:0/34411700 wait complete. 2024-05-08T06:09:58.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.382+0000 7feeeba32b80 1 -- start start 2024-05-08T06:09:58.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.382+0000 7feeeba32b80 1 Processor -- start 2024-05-08T06:09:58.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.382+0000 7feeeba32b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558ef37fe010 0x558ef37ffaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:58.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.382+0000 7feeeba32b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ef37fe930 0x558ef3800030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:58.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.382+0000 7feeeba32b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ef3802980 0x558ef3821180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:58.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.382+0000 7feeeba32b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558ef3818b90 con 0x558ef3802980 2024-05-08T06:09:58.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.382+0000 7feeeba32b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x558ef38184c0 con 0x558ef37fe010 2024-05-08T06:09:58.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.382+0000 7feeeba32b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558ef3819260 con 0x558ef37fe930 2024-05-08T06:09:58.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.383+0000 7feeeba32b80 10 client.0 _ll_register_callbacks cb 0x558ef36f7c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:09:58.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.383+0000 7feed9e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558ef37fe010 0x558ef37ffaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:58.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.383+0000 7feeda800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ef3802980 0x558ef3821180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:58.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.383+0000 7feed9400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ef37fe930 0x558ef3800030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:58.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.383+0000 7feed9e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558ef37fe010 0x558ef37ffaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:34806/0 (socket says 172.21.15.123:34806) 2024-05-08T06:09:58.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.383+0000 7feed9e00700 1 -- 172.21.15.123:0/2334968241 learned_addr learned my addr 172.21.15.123:0/2334968241 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:09:58.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.383+0000 7feed9e00700 1 -- 172.21.15.123:0/2334968241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ef37fe930 msgr2=0x558ef3800030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:58.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.383+0000 7feed9e00700 1 --2- 172.21.15.123:0/2334968241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ef37fe930 0x558ef3800030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:58.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.383+0000 7feed9e00700 1 -- 172.21.15.123:0/2334968241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ef3802980 msgr2=0x558ef3821180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:09:58.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.383+0000 7feed9e00700 1 --2- 172.21.15.123:0/2334968241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ef3802980 0x558ef3821180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:58.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.383+0000 7feed9e00700 1 -- 172.21.15.123:0/2334968241 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ef3813f80 con 0x558ef37fe010 2024-05-08T06:09:58.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.383+0000 7feed9400700 1 --2- 172.21.15.123:0/2334968241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ef37fe930 0x558ef3800030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:09:58.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.383+0000 7feeda800700 1 --2- 172.21.15.123:0/2334968241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ef3802980 0x558ef3821180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:09:58.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.384+0000 7feed9e00700 1 --2- 172.21.15.123:0/2334968241 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558ef37fe010 0x558ef37ffaf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7feecc00b840 tx=0x7feecc009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf3103d57924c13b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:09:58.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.384+0000 7feebbe00700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-05-08T06:09:58.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.384+0000 7feebbe00700 1 -- 172.21.15.123:0/2334968241 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc02b040 con 0x558ef37fe010 2024-05-08T06:09:58.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.384+0000 7feeeba32b80 1 -- 172.21.15.123:0/2334968241 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feec8015070 con 0x558ef37fe010 2024-05-08T06:09:58.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.384+0000 7feebbe00700 1 -- 172.21.15.123:0/2334968241 <== mon.2 v2:172.21.15.163:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7feecc003880 con 0x558ef37fe010 2024-05-08T06:09:58.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.384+0000 7feebbe00700 1 -- 172.21.15.123:0/2334968241 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7feecc003a80 con 0x558ef37fe010 2024-05-08T06:09:58.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.384+0000 7feeeba32b80 10 client.25007 fetch_fsmap learned FSMap version 23 2024-05-08T06:09:58.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.384+0000 7feeeba32b80 1 -- 172.21.15.123:0/2334968241 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x558ef37f95c0 con 0x558ef37fe010 2024-05-08T06:09:58.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.384+0000 7feebbe00700 1 -- 172.21.15.123:0/2334968241 <== mon.2 v2:172.21.15.163:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feec8015070 con 0x558ef37fe010 2024-05-08T06:09:58.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.385+0000 7feebbe00700 1 -- 172.21.15.123:0/2334968241 <== mon.2 v2:172.21.15.163:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7feecc00fc70 con 0x558ef37fe010 2024-05-08T06:09:58.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.385+0000 7feeeba32b80 10 client.25007 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T06:09:58.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.385+0000 7feeeba32b80 10 client.25007 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T06:09:58.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.385+0000 7feeeba32b80 20 client.25007 populate_metadata read hostname 'smithi123' 2024-05-08T06:09:58.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.385+0000 7feeeba32b80 1 --2- 172.21.15.123:0/2334968241 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x558ef3889210 0x558ef3887f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:09:58.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.385+0000 7feeeba32b80 4 client.25007 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T06:09:58.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.385+0000 7feeeba32b80 1 -- 172.21.15.123:0/2334968241 --> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] -- command(tid 0: {"prefix": "status"}) -- 0x558ef375e3b0 con 0x558ef3889210 2024-05-08T06:09:58.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.385+0000 7feed9400700 1 --2- 172.21.15.123:0/2334968241 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x558ef3889210 0x558ef3887f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:09:58.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.386+0000 7feed9400700 1 --2- 172.21.15.123:0/2334968241 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x558ef3889210 0x558ef3887f20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=75881ec7211a7b53 server_cookie=24831a4ba45ae513 in_seq=0 out_seq=0 2024-05-08T06:09:58.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.386+0000 7feebbe00700 10 client.25007 ms_handle_connect on v2:172.21.15.123:6840/1571726231 2024-05-08T06:09:58.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.386+0000 7feebbe00700 1 -- 172.21.15.123:0/2334968241 <== mds.1 v2:172.21.15.123:6840/1571726231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x558ef375e3b0 con 0x558ef3889210 2024-05-08T06:09:58.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.386+0000 7feebbe00700 10 client.25007 handle_command_reply: tid=0 2024-05-08T06:09:58.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.386+0000 7feeeba32b80 1 client.25007 shutdown 2024-05-08T06:09:58.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.386+0000 7feeeba32b80 1 -- 172.21.15.123:0/2334968241 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558ef37fe010 msgr2=0x558ef37ffaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:58.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.386+0000 7feeeba32b80 1 --2- 172.21.15.123:0/2334968241 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558ef37fe010 0x558ef37ffaf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7feecc00b840 tx=0x7feecc009ca0 comp rx=0 tx=0).stop 2024-05-08T06:09:58.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.386+0000 7feeeba32b80 1 -- 172.21.15.123:0/2334968241 shutdown_connections 2024-05-08T06:09:58.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.386+0000 7feeeba32b80 1 --2- 172.21.15.123:0/2334968241 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x558ef3889210 0x558ef3887f20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:58.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.386+0000 7feeeba32b80 1 --2- 172.21.15.123:0/2334968241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ef3802980 0x558ef3821180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:58.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.386+0000 7feeeba32b80 1 --2- 172.21.15.123:0/2334968241 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558ef37fe010 0x558ef37ffaf0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:58.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.386+0000 7feeeba32b80 1 --2- 172.21.15.123:0/2334968241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ef37fe930 0x558ef3800030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:58.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.386+0000 7feeeba32b80 1 -- 172.21.15.123:0/2334968241 >> 172.21.15.123:0/2334968241 conn(0x558ef372cb10 msgr2=0x558ef37f29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:58.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.386+0000 7feebbe00700 0 client.25007 ms_handle_reset on v2:172.21.15.123:6840/1571726231 2024-05-08T06:09:58.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.386+0000 7feeeba32b80 1 -- 172.21.15.123:0/2334968241 shutdown_connections 2024-05-08T06:09:58.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.387+0000 7feeeba32b80 1 -- 172.21.15.123:0/2334968241 wait complete. 2024-05-08T06:09:58.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.387+0000 7feeeba32b80 20 client.25007 trim_cache size 0 max 16384 2024-05-08T06:09:58.391 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-05-08T06:09:58.391 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_fsid": "85cfb2e6-0cff-11ef-bc97-c7b262605968", 2024-05-08T06:09:58.391 INFO:teuthology.orchestra.run.smithi123.stdout: "whoami": 1, 2024-05-08T06:09:58.391 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 24512, 2024-05-08T06:09:58.391 INFO:teuthology.orchestra.run.smithi123.stdout: "want_state": "up:active", 2024-05-08T06:09:58.391 INFO:teuthology.orchestra.run.smithi123.stdout: "state": "up:active", 2024-05-08T06:09:58.391 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs", 2024-05-08T06:09:58.391 INFO:teuthology.orchestra.run.smithi123.stdout: "rank_uptime": 225.00073909, 2024-05-08T06:09:58.391 INFO:teuthology.orchestra.run.smithi123.stdout: "mdsmap_epoch": 23, 2024-05-08T06:09:58.391 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch": 82, 2024-05-08T06:09:58.391 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch_barrier": 80, 2024-05-08T06:09:58.391 INFO:teuthology.orchestra.run.smithi123.stdout: "uptime": 271.48274005399998 2024-05-08T06:09:58.391 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-05-08T06:09:58.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.387+0000 7feee1000700 1 -- 172.21.15.123:0/4132117562 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7feec00b2900 msgr2=0x7feec00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:58.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.387+0000 7feee1000700 1 --2- 172.21.15.123:0/4132117562 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7feec00b2900 0x7feec00b4dc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7feecc00b2e0 tx=0x7feecc005fd0 comp rx=0 tx=0).stop 2024-05-08T06:09:58.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.387+0000 7feee1000700 1 -- 172.21.15.123:0/4132117562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feedc10a490 msgr2=0x7feedc1a06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:09:58.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.387+0000 7feee1000700 1 --2- 172.21.15.123:0/4132117562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feedc10a490 0x7feedc1a06f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7feec800d980 tx=0x7feec800dc90 comp rx=0 tx=0).stop 2024-05-08T06:09:58.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.387+0000 7feee1000700 1 -- 172.21.15.123:0/4132117562 shutdown_connections 2024-05-08T06:09:58.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.387+0000 7feee1000700 1 --2- 172.21.15.123:0/4132117562 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7feec00b2900 0x7feec00b4dc0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:58.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.387+0000 7feee1000700 1 --2- 172.21.15.123:0/4132117562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feedc109ad0 0x7feedc1a01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:58.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.387+0000 7feee1000700 1 --2- 172.21.15.123:0/4132117562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feedc10fb60 0x7feedc1a4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:58.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.387+0000 7feee1000700 1 --2- 172.21.15.123:0/4132117562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feedc10a490 0x7feedc1a06f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:09:58.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.387+0000 7feee1000700 1 -- 172.21.15.123:0/4132117562 >> 172.21.15.123:0/4132117562 conn(0x7feedc076c70 msgr2=0x7feedc077290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:09:58.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.388+0000 7feee1000700 1 -- 172.21.15.123:0/4132117562 shutdown_connections 2024-05-08T06:09:58.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:09:58.388+0000 7feee1000700 1 -- 172.21.15.123:0/4132117562 wait complete. 2024-05-08T06:09:59.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:09:58 smithi163 ceph-mon[34520]: pgmap v445: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:09:59.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:09:58 smithi192 ceph-mon[34664]: pgmap v445: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:09:59.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:09:58 smithi123 ceph-mon[29787]: pgmap v445: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:09:59.494 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-08T06:09:59.930 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:10:00.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:00 smithi123 ceph-mon[29787]: pgmap v446: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:10:00.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:00 smithi123 ceph-mon[29787]: overall HEALTH_OK 2024-05-08T06:10:01.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:00 smithi163 ceph-mon[34520]: pgmap v446: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:10:01.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:00 smithi163 ceph-mon[34520]: overall HEALTH_OK 2024-05-08T06:10:01.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:00 smithi192 ceph-mon[34664]: pgmap v446: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:10:01.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:00 smithi192 ceph-mon[34664]: overall HEALTH_OK 2024-05-08T06:10:02.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.511+0000 7fe870600700 1 -- 172.21.15.123:0/3748975992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe86810b6c0 msgr2=0x7fe86810baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:02.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.511+0000 7fe870600700 1 --2- 172.21.15.123:0/3748975992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe86810b6c0 0x7fe86810baa0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe85c009990 tx=0x7fe85c009ca0 comp rx=0 tx=0).stop 2024-05-08T06:10:02.512 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.512+0000 7fe870600700 1 -- 172.21.15.123:0/3748975992 shutdown_connections 2024-05-08T06:10:02.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.512+0000 7fe870600700 1 --2- 172.21.15.123:0/3748975992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe86810b6c0 0x7fe86810baa0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:02.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.512+0000 7fe870600700 1 --2- 172.21.15.123:0/3748975992 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe868106080 0x7fe868112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:02.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.512+0000 7fe870600700 1 --2- 172.21.15.123:0/3748975992 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8681056c0 0x7fe868105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:02.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.512+0000 7fe870600700 1 -- 172.21.15.123:0/3748975992 >> 172.21.15.123:0/3748975992 conn(0x7fe8681011b0 msgr2=0x7fe8681035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:02.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.512+0000 7fe870600700 1 -- 172.21.15.123:0/3748975992 shutdown_connections 2024-05-08T06:10:02.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.512+0000 7fe870600700 1 -- 172.21.15.123:0/3748975992 wait complete. 2024-05-08T06:10:02.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.513+0000 7fe870600700 1 Processor -- start 2024-05-08T06:10:02.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.513+0000 7fe870600700 1 -- start start 2024-05-08T06:10:02.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.514+0000 7fe870600700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8681056c0 0x7fe86819bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:02.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.514+0000 7fe870600700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe868106080 0x7fe86819c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:02.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.514+0000 7fe870600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe86810b6c0 0x7fe8681a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:02.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.514+0000 7fe870600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe8681158c0 con 0x7fe86810b6c0 2024-05-08T06:10:02.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.514+0000 7fe870600700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe868115740 con 0x7fe8681056c0 2024-05-08T06:10:02.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.514+0000 7fe870600700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe868115a40 con 0x7fe868106080 2024-05-08T06:10:02.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.514+0000 7fe86dc00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8681056c0 0x7fe86819bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:02.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.514+0000 7fe86dc00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8681056c0 0x7fe86819bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:34826/0 (socket says 172.21.15.123:34826) 2024-05-08T06:10:02.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.514+0000 7fe86d200700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe868106080 0x7fe86819c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:02.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.514+0000 7fe86dc00700 1 -- 172.21.15.123:0/530254851 learned_addr learned my addr 172.21.15.123:0/530254851 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:10:02.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.514+0000 7fe86e600700 1 --2- 172.21.15.123:0/530254851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe86810b6c0 0x7fe8681a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:02.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.514+0000 7fe86dc00700 1 -- 172.21.15.123:0/530254851 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe868106080 msgr2=0x7fe86819c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:02.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.514+0000 7fe86dc00700 1 --2- 172.21.15.123:0/530254851 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe868106080 0x7fe86819c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:02.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.514+0000 7fe86dc00700 1 -- 172.21.15.123:0/530254851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe86810b6c0 msgr2=0x7fe8681a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:02.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.514+0000 7fe86dc00700 1 --2- 172.21.15.123:0/530254851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe86810b6c0 0x7fe8681a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:02.515 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.514+0000 7fe86dc00700 1 -- 172.21.15.123:0/530254851 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8681a0d60 con 0x7fe8681056c0 2024-05-08T06:10:02.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.514+0000 7fe86e600700 1 --2- 172.21.15.123:0/530254851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe86810b6c0 0x7fe8681a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:10:02.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.515+0000 7fe86dc00700 1 --2- 172.21.15.123:0/530254851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8681056c0 0x7fe86819bd50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe85c0057f0 tx=0x7fe85c00bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:02.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.515+0000 7fe866000700 1 -- 172.21.15.123:0/530254851 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85c026070 con 0x7fe8681056c0 2024-05-08T06:10:02.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.515+0000 7fe870600700 1 -- 172.21.15.123:0/530254851 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8681a0ff0 con 0x7fe8681056c0 2024-05-08T06:10:02.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.515+0000 7fe870600700 1 -- 172.21.15.123:0/530254851 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8681a88d0 con 0x7fe8681056c0 2024-05-08T06:10:02.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.515+0000 7fe866000700 1 -- 172.21.15.123:0/530254851 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe85c021410 con 0x7fe8681056c0 2024-05-08T06:10:02.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.515+0000 7fe866000700 1 -- 172.21.15.123:0/530254851 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85c017650 con 0x7fe8681056c0 2024-05-08T06:10:02.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.520+0000 7fe866000700 1 -- 172.21.15.123:0/530254851 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fe85c029070 con 0x7fe8681056c0 2024-05-08T06:10:02.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.521+0000 7fe87aef9b80 1 Processor -- start 2024-05-08T06:10:02.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.521+0000 7fe87aef9b80 1 -- start start 2024-05-08T06:10:02.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.522+0000 7fe87aef9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b61baf3010 0x55b61baf33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:02.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.522+0000 7fe87aef9b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b61baf3930 0x55b61baf7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:02.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.522+0000 7fe87aef9b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b61baf7980 0x55b61baf7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:02.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.522+0000 7fe87aef9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b61ba459d0 con 0x55b61baf3010 2024-05-08T06:10:02.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.522+0000 7fe87aef9b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b61b9e3710 con 0x55b61baf3930 2024-05-08T06:10:02.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.522+0000 7fe87aef9b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b61ba41990 con 0x55b61baf7980 2024-05-08T06:10:02.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.522+0000 7fe86e600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b61baf3010 0x55b61baf33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:02.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.522+0000 7fe86e600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b61baf3010 0x55b61baf33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55246/0 (socket says 172.21.15.123:55246) 2024-05-08T06:10:02.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.522+0000 7fe86dc00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b61baf3930 0x55b61baf7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:02.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.522+0000 7fe86e600700 1 -- 172.21.15.123:0/802117335 learned_addr learned my addr 172.21.15.123:0/802117335 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:10:02.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.522+0000 7fe866000700 1 --2- 172.21.15.123:0/530254851 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe8500b2900 0x7fe8500b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:02.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.522+0000 7fe86d200700 1 --2- 172.21.15.123:0/802117335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b61baf7980 0x55b61baf7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:02.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.522+0000 7fe86e600700 1 --2- 172.21.15.123:0/530254851 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe8500b2900 0x7fe8500b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:02.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.522+0000 7fe86dc00700 1 -- 172.21.15.123:0/802117335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b61baf7980 msgr2=0x55b61baf7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:02.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.523+0000 7fe86dc00700 1 --2- 172.21.15.123:0/802117335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b61baf7980 0x55b61baf7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:02.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.523+0000 7fe866000700 1 -- 172.21.15.123:0/530254851 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fe85c0e2c10 con 0x7fe8681056c0 2024-05-08T06:10:02.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.523+0000 7fe86dc00700 1 -- 172.21.15.123:0/802117335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b61baf3010 msgr2=0x55b61baf33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:02.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.523+0000 7fe86dc00700 1 --2- 172.21.15.123:0/802117335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b61baf3010 0x55b61baf33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:02.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.523+0000 7fe86dc00700 1 -- 172.21.15.123:0/802117335 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b61b992880 con 0x55b61baf3930 2024-05-08T06:10:02.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.523+0000 7fe86dc00700 1 --2- 172.21.15.123:0/802117335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b61baf3930 0x55b61baf7440 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe85c0e0770 tx=0x7fe85c005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2c798a150d08e5a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:02.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.523+0000 7fe86e600700 1 --2- 172.21.15.123:0/530254851 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe8500b2900 0x7fe8500b4dc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe860003fd0 tx=0x7fe860004b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:02.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.523+0000 7fe85aa00700 1 -- 172.21.15.123:0/802117335 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85c0a9de0 con 0x55b61baf3930 2024-05-08T06:10:02.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.523+0000 7fe85aa00700 1 -- 172.21.15.123:0/802117335 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe85c0aa9d0 con 0x55b61baf3930 2024-05-08T06:10:02.524 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.523+0000 7fe85aa00700 1 -- 172.21.15.123:0/802117335 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85c0aa790 con 0x55b61baf3930 2024-05-08T06:10:02.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.524+0000 7fe87aef9b80 1 -- 172.21.15.123:0/802117335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b61baf3930 msgr2=0x55b61baf7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:02.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.524+0000 7fe87aef9b80 1 --2- 172.21.15.123:0/802117335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b61baf3930 0x55b61baf7440 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe85c0e0770 tx=0x7fe85c005fd0 comp rx=0 tx=0).stop 2024-05-08T06:10:02.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.524+0000 7fe87aef9b80 1 -- 172.21.15.123:0/802117335 shutdown_connections 2024-05-08T06:10:02.525 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.524+0000 7fe87aef9b80 1 --2- 172.21.15.123:0/802117335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b61baf3010 0x55b61baf33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:02.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.524+0000 7fe87aef9b80 1 --2- 172.21.15.123:0/802117335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b61baf7980 0x55b61baf7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:02.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.524+0000 7fe87aef9b80 1 --2- 172.21.15.123:0/802117335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b61baf3930 0x55b61baf7440 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:02.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.524+0000 7fe87aef9b80 1 -- 172.21.15.123:0/802117335 >> 172.21.15.123:0/802117335 conn(0x55b61ba21b10 msgr2=0x55b61baed750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:02.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.524+0000 7fe87aef9b80 1 -- 172.21.15.123:0/802117335 shutdown_connections 2024-05-08T06:10:02.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.525+0000 7fe87aef9b80 1 -- 172.21.15.123:0/802117335 wait complete. 2024-05-08T06:10:02.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.525+0000 7fe87aef9b80 1 -- start start 2024-05-08T06:10:02.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.525+0000 7fe87aef9b80 1 Processor -- start 2024-05-08T06:10:02.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.526+0000 7fe87aef9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b61baf3010 0x55b61bafed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:02.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.526+0000 7fe87aef9b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b61baf3930 0x55b61bafc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:02.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.526+0000 7fe87aef9b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b61baf7980 0x55b61bafc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:02.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.526+0000 7fe87aef9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b61baff950 con 0x55b61baf3010 2024-05-08T06:10:02.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.526+0000 7fe87aef9b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b61b9abad0 con 0x55b61baf3930 2024-05-08T06:10:02.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.526+0000 7fe87aef9b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b61baffad0 con 0x55b61baf7980 2024-05-08T06:10:02.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.526+0000 7fe87aef9b80 10 client.0 _ll_register_callbacks cb 0x55b61b9ecc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:10:02.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.526+0000 7fe86dc00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b61baf3010 0x55b61bafed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:02.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.526+0000 7fe86dc00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b61baf3010 0x55b61bafed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55256/0 (socket says 172.21.15.123:55256) 2024-05-08T06:10:02.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.526+0000 7fe86dc00700 1 -- 172.21.15.123:0/132412024 learned_addr learned my addr 172.21.15.123:0/132412024 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:10:02.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.526+0000 7fe86d200700 1 --2- 172.21.15.123:0/132412024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b61baf3930 0x55b61bafc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:02.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.526+0000 7fe86e600700 1 --2- 172.21.15.123:0/132412024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b61baf7980 0x55b61bafc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:02.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.527+0000 7fe86dc00700 1 -- 172.21.15.123:0/132412024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b61baf7980 msgr2=0x55b61bafc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:02.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.527+0000 7fe86dc00700 1 --2- 172.21.15.123:0/132412024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b61baf7980 0x55b61bafc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:02.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.527+0000 7fe86dc00700 1 -- 172.21.15.123:0/132412024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b61baf3930 msgr2=0x55b61bafc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:02.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.527+0000 7fe86dc00700 1 --2- 172.21.15.123:0/132412024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b61baf3930 0x55b61bafc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:02.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.527+0000 7fe86dc00700 1 -- 172.21.15.123:0/132412024 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b61ba0b3d0 con 0x55b61baf3010 2024-05-08T06:10:02.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.527+0000 7fe86dc00700 1 --2- 172.21.15.123:0/132412024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b61baf3010 0x55b61bafed90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe85c0aab80 tx=0x7fe85c0e2f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3100222e3b8820db server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:02.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.527+0000 7fe84f400700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-08T06:10:02.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.527+0000 7fe84f400700 1 -- 172.21.15.123:0/132412024 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85c0a9de0 con 0x55b61baf3010 2024-05-08T06:10:02.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.527+0000 7fe84f400700 1 -- 172.21.15.123:0/132412024 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fe85c0e0df0 con 0x55b61baf3010 2024-05-08T06:10:02.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.527+0000 7fe87aef9b80 1 -- 172.21.15.123:0/132412024 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe85c0aa9d0 con 0x55b61baf3010 2024-05-08T06:10:02.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.528+0000 7fe84f400700 1 -- 172.21.15.123:0/132412024 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe85c0b30a0 con 0x55b61baf3010 2024-05-08T06:10:02.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.528+0000 7fe87aef9b80 10 client.14904 fetch_fsmap learned FSMap version 23 2024-05-08T06:10:02.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.528+0000 7fe87aef9b80 1 -- 172.21.15.123:0/132412024 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55b61baff950 con 0x55b61baf3010 2024-05-08T06:10:02.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.528+0000 7fe84f400700 1 -- 172.21.15.123:0/132412024 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe85c0aa9d0 con 0x55b61baf3010 2024-05-08T06:10:02.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.528+0000 7fe84f400700 1 -- 172.21.15.123:0/132412024 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fe85c005110 con 0x55b61baf3010 2024-05-08T06:10:02.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.528+0000 7fe87aef9b80 10 client.14904 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T06:10:02.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.528+0000 7fe87aef9b80 10 client.14904 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T06:10:02.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.528+0000 7fe87aef9b80 20 client.14904 populate_metadata read hostname 'smithi123' 2024-05-08T06:10:02.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.528+0000 7fe87aef9b80 1 --2- 172.21.15.123:0/132412024 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55b61bb0c580 0x55b61bb0d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:02.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.528+0000 7fe87aef9b80 4 client.14904 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:10:02.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.528+0000 7fe87aef9b80 1 -- 172.21.15.123:0/132412024 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b61bac0050 con 0x55b61bb0c580 2024-05-08T06:10:02.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.529+0000 7fe86d200700 1 --2- 172.21.15.123:0/132412024 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55b61bb0c580 0x55b61bb0d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:02.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.529+0000 7fe86d200700 1 --2- 172.21.15.123:0/132412024 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55b61bb0c580 0x55b61bb0d9f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7eaa3b28397c9d61 server_cookie=7c4f254c6ef8b55e in_seq=0 out_seq=0 2024-05-08T06:10:02.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.529+0000 7fe84f400700 10 client.14904 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:10:02.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.530+0000 7fe84f400700 1 -- 172.21.15.123:0/132412024 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b61bac0050 con 0x55b61bb0c580 2024-05-08T06:10:02.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.530+0000 7fe84f400700 10 client.14904 handle_command_reply: tid=0 2024-05-08T06:10:02.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.530+0000 7fe87aef9b80 1 client.14904 shutdown 2024-05-08T06:10:02.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.530+0000 7fe87aef9b80 1 -- 172.21.15.123:0/132412024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b61baf3010 msgr2=0x55b61bafed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:02.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.530+0000 7fe87aef9b80 1 --2- 172.21.15.123:0/132412024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b61baf3010 0x55b61bafed90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe85c0aab80 tx=0x7fe85c0e2f10 comp rx=0 tx=0).stop 2024-05-08T06:10:02.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.530+0000 7fe87aef9b80 1 -- 172.21.15.123:0/132412024 shutdown_connections 2024-05-08T06:10:02.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.530+0000 7fe87aef9b80 1 --2- 172.21.15.123:0/132412024 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55b61bb0c580 0x55b61bb0d9f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:02.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.530+0000 7fe87aef9b80 1 --2- 172.21.15.123:0/132412024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b61baf3010 0x55b61bafed90 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:02.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.530+0000 7fe87aef9b80 1 --2- 172.21.15.123:0/132412024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b61baf7980 0x55b61bafc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:02.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.530+0000 7fe87aef9b80 1 --2- 172.21.15.123:0/132412024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b61baf3930 0x55b61bafc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:02.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.530+0000 7fe87aef9b80 1 -- 172.21.15.123:0/132412024 >> 172.21.15.123:0/132412024 conn(0x55b61ba21b10 msgr2=0x55b61baed710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:02.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.531+0000 7fe84f400700 0 client.14904 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:10:02.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.531+0000 7fe87aef9b80 1 -- 172.21.15.123:0/132412024 shutdown_connections 2024-05-08T06:10:02.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.531+0000 7fe87aef9b80 1 -- 172.21.15.123:0/132412024 wait complete. 2024-05-08T06:10:02.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.531+0000 7fe87aef9b80 20 client.14904 trim_cache size 0 max 16384 2024-05-08T06:10:02.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.554+0000 7fe87aef9b80 1 Processor -- start 2024-05-08T06:10:02.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.554+0000 7fe87aef9b80 1 -- start start 2024-05-08T06:10:02.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.555+0000 7fe87aef9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b61baf3010 0x55b61bafc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:02.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.555+0000 7fe87aef9b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b61baf3930 0x55b61bafcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:02.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.555+0000 7fe87aef9b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b61baf7980 0x55b61baf5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:02.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.555+0000 7fe87aef9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b61bb0bd30 con 0x55b61baf3010 2024-05-08T06:10:02.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.555+0000 7fe87aef9b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b61bb0c400 con 0x55b61baf3930 2024-05-08T06:10:02.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.555+0000 7fe87aef9b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b61bb0cad0 con 0x55b61baf7980 2024-05-08T06:10:02.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.555+0000 7fe86dc00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b61baf3010 0x55b61bafc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:02.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.555+0000 7fe86e600700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b61baf7980 0x55b61baf5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:02.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.555+0000 7fe86e600700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b61baf7980 0x55b61baf5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:34044/0 (socket says 172.21.15.123:34044) 2024-05-08T06:10:02.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.555+0000 7fe86e600700 1 -- 172.21.15.123:0/413688462 learned_addr learned my addr 172.21.15.123:0/413688462 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:10:02.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.555+0000 7fe86d200700 1 --2- 172.21.15.123:0/413688462 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b61baf3930 0x55b61bafcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:02.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.555+0000 7fe86e600700 1 -- 172.21.15.123:0/413688462 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b61baf3930 msgr2=0x55b61bafcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:02.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.555+0000 7fe86e600700 1 --2- 172.21.15.123:0/413688462 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b61baf3930 0x55b61bafcd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:02.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.555+0000 7fe86e600700 1 -- 172.21.15.123:0/413688462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b61baf3010 msgr2=0x55b61bafc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:02.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.555+0000 7fe86e600700 1 --2- 172.21.15.123:0/413688462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b61baf3010 0x55b61bafc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:02.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.555+0000 7fe86e600700 1 -- 172.21.15.123:0/413688462 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b61bafa6c0 con 0x55b61baf7980 2024-05-08T06:10:02.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.556+0000 7fe86e600700 1 --2- 172.21.15.123:0/413688462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b61baf7980 0x55b61baf5340 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe8600041b0 tx=0x7fe860009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5840999c9148dddc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:02.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.556+0000 7fe85a000700 1 -- 172.21.15.123:0/413688462 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86000eea0 con 0x55b61baf7980 2024-05-08T06:10:02.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.556+0000 7fe85a000700 1 -- 172.21.15.123:0/413688462 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe860004db0 con 0x55b61baf7980 2024-05-08T06:10:02.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.556+0000 7fe85a000700 1 -- 172.21.15.123:0/413688462 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86001b740 con 0x55b61baf7980 2024-05-08T06:10:02.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.557+0000 7fe87aef9b80 1 -- 172.21.15.123:0/413688462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b61baf7980 msgr2=0x55b61baf5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:02.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.557+0000 7fe87aef9b80 1 --2- 172.21.15.123:0/413688462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b61baf7980 0x55b61baf5340 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe8600041b0 tx=0x7fe860009920 comp rx=0 tx=0).stop 2024-05-08T06:10:02.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.557+0000 7fe87aef9b80 1 -- 172.21.15.123:0/413688462 shutdown_connections 2024-05-08T06:10:02.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.557+0000 7fe87aef9b80 1 --2- 172.21.15.123:0/413688462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b61baf3010 0x55b61bafc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:02.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.557+0000 7fe87aef9b80 1 --2- 172.21.15.123:0/413688462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b61baf7980 0x55b61baf5340 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:02.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.557+0000 7fe87aef9b80 1 --2- 172.21.15.123:0/413688462 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b61baf3930 0x55b61bafcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:02.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.557+0000 7fe87aef9b80 1 -- 172.21.15.123:0/413688462 >> 172.21.15.123:0/413688462 conn(0x55b61ba21b10 msgr2=0x55b61baed6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:02.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.557+0000 7fe87aef9b80 1 -- 172.21.15.123:0/413688462 shutdown_connections 2024-05-08T06:10:02.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.558+0000 7fe87aef9b80 1 -- 172.21.15.123:0/413688462 wait complete. 2024-05-08T06:10:02.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.558+0000 7fe87aef9b80 1 -- start start 2024-05-08T06:10:02.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.558+0000 7fe87aef9b80 1 Processor -- start 2024-05-08T06:10:02.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.559+0000 7fe87aef9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b61baf3010 0x55b61bb16ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:02.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.559+0000 7fe87aef9b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b61baf3930 0x55b61bb17430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:02.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.559+0000 7fe87aef9b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b61baf7980 0x55b61bb112e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:02.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.559+0000 7fe87aef9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b61bb0d870 con 0x55b61baf3010 2024-05-08T06:10:02.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.559+0000 7fe87aef9b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b61bb0d1a0 con 0x55b61baf3930 2024-05-08T06:10:02.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.559+0000 7fe87aef9b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b61bb0df40 con 0x55b61baf7980 2024-05-08T06:10:02.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.559+0000 7fe87aef9b80 10 client.0 _ll_register_callbacks cb 0x55b61b9ecc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:10:02.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.559+0000 7fe86dc00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b61baf3010 0x55b61bb16ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:02.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.559+0000 7fe86dc00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b61baf3010 0x55b61bb16ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55270/0 (socket says 172.21.15.123:55270) 2024-05-08T06:10:02.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.559+0000 7fe86dc00700 1 -- 172.21.15.123:0/180916541 learned_addr learned my addr 172.21.15.123:0/180916541 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:10:02.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.559+0000 7fe86d200700 1 --2- 172.21.15.123:0/180916541 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b61baf3930 0x55b61bb17430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:02.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.559+0000 7fe86e600700 1 --2- 172.21.15.123:0/180916541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b61baf7980 0x55b61bb112e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:02.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.559+0000 7fe86e600700 1 -- 172.21.15.123:0/180916541 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b61baf3930 msgr2=0x55b61bb17430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:02.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.559+0000 7fe86e600700 1 --2- 172.21.15.123:0/180916541 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b61baf3930 0x55b61bb17430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:02.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.559+0000 7fe86e600700 1 -- 172.21.15.123:0/180916541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b61baf3010 msgr2=0x55b61bb16ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:02.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.559+0000 7fe86e600700 1 --2- 172.21.15.123:0/180916541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b61baf3010 0x55b61bb16ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:02.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.559+0000 7fe86e600700 1 -- 172.21.15.123:0/180916541 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b61bb11b20 con 0x55b61baf7980 2024-05-08T06:10:02.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.559+0000 7fe86dc00700 1 --2- 172.21.15.123:0/180916541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b61baf3010 0x55b61bb16ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:10:02.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.560+0000 7fe86e600700 1 --2- 172.21.15.123:0/180916541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b61baf7980 0x55b61bb112e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe860004b10 tx=0x7fe860007b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f852d8462ca268bc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:02.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.560+0000 7fe84f400700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-05-08T06:10:02.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.560+0000 7fe84f400700 1 -- 172.21.15.123:0/180916541 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86000eea0 con 0x55b61baf7980 2024-05-08T06:10:02.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.560+0000 7fe87aef9b80 1 -- 172.21.15.123:0/180916541 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe860004db0 con 0x55b61baf7980 2024-05-08T06:10:02.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.560+0000 7fe84f400700 1 -- 172.21.15.123:0/180916541 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fe86001b660 con 0x55b61baf7980 2024-05-08T06:10:02.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.560+0000 7fe84f400700 1 -- 172.21.15.123:0/180916541 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe86001b880 con 0x55b61baf7980 2024-05-08T06:10:02.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.560+0000 7fe87aef9b80 10 client.25018 fetch_fsmap learned FSMap version 23 2024-05-08T06:10:02.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.560+0000 7fe87aef9b80 1 -- 172.21.15.123:0/180916541 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55b61bb80dc0 con 0x55b61baf7980 2024-05-08T06:10:02.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.561+0000 7fe84f400700 1 -- 172.21.15.123:0/180916541 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe860004db0 con 0x55b61baf7980 2024-05-08T06:10:02.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.561+0000 7fe84f400700 1 -- 172.21.15.123:0/180916541 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fe860026030 con 0x55b61baf7980 2024-05-08T06:10:02.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.561+0000 7fe87aef9b80 10 client.25018 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T06:10:02.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.561+0000 7fe87aef9b80 10 client.25018 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T06:10:02.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.561+0000 7fe87aef9b80 20 client.25018 populate_metadata read hostname 'smithi123' 2024-05-08T06:10:02.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.561+0000 7fe87aef9b80 1 --2- 172.21.15.123:0/180916541 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55b61bb856b0 0x55b61bb843c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:02.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.561+0000 7fe87aef9b80 4 client.25018 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T06:10:02.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.561+0000 7fe87aef9b80 1 -- 172.21.15.123:0/180916541 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "status"}) -- 0x55b61ba533b0 con 0x55b61bb856b0 2024-05-08T06:10:02.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.561+0000 7fe86dc00700 1 --2- 172.21.15.123:0/180916541 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55b61bb856b0 0x55b61bb843c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:02.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.562+0000 7fe86dc00700 1 --2- 172.21.15.123:0/180916541 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55b61bb856b0 0x55b61bb843c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b735c297f820d10e server_cookie=b58e70f1defb4360 in_seq=0 out_seq=0 2024-05-08T06:10:02.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.562+0000 7fe84f400700 10 client.25018 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:10:02.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.562+0000 7fe84f400700 1 -- 172.21.15.123:0/180916541 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b61ba533b0 con 0x55b61bb856b0 2024-05-08T06:10:02.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.562+0000 7fe84f400700 10 client.25018 handle_command_reply: tid=0 2024-05-08T06:10:02.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.562+0000 7fe87aef9b80 1 client.25018 shutdown 2024-05-08T06:10:02.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.562+0000 7fe87aef9b80 1 -- 172.21.15.123:0/180916541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b61baf7980 msgr2=0x55b61bb112e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:02.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.562+0000 7fe87aef9b80 1 --2- 172.21.15.123:0/180916541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b61baf7980 0x55b61bb112e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe860004b10 tx=0x7fe860007b90 comp rx=0 tx=0).stop 2024-05-08T06:10:02.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.563+0000 7fe87aef9b80 1 -- 172.21.15.123:0/180916541 shutdown_connections 2024-05-08T06:10:02.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.563+0000 7fe87aef9b80 1 --2- 172.21.15.123:0/180916541 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55b61bb856b0 0x55b61bb843c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:02.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.563+0000 7fe87aef9b80 1 --2- 172.21.15.123:0/180916541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b61baf3010 0x55b61bb16ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:02.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.563+0000 7fe87aef9b80 1 --2- 172.21.15.123:0/180916541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b61baf7980 0x55b61bb112e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:02.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.563+0000 7fe87aef9b80 1 --2- 172.21.15.123:0/180916541 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b61baf3930 0x55b61bb17430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:02.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.563+0000 7fe84f400700 0 client.25018 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:10:02.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.563+0000 7fe87aef9b80 1 -- 172.21.15.123:0/180916541 >> 172.21.15.123:0/180916541 conn(0x55b61ba21b10 msgr2=0x55b61bae79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:02.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.563+0000 7fe87aef9b80 1 -- 172.21.15.123:0/180916541 shutdown_connections 2024-05-08T06:10:02.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.563+0000 7fe87aef9b80 1 -- 172.21.15.123:0/180916541 wait complete. 2024-05-08T06:10:02.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.563+0000 7fe87aef9b80 20 client.25018 trim_cache size 0 max 16384 2024-05-08T06:10:02.566 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-05-08T06:10:02.566 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_fsid": "85cfb2e6-0cff-11ef-bc97-c7b262605968", 2024-05-08T06:10:02.566 INFO:teuthology.orchestra.run.smithi123.stdout: "whoami": 0, 2024-05-08T06:10:02.567 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 24524, 2024-05-08T06:10:02.567 INFO:teuthology.orchestra.run.smithi123.stdout: "want_state": "up:active", 2024-05-08T06:10:02.567 INFO:teuthology.orchestra.run.smithi123.stdout: "state": "up:active", 2024-05-08T06:10:02.567 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs", 2024-05-08T06:10:02.567 INFO:teuthology.orchestra.run.smithi123.stdout: "rank_uptime": 247.13127524500001, 2024-05-08T06:10:02.567 INFO:teuthology.orchestra.run.smithi123.stdout: "mdsmap_epoch": 23, 2024-05-08T06:10:02.567 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch": 82, 2024-05-08T06:10:02.567 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch_barrier": 80, 2024-05-08T06:10:02.567 INFO:teuthology.orchestra.run.smithi123.stdout: "uptime": 272.76238915099998 2024-05-08T06:10:02.567 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-05-08T06:10:02.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.564+0000 7fe870600700 1 -- 172.21.15.123:0/530254851 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe8500b2900 msgr2=0x7fe8500b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:02.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.564+0000 7fe870600700 1 --2- 172.21.15.123:0/530254851 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe8500b2900 0x7fe8500b4dc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe860003fd0 tx=0x7fe860004b40 comp rx=0 tx=0).stop 2024-05-08T06:10:02.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.564+0000 7fe870600700 1 -- 172.21.15.123:0/530254851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8681056c0 msgr2=0x7fe86819bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:02.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.564+0000 7fe870600700 1 --2- 172.21.15.123:0/530254851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8681056c0 0x7fe86819bd50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe85c0057f0 tx=0x7fe85c00bfd0 comp rx=0 tx=0).stop 2024-05-08T06:10:02.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.564+0000 7fe870600700 1 -- 172.21.15.123:0/530254851 shutdown_connections 2024-05-08T06:10:02.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.564+0000 7fe870600700 1 --2- 172.21.15.123:0/530254851 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe8500b2900 0x7fe8500b4dc0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:02.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.564+0000 7fe870600700 1 --2- 172.21.15.123:0/530254851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe86810b6c0 0x7fe8681a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:02.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.564+0000 7fe870600700 1 --2- 172.21.15.123:0/530254851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8681056c0 0x7fe86819bd50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:02.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.564+0000 7fe870600700 1 --2- 172.21.15.123:0/530254851 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe868106080 0x7fe86819c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:02.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.564+0000 7fe870600700 1 -- 172.21.15.123:0/530254851 >> 172.21.15.123:0/530254851 conn(0x7fe8681011b0 msgr2=0x7fe868101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:02.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.564+0000 7fe870600700 1 -- 172.21.15.123:0/530254851 shutdown_connections 2024-05-08T06:10:02.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:02.564+0000 7fe870600700 1 -- 172.21.15.123:0/530254851 wait complete. 2024-05-08T06:10:02.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:02 smithi123 ceph-mon[29787]: pgmap v447: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:10:03.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:02 smithi163 ceph-mon[34520]: pgmap v447: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:10:03.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:02 smithi192 ceph-mon[34664]: pgmap v447: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:10:03.676 INFO:teuthology.run_tasks:Running task sequential... 2024-05-08T06:10:03.685 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-08T06:10:03.685 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-08T06:10:03.686 INFO:teuthology.task.print:Enabling mgr modules 2024-05-08T06:10:03.686 INFO:teuthology.run_tasks:Running task kclient... 2024-05-08T06:10:03.723 INFO:tasks.kclient:Mounting kernel clients... 2024-05-08T06:10:03.723 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-08T06:10:03.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:10:03.724 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2024-05-08T06:10:03.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:10:03.783 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link delete ceph-brx 2024-05-08T06:10:03.873 INFO:teuthology.orchestra.run.smithi123.stderr:Cannot find device "ceph-brx" 2024-05-08T06:10:03.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T06:10:03.878 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-05-08T06:10:03.878 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T06:10:03.878 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T06:10:03.878 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-08T06:10:04.327 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:10:05.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:04 smithi163 ceph-mon[34520]: pgmap v448: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:10:05.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:04 smithi192 ceph-mon[34664]: pgmap v448: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:10:05.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:04 smithi123 ceph-mon[29787]: pgmap v448: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T06:10:06.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.747+0000 7f9eafe00700 1 -- 172.21.15.123:0/2339376774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ea8104100 msgr2=0x7f9ea8108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:06.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.747+0000 7f9eafe00700 1 --2- 172.21.15.123:0/2339376774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ea8104100 0x7f9ea8108120 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9e940098f0 tx=0x7f9e94009c00 comp rx=0 tx=0).stop 2024-05-08T06:10:06.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.747+0000 7f9eafe00700 1 -- 172.21.15.123:0/2339376774 shutdown_connections 2024-05-08T06:10:06.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.747+0000 7f9eafe00700 1 --2- 172.21.15.123:0/2339376774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ea8103750 0x7f9ea8103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:06.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.747+0000 7f9eafe00700 1 --2- 172.21.15.123:0/2339376774 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ea8108850 0x7f9ea81103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:06.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.747+0000 7f9eafe00700 1 --2- 172.21.15.123:0/2339376774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ea8104100 0x7f9ea8108120 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:06.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.747+0000 7f9eafe00700 1 -- 172.21.15.123:0/2339376774 >> 172.21.15.123:0/2339376774 conn(0x7f9ea807a600 msgr2=0x7f9ea807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:06.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.748+0000 7f9eafe00700 1 -- 172.21.15.123:0/2339376774 shutdown_connections 2024-05-08T06:10:06.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.748+0000 7f9eafe00700 1 -- 172.21.15.123:0/2339376774 wait complete. 2024-05-08T06:10:06.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.748+0000 7f9eafe00700 1 Processor -- start 2024-05-08T06:10:06.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.749+0000 7f9eafe00700 1 -- start start 2024-05-08T06:10:06.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.749+0000 7f9eafe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ea8103750 0x7f9ea80757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:06.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.749+0000 7f9eafe00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ea8104100 0x7f9ea8075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:06.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.749+0000 7f9eafe00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ea8108850 0x7f9ea8076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:06.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.749+0000 7f9eafe00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea8113680 con 0x7f9ea8103750 2024-05-08T06:10:06.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.749+0000 7f9eafe00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea8113500 con 0x7f9ea8108850 2024-05-08T06:10:06.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.749+0000 7f9eafe00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea8113800 con 0x7f9ea8104100 2024-05-08T06:10:06.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.749+0000 7f9ead400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ea8103750 0x7f9ea80757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:06.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.749+0000 7f9ead400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ea8103750 0x7f9ea80757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59402/0 (socket says 172.21.15.123:59402) 2024-05-08T06:10:06.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.749+0000 7f9ead400700 1 -- 172.21.15.123:0/3741322157 learned_addr learned my addr 172.21.15.123:0/3741322157 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:10:06.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.750+0000 7f9eade00700 1 --2- 172.21.15.123:0/3741322157 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ea8108850 0x7f9ea8076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:06.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.750+0000 7f9eaca00700 1 --2- 172.21.15.123:0/3741322157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ea8104100 0x7f9ea8075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:06.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.750+0000 7f9ead400700 1 -- 172.21.15.123:0/3741322157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ea8104100 msgr2=0x7f9ea8075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:06.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.750+0000 7f9ead400700 1 --2- 172.21.15.123:0/3741322157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ea8104100 0x7f9ea8075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:06.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.750+0000 7f9ead400700 1 -- 172.21.15.123:0/3741322157 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ea8108850 msgr2=0x7f9ea8076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:06.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.750+0000 7f9ead400700 1 --2- 172.21.15.123:0/3741322157 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ea8108850 0x7f9ea8076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:06.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.750+0000 7f9ead400700 1 -- 172.21.15.123:0/3741322157 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea81a8090 con 0x7f9ea8103750 2024-05-08T06:10:06.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.750+0000 7f9ead400700 1 --2- 172.21.15.123:0/3741322157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ea8103750 0x7f9ea80757a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00e3c0 tx=0x7f9e9c00e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:06.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.750+0000 7f9ea6000700 1 -- 172.21.15.123:0/3741322157 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c00ee40 con 0x7f9ea8103750 2024-05-08T06:10:06.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.751+0000 7f9ea6000700 1 -- 172.21.15.123:0/3741322157 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e9c003a00 con 0x7f9ea8103750 2024-05-08T06:10:06.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.751+0000 7f9ea6000700 1 -- 172.21.15.123:0/3741322157 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c004720 con 0x7f9ea8103750 2024-05-08T06:10:06.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.751+0000 7f9eafe00700 1 -- 172.21.15.123:0/3741322157 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ea81a8320 con 0x7f9ea8103750 2024-05-08T06:10:06.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.751+0000 7f9eafe00700 1 -- 172.21.15.123:0/3741322157 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ea81a87e0 con 0x7f9ea8103750 2024-05-08T06:10:06.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.752+0000 7f9eafe00700 1 -- 172.21.15.123:0/3741322157 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ea806a190 con 0x7f9ea8103750 2024-05-08T06:10:06.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.754+0000 7f9ea6000700 1 -- 172.21.15.123:0/3741322157 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f9e9c01b030 con 0x7f9ea8103750 2024-05-08T06:10:06.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.756+0000 7f9ea6000700 1 --2- 172.21.15.123:0/3741322157 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9e900b2900 0x7f9e900b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:06.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.756+0000 7f9ea6000700 1 -- 172.21.15.123:0/3741322157 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f9e9c0e1ad0 con 0x7f9ea8103750 2024-05-08T06:10:06.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.756+0000 7f9eaca00700 1 --2- 172.21.15.123:0/3741322157 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9e900b2900 0x7f9e900b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:06.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.757+0000 7f9eaca00700 1 --2- 172.21.15.123:0/3741322157 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9e900b2900 0x7f9e900b4dc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9e940098c0 tx=0x7f9e94019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:06.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:06.760+0000 7f9ea6000700 1 -- 172.21.15.123:0/3741322157 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9e9c022060 con 0x7f9ea8103750 2024-05-08T06:10:06.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:06 smithi123 ceph-mon[29787]: pgmap v449: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:10:07.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:06 smithi163 ceph-mon[34520]: pgmap v449: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:10:07.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:07.045+0000 7f9eafe00700 1 -- 172.21.15.123:0/3741322157 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9ea802e770 con 0x7f9ea8103750 2024-05-08T06:10:07.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:07.046+0000 7f9ea6000700 1 -- 172.21.15.123:0/3741322157 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) ==== 53+0+73 (secure 0 0 0) 0x7f9e9c010040 con 0x7f9ea8103750 2024-05-08T06:10:07.046 INFO:teuthology.orchestra.run.smithi123.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T06:10:07.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:07.049+0000 7f9eafe00700 1 -- 172.21.15.123:0/3741322157 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9e900b2900 msgr2=0x7f9e900b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:07.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:07.049+0000 7f9eafe00700 1 --2- 172.21.15.123:0/3741322157 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9e900b2900 0x7f9e900b4dc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9e940098c0 tx=0x7f9e94019040 comp rx=0 tx=0).stop 2024-05-08T06:10:07.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:07.049+0000 7f9eafe00700 1 -- 172.21.15.123:0/3741322157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ea8103750 msgr2=0x7f9ea80757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:07.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:07.049+0000 7f9eafe00700 1 --2- 172.21.15.123:0/3741322157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ea8103750 0x7f9ea80757a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00e3c0 tx=0x7f9e9c00e6d0 comp rx=0 tx=0).stop 2024-05-08T06:10:07.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:07.050+0000 7f9eafe00700 1 -- 172.21.15.123:0/3741322157 shutdown_connections 2024-05-08T06:10:07.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:07.050+0000 7f9eafe00700 1 --2- 172.21.15.123:0/3741322157 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9e900b2900 0x7f9e900b4dc0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:07.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:07.050+0000 7f9eafe00700 1 --2- 172.21.15.123:0/3741322157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ea8103750 0x7f9ea80757a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:07.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:07.050+0000 7f9eafe00700 1 --2- 172.21.15.123:0/3741322157 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ea8108850 0x7f9ea8076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:07.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:07.050+0000 7f9eafe00700 1 --2- 172.21.15.123:0/3741322157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ea8104100 0x7f9ea8075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:07.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:07.050+0000 7f9eafe00700 1 -- 172.21.15.123:0/3741322157 >> 172.21.15.123:0/3741322157 conn(0x7f9ea807a600 msgr2=0x7f9ea8101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:07.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:07.050+0000 7f9eafe00700 1 -- 172.21.15.123:0/3741322157 shutdown_connections 2024-05-08T06:10:07.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:07.050+0000 7f9eafe00700 1 -- 172.21.15.123:0/3741322157 wait complete. 2024-05-08T06:10:07.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:06 smithi192 ceph-mon[34664]: pgmap v449: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:10:08.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:07 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/3741322157' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T06:10:08.040 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:07 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/3741322157' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T06:10:08.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:07 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/3741322157' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T06:10:08.114 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T06:10:08.114 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T06:10:08.115 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi123.front.sepia.ceph.com 2024-05-08T06:10:08.115 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T06:10:08.115 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T06:10:08.115 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T06:10:08.115 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T06:10:08.115 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T06:10:08.115 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T06:10:08.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:10:08.115 DEBUG:teuthology.orchestra.run.smithi123:> ip addr 2024-05-08T06:10:08.172 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T06:10:08.172 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T06:10:08.172 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T06:10:08.172 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:10:08.172 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2024-05-08T06:10:08.172 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:10:08.173 INFO:teuthology.orchestra.run.smithi123.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T06:10:08.173 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6c brd ff:ff:ff:ff:ff:ff 2024-05-08T06:10:08.173 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f0 2024-05-08T06:10:08.173 INFO:teuthology.orchestra.run.smithi123.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T06:10:08.173 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6d brd ff:ff:ff:ff:ff:ff 2024-05-08T06:10:08.173 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f1 2024-05-08T06:10:08.173 INFO:teuthology.orchestra.run.smithi123.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T06:10:08.173 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f0 brd ff:ff:ff:ff:ff:ff 2024-05-08T06:10:08.173 INFO:teuthology.orchestra.run.smithi123.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T06:10:08.173 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f1 brd ff:ff:ff:ff:ff:ff 2024-05-08T06:10:08.173 INFO:teuthology.orchestra.run.smithi123.stdout: inet 172.21.15.123/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-08T06:10:08.173 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft 41828sec preferred_lft 41828sec 2024-05-08T06:10:08.173 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::ec4:7aff:fe8f:d3f1/64 scope link 2024-05-08T06:10:08.173 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-05-08T06:10:08.174 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-08T06:10:08.175 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T06:10:08.175 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-05-08T06:10:08.175 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add name ceph-brx type bridge 2024-05-08T06:10:08.175 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip addr flush dev ceph-brx 2024-05-08T06:10:08.175 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set ceph-brx up 2024-05-08T06:10:08.175 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-08T06:10:08.175 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-05-08T06:10:08.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T06:10:08.649 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:10:08.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:10:08.658 DEBUG:teuthology.orchestra.run.smithi123:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T06:10:08.728 INFO:teuthology.orchestra.run.smithi123.stdout:1 2024-05-08T06:10:08.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:10:08.731 DEBUG:teuthology.orchestra.run.smithi123:> route 2024-05-08T06:10:08.781 INFO:teuthology.orchestra.run.smithi123.stdout:Kernel IP routing table 2024-05-08T06:10:08.781 INFO:teuthology.orchestra.run.smithi123.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T06:10:08.781 INFO:teuthology.orchestra.run.smithi123.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T06:10:08.781 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-08T06:10:08.781 INFO:teuthology.orchestra.run.smithi123.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T06:10:08.783 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T06:10:08.783 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-05-08T06:10:08.783 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T06:10:08.783 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T06:10:08.783 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T06:10:08.783 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-05-08T06:10:08.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T06:10:08.903 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:08 smithi123 ceph-mon[29787]: pgmap v450: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:10:09.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:08 smithi163 ceph-mon[34520]: pgmap v450: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:10:09.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:08 smithi192 ceph-mon[34664]: pgmap v450: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:10:09.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:10:09.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:10:09.621 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2024-05-08T06:10:09.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:10:09.674 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list-id 2024-05-08T06:10:09.724 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T06:10:09.724 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-05-08T06:10:09.724 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T06:10:09.724 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-08T06:10:09.724 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-05-08T06:10:09.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T06:10:09.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:10:09.916 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-08T06:10:09.917 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T06:10:09.917 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-05-08T06:10:09.917 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-08T06:10:09.917 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-08T06:10:09.917 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-08T06:10:09.917 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-08T06:10:09.917 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-08T06:10:09.917 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-05-08T06:10:10.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T06:10:10.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:10:10.211 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T06:10:10.211 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-05-08T06:10:10.211 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set brx.0 up 2024-05-08T06:10:10.211 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set dev brx.0 master ceph-brx 2024-05-08T06:10:10.211 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-05-08T06:10:10.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T06:10:10.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:10:10.412 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T06:10:10.412 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T06:10:10.456 INFO:teuthology.orchestra.run.smithi123.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T06:10:10.456 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T06:10:10.456 DEBUG:teuthology.orchestra.run.smithi123:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T06:10:10.551 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-08T06:10:10.551 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-08T06:10:10.551 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/2af9b403-9f5f-4bf9-875d-231916530d9c 2024-05-08T06:10:10.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:10:10.552 DEBUG:teuthology.orchestra.run.smithi123:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/2af9b403-9f5f-4bf9-875d-231916530d9c /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-05-08T06:10:10.718 INFO:teuthology.orchestra.run.smithi123.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-05-08T06:10:10.719 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-05-08T06:10:11.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:10 smithi163 ceph-mon[34520]: pgmap v451: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:10:11.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:10 smithi192 ceph-mon[34664]: pgmap v451: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:10:11.175 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:10 smithi123 ceph-mon[29787]: pgmap v451: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:10:11.449 INFO:teuthology.orchestra.run.smithi123.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-05-08T06:10:11.449 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-05-08T06:10:11.449 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: resolved to: "172.21.15.123:3300,172.21.15.163:3300,172.21.15.192:3300" 2024-05-08T06:10:11.449 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: trying mount with new device syntax: 0@85cfb2e6-0cff-11ef-bc97-c7b262605968.cephfs=/volumes/qa/sv_1/2af9b403-9f5f-4bf9-875d-231916530d9c 2024-05-08T06:10:11.450 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync,name=0,key=0,mon_addr=172.21.15.123:3300/172.21.15.163:3300/172.21.15.192:3300" will pass to kernel 2024-05-08T06:10:11.457 INFO:teuthology.orchestra.run.smithi123.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-08T06:10:11.457 INFO:teuthology.orchestra.run.smithi123.stdout: You just mounted a file system that supports labels which does not 2024-05-08T06:10:11.457 INFO:teuthology.orchestra.run.smithi123.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-08T06:10:11.457 INFO:teuthology.orchestra.run.smithi123.stdout: applications will generate AVC messages and not be allowed access to 2024-05-08T06:10:11.457 INFO:teuthology.orchestra.run.smithi123.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-08T06:10:11.463 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T06:10:11.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:10:11.463 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T06:10:11.552 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T06:10:11.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T06:10:11.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:10:11.677 INFO:teuthology.orchestra.run.smithi123.stdout:client25027 2024-05-08T06:10:11.677 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-05-08T06:10:11.678 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T06:10:11.734 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T06:10:11.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T06:10:11.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:10:11.855 INFO:teuthology.orchestra.run.smithi123.stdout:client25027 2024-05-08T06:10:11.855 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T06:10:11.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T06:10:11.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:10:11.971 INFO:teuthology.orchestra.run.smithi123.stdout:client25027 2024-05-08T06:10:11.971 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968.client25027/status') 2024-05-08T06:10:12.076 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T06:10:12.105 INFO:teuthology.orchestra.run.smithi123.stdout:instance: client.25027 (3)172.21.15.123:0/3473506830 2024-05-08T06:10:12.106 INFO:teuthology.orchestra.run.smithi123.stdout:blocklisted: false 2024-05-08T06:10:12.106 INFO:teuthology.orchestra.run.smithi123.stderr:0+1 records in 2024-05-08T06:10:12.106 INFO:teuthology.orchestra.run.smithi123.stderr:0+1 records out 2024-05-08T06:10:12.106 INFO:teuthology.orchestra.run.smithi123.stderr:72 bytes copied, 0.000153346 s, 470 kB/s 2024-05-08T06:10:12.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T06:10:12.119 INFO:teuthology.run_tasks:Running task exec... 2024-05-08T06:10:12.133 INFO:teuthology.task.exec:Executing custom commands... 2024-05-08T06:10:12.133 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi123.front.sepia.ceph.com 2024-05-08T06:10:12.134 DEBUG:teuthology.orchestra.run.smithi123:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-05-08T06:10:12.337 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.337+0000 7f5cdfe00640 1 -- 172.21.15.123:0/3466463423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ce0109fb0 msgr2=0x7f5ce010a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:12.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.337+0000 7f5cdfe00640 1 --2- 172.21.15.123:0/3466463423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ce0109fb0 0x7f5ce010a430 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc009860 tx=0x7f5ccc02a340 comp rx=0 tx=0).stop 2024-05-08T06:10:12.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.337+0000 7f5cdfe00640 1 -- 172.21.15.123:0/3466463423 shutdown_connections 2024-05-08T06:10:12.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.337+0000 7f5cdfe00640 1 --2- 172.21.15.123:0/3466463423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ce010a970 0x7f5ce0111290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:12.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.337+0000 7f5cdfe00640 1 --2- 172.21.15.123:0/3466463423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ce0109fb0 0x7f5ce010a430 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:12.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.337+0000 7f5cdfe00640 1 --2- 172.21.15.123:0/3466463423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ce0108e60 0x7f5ce0109280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:12.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.337+0000 7f5cdfe00640 1 -- 172.21.15.123:0/3466463423 >> 172.21.15.123:0/3466463423 conn(0x7f5ce0077e40 msgr2=0x7f5ce007a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:12.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.337+0000 7f5cdfe00640 1 -- 172.21.15.123:0/3466463423 shutdown_connections 2024-05-08T06:10:12.338 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.337+0000 7f5cdfe00640 1 -- 172.21.15.123:0/3466463423 wait complete. 2024-05-08T06:10:12.340 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.338+0000 7f5cdfe00640 1 Processor -- start 2024-05-08T06:10:12.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.339+0000 7f5cdfe00640 1 -- start start 2024-05-08T06:10:12.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.339+0000 7f5cdfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ce0108e60 0x7f5ce019f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:12.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.339+0000 7f5cdfe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ce0109fb0 0x7f5ce019f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:12.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.339+0000 7f5cdfe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ce010a970 0x7f5ce01a6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:12.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.339+0000 7f5cdfe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce01149c0 con 0x7f5ce0108e60 2024-05-08T06:10:12.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.339+0000 7f5cdfe00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce0114840 con 0x7f5ce010a970 2024-05-08T06:10:12.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.339+0000 7f5cdfe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce0114b40 con 0x7f5ce0109fb0 2024-05-08T06:10:12.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.339+0000 7f5cdea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ce0108e60 0x7f5ce019f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:12.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.339+0000 7f5cdea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ce0108e60 0x7f5ce019f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59416/0 (socket says 172.21.15.123:59416) 2024-05-08T06:10:12.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.339+0000 7f5cdea00640 1 -- 172.21.15.123:0/695732091 learned_addr learned my addr 172.21.15.123:0/695732091 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:10:12.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.339+0000 7f5cdf400640 1 --2- 172.21.15.123:0/695732091 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ce010a970 0x7f5ce01a6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:12.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.340+0000 7f5cde000640 1 --2- 172.21.15.123:0/695732091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ce0109fb0 0x7f5ce019f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:12.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.340+0000 7f5cdf400640 1 -- 172.21.15.123:0/695732091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ce0109fb0 msgr2=0x7f5ce019f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:12.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.340+0000 7f5cdf400640 1 --2- 172.21.15.123:0/695732091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ce0109fb0 0x7f5ce019f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:12.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.340+0000 7f5cdf400640 1 -- 172.21.15.123:0/695732091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ce0108e60 msgr2=0x7f5ce019f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:12.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.340+0000 7f5cdf400640 1 --2- 172.21.15.123:0/695732091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ce0108e60 0x7f5ce019f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:12.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.340+0000 7f5cdf400640 1 -- 172.21.15.123:0/695732091 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce01a6c90 con 0x7f5ce010a970 2024-05-08T06:10:12.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.340+0000 7f5cdea00640 1 --2- 172.21.15.123:0/695732091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ce0108e60 0x7f5ce019f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:10:12.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.340+0000 7f5cdf400640 1 --2- 172.21.15.123:0/695732091 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ce010a970 0x7f5ce01a6620 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8009fd0 tx=0x7f5cd800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:12.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.340+0000 7f5cd7400640 1 -- 172.21.15.123:0/695732091 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd800ce20 con 0x7f5ce010a970 2024-05-08T06:10:12.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.341+0000 7f5cdfe00640 1 -- 172.21.15.123:0/695732091 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ce01a6f80 con 0x7f5ce010a970 2024-05-08T06:10:12.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.341+0000 7f5cd7400640 1 -- 172.21.15.123:0/695732091 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cd8005020 con 0x7f5ce010a970 2024-05-08T06:10:12.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.341+0000 7f5cd7400640 1 -- 172.21.15.123:0/695732091 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd8010690 con 0x7f5ce010a970 2024-05-08T06:10:12.342 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.341+0000 7f5cdfe00640 1 -- 172.21.15.123:0/695732091 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ce01a7440 con 0x7f5ce010a970 2024-05-08T06:10:12.343 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.342+0000 7f5cdfe00640 1 -- 172.21.15.123:0/695732091 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ce0110170 con 0x7f5ce010a970 2024-05-08T06:10:12.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.344+0000 7f5cd7400640 1 -- 172.21.15.123:0/695732091 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f5cd80054d0 con 0x7f5ce010a970 2024-05-08T06:10:12.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.346+0000 7f5cd7400640 1 --2- 172.21.15.123:0/695732091 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5cc40b27e0 0x7f5cc40b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:12.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.346+0000 7f5cd7400640 1 -- 172.21.15.123:0/695732091 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f5cd80e2c00 con 0x7f5ce010a970 2024-05-08T06:10:12.346 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.346+0000 7f5cdea00640 1 --2- 172.21.15.123:0/695732091 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5cc40b27e0 0x7f5cc40b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:12.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.347+0000 7f5cdea00640 1 --2- 172.21.15.123:0/695732091 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5cc40b27e0 0x7f5cc40b4ca0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5cd00058c0 tx=0x7f5cd0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:12.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.348+0000 7f5cd7400640 1 -- 172.21.15.123:0/695732091 <== mon.2 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5cd80aa8d0 con 0x7f5ce010a970 2024-05-08T06:10:12.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.598+0000 7f5cdfe00640 1 -- 172.21.15.123:0/695732091 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f5ce002db50 con 0x7f5ce010a970 2024-05-08T06:10:12.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.790+0000 7f5cd7400640 1 -- 172.21.15.123:0/695732091 <== mon.2 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v24) ==== 118+0+0 (secure 0 0 0) 0x7f5cd80aa1d0 con 0x7f5ce010a970 2024-05-08T06:10:12.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.794+0000 7f5cdfe00640 1 -- 172.21.15.123:0/695732091 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5cc40b27e0 msgr2=0x7f5cc40b4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:12.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.794+0000 7f5cdfe00640 1 --2- 172.21.15.123:0/695732091 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5cc40b27e0 0x7f5cc40b4ca0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5cd00058c0 tx=0x7f5cd0009260 comp rx=0 tx=0).stop 2024-05-08T06:10:12.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.794+0000 7f5cdfe00640 1 -- 172.21.15.123:0/695732091 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ce010a970 msgr2=0x7f5ce01a6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:12.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.794+0000 7f5cdfe00640 1 --2- 172.21.15.123:0/695732091 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ce010a970 0x7f5ce01a6620 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8009fd0 tx=0x7f5cd800ef10 comp rx=0 tx=0).stop 2024-05-08T06:10:12.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.794+0000 7f5cdfe00640 1 -- 172.21.15.123:0/695732091 shutdown_connections 2024-05-08T06:10:12.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.794+0000 7f5cdfe00640 1 --2- 172.21.15.123:0/695732091 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5cc40b27e0 0x7f5cc40b4ca0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:12.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.794+0000 7f5cdfe00640 1 --2- 172.21.15.123:0/695732091 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ce010a970 0x7f5ce01a6620 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:12.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.794+0000 7f5cdfe00640 1 --2- 172.21.15.123:0/695732091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ce0109fb0 0x7f5ce019f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:12.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.794+0000 7f5cdfe00640 1 --2- 172.21.15.123:0/695732091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ce0108e60 0x7f5ce019f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:12.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.794+0000 7f5cdfe00640 1 -- 172.21.15.123:0/695732091 >> 172.21.15.123:0/695732091 conn(0x7f5ce0077e40 msgr2=0x7f5ce010d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:12.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.794+0000 7f5cdfe00640 1 -- 172.21.15.123:0/695732091 shutdown_connections 2024-05-08T06:10:12.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:12.794+0000 7f5cdfe00640 1 -- 172.21.15.123:0/695732091 wait complete. 2024-05-08T06:10:12.816 DEBUG:teuthology.orchestra.run.smithi123:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-05-08T06:10:13.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.017+0000 7fdd18c00640 1 -- 172.21.15.123:0/1467537446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd14102a30 msgr2=0x7fdd14102e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:13.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.017+0000 7fdd18c00640 1 --2- 172.21.15.123:0/1467537446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd14102a30 0x7fdd14102e50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009ca0 tx=0x7fdd0402a8c0 comp rx=0 tx=0).stop 2024-05-08T06:10:13.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.017+0000 7fdd18c00640 1 -- 172.21.15.123:0/1467537446 shutdown_connections 2024-05-08T06:10:13.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.017+0000 7fdd18c00640 1 --2- 172.21.15.123:0/1467537446 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd141045e0 0x7fdd1410ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:13.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.017+0000 7fdd18c00640 1 --2- 172.21.15.123:0/1467537446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd14103c20 0x7fdd141040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:13.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.017+0000 7fdd18c00640 1 --2- 172.21.15.123:0/1467537446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd14102a30 0x7fdd14102e50 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:13.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.017+0000 7fdd18c00640 1 -- 172.21.15.123:0/1467537446 >> 172.21.15.123:0/1467537446 conn(0x7fdd140fdfa0 msgr2=0x7fdd141003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:13.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.017+0000 7fdd18c00640 1 -- 172.21.15.123:0/1467537446 shutdown_connections 2024-05-08T06:10:13.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.018+0000 7fdd18c00640 1 -- 172.21.15.123:0/1467537446 wait complete. 2024-05-08T06:10:13.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.018+0000 7fdd18c00640 1 Processor -- start 2024-05-08T06:10:13.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.019+0000 7fdd18c00640 1 -- start start 2024-05-08T06:10:13.020 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.019+0000 7fdd18c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd14102a30 0x7fdd1419ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:13.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.019+0000 7fdd18c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd14103c20 0x7fdd1419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:13.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.019+0000 7fdd18c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd141045e0 0x7fdd141a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:13.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.019+0000 7fdd18c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1410e5b0 con 0x7fdd14102a30 2024-05-08T06:10:13.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.019+0000 7fdd18c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1410e430 con 0x7fdd141045e0 2024-05-08T06:10:13.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.019+0000 7fdd18c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1410e730 con 0x7fdd14103c20 2024-05-08T06:10:13.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.019+0000 7fdd11e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd14102a30 0x7fdd1419ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:13.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.019+0000 7fdd11e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd14102a30 0x7fdd1419ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59430/0 (socket says 172.21.15.123:59430) 2024-05-08T06:10:13.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.019+0000 7fdd11e00640 1 -- 172.21.15.123:0/824690307 learned_addr learned my addr 172.21.15.123:0/824690307 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:10:13.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.019+0000 7fdd09400640 1 --2- 172.21.15.123:0/824690307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd14103c20 0x7fdd1419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:13.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.019+0000 7fdd12800640 1 --2- 172.21.15.123:0/824690307 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd141045e0 0x7fdd141a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:13.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.020+0000 7fdd11e00640 1 -- 172.21.15.123:0/824690307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd14103c20 msgr2=0x7fdd1419b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:13.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.020+0000 7fdd11e00640 1 --2- 172.21.15.123:0/824690307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd14103c20 0x7fdd1419b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:13.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.020+0000 7fdd11e00640 1 -- 172.21.15.123:0/824690307 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd141045e0 msgr2=0x7fdd141a23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:13.021 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.020+0000 7fdd11e00640 1 --2- 172.21.15.123:0/824690307 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd141045e0 0x7fdd141a23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:13.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.020+0000 7fdd11e00640 1 -- 172.21.15.123:0/824690307 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd141a2a20 con 0x7fdd14102a30 2024-05-08T06:10:13.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.020+0000 7fdd11e00640 1 --2- 172.21.15.123:0/824690307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd14102a30 0x7fdd1419ae00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdd04005e30 tx=0x7fdd040317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:13.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.020+0000 7fdd0b400640 1 -- 172.21.15.123:0/824690307 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd04003bf0 con 0x7fdd14102a30 2024-05-08T06:10:13.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.020+0000 7fdd18c00640 1 -- 172.21.15.123:0/824690307 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd141a2cb0 con 0x7fdd14102a30 2024-05-08T06:10:13.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.020+0000 7fdd0b400640 1 -- 172.21.15.123:0/824690307 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd04003d90 con 0x7fdd14102a30 2024-05-08T06:10:13.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.020+0000 7fdd0b400640 1 -- 172.21.15.123:0/824690307 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0402fc50 con 0x7fdd14102a30 2024-05-08T06:10:13.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.021+0000 7fdd18c00640 1 -- 172.21.15.123:0/824690307 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd141a31b0 con 0x7fdd14102a30 2024-05-08T06:10:13.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.022+0000 7fdd18c00640 1 -- 172.21.15.123:0/824690307 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd141a3440 con 0x7fdd14102a30 2024-05-08T06:10:13.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.024+0000 7fdd0b400640 1 -- 172.21.15.123:0/824690307 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fdd04004040 con 0x7fdd14102a30 2024-05-08T06:10:13.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.025+0000 7fdd0b400640 1 --2- 172.21.15.123:0/824690307 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fdcf80b28a0 0x7fdcf80b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:13.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.026+0000 7fdd0b400640 1 -- 172.21.15.123:0/824690307 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fdd041035c0 con 0x7fdd14102a30 2024-05-08T06:10:13.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.026+0000 7fdd09400640 1 --2- 172.21.15.123:0/824690307 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fdcf80b28a0 0x7fdcf80b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:13.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.027+0000 7fdd09400640 1 --2- 172.21.15.123:0/824690307 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fdcf80b28a0 0x7fdcf80b4d60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc0095f0 tx=0x7fdcfc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:13.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.030+0000 7fdd0b400640 1 -- 172.21.15.123:0/824690307 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdd040cb320 con 0x7fdd14102a30 2024-05-08T06:10:13.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:12 smithi192 ceph-mon[34664]: pgmap v452: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:10:13.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:12 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/695732091' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-08T06:10:13.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:12 smithi192 ceph-mon[34664]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-08T06:10:13.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:12 smithi123 ceph-mon[29787]: pgmap v452: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:10:13.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:12 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/695732091' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-08T06:10:13.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:12 smithi123 ceph-mon[29787]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-08T06:10:13.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.264+0000 7fdd18c00640 1 -- 172.21.15.123:0/824690307 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fdd14076300 con 0x7fdcf80b28a0 2024-05-08T06:10:13.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:12 smithi163 ceph-mon[34520]: pgmap v452: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:10:13.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:12 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/695732091' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-08T06:10:13.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:12 smithi163 ceph-mon[34520]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-08T06:10:13.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.294+0000 7fdd0b400640 1 -- 172.21.15.123:0/824690307 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7fdd14076300 con 0x7fdcf80b28a0 2024-05-08T06:10:13.294 INFO:teuthology.orchestra.run.smithi123.stdout:{} 2024-05-08T06:10:13.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.297+0000 7fdd18c00640 1 -- 172.21.15.123:0/824690307 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fdcf80b28a0 msgr2=0x7fdcf80b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:13.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.297+0000 7fdd18c00640 1 --2- 172.21.15.123:0/824690307 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fdcf80b28a0 0x7fdcf80b4d60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc0095f0 tx=0x7fdcfc005f90 comp rx=0 tx=0).stop 2024-05-08T06:10:13.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.297+0000 7fdd18c00640 1 -- 172.21.15.123:0/824690307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd14102a30 msgr2=0x7fdd1419ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:13.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.297+0000 7fdd18c00640 1 --2- 172.21.15.123:0/824690307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd14102a30 0x7fdd1419ae00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdd04005e30 tx=0x7fdd040317e0 comp rx=0 tx=0).stop 2024-05-08T06:10:13.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.297+0000 7fdd18c00640 1 -- 172.21.15.123:0/824690307 shutdown_connections 2024-05-08T06:10:13.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.297+0000 7fdd18c00640 1 --2- 172.21.15.123:0/824690307 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fdcf80b28a0 0x7fdcf80b4d60 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:13.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.297+0000 7fdd18c00640 1 --2- 172.21.15.123:0/824690307 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd141045e0 0x7fdd141a23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:13.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.297+0000 7fdd18c00640 1 --2- 172.21.15.123:0/824690307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd14103c20 0x7fdd1419b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:13.297 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.297+0000 7fdd18c00640 1 --2- 172.21.15.123:0/824690307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd14102a30 0x7fdd1419ae00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:13.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.297+0000 7fdd18c00640 1 -- 172.21.15.123:0/824690307 >> 172.21.15.123:0/824690307 conn(0x7fdd140fdfa0 msgr2=0x7fdd14074be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:13.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.297+0000 7fdd18c00640 1 -- 172.21.15.123:0/824690307 shutdown_connections 2024-05-08T06:10:13.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.297+0000 7fdd18c00640 1 -- 172.21.15.123:0/824690307 wait complete. 2024-05-08T06:10:13.315 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-08T06:10:13.326 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-05-08T06:10:13.351 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T06:10:13.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.551+0000 7f8b89400640 1 -- 172.21.15.123:0/3197776384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b84104c10 msgr2=0x7f8b84105030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:13.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.551+0000 7f8b89400640 1 --2- 172.21.15.123:0/3197776384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b84104c10 0x7f8b84105030 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8b74009ca0 tx=0x7f8b7402a8c0 comp rx=0 tx=0).stop 2024-05-08T06:10:13.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.551+0000 7f8b89400640 1 -- 172.21.15.123:0/3197776384 shutdown_connections 2024-05-08T06:10:13.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.551+0000 7f8b89400640 1 --2- 172.21.15.123:0/3197776384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b841067c0 0x7f8b8410d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:13.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.551+0000 7f8b89400640 1 --2- 172.21.15.123:0/3197776384 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b84105e00 0x7f8b84106280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:13.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.551+0000 7f8b89400640 1 --2- 172.21.15.123:0/3197776384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b84104c10 0x7f8b84105030 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:13.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.551+0000 7f8b89400640 1 -- 172.21.15.123:0/3197776384 >> 172.21.15.123:0/3197776384 conn(0x7f8b841001a0 msgr2=0x7f8b841025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:13.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.551+0000 7f8b89400640 1 -- 172.21.15.123:0/3197776384 shutdown_connections 2024-05-08T06:10:13.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.551+0000 7f8b89400640 1 -- 172.21.15.123:0/3197776384 wait complete. 2024-05-08T06:10:13.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.552+0000 7f8b89400640 1 Processor -- start 2024-05-08T06:10:13.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.552+0000 7f8b89400640 1 -- start start 2024-05-08T06:10:13.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.552+0000 7f8b89400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b84104c10 0x7f8b8419ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:13.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.552+0000 7f8b89400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b84105e00 0x7f8b8419b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:13.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.552+0000 7f8b89400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b841067c0 0x7f8b841a2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:13.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.552+0000 7f8b89400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8b84110710 con 0x7f8b841067c0 2024-05-08T06:10:13.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.552+0000 7f8b89400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8b84110590 con 0x7f8b84105e00 2024-05-08T06:10:13.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.552+0000 7f8b89400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8b84110890 con 0x7f8b84104c10 2024-05-08T06:10:13.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.553+0000 7f8b83200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b841067c0 0x7f8b841a2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:13.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.553+0000 7f8b79e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b84105e00 0x7f8b8419b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:13.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.553+0000 7f8b82800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b84104c10 0x7f8b8419ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:13.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.553+0000 7f8b79e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b84105e00 0x7f8b8419b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:37670/0 (socket says 172.21.15.123:37670) 2024-05-08T06:10:13.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.553+0000 7f8b79e00640 1 -- 172.21.15.123:0/2971977943 learned_addr learned my addr 172.21.15.123:0/2971977943 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:10:13.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.553+0000 7f8b82800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b84104c10 0x7f8b8419ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:36196/0 (socket says 172.21.15.123:36196) 2024-05-08T06:10:13.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.553+0000 7f8b79e00640 1 -- 172.21.15.123:0/2971977943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b84104c10 msgr2=0x7f8b8419ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:13.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.553+0000 7f8b79e00640 1 --2- 172.21.15.123:0/2971977943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b84104c10 0x7f8b8419ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:13.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.553+0000 7f8b79e00640 1 -- 172.21.15.123:0/2971977943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b841067c0 msgr2=0x7f8b841a2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:13.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.553+0000 7f8b79e00640 1 --2- 172.21.15.123:0/2971977943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b841067c0 0x7f8b841a2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:13.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.553+0000 7f8b79e00640 1 -- 172.21.15.123:0/2971977943 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b841a2a70 con 0x7f8b84105e00 2024-05-08T06:10:13.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.553+0000 7f8b79e00640 1 --2- 172.21.15.123:0/2971977943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b84105e00 0x7f8b8419b390 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c00e9f0 tx=0x7f8b6c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:13.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.554+0000 7f8b7be00640 1 -- 172.21.15.123:0/2971977943 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c00ccc0 con 0x7f8b84105e00 2024-05-08T06:10:13.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.554+0000 7f8b7be00640 1 -- 172.21.15.123:0/2971977943 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b6c0108e0 con 0x7f8b84105e00 2024-05-08T06:10:13.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.554+0000 7f8b89400640 1 -- 172.21.15.123:0/2971977943 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b841a2d60 con 0x7f8b84105e00 2024-05-08T06:10:13.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.554+0000 7f8b7be00640 1 -- 172.21.15.123:0/2971977943 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c018880 con 0x7f8b84105e00 2024-05-08T06:10:13.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.554+0000 7f8b89400640 1 -- 172.21.15.123:0/2971977943 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b841a3250 con 0x7f8b84105e00 2024-05-08T06:10:13.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.555+0000 7f8b89400640 1 -- 172.21.15.123:0/2971977943 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b8410c050 con 0x7f8b84105e00 2024-05-08T06:10:13.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.558+0000 7f8b7be00640 1 -- 172.21.15.123:0/2971977943 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f8b6c010430 con 0x7f8b84105e00 2024-05-08T06:10:13.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.559+0000 7f8b7be00640 1 --2- 172.21.15.123:0/2971977943 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f8b680b27f0 0x7f8b680b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:13.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.559+0000 7f8b7be00640 1 -- 172.21.15.123:0/2971977943 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f8b6c0e2ac0 con 0x7f8b84105e00 2024-05-08T06:10:13.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.559+0000 7f8b82800640 1 --2- 172.21.15.123:0/2971977943 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f8b680b27f0 0x7f8b680b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:13.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.560+0000 7f8b82800640 1 --2- 172.21.15.123:0/2971977943 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f8b680b27f0 0x7f8b680b4cb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8b74000f80 tx=0x7f8b74005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:13.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.561+0000 7f8b7be00640 1 -- 172.21.15.123:0/2971977943 <== mon.2 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8b6c0aa790 con 0x7f8b84105e00 2024-05-08T06:10:13.705 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 06:10:13 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T06:10:13.274+0000 7f780ca00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T06:10:13.705 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 06:10:13 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T06:10:13.274+0000 7f780ca00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T06:10:13.706 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 06:10:13 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T06:10:13.274+0000 7f780ca00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T06:10:13.706 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 06:10:13 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T06:10:13.274+0000 7f780ca00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T06:10:13.706 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 06:10:13 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T06:10:13.274+0000 7f780ca00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T06:10:13.769 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.769+0000 7f8b89400640 1 -- 172.21.15.123:0/2971977943 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b84063650 con 0x7f8b84105e00 2024-05-08T06:10:13.771 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.772+0000 7f8b7be00640 1 -- 172.21.15.123:0/2971977943 <== mon.2 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22719 (secure 0 0 0) 0x7f8b6c0aa090 con 0x7f8b84105e00 2024-05-08T06:10:13.772 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:10:13.772 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":82,"fsid":"85cfb2e6-0cff-11ef-bc97-c7b262605968","created":"2024-05-08T05:56:12.544750+0000","modified":"2024-05-08T06:06:32.471993+0000","last_up_change":"2024-05-08T06:04:33.654727+0000","last_in_change":"2024-05-08T06:04:13.809050+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T06:00:17.050518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T06:05:44.815334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T06:05:49.604457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a14fde9b-bfb1-40a7-a105-305e8981a778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6803","nonce":3608618058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6805","nonce":3608618058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6809","nonce":3608618058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6807","nonce":3608618058}]},"public_addr":"172.21.15.123:6803/3608618058","cluster_addr":"172.21.15.123:6805/3608618058","heartbeat_back_addr":"172.21.15.123:6809/3608618058","heartbeat_front_addr":"172.21.15.123:6807/3608618058","state":["exists","up"]},{"osd":1,"uuid":"e023ae3d-a82e-4bb6-8fef-889dd07fe183","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6801","nonce":1884753449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6803","nonce":1884753449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6807","nonce":1884753449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6805","nonce":1884753449}]},"public_addr":"172.21.15.163:6801/1884753449","cluster_addr":"172.21.15.163:6803/1884753449","heartbeat_back_addr":"172.21.15.163:6807/1884753449","heartbeat_front_addr":"172.21.15.163:6805/1884753449","state":["exists","up"]},{"osd":2,"uuid":"69f4cfca-63cb-4862-9480-d79891b0f338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6801","nonce":4161570404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6803","nonce":4161570404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6807","nonce":4161570404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6805","nonce":4161570404}]},"public_addr":"172.21.15.192:6801/4161570404","cluster_addr":"172.21.15.192:6803/4161570404","heartbeat_back_addr":"172.21.15.192:6807/4161570404","heartbeat_front_addr":"172.21.15.192:6805/4161570404","state":["exists","up"]},{"osd":3,"uuid":"71bb5ecd-c88c-45fb-8b51-0eefb53941c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6811","nonce":1243003258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6813","nonce":1243003258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6817","nonce":1243003258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6815","nonce":1243003258}]},"public_addr":"172.21.15.123:6811/1243003258","cluster_addr":"172.21.15.123:6813/1243003258","heartbeat_back_addr":"172.21.15.123:6817/1243003258","heartbeat_front_addr":"172.21.15.123:6815/1243003258","state":["exists","up"]},{"osd":4,"uuid":"5a7a3462-e74f-4d15-a85c-709b5d423745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6809","nonce":591680273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6811","nonce":591680273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6815","nonce":591680273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6813","nonce":591680273}]},"public_addr":"172.21.15.163:6809/591680273","cluster_addr":"172.21.15.163:6811/591680273","heartbeat_back_addr":"172.21.15.163:6815/591680273","heartbeat_front_addr":"172.21.15.163:6813/591680273","state":["exists","up"]},{"osd":5,"uuid":"85e3f1ab-f8c6-4025-bab9-9ddb6bd3e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6809","nonce":1734821180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6811","nonce":1734821180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6815","nonce":1734821180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6813","nonce":1734821180}]},"public_addr":"172.21.15.192:6809/1734821180","cluster_addr":"172.21.15.192:6811/1734821180","heartbeat_back_addr":"172.21.15.192:6815/1734821180","heartbeat_front_addr":"172.21.15.192:6813/1734821180","state":["exists","up"]},{"osd":6,"uuid":"f21a5624-6f8f-48be-bc6d-0a883d6b7b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6819","nonce":1002138878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6821","nonce":1002138878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6825","nonce":1002138878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6823","nonce":1002138878}]},"public_addr":"172.21.15.123:6819/1002138878","cluster_addr":"172.21.15.123:6821/1002138878","heartbeat_back_addr":"172.21.15.123:6825/1002138878","heartbeat_front_addr":"172.21.15.123:6823/1002138878","state":["exists","up"]},{"osd":7,"uuid":"6eb16729-d260-4041-9829-3e868795b111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6817","nonce":354411978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6819","nonce":354411978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6823","nonce":354411978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6821","nonce":354411978}]},"public_addr":"172.21.15.163:6817/354411978","cluster_addr":"172.21.15.163:6819/354411978","heartbeat_back_addr":"172.21.15.163:6823/354411978","heartbeat_front_addr":"172.21.15.163:6821/354411978","state":["exists","up"]},{"osd":8,"uuid":"122a281e-18b8-4728-97db-458e7432b2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6817","nonce":3669753689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6819","nonce":3669753689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6823","nonce":3669753689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6821","nonce":3669753689}]},"public_addr":"172.21.15.192:6817/3669753689","cluster_addr":"172.21.15.192:6819/3669753689","heartbeat_back_addr":"172.21.15.192:6823/3669753689","heartbeat_front_addr":"172.21.15.192:6821/3669753689","state":["exists","up"]},{"osd":9,"uuid":"9fa022a9-b873-4e07-806c-641a706a150f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6827","nonce":3104436473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6829","nonce":3104436473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6833","nonce":3104436473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6831","nonce":3104436473}]},"public_addr":"172.21.15.123:6827/3104436473","cluster_addr":"172.21.15.123:6829/3104436473","heartbeat_back_addr":"172.21.15.123:6833/3104436473","heartbeat_front_addr":"172.21.15.123:6831/3104436473","state":["exists","up"]},{"osd":10,"uuid":"fb59339b-e11b-4ffc-b202-7f67cb21da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6825","nonce":2628487883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6827","nonce":2628487883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6831","nonce":2628487883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6829","nonce":2628487883}]},"public_addr":"172.21.15.163:6825/2628487883","cluster_addr":"172.21.15.163:6827/2628487883","heartbeat_back_addr":"172.21.15.163:6831/2628487883","heartbeat_front_addr":"172.21.15.163:6829/2628487883","state":["exists","up"]},{"osd":11,"uuid":"be7a5f36-c1be-41d7-80ee-004d0546eb9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6825","nonce":1339159347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6827","nonce":1339159347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6831","nonce":1339159347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6829","nonce":1339159347}]},"public_addr":"172.21.15.192:6825/1339159347","cluster_addr":"172.21.15.192:6827/1339159347","heartbeat_back_addr":"172.21.15.192:6831/1339159347","heartbeat_front_addr":"172.21.15.192:6829/1339159347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:59:24.923234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:59:49.867071+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:00:13.454062+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:00:46.983594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:01:12.820452+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:01:37.451382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:02:13.325181+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:02:39.453436+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:03:03.837509+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:03:39.460579+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:04:06.314082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:04:31.723154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6801/3057688649":"2024-05-09T05:57:30.887936+0000","172.21.15.123:6800/3057688649":"2024-05-09T05:57:30.887936+0000","172.21.15.123:6800/2769724312":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/399884043":"2024-05-09T05:57:30.887936+0000","172.21.15.123:0/3406061897":"2024-05-09T05:57:30.887936+0000","172.21.15.123:0/644457268":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/3055092197":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/343390357":"2024-05-09T05:57:30.887936+0000","172.21.15.123:6801/2769724312":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/2263171562":"2024-05-09T05:56:55.849036+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T06:10:13.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.774+0000 7f8b89400640 1 -- 172.21.15.123:0/2971977943 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f8b680b27f0 msgr2=0x7f8b680b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:13.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.774+0000 7f8b89400640 1 --2- 172.21.15.123:0/2971977943 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f8b680b27f0 0x7f8b680b4cb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8b74000f80 tx=0x7f8b74005dc0 comp rx=0 tx=0).stop 2024-05-08T06:10:13.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.774+0000 7f8b89400640 1 -- 172.21.15.123:0/2971977943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b84105e00 msgr2=0x7f8b8419b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:13.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.774+0000 7f8b89400640 1 --2- 172.21.15.123:0/2971977943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b84105e00 0x7f8b8419b390 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c00e9f0 tx=0x7f8b6c00edf0 comp rx=0 tx=0).stop 2024-05-08T06:10:13.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.774+0000 7f8b89400640 1 -- 172.21.15.123:0/2971977943 shutdown_connections 2024-05-08T06:10:13.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.774+0000 7f8b89400640 1 --2- 172.21.15.123:0/2971977943 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f8b680b27f0 0x7f8b680b4cb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:13.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.774+0000 7f8b89400640 1 --2- 172.21.15.123:0/2971977943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b841067c0 0x7f8b841a2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:13.774 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.774+0000 7f8b89400640 1 --2- 172.21.15.123:0/2971977943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b84105e00 0x7f8b8419b390 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:13.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.774+0000 7f8b89400640 1 --2- 172.21.15.123:0/2971977943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b84104c10 0x7f8b8419ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:13.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.774+0000 7f8b89400640 1 -- 172.21.15.123:0/2971977943 >> 172.21.15.123:0/2971977943 conn(0x7f8b841001a0 msgr2=0x7f8b84109220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:13.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.774+0000 7f8b89400640 1 -- 172.21.15.123:0/2971977943 shutdown_connections 2024-05-08T06:10:13.775 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.774+0000 7f8b89400640 1 -- 172.21.15.123:0/2971977943 wait complete. 2024-05-08T06:10:13.794 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T06:00:17.050518+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T06:05:44.815334+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T06:05:49.604457+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T06:10:13.795 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T06:10:13.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.983+0000 7f62d3200640 1 -- 172.21.15.123:0/3737940741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62d4078f50 msgr2=0x7f62d40797f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:13.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.983+0000 7f62d3200640 1 --2- 172.21.15.123:0/3737940741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62d4078f50 0x7f62d40797f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009ca0 tx=0x7f62cc02a8c0 comp rx=0 tx=0).stop 2024-05-08T06:10:13.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.984+0000 7f62d3200640 1 -- 172.21.15.123:0/3737940741 shutdown_connections 2024-05-08T06:10:13.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.984+0000 7f62d3200640 1 --2- 172.21.15.123:0/3737940741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62d4078f50 0x7f62d40797f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:13.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.984+0000 7f62d3200640 1 --2- 172.21.15.123:0/3737940741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62d40785b0 0x7f62d4078a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:13.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.984+0000 7f62d3200640 1 --2- 172.21.15.123:0/3737940741 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62d4077370 0x7f62d4077790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:13.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.984+0000 7f62d3200640 1 -- 172.21.15.123:0/3737940741 >> 172.21.15.123:0/3737940741 conn(0x7f62d4100060 msgr2=0x7f62d41024a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:13.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.984+0000 7f62d3200640 1 -- 172.21.15.123:0/3737940741 shutdown_connections 2024-05-08T06:10:13.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.984+0000 7f62d3200640 1 -- 172.21.15.123:0/3737940741 wait complete. 2024-05-08T06:10:13.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.985+0000 7f62d3200640 1 Processor -- start 2024-05-08T06:10:13.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.985+0000 7f62d3200640 1 -- start start 2024-05-08T06:10:13.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.985+0000 7f62d3200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62d4077370 0x7f62d419f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:13.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.985+0000 7f62d3200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62d40785b0 0x7f62d419f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:13.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.985+0000 7f62d3200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62d4078f50 0x7f62d41a66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:13.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.985+0000 7f62d3200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f62d4114a70 con 0x7f62d4078f50 2024-05-08T06:10:13.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.985+0000 7f62d3200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f62d41148f0 con 0x7f62d40785b0 2024-05-08T06:10:13.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.985+0000 7f62d3200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f62d4114bf0 con 0x7f62d4077370 2024-05-08T06:10:13.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.985+0000 7f62d2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62d4078f50 0x7f62d41a66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:13.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.985+0000 7f62d1e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62d4077370 0x7f62d419f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:13.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.985+0000 7f62d2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62d4078f50 0x7f62d41a66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59486/0 (socket says 172.21.15.123:59486) 2024-05-08T06:10:13.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.985+0000 7f62d2800640 1 -- 172.21.15.123:0/1942192102 learned_addr learned my addr 172.21.15.123:0/1942192102 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:10:13.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.985+0000 7f62d1400640 1 --2- 172.21.15.123:0/1942192102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62d40785b0 0x7f62d419f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:13.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.986+0000 7f62d2800640 1 -- 172.21.15.123:0/1942192102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62d4077370 msgr2=0x7f62d419f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:13.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.986+0000 7f62d2800640 1 --2- 172.21.15.123:0/1942192102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62d4077370 0x7f62d419f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:13.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.986+0000 7f62d2800640 1 -- 172.21.15.123:0/1942192102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62d40785b0 msgr2=0x7f62d419f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:13.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.986+0000 7f62d2800640 1 --2- 172.21.15.123:0/1942192102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62d40785b0 0x7f62d419f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:13.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.986+0000 7f62d2800640 1 -- 172.21.15.123:0/1942192102 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d41a6d60 con 0x7f62d4078f50 2024-05-08T06:10:13.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.986+0000 7f62d1400640 1 --2- 172.21.15.123:0/1942192102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62d40785b0 0x7f62d419f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:10:13.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.986+0000 7f62d2800640 1 --2- 172.21.15.123:0/1942192102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62d4078f50 0x7f62d41a66f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009430 tx=0x7f62cc0316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:13.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.986+0000 7f62caa00640 1 -- 172.21.15.123:0/1942192102 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc003b70 con 0x7f62d4078f50 2024-05-08T06:10:13.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.986+0000 7f62caa00640 1 -- 172.21.15.123:0/1942192102 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62cc003d10 con 0x7f62d4078f50 2024-05-08T06:10:13.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.986+0000 7f62d3200640 1 -- 172.21.15.123:0/1942192102 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62d41a6ff0 con 0x7f62d4078f50 2024-05-08T06:10:13.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.986+0000 7f62caa00640 1 -- 172.21.15.123:0/1942192102 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc02fb40 con 0x7f62d4078f50 2024-05-08T06:10:13.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.987+0000 7f62d3200640 1 -- 172.21.15.123:0/1942192102 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62d41a74b0 con 0x7f62d4078f50 2024-05-08T06:10:13.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.988+0000 7f62d3200640 1 -- 172.21.15.123:0/1942192102 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62d4111460 con 0x7f62d4078f50 2024-05-08T06:10:13.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.990+0000 7f62caa00640 1 -- 172.21.15.123:0/1942192102 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 19) ==== 149704+0+0 (secure 0 0 0) 0x7f62cc004610 con 0x7f62d4078f50 2024-05-08T06:10:13.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.991+0000 7f62caa00640 1 --2- 172.21.15.123:0/1942192102 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f62b80b2880 0x7f62b80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:13.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.991+0000 7f62caa00640 1 -- 172.21.15.123:0/1942192102 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f62cc1036d0 con 0x7f62d4078f50 2024-05-08T06:10:13.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.991+0000 7f62d1e00640 1 --2- 172.21.15.123:0/1942192102 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f62b80b2880 0x7f62b80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:13.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.992+0000 7f62d1e00640 1 --2- 172.21.15.123:0/1942192102 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f62b80b2880 0x7f62b80b4d40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f62c4009bd0 tx=0x7f62c4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:13.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:13.996+0000 7f62caa00640 1 -- 172.21.15.123:0/1942192102 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f62cc0cb350 con 0x7f62d4078f50 2024-05-08T06:10:14.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.205+0000 7f62d3200640 1 -- 172.21.15.123:0/1942192102 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f62d402dda0 con 0x7f62d4078f50 2024-05-08T06:10:14.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:13 smithi123 ceph-mon[29787]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-08T06:10:14.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:13 smithi123 ceph-mon[29787]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T06:10:14.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:13 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/2971977943' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:10:14.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.206+0000 7f62caa00640 1 -- 172.21.15.123:0/1942192102 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f62cc0cac50 con 0x7f62d4078f50 2024-05-08T06:10:14.207 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-05-08T06:10:14.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.209+0000 7f62d3200640 1 -- 172.21.15.123:0/1942192102 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f62b80b2880 msgr2=0x7f62b80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:14.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.209+0000 7f62d3200640 1 --2- 172.21.15.123:0/1942192102 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f62b80b2880 0x7f62b80b4d40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f62c4009bd0 tx=0x7f62c4005f90 comp rx=0 tx=0).stop 2024-05-08T06:10:14.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.209+0000 7f62d3200640 1 -- 172.21.15.123:0/1942192102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62d4078f50 msgr2=0x7f62d41a66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:14.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.209+0000 7f62d3200640 1 --2- 172.21.15.123:0/1942192102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62d4078f50 0x7f62d41a66f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009430 tx=0x7f62cc0316f0 comp rx=0 tx=0).stop 2024-05-08T06:10:14.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.210+0000 7f62d3200640 1 -- 172.21.15.123:0/1942192102 shutdown_connections 2024-05-08T06:10:14.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.210+0000 7f62d3200640 1 --2- 172.21.15.123:0/1942192102 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f62b80b2880 0x7f62b80b4d40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:14.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.210+0000 7f62d3200640 1 --2- 172.21.15.123:0/1942192102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62d4078f50 0x7f62d41a66f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:14.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.210+0000 7f62d3200640 1 --2- 172.21.15.123:0/1942192102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62d40785b0 0x7f62d419f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:14.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.210+0000 7f62d3200640 1 --2- 172.21.15.123:0/1942192102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62d4077370 0x7f62d419f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:14.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.210+0000 7f62d3200640 1 -- 172.21.15.123:0/1942192102 >> 172.21.15.123:0/1942192102 conn(0x7f62d4100060 msgr2=0x7f62d4101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:14.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.210+0000 7f62d3200640 1 -- 172.21.15.123:0/1942192102 shutdown_connections 2024-05-08T06:10:14.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.210+0000 7f62d3200640 1 -- 172.21.15.123:0/1942192102 wait complete. 2024-05-08T06:10:14.227 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T06:10:14.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:13 smithi163 ceph-mon[34520]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-08T06:10:14.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:13 smithi163 ceph-mon[34520]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T06:10:14.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:13 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/2971977943' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:10:14.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:13 smithi192 ceph-mon[34664]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-08T06:10:14.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:13 smithi192 ceph-mon[34664]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T06:10:14.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:13 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/2971977943' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:10:14.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.430+0000 7f28b3200640 1 -- 172.21.15.123:0/3023028984 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28b410a080 msgr2=0x7f28b410a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:14.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.430+0000 7f28b3200640 1 --2- 172.21.15.123:0/3023028984 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28b410a080 0x7f28b410a500 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f289c009900 tx=0x7f289c02a400 comp rx=0 tx=0).stop 2024-05-08T06:10:14.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.430+0000 7f28b3200640 1 -- 172.21.15.123:0/3023028984 shutdown_connections 2024-05-08T06:10:14.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.430+0000 7f28b3200640 1 --2- 172.21.15.123:0/3023028984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28b410aa40 0x7f28b4111360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:14.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.430+0000 7f28b3200640 1 --2- 172.21.15.123:0/3023028984 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28b410a080 0x7f28b410a500 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:14.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.430+0000 7f28b3200640 1 --2- 172.21.15.123:0/3023028984 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28b4108ea0 0x7f28b41092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:14.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.430+0000 7f28b3200640 1 -- 172.21.15.123:0/3023028984 >> 172.21.15.123:0/3023028984 conn(0x7f28b4077e80 msgr2=0x7f28b407a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:14.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.430+0000 7f28b3200640 1 -- 172.21.15.123:0/3023028984 shutdown_connections 2024-05-08T06:10:14.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.430+0000 7f28b3200640 1 -- 172.21.15.123:0/3023028984 wait complete. 2024-05-08T06:10:14.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.431+0000 7f28b3200640 1 Processor -- start 2024-05-08T06:10:14.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.431+0000 7f28b3200640 1 -- start start 2024-05-08T06:10:14.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.431+0000 7f28b3200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28b4108ea0 0x7f28b419f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:14.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.431+0000 7f28b3200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28b410a080 0x7f28b419f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:14.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.431+0000 7f28b3200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28b410aa40 0x7f28b41a66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:14.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.431+0000 7f28b3200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f28b4114a20 con 0x7f28b410aa40 2024-05-08T06:10:14.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.431+0000 7f28b3200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f28b41148a0 con 0x7f28b4108ea0 2024-05-08T06:10:14.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.431+0000 7f28b3200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f28b4114ba0 con 0x7f28b410a080 2024-05-08T06:10:14.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.432+0000 7f28b1e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28b4108ea0 0x7f28b419f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:14.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.432+0000 7f28b1e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28b4108ea0 0x7f28b419f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:37716/0 (socket says 172.21.15.123:37716) 2024-05-08T06:10:14.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.432+0000 7f28b2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28b410aa40 0x7f28b41a66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:14.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.432+0000 7f28b1e00640 1 -- 172.21.15.123:0/1488691605 learned_addr learned my addr 172.21.15.123:0/1488691605 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:10:14.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.432+0000 7f28abe00640 1 --2- 172.21.15.123:0/1488691605 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28b410a080 0x7f28b419f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:14.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.432+0000 7f28b1e00640 1 -- 172.21.15.123:0/1488691605 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28b410a080 msgr2=0x7f28b419f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:14.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.432+0000 7f28b1e00640 1 --2- 172.21.15.123:0/1488691605 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28b410a080 0x7f28b419f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:14.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.432+0000 7f28b1e00640 1 -- 172.21.15.123:0/1488691605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28b410aa40 msgr2=0x7f28b41a66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:14.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.432+0000 7f28b1e00640 1 --2- 172.21.15.123:0/1488691605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28b410aa40 0x7f28b41a66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:14.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.432+0000 7f28b1e00640 1 -- 172.21.15.123:0/1488691605 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b41a6d10 con 0x7f28b4108ea0 2024-05-08T06:10:14.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.432+0000 7f28b2800640 1 --2- 172.21.15.123:0/1488691605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28b410aa40 0x7f28b41a66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:10:14.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.432+0000 7f28b1e00640 1 --2- 172.21.15.123:0/1488691605 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28b4108ea0 0x7f28b419f0f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f28a400e9c0 tx=0x7f28a400edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:14.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.433+0000 7f28aaa00640 1 -- 172.21.15.123:0/1488691605 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a400cd80 con 0x7f28b4108ea0 2024-05-08T06:10:14.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.433+0000 7f28aaa00640 1 -- 172.21.15.123:0/1488691605 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28a40108e0 con 0x7f28b4108ea0 2024-05-08T06:10:14.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.433+0000 7f28b3200640 1 -- 172.21.15.123:0/1488691605 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28b41a6fa0 con 0x7f28b4108ea0 2024-05-08T06:10:14.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.433+0000 7f28aaa00640 1 -- 172.21.15.123:0/1488691605 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a4018840 con 0x7f28b4108ea0 2024-05-08T06:10:14.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.433+0000 7f28b3200640 1 -- 172.21.15.123:0/1488691605 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28b41a7460 con 0x7f28b4108ea0 2024-05-08T06:10:14.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.434+0000 7f28b3200640 1 -- 172.21.15.123:0/1488691605 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28b4110240 con 0x7f28b4108ea0 2024-05-08T06:10:14.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.436+0000 7f28aaa00640 1 -- 172.21.15.123:0/1488691605 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 19) ==== 149704+0+0 (secure 0 0 0) 0x7f28a401f020 con 0x7f28b4108ea0 2024-05-08T06:10:14.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.438+0000 7f28aaa00640 1 --2- 172.21.15.123:0/1488691605 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f28980b2880 0x7f28980b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:14.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.438+0000 7f28aaa00640 1 -- 172.21.15.123:0/1488691605 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f28a40e2470 con 0x7f28b4108ea0 2024-05-08T06:10:14.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.438+0000 7f28abe00640 1 --2- 172.21.15.123:0/1488691605 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f28980b2880 0x7f28980b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:14.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.439+0000 7f28abe00640 1 --2- 172.21.15.123:0/1488691605 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f28980b2880 0x7f28980b4d40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f289c000c00 tx=0x7f289c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:14.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.442+0000 7f28aaa00640 1 -- 172.21.15.123:0/1488691605 <== mon.2 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f28a40a9fe0 con 0x7f28b4108ea0 2024-05-08T06:10:14.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.650+0000 7f28b3200640 1 -- 172.21.15.123:0/1488691605 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f28b40635b0 con 0x7f28b4108ea0 2024-05-08T06:10:14.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.651+0000 7f28aaa00640 1 -- 172.21.15.123:0/1488691605 <== mon.2 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) ==== 104+0+11 (secure 0 0 0) 0x7f28a40a98e0 con 0x7f28b4108ea0 2024-05-08T06:10:14.651 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-05-08T06:10:14.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.654+0000 7f28b3200640 1 -- 172.21.15.123:0/1488691605 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f28980b2880 msgr2=0x7f28980b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:14.654 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.654+0000 7f28b3200640 1 --2- 172.21.15.123:0/1488691605 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f28980b2880 0x7f28980b4d40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f289c000c00 tx=0x7f289c0314a0 comp rx=0 tx=0).stop 2024-05-08T06:10:14.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.654+0000 7f28b3200640 1 -- 172.21.15.123:0/1488691605 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28b4108ea0 msgr2=0x7f28b419f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:14.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.654+0000 7f28b3200640 1 --2- 172.21.15.123:0/1488691605 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28b4108ea0 0x7f28b419f0f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f28a400e9c0 tx=0x7f28a400edc0 comp rx=0 tx=0).stop 2024-05-08T06:10:14.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.654+0000 7f28b3200640 1 -- 172.21.15.123:0/1488691605 shutdown_connections 2024-05-08T06:10:14.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.654+0000 7f28b3200640 1 --2- 172.21.15.123:0/1488691605 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f28980b2880 0x7f28980b4d40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:14.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.654+0000 7f28b3200640 1 --2- 172.21.15.123:0/1488691605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28b410aa40 0x7f28b41a66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:14.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.654+0000 7f28b3200640 1 --2- 172.21.15.123:0/1488691605 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28b410a080 0x7f28b419f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:14.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.654+0000 7f28b3200640 1 --2- 172.21.15.123:0/1488691605 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28b4108ea0 0x7f28b419f0f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:14.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.654+0000 7f28b3200640 1 -- 172.21.15.123:0/1488691605 >> 172.21.15.123:0/1488691605 conn(0x7f28b4077e80 msgr2=0x7f28b410d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:14.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.654+0000 7f28b3200640 1 -- 172.21.15.123:0/1488691605 shutdown_connections 2024-05-08T06:10:14.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.654+0000 7f28b3200640 1 -- 172.21.15.123:0/1488691605 wait complete. 2024-05-08T06:10:14.673 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T06:10:14.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.868+0000 7f8cb5a00640 1 -- 172.21.15.123:0/261326065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cb0104af0 msgr2=0x7f8cb0104f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:14.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.868+0000 7f8cb5a00640 1 --2- 172.21.15.123:0/261326065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cb0104af0 0x7f8cb0104f10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0009900 tx=0x7f8ca002a3f0 comp rx=0 tx=0).stop 2024-05-08T06:10:14.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.868+0000 7f8cb5a00640 1 -- 172.21.15.123:0/261326065 shutdown_connections 2024-05-08T06:10:14.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.868+0000 7f8cb5a00640 1 --2- 172.21.15.123:0/261326065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cb01066a0 0x7f8cb010cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:14.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.868+0000 7f8cb5a00640 1 --2- 172.21.15.123:0/261326065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cb0105ce0 0x7f8cb0106160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:14.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.868+0000 7f8cb5a00640 1 --2- 172.21.15.123:0/261326065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cb0104af0 0x7f8cb0104f10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:14.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.868+0000 7f8cb5a00640 1 -- 172.21.15.123:0/261326065 >> 172.21.15.123:0/261326065 conn(0x7f8cb0100060 msgr2=0x7f8cb01024a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:14.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.868+0000 7f8cb5a00640 1 -- 172.21.15.123:0/261326065 shutdown_connections 2024-05-08T06:10:14.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.868+0000 7f8cb5a00640 1 -- 172.21.15.123:0/261326065 wait complete. 2024-05-08T06:10:14.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.869+0000 7f8cb5a00640 1 Processor -- start 2024-05-08T06:10:14.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.869+0000 7f8cb5a00640 1 -- start start 2024-05-08T06:10:14.869 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.869+0000 7f8cb5a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cb0104af0 0x7f8cb019ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:14.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.869+0000 7f8cb5a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cb0105ce0 0x7f8cb019b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:14.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.869+0000 7f8cb5a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cb01066a0 0x7f8cb01a2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:14.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.869+0000 7f8cb5a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb0110620 con 0x7f8cb01066a0 2024-05-08T06:10:14.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.869+0000 7f8cb5a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb01104a0 con 0x7f8cb0105ce0 2024-05-08T06:10:14.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.869+0000 7f8cb5a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb01107a0 con 0x7f8cb0104af0 2024-05-08T06:10:14.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.870+0000 7f8cb5000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cb01066a0 0x7f8cb01a2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:14.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.870+0000 7f8caf400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cb0105ce0 0x7f8cb019b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:14.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.870+0000 7f8cafe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cb0104af0 0x7f8cb019ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:14.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.870+0000 7f8caf400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cb0105ce0 0x7f8cb019b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:54176/0 (socket says 172.21.15.123:54176) 2024-05-08T06:10:14.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.870+0000 7f8cb5000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cb01066a0 0x7f8cb01a2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41626/0 (socket says 172.21.15.123:41626) 2024-05-08T06:10:14.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.870+0000 7f8caf400640 1 -- 172.21.15.123:0/4217251347 learned_addr learned my addr 172.21.15.123:0/4217251347 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:10:14.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.870+0000 7f8caf400640 1 -- 172.21.15.123:0/4217251347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cb0104af0 msgr2=0x7f8cb019ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:14.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.870+0000 7f8caf400640 1 --2- 172.21.15.123:0/4217251347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cb0104af0 0x7f8cb019ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:14.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.870+0000 7f8caf400640 1 -- 172.21.15.123:0/4217251347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cb01066a0 msgr2=0x7f8cb01a2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:14.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.870+0000 7f8caf400640 1 --2- 172.21.15.123:0/4217251347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cb01066a0 0x7f8cb01a2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:14.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.870+0000 7f8caf400640 1 -- 172.21.15.123:0/4217251347 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cb01a2920 con 0x7f8cb0105ce0 2024-05-08T06:10:14.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.870+0000 7f8caf400640 1 --2- 172.21.15.123:0/4217251347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cb0105ce0 0x7f8cb019b250 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00ea60 tx=0x7f8c9c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:14.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.871+0000 7f8cacc00640 1 -- 172.21.15.123:0/4217251347 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c00cde0 con 0x7f8cb0105ce0 2024-05-08T06:10:14.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.871+0000 7f8cb5a00640 1 -- 172.21.15.123:0/4217251347 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cb01a2c10 con 0x7f8cb0105ce0 2024-05-08T06:10:14.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.871+0000 7f8cacc00640 1 -- 172.21.15.123:0/4217251347 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c9c0035f0 con 0x7f8cb0105ce0 2024-05-08T06:10:14.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.871+0000 7f8cacc00640 1 -- 172.21.15.123:0/4217251347 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c010690 con 0x7f8cb0105ce0 2024-05-08T06:10:14.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.871+0000 7f8cb5a00640 1 -- 172.21.15.123:0/4217251347 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cb01a3160 con 0x7f8cb0105ce0 2024-05-08T06:10:14.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.872+0000 7f8cb5a00640 1 -- 172.21.15.123:0/4217251347 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cb010be90 con 0x7f8cb0105ce0 2024-05-08T06:10:14.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.876+0000 7f8cacc00640 1 -- 172.21.15.123:0/4217251347 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 19) ==== 149704+0+0 (secure 0 0 0) 0x7f8c9c0037a0 con 0x7f8cb0105ce0 2024-05-08T06:10:14.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.877+0000 7f8cacc00640 1 --2- 172.21.15.123:0/4217251347 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f8c940b2880 0x7f8c940b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:14.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.877+0000 7f8cacc00640 1 -- 172.21.15.123:0/4217251347 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f8c9c0e2740 con 0x7f8cb0105ce0 2024-05-08T06:10:14.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.877+0000 7f8cafe00640 1 --2- 172.21.15.123:0/4217251347 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f8c940b2880 0x7f8c940b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:14.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.878+0000 7f8cacc00640 1 -- 172.21.15.123:0/4217251347 <== mon.2 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8c9c0aa2b0 con 0x7f8cb0105ce0 2024-05-08T06:10:14.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:14.878+0000 7f8cafe00640 1 --2- 172.21.15.123:0/4217251347 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f8c940b2880 0x7f8c940b4d40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8cb019c1e0 tx=0x7f8ca00314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:14.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:14 smithi123 ceph-mon[29787]: pgmap v453: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-08T06:10:14.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:14 smithi123 ceph-mon[29787]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:10:14.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:14 smithi123 ceph-mon[29787]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-05-08T06:10:14.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:14 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/1942192102' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T06:10:14.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:14 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/1488691605' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T06:10:15.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:15.084+0000 7f8cb5a00640 1 -- 172.21.15.123:0/4217251347 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8cb002dda0 con 0x7f8cb0105ce0 2024-05-08T06:10:15.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:15.085+0000 7f8cacc00640 1 -- 172.21.15.123:0/4217251347 <== mon.2 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) ==== 100+0+11 (secure 0 0 0) 0x7f8c9c0a9bb0 con 0x7f8cb0105ce0 2024-05-08T06:10:15.085 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-05-08T06:10:15.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:15.088+0000 7f8cb5a00640 1 -- 172.21.15.123:0/4217251347 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f8c940b2880 msgr2=0x7f8c940b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:15.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:15.088+0000 7f8cb5a00640 1 --2- 172.21.15.123:0/4217251347 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f8c940b2880 0x7f8c940b4d40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8cb019c1e0 tx=0x7f8ca00314a0 comp rx=0 tx=0).stop 2024-05-08T06:10:15.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:15.088+0000 7f8cb5a00640 1 -- 172.21.15.123:0/4217251347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cb0105ce0 msgr2=0x7f8cb019b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:15.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:15.088+0000 7f8cb5a00640 1 --2- 172.21.15.123:0/4217251347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cb0105ce0 0x7f8cb019b250 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00ea60 tx=0x7f8c9c00ef20 comp rx=0 tx=0).stop 2024-05-08T06:10:15.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:15.088+0000 7f8cb5a00640 1 -- 172.21.15.123:0/4217251347 shutdown_connections 2024-05-08T06:10:15.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:15.088+0000 7f8cb5a00640 1 --2- 172.21.15.123:0/4217251347 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f8c940b2880 0x7f8c940b4d40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:15.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:15.088+0000 7f8cb5a00640 1 --2- 172.21.15.123:0/4217251347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cb01066a0 0x7f8cb01a2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:15.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:15.088+0000 7f8cb5a00640 1 --2- 172.21.15.123:0/4217251347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cb0105ce0 0x7f8cb019b250 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:15.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:15.088+0000 7f8cb5a00640 1 --2- 172.21.15.123:0/4217251347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cb0104af0 0x7f8cb019ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:15.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:15.088+0000 7f8cb5a00640 1 -- 172.21.15.123:0/4217251347 >> 172.21.15.123:0/4217251347 conn(0x7f8cb0100060 msgr2=0x7f8cb0109060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:15.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:15.088+0000 7f8cb5a00640 1 -- 172.21.15.123:0/4217251347 shutdown_connections 2024-05-08T06:10:15.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:15.088+0000 7f8cb5a00640 1 -- 172.21.15.123:0/4217251347 wait complete. 2024-05-08T06:10:15.107 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-05-08T06:10:15.108 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T06:10:15.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:14 smithi163 ceph-mon[34520]: pgmap v453: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-08T06:10:15.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:14 smithi163 ceph-mon[34520]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:10:15.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:14 smithi163 ceph-mon[34520]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-05-08T06:10:15.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:14 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/1942192102' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T06:10:15.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:14 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/1488691605' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T06:10:15.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:14 smithi192 ceph-mon[34664]: pgmap v453: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-08T06:10:15.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:14 smithi192 ceph-mon[34664]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:10:15.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:14 smithi192 ceph-mon[34664]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-05-08T06:10:15.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:14 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/1942192102' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T06:10:15.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:14 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/1488691605' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T06:10:15.557 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:10:15.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:15 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/4217251347' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T06:10:16.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:15 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/4217251347' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T06:10:16.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:15 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/4217251347' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T06:10:17.174 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:16 smithi123 ceph-mon[29787]: pgmap v454: 97 pgs: 97 active+clean; 645 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-08T06:10:17.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:16 smithi163 ceph-mon[34520]: pgmap v454: 97 pgs: 97 active+clean; 645 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-08T06:10:17.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:16 smithi192 ceph-mon[34664]: pgmap v454: 97 pgs: 97 active+clean; 645 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-08T06:10:18.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.078+0000 7fd57ec00700 1 -- 172.21.15.123:0/2193648941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd57806b180 msgr2=0x7fd57806b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:18.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.078+0000 7fd57ec00700 1 --2- 172.21.15.123:0/2193648941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd57806b180 0x7fd57806b600 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd560009990 tx=0x7fd560009ca0 comp rx=0 tx=0).stop 2024-05-08T06:10:18.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.079+0000 7fd57ec00700 1 -- 172.21.15.123:0/2193648941 shutdown_connections 2024-05-08T06:10:18.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.079+0000 7fd57ec00700 1 --2- 172.21.15.123:0/2193648941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd57806bb40 0x7fd578110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:18.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.079+0000 7fd57ec00700 1 --2- 172.21.15.123:0/2193648941 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd578109510 0x7fd5781098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:18.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.079+0000 7fd57ec00700 1 --2- 172.21.15.123:0/2193648941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd57806b180 0x7fd57806b600 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:18.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.079+0000 7fd57ec00700 1 -- 172.21.15.123:0/2193648941 >> 172.21.15.123:0/2193648941 conn(0x7fd578079350 msgr2=0x7fd578079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:18.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.079+0000 7fd57ec00700 1 -- 172.21.15.123:0/2193648941 shutdown_connections 2024-05-08T06:10:18.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.079+0000 7fd57ec00700 1 -- 172.21.15.123:0/2193648941 wait complete. 2024-05-08T06:10:18.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.080+0000 7fd57ec00700 1 Processor -- start 2024-05-08T06:10:18.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.080+0000 7fd57ec00700 1 -- start start 2024-05-08T06:10:18.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.081+0000 7fd57ec00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd57806b180 0x7fd57819bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:18.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.081+0000 7fd57ec00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd57806bb40 0x7fd57819c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:18.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.081+0000 7fd57ec00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd578109510 0x7fd5781a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:18.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.081+0000 7fd57ec00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd578113640 con 0x7fd57806b180 2024-05-08T06:10:18.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.081+0000 7fd57ec00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd5781134c0 con 0x7fd578109510 2024-05-08T06:10:18.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.081+0000 7fd57ec00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd5781137c0 con 0x7fd57806bb40 2024-05-08T06:10:18.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.081+0000 7fd577e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd57806b180 0x7fd57819bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:18.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.081+0000 7fd56f400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd57806bb40 0x7fd57819c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:18.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.081+0000 7fd56f400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd57806bb40 0x7fd57819c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:46876/0 (socket says 172.21.15.123:46876) 2024-05-08T06:10:18.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.081+0000 7fd56f400700 1 -- 172.21.15.123:0/2724349144 learned_addr learned my addr 172.21.15.123:0/2724349144 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:10:18.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.081+0000 7fd57cc00700 1 --2- 172.21.15.123:0/2724349144 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd578109510 0x7fd5781a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:18.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.082+0000 7fd56f400700 1 -- 172.21.15.123:0/2724349144 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd578109510 msgr2=0x7fd5781a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:18.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.082+0000 7fd56f400700 1 --2- 172.21.15.123:0/2724349144 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd578109510 0x7fd5781a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:18.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.082+0000 7fd56f400700 1 -- 172.21.15.123:0/2724349144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd57806b180 msgr2=0x7fd57819bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:18.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.082+0000 7fd56f400700 1 --2- 172.21.15.123:0/2724349144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd57806b180 0x7fd57819bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:18.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.082+0000 7fd56f400700 1 -- 172.21.15.123:0/2724349144 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5781a0d60 con 0x7fd57806bb40 2024-05-08T06:10:18.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.082+0000 7fd577e00700 1 --2- 172.21.15.123:0/2724349144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd57806b180 0x7fd57819bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:10:18.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.082+0000 7fd56f400700 1 --2- 172.21.15.123:0/2724349144 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd57806bb40 0x7fd57819c290 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd560000c00 tx=0x7fd56000bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:18.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.082+0000 7fd575600700 1 -- 172.21.15.123:0/2724349144 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd560027070 con 0x7fd57806bb40 2024-05-08T06:10:18.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.083+0000 7fd575600700 1 -- 172.21.15.123:0/2724349144 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd560022470 con 0x7fd57806bb40 2024-05-08T06:10:18.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.083+0000 7fd575600700 1 -- 172.21.15.123:0/2724349144 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd560017650 con 0x7fd57806bb40 2024-05-08T06:10:18.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.083+0000 7fd57ec00700 1 -- 172.21.15.123:0/2724349144 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5781a0ff0 con 0x7fd57806bb40 2024-05-08T06:10:18.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.083+0000 7fd57ec00700 1 -- 172.21.15.123:0/2724349144 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5781a88d0 con 0x7fd57806bb40 2024-05-08T06:10:18.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.084+0000 7fd57ec00700 1 -- 172.21.15.123:0/2724349144 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd57806a190 con 0x7fd57806bb40 2024-05-08T06:10:18.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.086+0000 7fd575600700 1 -- 172.21.15.123:0/2724349144 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) ==== 149704+0+0 (secure 0 0 0) 0x7fd56002a070 con 0x7fd57806bb40 2024-05-08T06:10:18.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.088+0000 7fd575600700 1 --2- 172.21.15.123:0/2724349144 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd55c0b2910 0x7fd55c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:18.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.088+0000 7fd575600700 1 -- 172.21.15.123:0/2724349144 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fd5600e3540 con 0x7fd57806bb40 2024-05-08T06:10:18.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.088+0000 7fd577e00700 1 --2- 172.21.15.123:0/2724349144 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd55c0b2910 0x7fd55c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:18.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.089+0000 7fd577e00700 1 --2- 172.21.15.123:0/2724349144 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd55c0b2910 0x7fd55c0b4dd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd568009480 tx=0x7fd56800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:18.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.090+0000 7fd575600700 1 -- 172.21.15.123:0/2724349144 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd560013070 con 0x7fd57806bb40 2024-05-08T06:10:18.373 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.371+0000 7fd57ec00700 1 -- 172.21.15.123:0/2724349144 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5781a8aa0 con 0x7fd57806bb40 2024-05-08T06:10:18.374 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.373+0000 7fd575600700 1 -- 172.21.15.123:0/2724349144 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11885 (secure 0 0 0) 0x7fd5600ab140 con 0x7fd57806bb40 2024-05-08T06:10:18.374 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:10:18.374 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3646476522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3646476522},{"type":"v1","addr":"172.21.15.123:6835","nonce":3646476522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2763706497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2763706497},{"type":"v1","addr":"172.21.15.123:6837","nonce":2763706497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/274635265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":274635265},{"type":"v1","addr":"172.21.15.123:6839","nonce":274635265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/175423115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":175423115},{"type":"v1","addr":"172.21.15.192:6833","nonce":175423115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/3263009472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3263009472},{"type":"v1","addr":"172.21.15.163:6833","nonce":3263009472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3308612826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3308612826},{"type":"v1","addr":"172.21.15.163:6835","nonce":3308612826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T06:05:54.755550+0000","modified":"2024-05-08T06:10:12.600417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24512,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/202513680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":202513680},{"type":"v1","addr":"172.21.15.192:6835","nonce":202513680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/2483545969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2483545969},{"type":"v1","addr":"172.21.15.192:6837","nonce":2483545969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/660511838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":660511838},{"type":"v1","addr":"172.21.15.163:6837","nonce":660511838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.192:6839/3598682360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":3598682360},{"type":"v1","addr":"172.21.15.192:6839","nonce":3598682360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6841/1571726231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":1571726231},{"type":"v1","addr":"172.21.15.123:6841","nonce":1571726231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.163:6839/1568161515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":1568161515},{"type":"v1","addr":"172.21.15.163:6839","nonce":1568161515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24512,24496,24524]},"id":1}]} 2024-05-08T06:10:18.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.376+0000 7fd57ec00700 1 -- 172.21.15.123:0/2724349144 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd55c0b2910 msgr2=0x7fd55c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:18.377 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.376+0000 7fd57ec00700 1 --2- 172.21.15.123:0/2724349144 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd55c0b2910 0x7fd55c0b4dd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd568009480 tx=0x7fd56800b040 comp rx=0 tx=0).stop 2024-05-08T06:10:18.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.376+0000 7fd57ec00700 1 -- 172.21.15.123:0/2724349144 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd57806bb40 msgr2=0x7fd57819c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:18.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.376+0000 7fd57ec00700 1 --2- 172.21.15.123:0/2724349144 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd57806bb40 0x7fd57819c290 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd560000c00 tx=0x7fd56000bfd0 comp rx=0 tx=0).stop 2024-05-08T06:10:18.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.376+0000 7fd57ec00700 1 -- 172.21.15.123:0/2724349144 shutdown_connections 2024-05-08T06:10:18.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.376+0000 7fd57ec00700 1 --2- 172.21.15.123:0/2724349144 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd55c0b2910 0x7fd55c0b4dd0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:18.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.376+0000 7fd57ec00700 1 --2- 172.21.15.123:0/2724349144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd57806b180 0x7fd57819bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:18.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.376+0000 7fd57ec00700 1 --2- 172.21.15.123:0/2724349144 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd578109510 0x7fd5781a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:18.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.376+0000 7fd57ec00700 1 --2- 172.21.15.123:0/2724349144 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd57806bb40 0x7fd57819c290 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:18.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.376+0000 7fd57ec00700 1 -- 172.21.15.123:0/2724349144 >> 172.21.15.123:0/2724349144 conn(0x7fd578079350 msgr2=0x7fd57810e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:18.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.376+0000 7fd57ec00700 1 -- 172.21.15.123:0/2724349144 shutdown_connections 2024-05-08T06:10:18.378 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:18.377+0000 7fd57ec00700 1 -- 172.21.15.123:0/2724349144 wait complete. 2024-05-08T06:10:18.379 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 24 2024-05-08T06:10:19.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:18 smithi123 ceph-mon[29787]: pgmap v455: 97 pgs: 97 active+clean; 645 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 852 B/s wr, 7 op/s 2024-05-08T06:10:19.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:18 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/2724349144' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:10:19.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:18 smithi163 ceph-mon[34520]: pgmap v455: 97 pgs: 97 active+clean; 645 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 852 B/s wr, 7 op/s 2024-05-08T06:10:19.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:18 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/2724349144' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:10:19.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:18 smithi192 ceph-mon[34664]: pgmap v455: 97 pgs: 97 active+clean; 645 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 852 B/s wr, 7 op/s 2024-05-08T06:10:19.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:18 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/2724349144' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:10:19.423 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-05-08T06:10:19.423 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-05-08T06:10:19.424 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T06:10:19.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.633+0000 7f3c28000640 1 -- 172.21.15.123:0/2551432652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c200785b0 msgr2=0x7f3c20078a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:19.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.633+0000 7f3c28000640 1 --2- 172.21.15.123:0/2551432652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c200785b0 0x7f3c20078a10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c009900 tx=0x7f3c0c02a3f0 comp rx=0 tx=0).stop 2024-05-08T06:10:19.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.634+0000 7f3c28000640 1 -- 172.21.15.123:0/2551432652 shutdown_connections 2024-05-08T06:10:19.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.634+0000 7f3c28000640 1 --2- 172.21.15.123:0/2551432652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3c20078f50 0x7f3c200797f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:19.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.634+0000 7f3c28000640 1 --2- 172.21.15.123:0/2551432652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c200785b0 0x7f3c20078a10 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:19.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.634+0000 7f3c28000640 1 --2- 172.21.15.123:0/2551432652 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c20077370 0x7f3c20077790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:19.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.634+0000 7f3c28000640 1 -- 172.21.15.123:0/2551432652 >> 172.21.15.123:0/2551432652 conn(0x7f3c201001a0 msgr2=0x7f3c201025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:19.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.634+0000 7f3c28000640 1 -- 172.21.15.123:0/2551432652 shutdown_connections 2024-05-08T06:10:19.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.634+0000 7f3c28000640 1 -- 172.21.15.123:0/2551432652 wait complete. 2024-05-08T06:10:19.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.635+0000 7f3c28000640 1 Processor -- start 2024-05-08T06:10:19.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.635+0000 7f3c28000640 1 -- start start 2024-05-08T06:10:19.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.635+0000 7f3c28000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c20077370 0x7f3c2019f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:19.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.635+0000 7f3c28000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c200785b0 0x7f3c2019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:19.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.635+0000 7f3c28000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3c20078f50 0x7f3c201a6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:19.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.635+0000 7f3c28000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3c20114b70 con 0x7f3c200785b0 2024-05-08T06:10:19.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.635+0000 7f3c28000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f3c201149f0 con 0x7f3c20078f50 2024-05-08T06:10:19.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.635+0000 7f3c28000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3c20114cf0 con 0x7f3c20077370 2024-05-08T06:10:19.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.636+0000 7f3c24c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c200785b0 0x7f3c2019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:19.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.636+0000 7f3c24c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c200785b0 0x7f3c2019f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41654/0 (socket says 172.21.15.123:41654) 2024-05-08T06:10:19.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.636+0000 7f3c24c00640 1 -- 172.21.15.123:0/2256934025 learned_addr learned my addr 172.21.15.123:0/2256934025 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:10:19.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.636+0000 7f3c26000640 1 --2- 172.21.15.123:0/2256934025 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3c20078f50 0x7f3c201a6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:19.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.636+0000 7f3c25600640 1 --2- 172.21.15.123:0/2256934025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c20077370 0x7f3c2019f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:19.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.636+0000 7f3c24c00640 1 -- 172.21.15.123:0/2256934025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c20077370 msgr2=0x7f3c2019f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:19.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.636+0000 7f3c24c00640 1 --2- 172.21.15.123:0/2256934025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c20077370 0x7f3c2019f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:19.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.636+0000 7f3c24c00640 1 -- 172.21.15.123:0/2256934025 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3c20078f50 msgr2=0x7f3c201a6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:19.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.636+0000 7f3c24c00640 1 --2- 172.21.15.123:0/2256934025 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3c20078f50 0x7f3c201a6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:19.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.636+0000 7f3c24c00640 1 -- 172.21.15.123:0/2256934025 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c201a6ea0 con 0x7f3c200785b0 2024-05-08T06:10:19.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.636+0000 7f3c26000640 1 --2- 172.21.15.123:0/2256934025 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3c20078f50 0x7f3c201a6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:10:19.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.636+0000 7f3c24c00640 1 --2- 172.21.15.123:0/2256934025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c200785b0 0x7f3c2019f7c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c005940 tx=0x7f3c0c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:19.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.637+0000 7f3c1e000640 1 -- 172.21.15.123:0/2256934025 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c03f030 con 0x7f3c200785b0 2024-05-08T06:10:19.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.637+0000 7f3c28000640 1 -- 172.21.15.123:0/2256934025 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c201a7130 con 0x7f3c200785b0 2024-05-08T06:10:19.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.637+0000 7f3c1e000640 1 -- 172.21.15.123:0/2256934025 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c0c003cb0 con 0x7f3c200785b0 2024-05-08T06:10:19.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.637+0000 7f3c1e000640 1 -- 172.21.15.123:0/2256934025 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c035070 con 0x7f3c200785b0 2024-05-08T06:10:19.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.637+0000 7f3c28000640 1 -- 172.21.15.123:0/2256934025 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c201a7670 con 0x7f3c200785b0 2024-05-08T06:10:19.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.638+0000 7f3c28000640 1 -- 172.21.15.123:0/2256934025 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c20111820 con 0x7f3c200785b0 2024-05-08T06:10:19.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.640+0000 7f3c1e000640 1 -- 172.21.15.123:0/2256934025 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 19) ==== 149704+0+0 (secure 0 0 0) 0x7f3c0c042030 con 0x7f3c200785b0 2024-05-08T06:10:19.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.642+0000 7f3c1e000640 1 --2- 172.21.15.123:0/2256934025 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f3c080b2880 0x7f3c080b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:19.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.642+0000 7f3c1e000640 1 -- 172.21.15.123:0/2256934025 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f3c0c1027b0 con 0x7f3c200785b0 2024-05-08T06:10:19.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.642+0000 7f3c25600640 1 --2- 172.21.15.123:0/2256934025 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f3c080b2880 0x7f3c080b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:19.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.643+0000 7f3c25600640 1 --2- 172.21.15.123:0/2256934025 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f3c080b2880 0x7f3c080b4d40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3c14009bd0 tx=0x7f3c14005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:19.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.647+0000 7f3c1e000640 1 -- 172.21.15.123:0/2256934025 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3c0c003e60 con 0x7f3c200785b0 2024-05-08T06:10:19.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.852+0000 7f3c28000640 1 -- 172.21.15.123:0/2256934025 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3c20062df0 con 0x7f3c080b2880 2024-05-08T06:10:19.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.874+0000 7f3c1e000640 1 -- 172.21.15.123:0/2256934025 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259055 (secure 0 0 0) 0x7f3c20062df0 con 0x7f3c080b2880 2024-05-08T06:10:19.880 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:10:19.881 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-05-08T06:10:19.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.881+0000 7f3c28000640 1 -- 172.21.15.123:0/2256934025 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f3c080b2880 msgr2=0x7f3c080b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:19.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.881+0000 7f3c28000640 1 --2- 172.21.15.123:0/2256934025 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f3c080b2880 0x7f3c080b4d40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3c14009bd0 tx=0x7f3c14005f90 comp rx=0 tx=0).stop 2024-05-08T06:10:19.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.881+0000 7f3c28000640 1 -- 172.21.15.123:0/2256934025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c200785b0 msgr2=0x7f3c2019f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:19.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.881+0000 7f3c28000640 1 --2- 172.21.15.123:0/2256934025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c200785b0 0x7f3c2019f7c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c005940 tx=0x7f3c0c0042f0 comp rx=0 tx=0).stop 2024-05-08T06:10:19.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.881+0000 7f3c28000640 1 -- 172.21.15.123:0/2256934025 shutdown_connections 2024-05-08T06:10:19.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.881+0000 7f3c28000640 1 --2- 172.21.15.123:0/2256934025 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f3c080b2880 0x7f3c080b4d40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:19.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.881+0000 7f3c28000640 1 --2- 172.21.15.123:0/2256934025 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3c20078f50 0x7f3c201a6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:19.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.881+0000 7f3c28000640 1 --2- 172.21.15.123:0/2256934025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c200785b0 0x7f3c2019f7c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:19.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.881+0000 7f3c28000640 1 --2- 172.21.15.123:0/2256934025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c20077370 0x7f3c2019f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:19.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.881+0000 7f3c28000640 1 -- 172.21.15.123:0/2256934025 >> 172.21.15.123:0/2256934025 conn(0x7f3c201001a0 msgr2=0x7f3c20101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:19.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.881+0000 7f3c28000640 1 -- 172.21.15.123:0/2256934025 shutdown_connections 2024-05-08T06:10:19.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:19.881+0000 7f3c28000640 1 -- 172.21.15.123:0/2256934025 wait complete. 2024-05-08T06:10:19.903 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":456,"stamp":"2024-05-08T06:10:19.154293+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668889,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1542,"num_read_kb":882,"num_write":315,"num_write_kb":2293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":2951896,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":755,"ondisk_log_size":755,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372504,"kb_used_data":10872,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569320,"statfs":{"total":1151940427776,"available":1151558983680,"internally_reserved":0,"allocated":11132928,"data_stored":7181319,"data_compressed":113232,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9686,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":81,"num_read_kb":53,"num_write":15,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005348"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.064237+0000","last_change":"2024-05-08T06:05:51.431744+0000","last_active":"2024-05-08T06:08:01.064237+0000","last_peered":"2024-05-08T06:08:01.064237+0000","last_clean":"2024-05-08T06:08:01.064237+0000","last_became_active":"2024-05-08T06:05:51.431405+0000","last_became_peered":"2024-05-08T06:05:51.431405+0000","last_unstale":"2024-05-08T06:08:01.064237+0000","last_undegraded":"2024-05-08T06:08:01.064237+0000","last_fullsized":"2024-05-08T06:08:01.064237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:56:40.503607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.062583+0000","last_change":"2024-05-08T06:05:51.426473+0000","last_active":"2024-05-08T06:08:01.062583+0000","last_peered":"2024-05-08T06:08:01.062583+0000","last_clean":"2024-05-08T06:08:01.062583+0000","last_became_active":"2024-05-08T06:05:51.424477+0000","last_became_peered":"2024-05-08T06:05:51.424477+0000","last_unstale":"2024-05-08T06:08:01.062583+0000","last_undegraded":"2024-05-08T06:08:01.062583+0000","last_fullsized":"2024-05-08T06:08:01.062583+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:26:55.747808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.060622+0000","last_change":"2024-05-08T06:05:51.423767+0000","last_active":"2024-05-08T06:08:01.060622+0000","last_peered":"2024-05-08T06:08:01.060622+0000","last_clean":"2024-05-08T06:08:01.060622+0000","last_became_active":"2024-05-08T06:05:51.423273+0000","last_became_peered":"2024-05-08T06:05:51.423273+0000","last_unstale":"2024-05-08T06:08:01.060622+0000","last_undegraded":"2024-05-08T06:08:01.060622+0000","last_fullsized":"2024-05-08T06:08:01.060622+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:24:44.099743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.047241+0000","last_change":"2024-05-08T06:05:51.416604+0000","last_active":"2024-05-08T06:08:01.047241+0000","last_peered":"2024-05-08T06:08:01.047241+0000","last_clean":"2024-05-08T06:08:01.047241+0000","last_became_active":"2024-05-08T06:05:51.416284+0000","last_became_peered":"2024-05-08T06:05:51.416284+0000","last_unstale":"2024-05-08T06:08:01.047241+0000","last_undegraded":"2024-05-08T06:08:01.047241+0000","last_fullsized":"2024-05-08T06:08:01.047241+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:05:26.014323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.059697+0000","last_change":"2024-05-08T06:05:51.423915+0000","last_active":"2024-05-08T06:08:01.059697+0000","last_peered":"2024-05-08T06:08:01.059697+0000","last_clean":"2024-05-08T06:08:01.059697+0000","last_became_active":"2024-05-08T06:05:51.423534+0000","last_became_peered":"2024-05-08T06:05:51.423534+0000","last_unstale":"2024-05-08T06:08:01.059697+0000","last_undegraded":"2024-05-08T06:08:01.059697+0000","last_fullsized":"2024-05-08T06:08:01.059697+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:06:46.197071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.041181+0000","last_change":"2024-05-08T06:05:51.411102+0000","last_active":"2024-05-08T06:08:01.041181+0000","last_peered":"2024-05-08T06:08:01.041181+0000","last_clean":"2024-05-08T06:08:01.041181+0000","last_became_active":"2024-05-08T06:05:51.410791+0000","last_became_peered":"2024-05-08T06:05:51.410791+0000","last_unstale":"2024-05-08T06:08:01.041181+0000","last_undegraded":"2024-05-08T06:08:01.041181+0000","last_fullsized":"2024-05-08T06:08:01.041181+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:31:47.861663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.041524+0000","last_change":"2024-05-08T06:05:51.417756+0000","last_active":"2024-05-08T06:08:01.041524+0000","last_peered":"2024-05-08T06:08:01.041524+0000","last_clean":"2024-05-08T06:08:01.041524+0000","last_became_active":"2024-05-08T06:05:51.417373+0000","last_became_peered":"2024-05-08T06:05:51.417373+0000","last_unstale":"2024-05-08T06:08:01.041524+0000","last_undegraded":"2024-05-08T06:08:01.041524+0000","last_fullsized":"2024-05-08T06:08:01.041524+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:07:48.330401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.063489+0000","last_change":"2024-05-08T06:05:51.413883+0000","last_active":"2024-05-08T06:08:01.063489+0000","last_peered":"2024-05-08T06:08:01.063489+0000","last_clean":"2024-05-08T06:08:01.063489+0000","last_became_active":"2024-05-08T06:05:51.413565+0000","last_became_peered":"2024-05-08T06:05:51.413565+0000","last_unstale":"2024-05-08T06:08:01.063489+0000","last_undegraded":"2024-05-08T06:08:01.063489+0000","last_fullsized":"2024-05-08T06:08:01.063489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:06:38.186225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.053054+0000","last_change":"2024-05-08T06:05:51.416300+0000","last_active":"2024-05-08T06:08:01.053054+0000","last_peered":"2024-05-08T06:08:01.053054+0000","last_clean":"2024-05-08T06:08:01.053054+0000","last_became_active":"2024-05-08T06:05:51.415970+0000","last_became_peered":"2024-05-08T06:05:51.415970+0000","last_unstale":"2024-05-08T06:08:01.053054+0000","last_undegraded":"2024-05-08T06:08:01.053054+0000","last_fullsized":"2024-05-08T06:08:01.053054+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:44:51.853339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.039789+0000","last_change":"2024-05-08T06:05:51.410396+0000","last_active":"2024-05-08T06:08:01.039789+0000","last_peered":"2024-05-08T06:08:01.039789+0000","last_clean":"2024-05-08T06:08:01.039789+0000","last_became_active":"2024-05-08T06:05:51.410067+0000","last_became_peered":"2024-05-08T06:05:51.410067+0000","last_unstale":"2024-05-08T06:08:01.039789+0000","last_undegraded":"2024-05-08T06:08:01.039789+0000","last_fullsized":"2024-05-08T06:08:01.039789+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:32:00.217217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.052539+0000","last_change":"2024-05-08T06:05:51.409276+0000","last_active":"2024-05-08T06:08:01.052539+0000","last_peered":"2024-05-08T06:08:01.052539+0000","last_clean":"2024-05-08T06:08:01.052539+0000","last_became_active":"2024-05-08T06:05:51.408768+0000","last_became_peered":"2024-05-08T06:05:51.408768+0000","last_unstale":"2024-05-08T06:08:01.052539+0000","last_undegraded":"2024-05-08T06:08:01.052539+0000","last_fullsized":"2024-05-08T06:08:01.052539+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:22:57.483944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.048049+0000","last_change":"2024-05-08T06:05:51.419083+0000","last_active":"2024-05-08T06:08:01.048049+0000","last_peered":"2024-05-08T06:08:01.048049+0000","last_clean":"2024-05-08T06:08:01.048049+0000","last_became_active":"2024-05-08T06:05:51.418685+0000","last_became_peered":"2024-05-08T06:05:51.418685+0000","last_unstale":"2024-05-08T06:08:01.048049+0000","last_undegraded":"2024-05-08T06:08:01.048049+0000","last_fullsized":"2024-05-08T06:08:01.048049+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:25:49.256242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.057514+0000","last_change":"2024-05-08T06:05:51.420399+0000","last_active":"2024-05-08T06:08:01.057514+0000","last_peered":"2024-05-08T06:08:01.057514+0000","last_clean":"2024-05-08T06:08:01.057514+0000","last_became_active":"2024-05-08T06:05:51.420076+0000","last_became_peered":"2024-05-08T06:05:51.420076+0000","last_unstale":"2024-05-08T06:08:01.057514+0000","last_undegraded":"2024-05-08T06:08:01.057514+0000","last_fullsized":"2024-05-08T06:08:01.057514+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:51:38.579280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.059465+0000","last_change":"2024-05-08T06:05:51.431518+0000","last_active":"2024-05-08T06:08:01.059465+0000","last_peered":"2024-05-08T06:08:01.059465+0000","last_clean":"2024-05-08T06:08:01.059465+0000","last_became_active":"2024-05-08T06:05:51.431029+0000","last_became_peered":"2024-05-08T06:05:51.431029+0000","last_unstale":"2024-05-08T06:08:01.059465+0000","last_undegraded":"2024-05-08T06:08:01.059465+0000","last_fullsized":"2024-05-08T06:08:01.059465+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:11:09.596221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.061333+0000","last_change":"2024-05-08T06:06:32.498002+0000","last_active":"2024-05-08T06:08:01.061333+0000","last_peered":"2024-05-08T06:08:01.061333+0000","last_clean":"2024-05-08T06:08:01.061333+0000","last_became_active":"2024-05-08T06:06:32.497367+0000","last_became_peered":"2024-05-08T06:06:32.497367+0000","last_unstale":"2024-05-08T06:08:01.061333+0000","last_undegraded":"2024-05-08T06:08:01.061333+0000","last_fullsized":"2024-05-08T06:08:01.061333+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:32:36.461777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.064064+0000","last_change":"2024-05-08T06:05:51.423857+0000","last_active":"2024-05-08T06:08:01.064064+0000","last_peered":"2024-05-08T06:08:01.064064+0000","last_clean":"2024-05-08T06:08:01.064064+0000","last_became_active":"2024-05-08T06:05:51.423487+0000","last_became_peered":"2024-05-08T06:05:51.423487+0000","last_unstale":"2024-05-08T06:08:01.064064+0000","last_undegraded":"2024-05-08T06:08:01.064064+0000","last_fullsized":"2024-05-08T06:08:01.064064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:17:33.269354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.064245+0000","last_change":"2024-05-08T06:05:51.431795+0000","last_active":"2024-05-08T06:08:01.064245+0000","last_peered":"2024-05-08T06:08:01.064245+0000","last_clean":"2024-05-08T06:08:01.064245+0000","last_became_active":"2024-05-08T06:05:51.431432+0000","last_became_peered":"2024-05-08T06:05:51.431432+0000","last_unstale":"2024-05-08T06:08:01.064245+0000","last_undegraded":"2024-05-08T06:08:01.064245+0000","last_fullsized":"2024-05-08T06:08:01.064245+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:11:16.561478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.048666+0000","last_change":"2024-05-08T06:05:51.414805+0000","last_active":"2024-05-08T06:08:01.048666+0000","last_peered":"2024-05-08T06:08:01.048666+0000","last_clean":"2024-05-08T06:08:01.048666+0000","last_became_active":"2024-05-08T06:05:51.414482+0000","last_became_peered":"2024-05-08T06:05:51.414482+0000","last_unstale":"2024-05-08T06:08:01.048666+0000","last_undegraded":"2024-05-08T06:08:01.048666+0000","last_fullsized":"2024-05-08T06:08:01.048666+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:08:13.444591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.064433+0000","last_change":"2024-05-08T06:05:51.418516+0000","last_active":"2024-05-08T06:08:01.064433+0000","last_peered":"2024-05-08T06:08:01.064433+0000","last_clean":"2024-05-08T06:08:01.064433+0000","last_became_active":"2024-05-08T06:05:51.418182+0000","last_became_peered":"2024-05-08T06:05:51.418182+0000","last_unstale":"2024-05-08T06:08:01.064433+0000","last_undegraded":"2024-05-08T06:08:01.064433+0000","last_fullsized":"2024-05-08T06:08:01.064433+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:32:33.338311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:07:51.948755+0000","last_change":"2024-05-08T06:05:46.350509+0000","last_active":"2024-05-08T06:07:51.948755+0000","last_peered":"2024-05-08T06:07:51.948755+0000","last_clean":"2024-05-08T06:07:51.948755+0000","last_became_active":"2024-05-08T06:05:46.349986+0000","last_became_peered":"2024-05-08T06:05:46.349986+0000","last_unstale":"2024-05-08T06:07:51.948755+0000","last_undegraded":"2024-05-08T06:07:51.948755+0000","last_fullsized":"2024-05-08T06:07:51.948755+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:53:00.928971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.061073+0000","last_change":"2024-05-08T06:05:51.413621+0000","last_active":"2024-05-08T06:08:01.061073+0000","last_peered":"2024-05-08T06:08:01.061073+0000","last_clean":"2024-05-08T06:08:01.061073+0000","last_became_active":"2024-05-08T06:05:51.413188+0000","last_became_peered":"2024-05-08T06:05:51.413188+0000","last_unstale":"2024-05-08T06:08:01.061073+0000","last_undegraded":"2024-05-08T06:08:01.061073+0000","last_fullsized":"2024-05-08T06:08:01.061073+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:13:21.413696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:07:50.005420+0000","last_change":"2024-05-08T06:05:46.552444+0000","last_active":"2024-05-08T06:07:50.005420+0000","last_peered":"2024-05-08T06:07:50.005420+0000","last_clean":"2024-05-08T06:07:50.005420+0000","last_became_active":"2024-05-08T06:05:46.551310+0000","last_became_peered":"2024-05-08T06:05:46.551310+0000","last_unstale":"2024-05-08T06:07:50.005420+0000","last_undegraded":"2024-05-08T06:07:50.005420+0000","last_fullsized":"2024-05-08T06:07:50.005420+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:58:17.632832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.062434+0000","last_change":"2024-05-08T06:05:51.425649+0000","last_active":"2024-05-08T06:08:01.062434+0000","last_peered":"2024-05-08T06:08:01.062434+0000","last_clean":"2024-05-08T06:08:01.062434+0000","last_became_active":"2024-05-08T06:05:51.425288+0000","last_became_peered":"2024-05-08T06:05:51.425288+0000","last_unstale":"2024-05-08T06:08:01.062434+0000","last_undegraded":"2024-05-08T06:08:01.062434+0000","last_fullsized":"2024-05-08T06:08:01.062434+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:42:40.580641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480655+0000","last_change":"2024-05-08T06:05:46.350300+0000","last_active":"2024-05-08T06:06:32.480655+0000","last_peered":"2024-05-08T06:06:32.480655+0000","last_clean":"2024-05-08T06:06:32.480655+0000","last_became_active":"2024-05-08T06:05:46.349918+0000","last_became_peered":"2024-05-08T06:05:46.349918+0000","last_unstale":"2024-05-08T06:06:32.480655+0000","last_undegraded":"2024-05-08T06:06:32.480655+0000","last_fullsized":"2024-05-08T06:06:32.480655+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:32:41.952772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.057546+0000","last_change":"2024-05-08T06:05:51.430130+0000","last_active":"2024-05-08T06:08:01.057546+0000","last_peered":"2024-05-08T06:08:01.057546+0000","last_clean":"2024-05-08T06:08:01.057546+0000","last_became_active":"2024-05-08T06:05:51.429794+0000","last_became_peered":"2024-05-08T06:05:51.429794+0000","last_unstale":"2024-05-08T06:08:01.057546+0000","last_undegraded":"2024-05-08T06:08:01.057546+0000","last_fullsized":"2024-05-08T06:08:01.057546+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:21:55.773309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480539+0000","last_change":"2024-05-08T06:05:46.351664+0000","last_active":"2024-05-08T06:06:32.480539+0000","last_peered":"2024-05-08T06:06:32.480539+0000","last_clean":"2024-05-08T06:06:32.480539+0000","last_became_active":"2024-05-08T06:05:46.351305+0000","last_became_peered":"2024-05-08T06:05:46.351305+0000","last_unstale":"2024-05-08T06:06:32.480539+0000","last_undegraded":"2024-05-08T06:06:32.480539+0000","last_fullsized":"2024-05-08T06:06:32.480539+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:24:36.164315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.043418+0000","last_change":"2024-05-08T06:05:51.414793+0000","last_active":"2024-05-08T06:08:01.043418+0000","last_peered":"2024-05-08T06:08:01.043418+0000","last_clean":"2024-05-08T06:08:01.043418+0000","last_became_active":"2024-05-08T06:05:51.414317+0000","last_became_peered":"2024-05-08T06:05:51.414317+0000","last_unstale":"2024-05-08T06:08:01.043418+0000","last_undegraded":"2024-05-08T06:08:01.043418+0000","last_fullsized":"2024-05-08T06:08:01.043418+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:57:56.152836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":274,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:10:13.723637+0000","last_change":"2024-05-08T06:05:46.352683+0000","last_active":"2024-05-08T06:10:13.723637+0000","last_peered":"2024-05-08T06:10:13.723637+0000","last_clean":"2024-05-08T06:10:13.723637+0000","last_became_active":"2024-05-08T06:05:46.352278+0000","last_became_peered":"2024-05-08T06:05:46.352278+0000","last_unstale":"2024-05-08T06:10:13.723637+0000","last_undegraded":"2024-05-08T06:10:13.723637+0000","last_fullsized":"2024-05-08T06:10:13.723637+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:39:22.655516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1323,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":240,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.037458+0000","last_change":"2024-05-08T06:05:51.415822+0000","last_active":"2024-05-08T06:08:01.037458+0000","last_peered":"2024-05-08T06:08:01.037458+0000","last_clean":"2024-05-08T06:08:01.037458+0000","last_became_active":"2024-05-08T06:05:51.415449+0000","last_became_peered":"2024-05-08T06:05:51.415449+0000","last_unstale":"2024-05-08T06:08:01.037458+0000","last_undegraded":"2024-05-08T06:08:01.037458+0000","last_fullsized":"2024-05-08T06:08:01.037458+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:44:49.127335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.478073+0000","last_change":"2024-05-08T06:05:46.361288+0000","last_active":"2024-05-08T06:06:32.478073+0000","last_peered":"2024-05-08T06:06:32.478073+0000","last_clean":"2024-05-08T06:06:32.478073+0000","last_became_active":"2024-05-08T06:05:46.360761+0000","last_became_peered":"2024-05-08T06:05:46.360761+0000","last_unstale":"2024-05-08T06:06:32.478073+0000","last_undegraded":"2024-05-08T06:06:32.478073+0000","last_fullsized":"2024-05-08T06:06:32.478073+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:14:59.858935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.063490+0000","last_change":"2024-05-08T06:05:51.418547+0000","last_active":"2024-05-08T06:08:01.063490+0000","last_peered":"2024-05-08T06:08:01.063490+0000","last_clean":"2024-05-08T06:08:01.063490+0000","last_became_active":"2024-05-08T06:05:51.417588+0000","last_became_peered":"2024-05-08T06:05:51.417588+0000","last_unstale":"2024-05-08T06:08:01.063490+0000","last_undegraded":"2024-05-08T06:08:01.063490+0000","last_fullsized":"2024-05-08T06:08:01.063490+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:16:41.451351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.492080+0000","last_change":"2024-05-08T06:05:46.362077+0000","last_active":"2024-05-08T06:06:32.492080+0000","last_peered":"2024-05-08T06:06:32.492080+0000","last_clean":"2024-05-08T06:06:32.492080+0000","last_became_active":"2024-05-08T06:05:46.360823+0000","last_became_peered":"2024-05-08T06:05:46.360823+0000","last_unstale":"2024-05-08T06:06:32.492080+0000","last_undegraded":"2024-05-08T06:06:32.492080+0000","last_fullsized":"2024-05-08T06:06:32.492080+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:14.169822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.063350+0000","last_change":"2024-05-08T06:05:51.423687+0000","last_active":"2024-05-08T06:08:01.063350+0000","last_peered":"2024-05-08T06:08:01.063350+0000","last_clean":"2024-05-08T06:08:01.063350+0000","last_became_active":"2024-05-08T06:05:51.417656+0000","last_became_peered":"2024-05-08T06:05:51.417656+0000","last_unstale":"2024-05-08T06:08:01.063350+0000","last_undegraded":"2024-05-08T06:08:01.063350+0000","last_fullsized":"2024-05-08T06:08:01.063350+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:59:31.319915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.595803+0000","last_change":"2024-05-08T06:05:46.351892+0000","last_active":"2024-05-08T06:06:32.595803+0000","last_peered":"2024-05-08T06:06:32.595803+0000","last_clean":"2024-05-08T06:06:32.595803+0000","last_became_active":"2024-05-08T06:05:46.351523+0000","last_became_peered":"2024-05-08T06:05:46.351523+0000","last_unstale":"2024-05-08T06:06:32.595803+0000","last_undegraded":"2024-05-08T06:06:32.595803+0000","last_fullsized":"2024-05-08T06:06:32.595803+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:11:23.794313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.063933+0000","last_change":"2024-05-08T06:05:51.407681+0000","last_active":"2024-05-08T06:08:01.063933+0000","last_peered":"2024-05-08T06:08:01.063933+0000","last_clean":"2024-05-08T06:08:01.063933+0000","last_became_active":"2024-05-08T06:05:51.407243+0000","last_became_peered":"2024-05-08T06:05:51.407243+0000","last_unstale":"2024-05-08T06:08:01.063933+0000","last_undegraded":"2024-05-08T06:08:01.063933+0000","last_fullsized":"2024-05-08T06:08:01.063933+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:06:30.490268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":271,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:10:16.728337+0000","last_change":"2024-05-08T06:05:46.356675+0000","last_active":"2024-05-08T06:10:16.728337+0000","last_peered":"2024-05-08T06:10:16.728337+0000","last_clean":"2024-05-08T06:10:16.728337+0000","last_became_active":"2024-05-08T06:05:46.356360+0000","last_became_peered":"2024-05-08T06:05:46.356360+0000","last_unstale":"2024-05-08T06:10:16.728337+0000","last_undegraded":"2024-05-08T06:10:16.728337+0000","last_fullsized":"2024-05-08T06:10:16.728337+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:46:48.131551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":240,"num_read_kb":240,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.047752+0000","last_change":"2024-05-08T06:05:51.421325+0000","last_active":"2024-05-08T06:08:01.047752+0000","last_peered":"2024-05-08T06:08:01.047752+0000","last_clean":"2024-05-08T06:08:01.047752+0000","last_became_active":"2024-05-08T06:05:51.420905+0000","last_became_peered":"2024-05-08T06:05:51.420905+0000","last_unstale":"2024-05-08T06:08:01.047752+0000","last_undegraded":"2024-05-08T06:08:01.047752+0000","last_fullsized":"2024-05-08T06:08:01.047752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:37:27.324920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482723+0000","last_change":"2024-05-08T06:05:46.351958+0000","last_active":"2024-05-08T06:06:32.482723+0000","last_peered":"2024-05-08T06:06:32.482723+0000","last_clean":"2024-05-08T06:06:32.482723+0000","last_became_active":"2024-05-08T06:05:46.351561+0000","last_became_peered":"2024-05-08T06:05:46.351561+0000","last_unstale":"2024-05-08T06:06:32.482723+0000","last_undegraded":"2024-05-08T06:06:32.482723+0000","last_fullsized":"2024-05-08T06:06:32.482723+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:11:27.727029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.061468+0000","last_change":"2024-05-08T06:05:51.431809+0000","last_active":"2024-05-08T06:08:01.061468+0000","last_peered":"2024-05-08T06:08:01.061468+0000","last_clean":"2024-05-08T06:08:01.061468+0000","last_became_active":"2024-05-08T06:05:51.431416+0000","last_became_peered":"2024-05-08T06:05:51.431416+0000","last_unstale":"2024-05-08T06:08:01.061468+0000","last_undegraded":"2024-05-08T06:08:01.061468+0000","last_fullsized":"2024-05-08T06:08:01.061468+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:48:16.704134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.596352+0000","last_change":"2024-05-08T06:05:46.550763+0000","last_active":"2024-05-08T06:06:32.596352+0000","last_peered":"2024-05-08T06:06:32.596352+0000","last_clean":"2024-05-08T06:06:32.596352+0000","last_became_active":"2024-05-08T06:05:46.550297+0000","last_became_peered":"2024-05-08T06:05:46.550297+0000","last_unstale":"2024-05-08T06:06:32.596352+0000","last_undegraded":"2024-05-08T06:06:32.596352+0000","last_fullsized":"2024-05-08T06:06:32.596352+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:32:19.907220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.051960+0000","last_change":"2024-05-08T06:05:51.414738+0000","last_active":"2024-05-08T06:08:01.051960+0000","last_peered":"2024-05-08T06:08:01.051960+0000","last_clean":"2024-05-08T06:08:01.051960+0000","last_became_active":"2024-05-08T06:05:51.414369+0000","last_became_peered":"2024-05-08T06:05:51.414369+0000","last_unstale":"2024-05-08T06:08:01.051960+0000","last_undegraded":"2024-05-08T06:08:01.051960+0000","last_fullsized":"2024-05-08T06:08:01.051960+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:57:44.215343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'39","reported_seq":560,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:10:17.729921+0000","last_change":"2024-05-08T06:05:46.349825+0000","last_active":"2024-05-08T06:10:17.729921+0000","last_peered":"2024-05-08T06:10:17.729921+0000","last_clean":"2024-05-08T06:10:17.729921+0000","last_became_active":"2024-05-08T06:05:46.349425+0000","last_became_peered":"2024-05-08T06:05:46.349425+0000","last_unstale":"2024-05-08T06:10:17.729921+0000","last_undegraded":"2024-05-08T06:10:17.729921+0000","last_fullsized":"2024-05-08T06:10:17.729921+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:29:39.123817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73053,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":494,"num_read_kb":319,"num_write":40,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.022668+0000","last_change":"2024-05-08T06:05:51.411341+0000","last_active":"2024-05-08T06:08:01.022668+0000","last_peered":"2024-05-08T06:08:01.022668+0000","last_clean":"2024-05-08T06:08:01.022668+0000","last_became_active":"2024-05-08T06:05:51.411002+0000","last_became_peered":"2024-05-08T06:05:51.411002+0000","last_unstale":"2024-05-08T06:08:01.022668+0000","last_undegraded":"2024-05-08T06:08:01.022668+0000","last_fullsized":"2024-05-08T06:08:01.022668+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:27:07.013809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:34.244232+0000","last_change":"2024-05-08T06:06:32.508904+0000","last_active":"2024-05-08T06:06:34.244232+0000","last_peered":"2024-05-08T06:06:34.244232+0000","last_clean":"2024-05-08T06:06:34.244232+0000","last_became_active":"2024-05-08T06:06:32.496001+0000","last_became_peered":"2024-05-08T06:06:32.496001+0000","last_unstale":"2024-05-08T06:06:34.244232+0000","last_undegraded":"2024-05-08T06:06:34.244232+0000","last_fullsized":"2024-05-08T06:06:34.244232+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:34:32.417105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.065212+0000","last_change":"2024-05-08T06:05:51.418395+0000","last_active":"2024-05-08T06:08:01.065212+0000","last_peered":"2024-05-08T06:08:01.065212+0000","last_clean":"2024-05-08T06:08:01.065212+0000","last_became_active":"2024-05-08T06:05:51.418035+0000","last_became_peered":"2024-05-08T06:05:51.418035+0000","last_unstale":"2024-05-08T06:08:01.065212+0000","last_undegraded":"2024-05-08T06:08:01.065212+0000","last_fullsized":"2024-05-08T06:08:01.065212+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:58:16.529144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.595996+0000","last_change":"2024-05-08T06:05:46.363088+0000","last_active":"2024-05-08T06:06:32.595996+0000","last_peered":"2024-05-08T06:06:32.595996+0000","last_clean":"2024-05-08T06:06:32.595996+0000","last_became_active":"2024-05-08T06:05:46.362488+0000","last_became_peered":"2024-05-08T06:05:46.362488+0000","last_unstale":"2024-05-08T06:06:32.595996+0000","last_undegraded":"2024-05-08T06:06:32.595996+0000","last_fullsized":"2024-05-08T06:06:32.595996+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:28:10.039647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.059203+0000","last_change":"2024-05-08T06:05:51.415567+0000","last_active":"2024-05-08T06:08:01.059203+0000","last_peered":"2024-05-08T06:08:01.059203+0000","last_clean":"2024-05-08T06:08:01.059203+0000","last_became_active":"2024-05-08T06:05:51.415148+0000","last_became_peered":"2024-05-08T06:05:51.415148+0000","last_unstale":"2024-05-08T06:08:01.059203+0000","last_undegraded":"2024-05-08T06:08:01.059203+0000","last_fullsized":"2024-05-08T06:08:01.059203+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:51:48.871149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'6","reported_seq":275,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:10:13.925119+0000","last_change":"2024-05-08T06:05:46.348769+0000","last_active":"2024-05-08T06:10:13.925119+0000","last_peered":"2024-05-08T06:10:13.925119+0000","last_clean":"2024-05-08T06:10:13.925119+0000","last_became_active":"2024-05-08T06:05:46.348072+0000","last_became_peered":"2024-05-08T06:05:46.348072+0000","last_unstale":"2024-05-08T06:10:13.925119+0000","last_undegraded":"2024-05-08T06:10:13.925119+0000","last_fullsized":"2024-05-08T06:10:13.925119+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:45.292436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":243,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.049147+0000","last_change":"2024-05-08T06:05:51.414623+0000","last_active":"2024-05-08T06:08:01.049147+0000","last_peered":"2024-05-08T06:08:01.049147+0000","last_clean":"2024-05-08T06:08:01.049147+0000","last_became_active":"2024-05-08T06:05:51.414258+0000","last_became_peered":"2024-05-08T06:05:51.414258+0000","last_unstale":"2024-05-08T06:08:01.049147+0000","last_undegraded":"2024-05-08T06:08:01.049147+0000","last_fullsized":"2024-05-08T06:08:01.049147+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:38:48.297900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":43,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480030+0000","last_change":"2024-05-08T06:05:46.546936+0000","last_active":"2024-05-08T06:06:32.480030+0000","last_peered":"2024-05-08T06:06:32.480030+0000","last_clean":"2024-05-08T06:06:32.480030+0000","last_became_active":"2024-05-08T06:05:46.546574+0000","last_became_peered":"2024-05-08T06:05:46.546574+0000","last_unstale":"2024-05-08T06:06:32.480030+0000","last_undegraded":"2024-05-08T06:06:32.480030+0000","last_fullsized":"2024-05-08T06:06:32.480030+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:16:29.720304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.029792+0000","last_change":"2024-05-08T06:05:51.418012+0000","last_active":"2024-05-08T06:08:01.029792+0000","last_peered":"2024-05-08T06:08:01.029792+0000","last_clean":"2024-05-08T06:08:01.029792+0000","last_became_active":"2024-05-08T06:05:51.417646+0000","last_became_peered":"2024-05-08T06:05:51.417646+0000","last_unstale":"2024-05-08T06:08:01.029792+0000","last_undegraded":"2024-05-08T06:08:01.029792+0000","last_fullsized":"2024-05-08T06:08:01.029792+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:43:02.733967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:00.689473+0000","last_change":"2024-05-08T06:05:46.347748+0000","last_active":"2024-05-08T06:08:00.689473+0000","last_peered":"2024-05-08T06:08:00.689473+0000","last_clean":"2024-05-08T06:08:00.689473+0000","last_became_active":"2024-05-08T06:05:46.347367+0000","last_became_peered":"2024-05-08T06:05:46.347367+0000","last_unstale":"2024-05-08T06:08:00.689473+0000","last_undegraded":"2024-05-08T06:08:00.689473+0000","last_fullsized":"2024-05-08T06:08:00.689473+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:51:50.973855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.038216+0000","last_change":"2024-05-08T06:06:32.597018+0000","last_active":"2024-05-08T06:08:01.038216+0000","last_peered":"2024-05-08T06:08:01.038216+0000","last_clean":"2024-05-08T06:08:01.038216+0000","last_became_active":"2024-05-08T06:06:32.596677+0000","last_became_peered":"2024-05-08T06:06:32.596677+0000","last_unstale":"2024-05-08T06:08:01.038216+0000","last_undegraded":"2024-05-08T06:08:01.038216+0000","last_fullsized":"2024-05-08T06:08:01.038216+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:02:46.223949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:07:51.948701+0000","last_change":"2024-05-08T06:05:46.554443+0000","last_active":"2024-05-08T06:07:51.948701+0000","last_peered":"2024-05-08T06:07:51.948701+0000","last_clean":"2024-05-08T06:07:51.948701+0000","last_became_active":"2024-05-08T06:05:46.553226+0000","last_became_peered":"2024-05-08T06:05:46.553226+0000","last_unstale":"2024-05-08T06:07:51.948701+0000","last_undegraded":"2024-05-08T06:07:51.948701+0000","last_fullsized":"2024-05-08T06:07:51.948701+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:56:39.784652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.060932+0000","last_change":"2024-05-08T06:05:51.413121+0000","last_active":"2024-05-08T06:08:01.060932+0000","last_peered":"2024-05-08T06:08:01.060932+0000","last_clean":"2024-05-08T06:08:01.060932+0000","last_became_active":"2024-05-08T06:05:51.412767+0000","last_became_peered":"2024-05-08T06:05:51.412767+0000","last_unstale":"2024-05-08T06:08:01.060932+0000","last_undegraded":"2024-05-08T06:08:01.060932+0000","last_fullsized":"2024-05-08T06:08:01.060932+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:59:13.416981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:07:51.431460+0000","last_change":"2024-05-08T06:05:46.361349+0000","last_active":"2024-05-08T06:07:51.431460+0000","last_peered":"2024-05-08T06:07:51.431460+0000","last_clean":"2024-05-08T06:07:51.431460+0000","last_became_active":"2024-05-08T06:05:46.360654+0000","last_became_peered":"2024-05-08T06:05:46.360654+0000","last_unstale":"2024-05-08T06:07:51.431460+0000","last_undegraded":"2024-05-08T06:07:51.431460+0000","last_fullsized":"2024-05-08T06:07:51.431460+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:19:22.144874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.062214+0000","last_change":"2024-05-08T06:05:51.423836+0000","last_active":"2024-05-08T06:08:01.062214+0000","last_peered":"2024-05-08T06:08:01.062214+0000","last_clean":"2024-05-08T06:08:01.062214+0000","last_became_active":"2024-05-08T06:05:51.423424+0000","last_became_peered":"2024-05-08T06:05:51.423424+0000","last_unstale":"2024-05-08T06:08:01.062214+0000","last_undegraded":"2024-05-08T06:08:01.062214+0000","last_fullsized":"2024-05-08T06:08:01.062214+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:56:14.779765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.048549+0000","last_change":"2024-05-08T06:06:32.598978+0000","last_active":"2024-05-08T06:08:01.048549+0000","last_peered":"2024-05-08T06:08:01.048549+0000","last_clean":"2024-05-08T06:08:01.048549+0000","last_became_active":"2024-05-08T06:06:32.598158+0000","last_became_peered":"2024-05-08T06:06:32.598158+0000","last_unstale":"2024-05-08T06:08:01.048549+0000","last_undegraded":"2024-05-08T06:08:01.048549+0000","last_fullsized":"2024-05-08T06:08:01.048549+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:51:09.270066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":271,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:10:13.923393+0000","last_change":"2024-05-08T06:05:46.547450+0000","last_active":"2024-05-08T06:10:13.923393+0000","last_peered":"2024-05-08T06:10:13.923393+0000","last_clean":"2024-05-08T06:10:13.923393+0000","last_became_active":"2024-05-08T06:05:46.546444+0000","last_became_peered":"2024-05-08T06:05:46.546444+0000","last_unstale":"2024-05-08T06:10:13.923393+0000","last_undegraded":"2024-05-08T06:10:13.923393+0000","last_fullsized":"2024-05-08T06:10:13.923393+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:17:14.124224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":237,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.065294+0000","last_change":"2024-05-08T06:05:51.423191+0000","last_active":"2024-05-08T06:08:01.065294+0000","last_peered":"2024-05-08T06:08:01.065294+0000","last_clean":"2024-05-08T06:08:01.065294+0000","last_became_active":"2024-05-08T06:05:51.420912+0000","last_became_peered":"2024-05-08T06:05:51.420912+0000","last_unstale":"2024-05-08T06:08:01.065294+0000","last_undegraded":"2024-05-08T06:08:01.065294+0000","last_fullsized":"2024-05-08T06:08:01.065294+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:27:57.678527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:07:50.743006+0000","last_change":"2024-05-08T06:05:46.348650+0000","last_active":"2024-05-08T06:07:50.743006+0000","last_peered":"2024-05-08T06:07:50.743006+0000","last_clean":"2024-05-08T06:07:50.743006+0000","last_became_active":"2024-05-08T06:05:46.347266+0000","last_became_peered":"2024-05-08T06:05:46.347266+0000","last_unstale":"2024-05-08T06:07:50.743006+0000","last_undegraded":"2024-05-08T06:07:50.743006+0000","last_fullsized":"2024-05-08T06:07:50.743006+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:37:30.623580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":101,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:47.536769+0000","last_change":"2024-05-08T06:04:09.404663+0000","last_active":"2024-05-08T06:06:47.536769+0000","last_peered":"2024-05-08T06:06:47.536769+0000","last_clean":"2024-05-08T06:06:47.536769+0000","last_became_active":"2024-05-08T06:04:09.400115+0000","last_became_peered":"2024-05-08T06:04:09.400115+0000","last_unstale":"2024-05-08T06:06:47.536769+0000","last_undegraded":"2024-05-08T06:06:47.536769+0000","last_fullsized":"2024-05-08T06:06:47.536769+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:00:17.419712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:00:17.419712+0000","last_clean_scrub_stamp":"2024-05-08T06:00:17.419712+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:27:39.154694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.053609+0000","last_change":"2024-05-08T06:05:51.421826+0000","last_active":"2024-05-08T06:08:01.053609+0000","last_peered":"2024-05-08T06:08:01.053609+0000","last_clean":"2024-05-08T06:08:01.053609+0000","last_became_active":"2024-05-08T06:05:51.420706+0000","last_became_peered":"2024-05-08T06:05:51.420706+0000","last_unstale":"2024-05-08T06:08:01.053609+0000","last_undegraded":"2024-05-08T06:08:01.053609+0000","last_fullsized":"2024-05-08T06:08:01.053609+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:06:16.242332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:07:47.542941+0000","last_change":"2024-05-08T06:05:46.351365+0000","last_active":"2024-05-08T06:07:47.542941+0000","last_peered":"2024-05-08T06:07:47.542941+0000","last_clean":"2024-05-08T06:07:47.542941+0000","last_became_active":"2024-05-08T06:05:46.351022+0000","last_became_peered":"2024-05-08T06:05:46.351022+0000","last_unstale":"2024-05-08T06:07:47.542941+0000","last_undegraded":"2024-05-08T06:07:47.542941+0000","last_fullsized":"2024-05-08T06:07:47.542941+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:32:44.134994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:34.244260+0000","last_change":"2024-05-08T06:06:32.501263+0000","last_active":"2024-05-08T06:06:34.244260+0000","last_peered":"2024-05-08T06:06:34.244260+0000","last_clean":"2024-05-08T06:06:34.244260+0000","last_became_active":"2024-05-08T06:06:32.493795+0000","last_became_peered":"2024-05-08T06:06:32.493795+0000","last_unstale":"2024-05-08T06:06:34.244260+0000","last_undegraded":"2024-05-08T06:06:34.244260+0000","last_fullsized":"2024-05-08T06:06:34.244260+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:49:03.898162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.056756+0000","last_change":"2024-05-08T06:05:51.428382+0000","last_active":"2024-05-08T06:08:01.056756+0000","last_peered":"2024-05-08T06:08:01.056756+0000","last_clean":"2024-05-08T06:08:01.056756+0000","last_became_active":"2024-05-08T06:05:51.421443+0000","last_became_peered":"2024-05-08T06:05:51.421443+0000","last_unstale":"2024-05-08T06:08:01.056756+0000","last_undegraded":"2024-05-08T06:08:01.056756+0000","last_fullsized":"2024-05-08T06:08:01.056756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:26:10.215020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'15","reported_seq":43,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:10:11.475242+0000","last_change":"2024-05-08T06:05:46.364306+0000","last_active":"2024-05-08T06:10:11.475242+0000","last_peered":"2024-05-08T06:10:11.475242+0000","last_clean":"2024-05-08T06:10:11.475242+0000","last_became_active":"2024-05-08T06:05:46.363871+0000","last_became_peered":"2024-05-08T06:05:46.363871+0000","last_unstale":"2024-05-08T06:10:11.475242+0000","last_undegraded":"2024-05-08T06:10:11.475242+0000","last_fullsized":"2024-05-08T06:10:11.475242+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:38:34.531162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:05.007928+0000","last_change":"2024-05-08T06:05:51.415020+0000","last_active":"2024-05-08T06:08:05.007928+0000","last_peered":"2024-05-08T06:08:05.007928+0000","last_clean":"2024-05-08T06:08:05.007928+0000","last_became_active":"2024-05-08T06:05:51.413690+0000","last_became_peered":"2024-05-08T06:05:51.413690+0000","last_unstale":"2024-05-08T06:08:05.007928+0000","last_undegraded":"2024-05-08T06:08:05.007928+0000","last_fullsized":"2024-05-08T06:08:05.007928+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:55:38.121492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:07:55.743454+0000","last_change":"2024-05-08T06:05:46.359100+0000","last_active":"2024-05-08T06:07:55.743454+0000","last_peered":"2024-05-08T06:07:55.743454+0000","last_clean":"2024-05-08T06:07:55.743454+0000","last_became_active":"2024-05-08T06:05:46.358815+0000","last_became_peered":"2024-05-08T06:05:46.358815+0000","last_unstale":"2024-05-08T06:07:55.743454+0000","last_undegraded":"2024-05-08T06:07:55.743454+0000","last_fullsized":"2024-05-08T06:07:55.743454+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:22:58.532358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:02.879400+0000","last_change":"2024-05-08T06:05:51.429222+0000","last_active":"2024-05-08T06:08:02.879400+0000","last_peered":"2024-05-08T06:08:02.879400+0000","last_clean":"2024-05-08T06:08:02.879400+0000","last_became_active":"2024-05-08T06:05:51.428845+0000","last_became_peered":"2024-05-08T06:05:51.428845+0000","last_unstale":"2024-05-08T06:08:02.879400+0000","last_undegraded":"2024-05-08T06:08:02.879400+0000","last_fullsized":"2024-05-08T06:08:02.879400+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:36:41.435666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:07:57.210823+0000","last_change":"2024-05-08T06:05:46.547931+0000","last_active":"2024-05-08T06:07:57.210823+0000","last_peered":"2024-05-08T06:07:57.210823+0000","last_clean":"2024-05-08T06:07:57.210823+0000","last_became_active":"2024-05-08T06:05:46.547547+0000","last_became_peered":"2024-05-08T06:05:46.547547+0000","last_unstale":"2024-05-08T06:07:57.210823+0000","last_undegraded":"2024-05-08T06:07:57.210823+0000","last_fullsized":"2024-05-08T06:07:57.210823+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:27:03.502153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.020367+0000","last_change":"2024-05-08T06:05:51.413259+0000","last_active":"2024-05-08T06:08:01.020367+0000","last_peered":"2024-05-08T06:08:01.020367+0000","last_clean":"2024-05-08T06:08:01.020367+0000","last_became_active":"2024-05-08T06:05:51.412890+0000","last_became_peered":"2024-05-08T06:05:51.412890+0000","last_unstale":"2024-05-08T06:08:01.020367+0000","last_undegraded":"2024-05-08T06:08:01.020367+0000","last_fullsized":"2024-05-08T06:08:01.020367+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:04:56.690098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:00.007037+0000","last_change":"2024-05-08T06:05:46.365409+0000","last_active":"2024-05-08T06:08:00.007037+0000","last_peered":"2024-05-08T06:08:00.007037+0000","last_clean":"2024-05-08T06:08:00.007037+0000","last_became_active":"2024-05-08T06:05:46.361277+0000","last_became_peered":"2024-05-08T06:05:46.361277+0000","last_unstale":"2024-05-08T06:08:00.007037+0000","last_undegraded":"2024-05-08T06:08:00.007037+0000","last_fullsized":"2024-05-08T06:08:00.007037+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:06.927385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:04.253756+0000","last_change":"2024-05-08T06:05:51.423749+0000","last_active":"2024-05-08T06:08:04.253756+0000","last_peered":"2024-05-08T06:08:04.253756+0000","last_clean":"2024-05-08T06:08:04.253756+0000","last_became_active":"2024-05-08T06:05:51.423386+0000","last_became_peered":"2024-05-08T06:05:51.423386+0000","last_unstale":"2024-05-08T06:08:04.253756+0000","last_undegraded":"2024-05-08T06:08:04.253756+0000","last_fullsized":"2024-05-08T06:08:04.253756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:41:26.261683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'2","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:10:06.033061+0000","last_change":"2024-05-08T06:05:46.362497+0000","last_active":"2024-05-08T06:10:06.033061+0000","last_peered":"2024-05-08T06:10:06.033061+0000","last_clean":"2024-05-08T06:10:06.033061+0000","last_became_active":"2024-05-08T06:05:46.362110+0000","last_became_peered":"2024-05-08T06:05:46.362110+0000","last_unstale":"2024-05-08T06:10:06.033061+0000","last_undegraded":"2024-05-08T06:10:06.033061+0000","last_fullsized":"2024-05-08T06:10:06.033061+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:25:06.205380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.060826+0000","last_change":"2024-05-08T06:06:32.495078+0000","last_active":"2024-05-08T06:08:01.060826+0000","last_peered":"2024-05-08T06:08:01.060826+0000","last_clean":"2024-05-08T06:08:01.060826+0000","last_became_active":"2024-05-08T06:06:32.493855+0000","last_became_peered":"2024-05-08T06:06:32.493855+0000","last_unstale":"2024-05-08T06:08:01.060826+0000","last_undegraded":"2024-05-08T06:08:01.060826+0000","last_fullsized":"2024-05-08T06:08:01.060826+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:56:54.143080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:00.689448+0000","last_change":"2024-05-08T06:06:32.601826+0000","last_active":"2024-05-08T06:08:00.689448+0000","last_peered":"2024-05-08T06:08:00.689448+0000","last_clean":"2024-05-08T06:08:00.689448+0000","last_became_active":"2024-05-08T06:06:32.597823+0000","last_became_peered":"2024-05-08T06:06:32.597823+0000","last_unstale":"2024-05-08T06:08:00.689448+0000","last_undegraded":"2024-05-08T06:08:00.689448+0000","last_fullsized":"2024-05-08T06:08:00.689448+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:59:09.741096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:02.544515+0000","last_change":"2024-05-08T06:05:51.413744+0000","last_active":"2024-05-08T06:08:02.544515+0000","last_peered":"2024-05-08T06:08:02.544515+0000","last_clean":"2024-05-08T06:08:02.544515+0000","last_became_active":"2024-05-08T06:05:51.413346+0000","last_became_peered":"2024-05-08T06:05:51.413346+0000","last_unstale":"2024-05-08T06:08:02.544515+0000","last_undegraded":"2024-05-08T06:08:02.544515+0000","last_fullsized":"2024-05-08T06:08:02.544515+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:46:56.109755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.019506+0000","last_change":"2024-05-08T06:05:46.552604+0000","last_active":"2024-05-08T06:08:01.019506+0000","last_peered":"2024-05-08T06:08:01.019506+0000","last_clean":"2024-05-08T06:08:01.019506+0000","last_became_active":"2024-05-08T06:05:46.552248+0000","last_became_peered":"2024-05-08T06:05:46.552248+0000","last_unstale":"2024-05-08T06:08:01.019506+0000","last_undegraded":"2024-05-08T06:08:01.019506+0000","last_fullsized":"2024-05-08T06:08:01.019506+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:39:10.306692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.054186+0000","last_change":"2024-05-08T06:05:51.404988+0000","last_active":"2024-05-08T06:08:01.054186+0000","last_peered":"2024-05-08T06:08:01.054186+0000","last_clean":"2024-05-08T06:08:01.054186+0000","last_became_active":"2024-05-08T06:05:51.404625+0000","last_became_peered":"2024-05-08T06:05:51.404625+0000","last_unstale":"2024-05-08T06:08:01.054186+0000","last_undegraded":"2024-05-08T06:08:01.054186+0000","last_fullsized":"2024-05-08T06:08:01.054186+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:33:27.919929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.596178+0000","last_change":"2024-05-08T06:05:46.361466+0000","last_active":"2024-05-08T06:06:32.596178+0000","last_peered":"2024-05-08T06:06:32.596178+0000","last_clean":"2024-05-08T06:06:32.596178+0000","last_became_active":"2024-05-08T06:05:46.361107+0000","last_became_peered":"2024-05-08T06:05:46.361107+0000","last_unstale":"2024-05-08T06:06:32.596178+0000","last_undegraded":"2024-05-08T06:06:32.596178+0000","last_fullsized":"2024-05-08T06:06:32.596178+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:15:31.314226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.062827+0000","last_change":"2024-05-08T06:05:51.416437+0000","last_active":"2024-05-08T06:08:01.062827+0000","last_peered":"2024-05-08T06:08:01.062827+0000","last_clean":"2024-05-08T06:08:01.062827+0000","last_became_active":"2024-05-08T06:05:51.416070+0000","last_became_peered":"2024-05-08T06:05:51.416070+0000","last_unstale":"2024-05-08T06:08:01.062827+0000","last_undegraded":"2024-05-08T06:08:01.062827+0000","last_fullsized":"2024-05-08T06:08:01.062827+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:35:23.652373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:00.709889+0000","last_change":"2024-05-08T06:05:46.362475+0000","last_active":"2024-05-08T06:08:00.709889+0000","last_peered":"2024-05-08T06:08:00.709889+0000","last_clean":"2024-05-08T06:08:00.709889+0000","last_became_active":"2024-05-08T06:05:46.361852+0000","last_became_peered":"2024-05-08T06:05:46.361852+0000","last_unstale":"2024-05-08T06:08:00.709889+0000","last_undegraded":"2024-05-08T06:08:00.709889+0000","last_fullsized":"2024-05-08T06:08:00.709889+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:16:54.536330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.049399+0000","last_change":"2024-05-08T06:05:51.410718+0000","last_active":"2024-05-08T06:08:01.049399+0000","last_peered":"2024-05-08T06:08:01.049399+0000","last_clean":"2024-05-08T06:08:01.049399+0000","last_became_active":"2024-05-08T06:05:51.408902+0000","last_became_peered":"2024-05-08T06:05:51.408902+0000","last_unstale":"2024-05-08T06:08:01.049399+0000","last_undegraded":"2024-05-08T06:08:01.049399+0000","last_fullsized":"2024-05-08T06:08:01.049399+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:23:33.857677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.039067+0000","last_change":"2024-05-08T06:05:51.414362+0000","last_active":"2024-05-08T06:08:01.039067+0000","last_peered":"2024-05-08T06:08:01.039067+0000","last_clean":"2024-05-08T06:08:01.039067+0000","last_became_active":"2024-05-08T06:05:51.413998+0000","last_became_peered":"2024-05-08T06:05:51.413998+0000","last_unstale":"2024-05-08T06:08:01.039067+0000","last_undegraded":"2024-05-08T06:08:01.039067+0000","last_fullsized":"2024-05-08T06:08:01.039067+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:38:22.198219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.032578+0000","last_change":"2024-05-08T06:05:51.415508+0000","last_active":"2024-05-08T06:08:01.032578+0000","last_peered":"2024-05-08T06:08:01.032578+0000","last_clean":"2024-05-08T06:08:01.032578+0000","last_became_active":"2024-05-08T06:05:51.415137+0000","last_became_peered":"2024-05-08T06:05:51.415137+0000","last_unstale":"2024-05-08T06:08:01.032578+0000","last_undegraded":"2024-05-08T06:08:01.032578+0000","last_fullsized":"2024-05-08T06:08:01.032578+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:15.007996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.064401+0000","last_change":"2024-05-08T06:05:51.422093+0000","last_active":"2024-05-08T06:08:01.064401+0000","last_peered":"2024-05-08T06:08:01.064401+0000","last_clean":"2024-05-08T06:08:01.064401+0000","last_became_active":"2024-05-08T06:05:51.421733+0000","last_became_peered":"2024-05-08T06:05:51.421733+0000","last_unstale":"2024-05-08T06:08:01.064401+0000","last_undegraded":"2024-05-08T06:08:01.064401+0000","last_fullsized":"2024-05-08T06:08:01.064401+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:35:55.950229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.062534+0000","last_change":"2024-05-08T06:05:51.434705+0000","last_active":"2024-05-08T06:08:01.062534+0000","last_peered":"2024-05-08T06:08:01.062534+0000","last_clean":"2024-05-08T06:08:01.062534+0000","last_became_active":"2024-05-08T06:05:51.434324+0000","last_became_peered":"2024-05-08T06:05:51.434324+0000","last_unstale":"2024-05-08T06:08:01.062534+0000","last_undegraded":"2024-05-08T06:08:01.062534+0000","last_fullsized":"2024-05-08T06:08:01.062534+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:38:23.101247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.064059+0000","last_change":"2024-05-08T06:06:32.499837+0000","last_active":"2024-05-08T06:08:01.064059+0000","last_peered":"2024-05-08T06:08:01.064059+0000","last_clean":"2024-05-08T06:08:01.064059+0000","last_became_active":"2024-05-08T06:06:32.499026+0000","last_became_peered":"2024-05-08T06:06:32.499026+0000","last_unstale":"2024-05-08T06:08:01.064059+0000","last_undegraded":"2024-05-08T06:08:01.064059+0000","last_fullsized":"2024-05-08T06:08:01.064059+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:48:51.910384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:04.253787+0000","last_change":"2024-05-08T06:05:51.412370+0000","last_active":"2024-05-08T06:08:04.253787+0000","last_peered":"2024-05-08T06:08:04.253787+0000","last_clean":"2024-05-08T06:08:04.253787+0000","last_became_active":"2024-05-08T06:05:51.412057+0000","last_became_peered":"2024-05-08T06:05:51.412057+0000","last_unstale":"2024-05-08T06:08:04.253787+0000","last_undegraded":"2024-05-08T06:08:04.253787+0000","last_fullsized":"2024-05-08T06:08:04.253787+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:20:51.549926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.048132+0000","last_change":"2024-05-08T06:05:51.408393+0000","last_active":"2024-05-08T06:08:01.048132+0000","last_peered":"2024-05-08T06:08:01.048132+0000","last_clean":"2024-05-08T06:08:01.048132+0000","last_became_active":"2024-05-08T06:05:51.408023+0000","last_became_peered":"2024-05-08T06:05:51.408023+0000","last_unstale":"2024-05-08T06:08:01.048132+0000","last_undegraded":"2024-05-08T06:08:01.048132+0000","last_fullsized":"2024-05-08T06:08:01.048132+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:53.898929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:05.007852+0000","last_change":"2024-05-08T06:05:51.418055+0000","last_active":"2024-05-08T06:08:05.007852+0000","last_peered":"2024-05-08T06:08:05.007852+0000","last_clean":"2024-05-08T06:08:05.007852+0000","last_became_active":"2024-05-08T06:05:51.417067+0000","last_became_peered":"2024-05-08T06:05:51.417067+0000","last_unstale":"2024-05-08T06:08:05.007852+0000","last_undegraded":"2024-05-08T06:08:05.007852+0000","last_fullsized":"2024-05-08T06:08:05.007852+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:07:35.413672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:04.506147+0000","last_change":"2024-05-08T06:05:51.421025+0000","last_active":"2024-05-08T06:08:04.506147+0000","last_peered":"2024-05-08T06:08:04.506147+0000","last_clean":"2024-05-08T06:08:04.506147+0000","last_became_active":"2024-05-08T06:05:51.420794+0000","last_became_peered":"2024-05-08T06:05:51.420794+0000","last_unstale":"2024-05-08T06:08:04.506147+0000","last_undegraded":"2024-05-08T06:08:04.506147+0000","last_fullsized":"2024-05-08T06:08:04.506147+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:45:01.852405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.060497+0000","last_change":"2024-05-08T06:05:51.417709+0000","last_active":"2024-05-08T06:08:01.060497+0000","last_peered":"2024-05-08T06:08:01.060497+0000","last_clean":"2024-05-08T06:08:01.060497+0000","last_became_active":"2024-05-08T06:05:51.417337+0000","last_became_peered":"2024-05-08T06:05:51.417337+0000","last_unstale":"2024-05-08T06:08:01.060497+0000","last_undegraded":"2024-05-08T06:08:01.060497+0000","last_fullsized":"2024-05-08T06:08:01.060497+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:32:24.551768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.065666+0000","last_change":"2024-05-08T06:05:51.425318+0000","last_active":"2024-05-08T06:08:01.065666+0000","last_peered":"2024-05-08T06:08:01.065666+0000","last_clean":"2024-05-08T06:08:01.065666+0000","last_became_active":"2024-05-08T06:05:51.424931+0000","last_became_peered":"2024-05-08T06:05:51.424931+0000","last_unstale":"2024-05-08T06:08:01.065666+0000","last_undegraded":"2024-05-08T06:08:01.065666+0000","last_fullsized":"2024-05-08T06:08:01.065666+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:59:45.110201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.064188+0000","last_change":"2024-05-08T06:05:51.433529+0000","last_active":"2024-05-08T06:08:01.064188+0000","last_peered":"2024-05-08T06:08:01.064188+0000","last_clean":"2024-05-08T06:08:01.064188+0000","last_became_active":"2024-05-08T06:05:51.433173+0000","last_became_peered":"2024-05-08T06:05:51.433173+0000","last_unstale":"2024-05-08T06:08:01.064188+0000","last_undegraded":"2024-05-08T06:08:01.064188+0000","last_fullsized":"2024-05-08T06:08:01.064188+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:00:29.042188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.045237+0000","last_change":"2024-05-08T06:05:51.411418+0000","last_active":"2024-05-08T06:08:01.045237+0000","last_peered":"2024-05-08T06:08:01.045237+0000","last_clean":"2024-05-08T06:08:01.045237+0000","last_became_active":"2024-05-08T06:05:51.411084+0000","last_became_peered":"2024-05-08T06:05:51.411084+0000","last_unstale":"2024-05-08T06:08:01.045237+0000","last_undegraded":"2024-05-08T06:08:01.045237+0000","last_fullsized":"2024-05-08T06:08:01.045237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:08:11.522282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282383,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":78287,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1466,"num_read_kb":818,"num_write":190,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":234861,"data_compressed":5508,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645383,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":888832,"data_stored":431859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841613,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":1097260,"data_compressed":37744,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005139,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":897024,"data_stored":432039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201369,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":892928,"data_stored":431937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397598,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":1025122,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561123,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":884736,"data_stored":431428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574976,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":504274,"data_compressed":1836,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738502,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":432699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411451,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":974848,"data_stored":506291,"data_compressed":1836,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117109,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":431739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920879,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":888832,"data_stored":432788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757353,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":1023883,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34281,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34281,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34281,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":73053,"data_compressed":1836,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":75070,"data_compressed":1836,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":75554,"data_compressed":1836,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T06:10:19.908 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T06:10:20.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.153+0000 7f01c2c00640 1 -- 172.21.15.123:0/2429540796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01bc077370 msgr2=0x7f01bc077790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:20.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.153+0000 7f01c2c00640 1 --2- 172.21.15.123:0/2429540796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01bc077370 0x7f01bc077790 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f01b0009900 tx=0x7f01b002a410 comp rx=0 tx=0).stop 2024-05-08T06:10:20.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.153+0000 7f01c2c00640 1 -- 172.21.15.123:0/2429540796 shutdown_connections 2024-05-08T06:10:20.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.153+0000 7f01c2c00640 1 --2- 172.21.15.123:0/2429540796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01bc078f50 0x7f01bc0797f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:20.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.153+0000 7f01c2c00640 1 --2- 172.21.15.123:0/2429540796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01bc0785b0 0x7f01bc078a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:20.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.153+0000 7f01c2c00640 1 --2- 172.21.15.123:0/2429540796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01bc077370 0x7f01bc077790 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:20.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.153+0000 7f01c2c00640 1 -- 172.21.15.123:0/2429540796 >> 172.21.15.123:0/2429540796 conn(0x7f01bc0fffc0 msgr2=0x7f01bc1023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:20.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.153+0000 7f01c2c00640 1 -- 172.21.15.123:0/2429540796 shutdown_connections 2024-05-08T06:10:20.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.153+0000 7f01c2c00640 1 -- 172.21.15.123:0/2429540796 wait complete. 2024-05-08T06:10:20.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.154+0000 7f01c2c00640 1 Processor -- start 2024-05-08T06:10:20.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.154+0000 7f01c2c00640 1 -- start start 2024-05-08T06:10:20.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.154+0000 7f01c2c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01bc077370 0x7f01bc19f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:20.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.154+0000 7f01c2c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01bc0785b0 0x7f01bc19f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:20.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.154+0000 7f01c2c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01bc078f50 0x7f01bc1a6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:20.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.154+0000 7f01c2c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f01bc1149b0 con 0x7f01bc0785b0 2024-05-08T06:10:20.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.154+0000 7f01c2c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f01bc114830 con 0x7f01bc078f50 2024-05-08T06:10:20.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.154+0000 7f01c2c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f01bc114b30 con 0x7f01bc077370 2024-05-08T06:10:20.157 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.155+0000 7f01c0e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01bc0785b0 0x7f01bc19f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:20.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.155+0000 7f01c1800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01bc077370 0x7f01bc19f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:20.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.155+0000 7f01c1800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01bc077370 0x7f01bc19f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:46908/0 (socket says 172.21.15.123:46908) 2024-05-08T06:10:20.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.155+0000 7f01c1800640 1 -- 172.21.15.123:0/2940083322 learned_addr learned my addr 172.21.15.123:0/2940083322 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:10:20.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.155+0000 7f01c2200640 1 --2- 172.21.15.123:0/2940083322 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01bc078f50 0x7f01bc1a6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:20.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.155+0000 7f01c0e00640 1 -- 172.21.15.123:0/2940083322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01bc077370 msgr2=0x7f01bc19f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:20.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.155+0000 7f01c0e00640 1 --2- 172.21.15.123:0/2940083322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01bc077370 0x7f01bc19f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:20.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.155+0000 7f01c0e00640 1 -- 172.21.15.123:0/2940083322 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01bc078f50 msgr2=0x7f01bc1a6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:20.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.155+0000 7f01c0e00640 1 --2- 172.21.15.123:0/2940083322 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01bc078f50 0x7f01bc1a6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:20.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.155+0000 7f01c0e00640 1 -- 172.21.15.123:0/2940083322 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01bc1a6cb0 con 0x7f01bc0785b0 2024-05-08T06:10:20.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.155+0000 7f01c0e00640 1 --2- 172.21.15.123:0/2940083322 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01bc0785b0 0x7f01bc19f5d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f01a800e9f0 tx=0x7f01a800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:20.158 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.156+0000 7f01ba000640 1 -- 172.21.15.123:0/2940083322 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a800ccc0 con 0x7f01bc0785b0 2024-05-08T06:10:20.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.156+0000 7f01ba000640 1 -- 172.21.15.123:0/2940083322 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01a8005020 con 0x7f01bc0785b0 2024-05-08T06:10:20.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.156+0000 7f01ba000640 1 -- 172.21.15.123:0/2940083322 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a800ce60 con 0x7f01bc0785b0 2024-05-08T06:10:20.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.156+0000 7f01c2c00640 1 -- 172.21.15.123:0/2940083322 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01bc1a6fa0 con 0x7f01bc0785b0 2024-05-08T06:10:20.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.156+0000 7f01c2c00640 1 -- 172.21.15.123:0/2940083322 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01bc1a7410 con 0x7f01bc0785b0 2024-05-08T06:10:20.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.157+0000 7f01c2c00640 1 -- 172.21.15.123:0/2940083322 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01bc1115e0 con 0x7f01bc0785b0 2024-05-08T06:10:20.160 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.159+0000 7f01ba000640 1 -- 172.21.15.123:0/2940083322 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 19) ==== 149704+0+0 (secure 0 0 0) 0x7f01a80051d0 con 0x7f01bc0785b0 2024-05-08T06:10:20.161 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.161+0000 7f01ba000640 1 --2- 172.21.15.123:0/2940083322 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f01a40b2900 0x7f01a40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:20.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.161+0000 7f01ba000640 1 -- 172.21.15.123:0/2940083322 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f01a80e22a0 con 0x7f01bc0785b0 2024-05-08T06:10:20.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.161+0000 7f01c1800640 1 --2- 172.21.15.123:0/2940083322 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f01a40b2900 0x7f01a40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:20.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.162+0000 7f01c1800640 1 --2- 172.21.15.123:0/2940083322 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f01a40b2900 0x7f01a40b4dc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f01b00096c0 tx=0x7f01b0005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:20.165 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.165+0000 7f01ba000640 1 -- 172.21.15.123:0/2940083322 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f01a80a9e10 con 0x7f01bc0785b0 2024-05-08T06:10:20.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.368+0000 7f01c2c00640 1 -- 172.21.15.123:0/2940083322 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f01bc062df0 con 0x7f01a40b2900 2024-05-08T06:10:20.380 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.379+0000 7f01ba000640 1 -- 172.21.15.123:0/2940083322 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259055 (secure 0 0 0) 0x7f01bc062df0 con 0x7f01a40b2900 2024-05-08T06:10:20.384 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:10:20.386 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-05-08T06:10:20.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.386+0000 7f01c2c00640 1 -- 172.21.15.123:0/2940083322 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f01a40b2900 msgr2=0x7f01a40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:20.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.386+0000 7f01c2c00640 1 --2- 172.21.15.123:0/2940083322 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f01a40b2900 0x7f01a40b4dc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f01b00096c0 tx=0x7f01b0005a20 comp rx=0 tx=0).stop 2024-05-08T06:10:20.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.386+0000 7f01c2c00640 1 -- 172.21.15.123:0/2940083322 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01bc0785b0 msgr2=0x7f01bc19f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:20.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.386+0000 7f01c2c00640 1 --2- 172.21.15.123:0/2940083322 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01bc0785b0 0x7f01bc19f5d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f01a800e9f0 tx=0x7f01a800edf0 comp rx=0 tx=0).stop 2024-05-08T06:10:20.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.386+0000 7f01c2c00640 1 -- 172.21.15.123:0/2940083322 shutdown_connections 2024-05-08T06:10:20.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.386+0000 7f01c2c00640 1 --2- 172.21.15.123:0/2940083322 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f01a40b2900 0x7f01a40b4dc0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:20.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.386+0000 7f01c2c00640 1 --2- 172.21.15.123:0/2940083322 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01bc078f50 0x7f01bc1a6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:20.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.386+0000 7f01c2c00640 1 --2- 172.21.15.123:0/2940083322 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01bc0785b0 0x7f01bc19f5d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:20.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.386+0000 7f01c2c00640 1 --2- 172.21.15.123:0/2940083322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01bc077370 0x7f01bc19f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:20.388 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.386+0000 7f01c2c00640 1 -- 172.21.15.123:0/2940083322 >> 172.21.15.123:0/2940083322 conn(0x7f01bc0fffc0 msgr2=0x7f01bc101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:20.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.386+0000 7f01c2c00640 1 -- 172.21.15.123:0/2940083322 shutdown_connections 2024-05-08T06:10:20.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.386+0000 7f01c2c00640 1 -- 172.21.15.123:0/2940083322 wait complete. 2024-05-08T06:10:20.408 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":456,"stamp":"2024-05-08T06:10:19.154293+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668889,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1542,"num_read_kb":882,"num_write":315,"num_write_kb":2293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":2951896,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":755,"ondisk_log_size":755,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372504,"kb_used_data":10872,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569320,"statfs":{"total":1151940427776,"available":1151558983680,"internally_reserved":0,"allocated":11132928,"data_stored":7181319,"data_compressed":113232,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9686,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":81,"num_read_kb":53,"num_write":15,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005348"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.064237+0000","last_change":"2024-05-08T06:05:51.431744+0000","last_active":"2024-05-08T06:08:01.064237+0000","last_peered":"2024-05-08T06:08:01.064237+0000","last_clean":"2024-05-08T06:08:01.064237+0000","last_became_active":"2024-05-08T06:05:51.431405+0000","last_became_peered":"2024-05-08T06:05:51.431405+0000","last_unstale":"2024-05-08T06:08:01.064237+0000","last_undegraded":"2024-05-08T06:08:01.064237+0000","last_fullsized":"2024-05-08T06:08:01.064237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:56:40.503607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.062583+0000","last_change":"2024-05-08T06:05:51.426473+0000","last_active":"2024-05-08T06:08:01.062583+0000","last_peered":"2024-05-08T06:08:01.062583+0000","last_clean":"2024-05-08T06:08:01.062583+0000","last_became_active":"2024-05-08T06:05:51.424477+0000","last_became_peered":"2024-05-08T06:05:51.424477+0000","last_unstale":"2024-05-08T06:08:01.062583+0000","last_undegraded":"2024-05-08T06:08:01.062583+0000","last_fullsized":"2024-05-08T06:08:01.062583+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:26:55.747808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.060622+0000","last_change":"2024-05-08T06:05:51.423767+0000","last_active":"2024-05-08T06:08:01.060622+0000","last_peered":"2024-05-08T06:08:01.060622+0000","last_clean":"2024-05-08T06:08:01.060622+0000","last_became_active":"2024-05-08T06:05:51.423273+0000","last_became_peered":"2024-05-08T06:05:51.423273+0000","last_unstale":"2024-05-08T06:08:01.060622+0000","last_undegraded":"2024-05-08T06:08:01.060622+0000","last_fullsized":"2024-05-08T06:08:01.060622+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:24:44.099743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.047241+0000","last_change":"2024-05-08T06:05:51.416604+0000","last_active":"2024-05-08T06:08:01.047241+0000","last_peered":"2024-05-08T06:08:01.047241+0000","last_clean":"2024-05-08T06:08:01.047241+0000","last_became_active":"2024-05-08T06:05:51.416284+0000","last_became_peered":"2024-05-08T06:05:51.416284+0000","last_unstale":"2024-05-08T06:08:01.047241+0000","last_undegraded":"2024-05-08T06:08:01.047241+0000","last_fullsized":"2024-05-08T06:08:01.047241+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:05:26.014323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.059697+0000","last_change":"2024-05-08T06:05:51.423915+0000","last_active":"2024-05-08T06:08:01.059697+0000","last_peered":"2024-05-08T06:08:01.059697+0000","last_clean":"2024-05-08T06:08:01.059697+0000","last_became_active":"2024-05-08T06:05:51.423534+0000","last_became_peered":"2024-05-08T06:05:51.423534+0000","last_unstale":"2024-05-08T06:08:01.059697+0000","last_undegraded":"2024-05-08T06:08:01.059697+0000","last_fullsized":"2024-05-08T06:08:01.059697+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:06:46.197071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.041181+0000","last_change":"2024-05-08T06:05:51.411102+0000","last_active":"2024-05-08T06:08:01.041181+0000","last_peered":"2024-05-08T06:08:01.041181+0000","last_clean":"2024-05-08T06:08:01.041181+0000","last_became_active":"2024-05-08T06:05:51.410791+0000","last_became_peered":"2024-05-08T06:05:51.410791+0000","last_unstale":"2024-05-08T06:08:01.041181+0000","last_undegraded":"2024-05-08T06:08:01.041181+0000","last_fullsized":"2024-05-08T06:08:01.041181+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:31:47.861663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.041524+0000","last_change":"2024-05-08T06:05:51.417756+0000","last_active":"2024-05-08T06:08:01.041524+0000","last_peered":"2024-05-08T06:08:01.041524+0000","last_clean":"2024-05-08T06:08:01.041524+0000","last_became_active":"2024-05-08T06:05:51.417373+0000","last_became_peered":"2024-05-08T06:05:51.417373+0000","last_unstale":"2024-05-08T06:08:01.041524+0000","last_undegraded":"2024-05-08T06:08:01.041524+0000","last_fullsized":"2024-05-08T06:08:01.041524+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:07:48.330401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.063489+0000","last_change":"2024-05-08T06:05:51.413883+0000","last_active":"2024-05-08T06:08:01.063489+0000","last_peered":"2024-05-08T06:08:01.063489+0000","last_clean":"2024-05-08T06:08:01.063489+0000","last_became_active":"2024-05-08T06:05:51.413565+0000","last_became_peered":"2024-05-08T06:05:51.413565+0000","last_unstale":"2024-05-08T06:08:01.063489+0000","last_undegraded":"2024-05-08T06:08:01.063489+0000","last_fullsized":"2024-05-08T06:08:01.063489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:06:38.186225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.053054+0000","last_change":"2024-05-08T06:05:51.416300+0000","last_active":"2024-05-08T06:08:01.053054+0000","last_peered":"2024-05-08T06:08:01.053054+0000","last_clean":"2024-05-08T06:08:01.053054+0000","last_became_active":"2024-05-08T06:05:51.415970+0000","last_became_peered":"2024-05-08T06:05:51.415970+0000","last_unstale":"2024-05-08T06:08:01.053054+0000","last_undegraded":"2024-05-08T06:08:01.053054+0000","last_fullsized":"2024-05-08T06:08:01.053054+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:44:51.853339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.039789+0000","last_change":"2024-05-08T06:05:51.410396+0000","last_active":"2024-05-08T06:08:01.039789+0000","last_peered":"2024-05-08T06:08:01.039789+0000","last_clean":"2024-05-08T06:08:01.039789+0000","last_became_active":"2024-05-08T06:05:51.410067+0000","last_became_peered":"2024-05-08T06:05:51.410067+0000","last_unstale":"2024-05-08T06:08:01.039789+0000","last_undegraded":"2024-05-08T06:08:01.039789+0000","last_fullsized":"2024-05-08T06:08:01.039789+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:32:00.217217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.052539+0000","last_change":"2024-05-08T06:05:51.409276+0000","last_active":"2024-05-08T06:08:01.052539+0000","last_peered":"2024-05-08T06:08:01.052539+0000","last_clean":"2024-05-08T06:08:01.052539+0000","last_became_active":"2024-05-08T06:05:51.408768+0000","last_became_peered":"2024-05-08T06:05:51.408768+0000","last_unstale":"2024-05-08T06:08:01.052539+0000","last_undegraded":"2024-05-08T06:08:01.052539+0000","last_fullsized":"2024-05-08T06:08:01.052539+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:22:57.483944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.048049+0000","last_change":"2024-05-08T06:05:51.419083+0000","last_active":"2024-05-08T06:08:01.048049+0000","last_peered":"2024-05-08T06:08:01.048049+0000","last_clean":"2024-05-08T06:08:01.048049+0000","last_became_active":"2024-05-08T06:05:51.418685+0000","last_became_peered":"2024-05-08T06:05:51.418685+0000","last_unstale":"2024-05-08T06:08:01.048049+0000","last_undegraded":"2024-05-08T06:08:01.048049+0000","last_fullsized":"2024-05-08T06:08:01.048049+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:25:49.256242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.057514+0000","last_change":"2024-05-08T06:05:51.420399+0000","last_active":"2024-05-08T06:08:01.057514+0000","last_peered":"2024-05-08T06:08:01.057514+0000","last_clean":"2024-05-08T06:08:01.057514+0000","last_became_active":"2024-05-08T06:05:51.420076+0000","last_became_peered":"2024-05-08T06:05:51.420076+0000","last_unstale":"2024-05-08T06:08:01.057514+0000","last_undegraded":"2024-05-08T06:08:01.057514+0000","last_fullsized":"2024-05-08T06:08:01.057514+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:51:38.579280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.059465+0000","last_change":"2024-05-08T06:05:51.431518+0000","last_active":"2024-05-08T06:08:01.059465+0000","last_peered":"2024-05-08T06:08:01.059465+0000","last_clean":"2024-05-08T06:08:01.059465+0000","last_became_active":"2024-05-08T06:05:51.431029+0000","last_became_peered":"2024-05-08T06:05:51.431029+0000","last_unstale":"2024-05-08T06:08:01.059465+0000","last_undegraded":"2024-05-08T06:08:01.059465+0000","last_fullsized":"2024-05-08T06:08:01.059465+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:11:09.596221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.061333+0000","last_change":"2024-05-08T06:06:32.498002+0000","last_active":"2024-05-08T06:08:01.061333+0000","last_peered":"2024-05-08T06:08:01.061333+0000","last_clean":"2024-05-08T06:08:01.061333+0000","last_became_active":"2024-05-08T06:06:32.497367+0000","last_became_peered":"2024-05-08T06:06:32.497367+0000","last_unstale":"2024-05-08T06:08:01.061333+0000","last_undegraded":"2024-05-08T06:08:01.061333+0000","last_fullsized":"2024-05-08T06:08:01.061333+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:32:36.461777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.064064+0000","last_change":"2024-05-08T06:05:51.423857+0000","last_active":"2024-05-08T06:08:01.064064+0000","last_peered":"2024-05-08T06:08:01.064064+0000","last_clean":"2024-05-08T06:08:01.064064+0000","last_became_active":"2024-05-08T06:05:51.423487+0000","last_became_peered":"2024-05-08T06:05:51.423487+0000","last_unstale":"2024-05-08T06:08:01.064064+0000","last_undegraded":"2024-05-08T06:08:01.064064+0000","last_fullsized":"2024-05-08T06:08:01.064064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:17:33.269354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.064245+0000","last_change":"2024-05-08T06:05:51.431795+0000","last_active":"2024-05-08T06:08:01.064245+0000","last_peered":"2024-05-08T06:08:01.064245+0000","last_clean":"2024-05-08T06:08:01.064245+0000","last_became_active":"2024-05-08T06:05:51.431432+0000","last_became_peered":"2024-05-08T06:05:51.431432+0000","last_unstale":"2024-05-08T06:08:01.064245+0000","last_undegraded":"2024-05-08T06:08:01.064245+0000","last_fullsized":"2024-05-08T06:08:01.064245+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:11:16.561478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.048666+0000","last_change":"2024-05-08T06:05:51.414805+0000","last_active":"2024-05-08T06:08:01.048666+0000","last_peered":"2024-05-08T06:08:01.048666+0000","last_clean":"2024-05-08T06:08:01.048666+0000","last_became_active":"2024-05-08T06:05:51.414482+0000","last_became_peered":"2024-05-08T06:05:51.414482+0000","last_unstale":"2024-05-08T06:08:01.048666+0000","last_undegraded":"2024-05-08T06:08:01.048666+0000","last_fullsized":"2024-05-08T06:08:01.048666+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:08:13.444591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.064433+0000","last_change":"2024-05-08T06:05:51.418516+0000","last_active":"2024-05-08T06:08:01.064433+0000","last_peered":"2024-05-08T06:08:01.064433+0000","last_clean":"2024-05-08T06:08:01.064433+0000","last_became_active":"2024-05-08T06:05:51.418182+0000","last_became_peered":"2024-05-08T06:05:51.418182+0000","last_unstale":"2024-05-08T06:08:01.064433+0000","last_undegraded":"2024-05-08T06:08:01.064433+0000","last_fullsized":"2024-05-08T06:08:01.064433+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:32:33.338311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:07:51.948755+0000","last_change":"2024-05-08T06:05:46.350509+0000","last_active":"2024-05-08T06:07:51.948755+0000","last_peered":"2024-05-08T06:07:51.948755+0000","last_clean":"2024-05-08T06:07:51.948755+0000","last_became_active":"2024-05-08T06:05:46.349986+0000","last_became_peered":"2024-05-08T06:05:46.349986+0000","last_unstale":"2024-05-08T06:07:51.948755+0000","last_undegraded":"2024-05-08T06:07:51.948755+0000","last_fullsized":"2024-05-08T06:07:51.948755+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:53:00.928971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.061073+0000","last_change":"2024-05-08T06:05:51.413621+0000","last_active":"2024-05-08T06:08:01.061073+0000","last_peered":"2024-05-08T06:08:01.061073+0000","last_clean":"2024-05-08T06:08:01.061073+0000","last_became_active":"2024-05-08T06:05:51.413188+0000","last_became_peered":"2024-05-08T06:05:51.413188+0000","last_unstale":"2024-05-08T06:08:01.061073+0000","last_undegraded":"2024-05-08T06:08:01.061073+0000","last_fullsized":"2024-05-08T06:08:01.061073+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:13:21.413696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:07:50.005420+0000","last_change":"2024-05-08T06:05:46.552444+0000","last_active":"2024-05-08T06:07:50.005420+0000","last_peered":"2024-05-08T06:07:50.005420+0000","last_clean":"2024-05-08T06:07:50.005420+0000","last_became_active":"2024-05-08T06:05:46.551310+0000","last_became_peered":"2024-05-08T06:05:46.551310+0000","last_unstale":"2024-05-08T06:07:50.005420+0000","last_undegraded":"2024-05-08T06:07:50.005420+0000","last_fullsized":"2024-05-08T06:07:50.005420+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:58:17.632832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.062434+0000","last_change":"2024-05-08T06:05:51.425649+0000","last_active":"2024-05-08T06:08:01.062434+0000","last_peered":"2024-05-08T06:08:01.062434+0000","last_clean":"2024-05-08T06:08:01.062434+0000","last_became_active":"2024-05-08T06:05:51.425288+0000","last_became_peered":"2024-05-08T06:05:51.425288+0000","last_unstale":"2024-05-08T06:08:01.062434+0000","last_undegraded":"2024-05-08T06:08:01.062434+0000","last_fullsized":"2024-05-08T06:08:01.062434+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:42:40.580641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480655+0000","last_change":"2024-05-08T06:05:46.350300+0000","last_active":"2024-05-08T06:06:32.480655+0000","last_peered":"2024-05-08T06:06:32.480655+0000","last_clean":"2024-05-08T06:06:32.480655+0000","last_became_active":"2024-05-08T06:05:46.349918+0000","last_became_peered":"2024-05-08T06:05:46.349918+0000","last_unstale":"2024-05-08T06:06:32.480655+0000","last_undegraded":"2024-05-08T06:06:32.480655+0000","last_fullsized":"2024-05-08T06:06:32.480655+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:32:41.952772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.057546+0000","last_change":"2024-05-08T06:05:51.430130+0000","last_active":"2024-05-08T06:08:01.057546+0000","last_peered":"2024-05-08T06:08:01.057546+0000","last_clean":"2024-05-08T06:08:01.057546+0000","last_became_active":"2024-05-08T06:05:51.429794+0000","last_became_peered":"2024-05-08T06:05:51.429794+0000","last_unstale":"2024-05-08T06:08:01.057546+0000","last_undegraded":"2024-05-08T06:08:01.057546+0000","last_fullsized":"2024-05-08T06:08:01.057546+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:21:55.773309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480539+0000","last_change":"2024-05-08T06:05:46.351664+0000","last_active":"2024-05-08T06:06:32.480539+0000","last_peered":"2024-05-08T06:06:32.480539+0000","last_clean":"2024-05-08T06:06:32.480539+0000","last_became_active":"2024-05-08T06:05:46.351305+0000","last_became_peered":"2024-05-08T06:05:46.351305+0000","last_unstale":"2024-05-08T06:06:32.480539+0000","last_undegraded":"2024-05-08T06:06:32.480539+0000","last_fullsized":"2024-05-08T06:06:32.480539+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:24:36.164315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.043418+0000","last_change":"2024-05-08T06:05:51.414793+0000","last_active":"2024-05-08T06:08:01.043418+0000","last_peered":"2024-05-08T06:08:01.043418+0000","last_clean":"2024-05-08T06:08:01.043418+0000","last_became_active":"2024-05-08T06:05:51.414317+0000","last_became_peered":"2024-05-08T06:05:51.414317+0000","last_unstale":"2024-05-08T06:08:01.043418+0000","last_undegraded":"2024-05-08T06:08:01.043418+0000","last_fullsized":"2024-05-08T06:08:01.043418+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:57:56.152836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":274,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:10:13.723637+0000","last_change":"2024-05-08T06:05:46.352683+0000","last_active":"2024-05-08T06:10:13.723637+0000","last_peered":"2024-05-08T06:10:13.723637+0000","last_clean":"2024-05-08T06:10:13.723637+0000","last_became_active":"2024-05-08T06:05:46.352278+0000","last_became_peered":"2024-05-08T06:05:46.352278+0000","last_unstale":"2024-05-08T06:10:13.723637+0000","last_undegraded":"2024-05-08T06:10:13.723637+0000","last_fullsized":"2024-05-08T06:10:13.723637+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:39:22.655516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1323,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":240,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.037458+0000","last_change":"2024-05-08T06:05:51.415822+0000","last_active":"2024-05-08T06:08:01.037458+0000","last_peered":"2024-05-08T06:08:01.037458+0000","last_clean":"2024-05-08T06:08:01.037458+0000","last_became_active":"2024-05-08T06:05:51.415449+0000","last_became_peered":"2024-05-08T06:05:51.415449+0000","last_unstale":"2024-05-08T06:08:01.037458+0000","last_undegraded":"2024-05-08T06:08:01.037458+0000","last_fullsized":"2024-05-08T06:08:01.037458+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:44:49.127335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.478073+0000","last_change":"2024-05-08T06:05:46.361288+0000","last_active":"2024-05-08T06:06:32.478073+0000","last_peered":"2024-05-08T06:06:32.478073+0000","last_clean":"2024-05-08T06:06:32.478073+0000","last_became_active":"2024-05-08T06:05:46.360761+0000","last_became_peered":"2024-05-08T06:05:46.360761+0000","last_unstale":"2024-05-08T06:06:32.478073+0000","last_undegraded":"2024-05-08T06:06:32.478073+0000","last_fullsized":"2024-05-08T06:06:32.478073+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:14:59.858935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.063490+0000","last_change":"2024-05-08T06:05:51.418547+0000","last_active":"2024-05-08T06:08:01.063490+0000","last_peered":"2024-05-08T06:08:01.063490+0000","last_clean":"2024-05-08T06:08:01.063490+0000","last_became_active":"2024-05-08T06:05:51.417588+0000","last_became_peered":"2024-05-08T06:05:51.417588+0000","last_unstale":"2024-05-08T06:08:01.063490+0000","last_undegraded":"2024-05-08T06:08:01.063490+0000","last_fullsized":"2024-05-08T06:08:01.063490+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:16:41.451351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.492080+0000","last_change":"2024-05-08T06:05:46.362077+0000","last_active":"2024-05-08T06:06:32.492080+0000","last_peered":"2024-05-08T06:06:32.492080+0000","last_clean":"2024-05-08T06:06:32.492080+0000","last_became_active":"2024-05-08T06:05:46.360823+0000","last_became_peered":"2024-05-08T06:05:46.360823+0000","last_unstale":"2024-05-08T06:06:32.492080+0000","last_undegraded":"2024-05-08T06:06:32.492080+0000","last_fullsized":"2024-05-08T06:06:32.492080+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:14.169822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.063350+0000","last_change":"2024-05-08T06:05:51.423687+0000","last_active":"2024-05-08T06:08:01.063350+0000","last_peered":"2024-05-08T06:08:01.063350+0000","last_clean":"2024-05-08T06:08:01.063350+0000","last_became_active":"2024-05-08T06:05:51.417656+0000","last_became_peered":"2024-05-08T06:05:51.417656+0000","last_unstale":"2024-05-08T06:08:01.063350+0000","last_undegraded":"2024-05-08T06:08:01.063350+0000","last_fullsized":"2024-05-08T06:08:01.063350+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:59:31.319915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.595803+0000","last_change":"2024-05-08T06:05:46.351892+0000","last_active":"2024-05-08T06:06:32.595803+0000","last_peered":"2024-05-08T06:06:32.595803+0000","last_clean":"2024-05-08T06:06:32.595803+0000","last_became_active":"2024-05-08T06:05:46.351523+0000","last_became_peered":"2024-05-08T06:05:46.351523+0000","last_unstale":"2024-05-08T06:06:32.595803+0000","last_undegraded":"2024-05-08T06:06:32.595803+0000","last_fullsized":"2024-05-08T06:06:32.595803+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:11:23.794313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.063933+0000","last_change":"2024-05-08T06:05:51.407681+0000","last_active":"2024-05-08T06:08:01.063933+0000","last_peered":"2024-05-08T06:08:01.063933+0000","last_clean":"2024-05-08T06:08:01.063933+0000","last_became_active":"2024-05-08T06:05:51.407243+0000","last_became_peered":"2024-05-08T06:05:51.407243+0000","last_unstale":"2024-05-08T06:08:01.063933+0000","last_undegraded":"2024-05-08T06:08:01.063933+0000","last_fullsized":"2024-05-08T06:08:01.063933+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:06:30.490268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":271,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:10:16.728337+0000","last_change":"2024-05-08T06:05:46.356675+0000","last_active":"2024-05-08T06:10:16.728337+0000","last_peered":"2024-05-08T06:10:16.728337+0000","last_clean":"2024-05-08T06:10:16.728337+0000","last_became_active":"2024-05-08T06:05:46.356360+0000","last_became_peered":"2024-05-08T06:05:46.356360+0000","last_unstale":"2024-05-08T06:10:16.728337+0000","last_undegraded":"2024-05-08T06:10:16.728337+0000","last_fullsized":"2024-05-08T06:10:16.728337+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:46:48.131551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":240,"num_read_kb":240,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.047752+0000","last_change":"2024-05-08T06:05:51.421325+0000","last_active":"2024-05-08T06:08:01.047752+0000","last_peered":"2024-05-08T06:08:01.047752+0000","last_clean":"2024-05-08T06:08:01.047752+0000","last_became_active":"2024-05-08T06:05:51.420905+0000","last_became_peered":"2024-05-08T06:05:51.420905+0000","last_unstale":"2024-05-08T06:08:01.047752+0000","last_undegraded":"2024-05-08T06:08:01.047752+0000","last_fullsized":"2024-05-08T06:08:01.047752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:37:27.324920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482723+0000","last_change":"2024-05-08T06:05:46.351958+0000","last_active":"2024-05-08T06:06:32.482723+0000","last_peered":"2024-05-08T06:06:32.482723+0000","last_clean":"2024-05-08T06:06:32.482723+0000","last_became_active":"2024-05-08T06:05:46.351561+0000","last_became_peered":"2024-05-08T06:05:46.351561+0000","last_unstale":"2024-05-08T06:06:32.482723+0000","last_undegraded":"2024-05-08T06:06:32.482723+0000","last_fullsized":"2024-05-08T06:06:32.482723+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:11:27.727029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.061468+0000","last_change":"2024-05-08T06:05:51.431809+0000","last_active":"2024-05-08T06:08:01.061468+0000","last_peered":"2024-05-08T06:08:01.061468+0000","last_clean":"2024-05-08T06:08:01.061468+0000","last_became_active":"2024-05-08T06:05:51.431416+0000","last_became_peered":"2024-05-08T06:05:51.431416+0000","last_unstale":"2024-05-08T06:08:01.061468+0000","last_undegraded":"2024-05-08T06:08:01.061468+0000","last_fullsized":"2024-05-08T06:08:01.061468+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:48:16.704134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.596352+0000","last_change":"2024-05-08T06:05:46.550763+0000","last_active":"2024-05-08T06:06:32.596352+0000","last_peered":"2024-05-08T06:06:32.596352+0000","last_clean":"2024-05-08T06:06:32.596352+0000","last_became_active":"2024-05-08T06:05:46.550297+0000","last_became_peered":"2024-05-08T06:05:46.550297+0000","last_unstale":"2024-05-08T06:06:32.596352+0000","last_undegraded":"2024-05-08T06:06:32.596352+0000","last_fullsized":"2024-05-08T06:06:32.596352+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:32:19.907220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.051960+0000","last_change":"2024-05-08T06:05:51.414738+0000","last_active":"2024-05-08T06:08:01.051960+0000","last_peered":"2024-05-08T06:08:01.051960+0000","last_clean":"2024-05-08T06:08:01.051960+0000","last_became_active":"2024-05-08T06:05:51.414369+0000","last_became_peered":"2024-05-08T06:05:51.414369+0000","last_unstale":"2024-05-08T06:08:01.051960+0000","last_undegraded":"2024-05-08T06:08:01.051960+0000","last_fullsized":"2024-05-08T06:08:01.051960+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:57:44.215343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'39","reported_seq":560,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:10:17.729921+0000","last_change":"2024-05-08T06:05:46.349825+0000","last_active":"2024-05-08T06:10:17.729921+0000","last_peered":"2024-05-08T06:10:17.729921+0000","last_clean":"2024-05-08T06:10:17.729921+0000","last_became_active":"2024-05-08T06:05:46.349425+0000","last_became_peered":"2024-05-08T06:05:46.349425+0000","last_unstale":"2024-05-08T06:10:17.729921+0000","last_undegraded":"2024-05-08T06:10:17.729921+0000","last_fullsized":"2024-05-08T06:10:17.729921+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:29:39.123817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73053,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":494,"num_read_kb":319,"num_write":40,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.022668+0000","last_change":"2024-05-08T06:05:51.411341+0000","last_active":"2024-05-08T06:08:01.022668+0000","last_peered":"2024-05-08T06:08:01.022668+0000","last_clean":"2024-05-08T06:08:01.022668+0000","last_became_active":"2024-05-08T06:05:51.411002+0000","last_became_peered":"2024-05-08T06:05:51.411002+0000","last_unstale":"2024-05-08T06:08:01.022668+0000","last_undegraded":"2024-05-08T06:08:01.022668+0000","last_fullsized":"2024-05-08T06:08:01.022668+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:27:07.013809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:34.244232+0000","last_change":"2024-05-08T06:06:32.508904+0000","last_active":"2024-05-08T06:06:34.244232+0000","last_peered":"2024-05-08T06:06:34.244232+0000","last_clean":"2024-05-08T06:06:34.244232+0000","last_became_active":"2024-05-08T06:06:32.496001+0000","last_became_peered":"2024-05-08T06:06:32.496001+0000","last_unstale":"2024-05-08T06:06:34.244232+0000","last_undegraded":"2024-05-08T06:06:34.244232+0000","last_fullsized":"2024-05-08T06:06:34.244232+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:34:32.417105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.065212+0000","last_change":"2024-05-08T06:05:51.418395+0000","last_active":"2024-05-08T06:08:01.065212+0000","last_peered":"2024-05-08T06:08:01.065212+0000","last_clean":"2024-05-08T06:08:01.065212+0000","last_became_active":"2024-05-08T06:05:51.418035+0000","last_became_peered":"2024-05-08T06:05:51.418035+0000","last_unstale":"2024-05-08T06:08:01.065212+0000","last_undegraded":"2024-05-08T06:08:01.065212+0000","last_fullsized":"2024-05-08T06:08:01.065212+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:58:16.529144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.595996+0000","last_change":"2024-05-08T06:05:46.363088+0000","last_active":"2024-05-08T06:06:32.595996+0000","last_peered":"2024-05-08T06:06:32.595996+0000","last_clean":"2024-05-08T06:06:32.595996+0000","last_became_active":"2024-05-08T06:05:46.362488+0000","last_became_peered":"2024-05-08T06:05:46.362488+0000","last_unstale":"2024-05-08T06:06:32.595996+0000","last_undegraded":"2024-05-08T06:06:32.595996+0000","last_fullsized":"2024-05-08T06:06:32.595996+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:28:10.039647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.059203+0000","last_change":"2024-05-08T06:05:51.415567+0000","last_active":"2024-05-08T06:08:01.059203+0000","last_peered":"2024-05-08T06:08:01.059203+0000","last_clean":"2024-05-08T06:08:01.059203+0000","last_became_active":"2024-05-08T06:05:51.415148+0000","last_became_peered":"2024-05-08T06:05:51.415148+0000","last_unstale":"2024-05-08T06:08:01.059203+0000","last_undegraded":"2024-05-08T06:08:01.059203+0000","last_fullsized":"2024-05-08T06:08:01.059203+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:51:48.871149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'6","reported_seq":275,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:10:13.925119+0000","last_change":"2024-05-08T06:05:46.348769+0000","last_active":"2024-05-08T06:10:13.925119+0000","last_peered":"2024-05-08T06:10:13.925119+0000","last_clean":"2024-05-08T06:10:13.925119+0000","last_became_active":"2024-05-08T06:05:46.348072+0000","last_became_peered":"2024-05-08T06:05:46.348072+0000","last_unstale":"2024-05-08T06:10:13.925119+0000","last_undegraded":"2024-05-08T06:10:13.925119+0000","last_fullsized":"2024-05-08T06:10:13.925119+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:45.292436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":243,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.049147+0000","last_change":"2024-05-08T06:05:51.414623+0000","last_active":"2024-05-08T06:08:01.049147+0000","last_peered":"2024-05-08T06:08:01.049147+0000","last_clean":"2024-05-08T06:08:01.049147+0000","last_became_active":"2024-05-08T06:05:51.414258+0000","last_became_peered":"2024-05-08T06:05:51.414258+0000","last_unstale":"2024-05-08T06:08:01.049147+0000","last_undegraded":"2024-05-08T06:08:01.049147+0000","last_fullsized":"2024-05-08T06:08:01.049147+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:38:48.297900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":43,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480030+0000","last_change":"2024-05-08T06:05:46.546936+0000","last_active":"2024-05-08T06:06:32.480030+0000","last_peered":"2024-05-08T06:06:32.480030+0000","last_clean":"2024-05-08T06:06:32.480030+0000","last_became_active":"2024-05-08T06:05:46.546574+0000","last_became_peered":"2024-05-08T06:05:46.546574+0000","last_unstale":"2024-05-08T06:06:32.480030+0000","last_undegraded":"2024-05-08T06:06:32.480030+0000","last_fullsized":"2024-05-08T06:06:32.480030+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:16:29.720304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.029792+0000","last_change":"2024-05-08T06:05:51.418012+0000","last_active":"2024-05-08T06:08:01.029792+0000","last_peered":"2024-05-08T06:08:01.029792+0000","last_clean":"2024-05-08T06:08:01.029792+0000","last_became_active":"2024-05-08T06:05:51.417646+0000","last_became_peered":"2024-05-08T06:05:51.417646+0000","last_unstale":"2024-05-08T06:08:01.029792+0000","last_undegraded":"2024-05-08T06:08:01.029792+0000","last_fullsized":"2024-05-08T06:08:01.029792+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:43:02.733967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:00.689473+0000","last_change":"2024-05-08T06:05:46.347748+0000","last_active":"2024-05-08T06:08:00.689473+0000","last_peered":"2024-05-08T06:08:00.689473+0000","last_clean":"2024-05-08T06:08:00.689473+0000","last_became_active":"2024-05-08T06:05:46.347367+0000","last_became_peered":"2024-05-08T06:05:46.347367+0000","last_unstale":"2024-05-08T06:08:00.689473+0000","last_undegraded":"2024-05-08T06:08:00.689473+0000","last_fullsized":"2024-05-08T06:08:00.689473+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:51:50.973855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.038216+0000","last_change":"2024-05-08T06:06:32.597018+0000","last_active":"2024-05-08T06:08:01.038216+0000","last_peered":"2024-05-08T06:08:01.038216+0000","last_clean":"2024-05-08T06:08:01.038216+0000","last_became_active":"2024-05-08T06:06:32.596677+0000","last_became_peered":"2024-05-08T06:06:32.596677+0000","last_unstale":"2024-05-08T06:08:01.038216+0000","last_undegraded":"2024-05-08T06:08:01.038216+0000","last_fullsized":"2024-05-08T06:08:01.038216+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:02:46.223949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:07:51.948701+0000","last_change":"2024-05-08T06:05:46.554443+0000","last_active":"2024-05-08T06:07:51.948701+0000","last_peered":"2024-05-08T06:07:51.948701+0000","last_clean":"2024-05-08T06:07:51.948701+0000","last_became_active":"2024-05-08T06:05:46.553226+0000","last_became_peered":"2024-05-08T06:05:46.553226+0000","last_unstale":"2024-05-08T06:07:51.948701+0000","last_undegraded":"2024-05-08T06:07:51.948701+0000","last_fullsized":"2024-05-08T06:07:51.948701+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:56:39.784652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.060932+0000","last_change":"2024-05-08T06:05:51.413121+0000","last_active":"2024-05-08T06:08:01.060932+0000","last_peered":"2024-05-08T06:08:01.060932+0000","last_clean":"2024-05-08T06:08:01.060932+0000","last_became_active":"2024-05-08T06:05:51.412767+0000","last_became_peered":"2024-05-08T06:05:51.412767+0000","last_unstale":"2024-05-08T06:08:01.060932+0000","last_undegraded":"2024-05-08T06:08:01.060932+0000","last_fullsized":"2024-05-08T06:08:01.060932+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:59:13.416981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:07:51.431460+0000","last_change":"2024-05-08T06:05:46.361349+0000","last_active":"2024-05-08T06:07:51.431460+0000","last_peered":"2024-05-08T06:07:51.431460+0000","last_clean":"2024-05-08T06:07:51.431460+0000","last_became_active":"2024-05-08T06:05:46.360654+0000","last_became_peered":"2024-05-08T06:05:46.360654+0000","last_unstale":"2024-05-08T06:07:51.431460+0000","last_undegraded":"2024-05-08T06:07:51.431460+0000","last_fullsized":"2024-05-08T06:07:51.431460+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:19:22.144874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.062214+0000","last_change":"2024-05-08T06:05:51.423836+0000","last_active":"2024-05-08T06:08:01.062214+0000","last_peered":"2024-05-08T06:08:01.062214+0000","last_clean":"2024-05-08T06:08:01.062214+0000","last_became_active":"2024-05-08T06:05:51.423424+0000","last_became_peered":"2024-05-08T06:05:51.423424+0000","last_unstale":"2024-05-08T06:08:01.062214+0000","last_undegraded":"2024-05-08T06:08:01.062214+0000","last_fullsized":"2024-05-08T06:08:01.062214+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:56:14.779765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.048549+0000","last_change":"2024-05-08T06:06:32.598978+0000","last_active":"2024-05-08T06:08:01.048549+0000","last_peered":"2024-05-08T06:08:01.048549+0000","last_clean":"2024-05-08T06:08:01.048549+0000","last_became_active":"2024-05-08T06:06:32.598158+0000","last_became_peered":"2024-05-08T06:06:32.598158+0000","last_unstale":"2024-05-08T06:08:01.048549+0000","last_undegraded":"2024-05-08T06:08:01.048549+0000","last_fullsized":"2024-05-08T06:08:01.048549+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:51:09.270066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":271,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:10:13.923393+0000","last_change":"2024-05-08T06:05:46.547450+0000","last_active":"2024-05-08T06:10:13.923393+0000","last_peered":"2024-05-08T06:10:13.923393+0000","last_clean":"2024-05-08T06:10:13.923393+0000","last_became_active":"2024-05-08T06:05:46.546444+0000","last_became_peered":"2024-05-08T06:05:46.546444+0000","last_unstale":"2024-05-08T06:10:13.923393+0000","last_undegraded":"2024-05-08T06:10:13.923393+0000","last_fullsized":"2024-05-08T06:10:13.923393+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:17:14.124224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":237,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.065294+0000","last_change":"2024-05-08T06:05:51.423191+0000","last_active":"2024-05-08T06:08:01.065294+0000","last_peered":"2024-05-08T06:08:01.065294+0000","last_clean":"2024-05-08T06:08:01.065294+0000","last_became_active":"2024-05-08T06:05:51.420912+0000","last_became_peered":"2024-05-08T06:05:51.420912+0000","last_unstale":"2024-05-08T06:08:01.065294+0000","last_undegraded":"2024-05-08T06:08:01.065294+0000","last_fullsized":"2024-05-08T06:08:01.065294+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:27:57.678527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:07:50.743006+0000","last_change":"2024-05-08T06:05:46.348650+0000","last_active":"2024-05-08T06:07:50.743006+0000","last_peered":"2024-05-08T06:07:50.743006+0000","last_clean":"2024-05-08T06:07:50.743006+0000","last_became_active":"2024-05-08T06:05:46.347266+0000","last_became_peered":"2024-05-08T06:05:46.347266+0000","last_unstale":"2024-05-08T06:07:50.743006+0000","last_undegraded":"2024-05-08T06:07:50.743006+0000","last_fullsized":"2024-05-08T06:07:50.743006+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:37:30.623580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":101,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:47.536769+0000","last_change":"2024-05-08T06:04:09.404663+0000","last_active":"2024-05-08T06:06:47.536769+0000","last_peered":"2024-05-08T06:06:47.536769+0000","last_clean":"2024-05-08T06:06:47.536769+0000","last_became_active":"2024-05-08T06:04:09.400115+0000","last_became_peered":"2024-05-08T06:04:09.400115+0000","last_unstale":"2024-05-08T06:06:47.536769+0000","last_undegraded":"2024-05-08T06:06:47.536769+0000","last_fullsized":"2024-05-08T06:06:47.536769+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:00:17.419712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:00:17.419712+0000","last_clean_scrub_stamp":"2024-05-08T06:00:17.419712+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:27:39.154694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.053609+0000","last_change":"2024-05-08T06:05:51.421826+0000","last_active":"2024-05-08T06:08:01.053609+0000","last_peered":"2024-05-08T06:08:01.053609+0000","last_clean":"2024-05-08T06:08:01.053609+0000","last_became_active":"2024-05-08T06:05:51.420706+0000","last_became_peered":"2024-05-08T06:05:51.420706+0000","last_unstale":"2024-05-08T06:08:01.053609+0000","last_undegraded":"2024-05-08T06:08:01.053609+0000","last_fullsized":"2024-05-08T06:08:01.053609+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:06:16.242332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:07:47.542941+0000","last_change":"2024-05-08T06:05:46.351365+0000","last_active":"2024-05-08T06:07:47.542941+0000","last_peered":"2024-05-08T06:07:47.542941+0000","last_clean":"2024-05-08T06:07:47.542941+0000","last_became_active":"2024-05-08T06:05:46.351022+0000","last_became_peered":"2024-05-08T06:05:46.351022+0000","last_unstale":"2024-05-08T06:07:47.542941+0000","last_undegraded":"2024-05-08T06:07:47.542941+0000","last_fullsized":"2024-05-08T06:07:47.542941+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:32:44.134994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:34.244260+0000","last_change":"2024-05-08T06:06:32.501263+0000","last_active":"2024-05-08T06:06:34.244260+0000","last_peered":"2024-05-08T06:06:34.244260+0000","last_clean":"2024-05-08T06:06:34.244260+0000","last_became_active":"2024-05-08T06:06:32.493795+0000","last_became_peered":"2024-05-08T06:06:32.493795+0000","last_unstale":"2024-05-08T06:06:34.244260+0000","last_undegraded":"2024-05-08T06:06:34.244260+0000","last_fullsized":"2024-05-08T06:06:34.244260+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:49:03.898162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.056756+0000","last_change":"2024-05-08T06:05:51.428382+0000","last_active":"2024-05-08T06:08:01.056756+0000","last_peered":"2024-05-08T06:08:01.056756+0000","last_clean":"2024-05-08T06:08:01.056756+0000","last_became_active":"2024-05-08T06:05:51.421443+0000","last_became_peered":"2024-05-08T06:05:51.421443+0000","last_unstale":"2024-05-08T06:08:01.056756+0000","last_undegraded":"2024-05-08T06:08:01.056756+0000","last_fullsized":"2024-05-08T06:08:01.056756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:26:10.215020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'15","reported_seq":43,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:10:11.475242+0000","last_change":"2024-05-08T06:05:46.364306+0000","last_active":"2024-05-08T06:10:11.475242+0000","last_peered":"2024-05-08T06:10:11.475242+0000","last_clean":"2024-05-08T06:10:11.475242+0000","last_became_active":"2024-05-08T06:05:46.363871+0000","last_became_peered":"2024-05-08T06:05:46.363871+0000","last_unstale":"2024-05-08T06:10:11.475242+0000","last_undegraded":"2024-05-08T06:10:11.475242+0000","last_fullsized":"2024-05-08T06:10:11.475242+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:38:34.531162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:05.007928+0000","last_change":"2024-05-08T06:05:51.415020+0000","last_active":"2024-05-08T06:08:05.007928+0000","last_peered":"2024-05-08T06:08:05.007928+0000","last_clean":"2024-05-08T06:08:05.007928+0000","last_became_active":"2024-05-08T06:05:51.413690+0000","last_became_peered":"2024-05-08T06:05:51.413690+0000","last_unstale":"2024-05-08T06:08:05.007928+0000","last_undegraded":"2024-05-08T06:08:05.007928+0000","last_fullsized":"2024-05-08T06:08:05.007928+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:55:38.121492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:07:55.743454+0000","last_change":"2024-05-08T06:05:46.359100+0000","last_active":"2024-05-08T06:07:55.743454+0000","last_peered":"2024-05-08T06:07:55.743454+0000","last_clean":"2024-05-08T06:07:55.743454+0000","last_became_active":"2024-05-08T06:05:46.358815+0000","last_became_peered":"2024-05-08T06:05:46.358815+0000","last_unstale":"2024-05-08T06:07:55.743454+0000","last_undegraded":"2024-05-08T06:07:55.743454+0000","last_fullsized":"2024-05-08T06:07:55.743454+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:22:58.532358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:02.879400+0000","last_change":"2024-05-08T06:05:51.429222+0000","last_active":"2024-05-08T06:08:02.879400+0000","last_peered":"2024-05-08T06:08:02.879400+0000","last_clean":"2024-05-08T06:08:02.879400+0000","last_became_active":"2024-05-08T06:05:51.428845+0000","last_became_peered":"2024-05-08T06:05:51.428845+0000","last_unstale":"2024-05-08T06:08:02.879400+0000","last_undegraded":"2024-05-08T06:08:02.879400+0000","last_fullsized":"2024-05-08T06:08:02.879400+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:36:41.435666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:07:57.210823+0000","last_change":"2024-05-08T06:05:46.547931+0000","last_active":"2024-05-08T06:07:57.210823+0000","last_peered":"2024-05-08T06:07:57.210823+0000","last_clean":"2024-05-08T06:07:57.210823+0000","last_became_active":"2024-05-08T06:05:46.547547+0000","last_became_peered":"2024-05-08T06:05:46.547547+0000","last_unstale":"2024-05-08T06:07:57.210823+0000","last_undegraded":"2024-05-08T06:07:57.210823+0000","last_fullsized":"2024-05-08T06:07:57.210823+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:27:03.502153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.020367+0000","last_change":"2024-05-08T06:05:51.413259+0000","last_active":"2024-05-08T06:08:01.020367+0000","last_peered":"2024-05-08T06:08:01.020367+0000","last_clean":"2024-05-08T06:08:01.020367+0000","last_became_active":"2024-05-08T06:05:51.412890+0000","last_became_peered":"2024-05-08T06:05:51.412890+0000","last_unstale":"2024-05-08T06:08:01.020367+0000","last_undegraded":"2024-05-08T06:08:01.020367+0000","last_fullsized":"2024-05-08T06:08:01.020367+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:04:56.690098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:00.007037+0000","last_change":"2024-05-08T06:05:46.365409+0000","last_active":"2024-05-08T06:08:00.007037+0000","last_peered":"2024-05-08T06:08:00.007037+0000","last_clean":"2024-05-08T06:08:00.007037+0000","last_became_active":"2024-05-08T06:05:46.361277+0000","last_became_peered":"2024-05-08T06:05:46.361277+0000","last_unstale":"2024-05-08T06:08:00.007037+0000","last_undegraded":"2024-05-08T06:08:00.007037+0000","last_fullsized":"2024-05-08T06:08:00.007037+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:06.927385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:04.253756+0000","last_change":"2024-05-08T06:05:51.423749+0000","last_active":"2024-05-08T06:08:04.253756+0000","last_peered":"2024-05-08T06:08:04.253756+0000","last_clean":"2024-05-08T06:08:04.253756+0000","last_became_active":"2024-05-08T06:05:51.423386+0000","last_became_peered":"2024-05-08T06:05:51.423386+0000","last_unstale":"2024-05-08T06:08:04.253756+0000","last_undegraded":"2024-05-08T06:08:04.253756+0000","last_fullsized":"2024-05-08T06:08:04.253756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:41:26.261683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'2","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:10:06.033061+0000","last_change":"2024-05-08T06:05:46.362497+0000","last_active":"2024-05-08T06:10:06.033061+0000","last_peered":"2024-05-08T06:10:06.033061+0000","last_clean":"2024-05-08T06:10:06.033061+0000","last_became_active":"2024-05-08T06:05:46.362110+0000","last_became_peered":"2024-05-08T06:05:46.362110+0000","last_unstale":"2024-05-08T06:10:06.033061+0000","last_undegraded":"2024-05-08T06:10:06.033061+0000","last_fullsized":"2024-05-08T06:10:06.033061+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:25:06.205380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.060826+0000","last_change":"2024-05-08T06:06:32.495078+0000","last_active":"2024-05-08T06:08:01.060826+0000","last_peered":"2024-05-08T06:08:01.060826+0000","last_clean":"2024-05-08T06:08:01.060826+0000","last_became_active":"2024-05-08T06:06:32.493855+0000","last_became_peered":"2024-05-08T06:06:32.493855+0000","last_unstale":"2024-05-08T06:08:01.060826+0000","last_undegraded":"2024-05-08T06:08:01.060826+0000","last_fullsized":"2024-05-08T06:08:01.060826+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:56:54.143080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:00.689448+0000","last_change":"2024-05-08T06:06:32.601826+0000","last_active":"2024-05-08T06:08:00.689448+0000","last_peered":"2024-05-08T06:08:00.689448+0000","last_clean":"2024-05-08T06:08:00.689448+0000","last_became_active":"2024-05-08T06:06:32.597823+0000","last_became_peered":"2024-05-08T06:06:32.597823+0000","last_unstale":"2024-05-08T06:08:00.689448+0000","last_undegraded":"2024-05-08T06:08:00.689448+0000","last_fullsized":"2024-05-08T06:08:00.689448+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:59:09.741096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:02.544515+0000","last_change":"2024-05-08T06:05:51.413744+0000","last_active":"2024-05-08T06:08:02.544515+0000","last_peered":"2024-05-08T06:08:02.544515+0000","last_clean":"2024-05-08T06:08:02.544515+0000","last_became_active":"2024-05-08T06:05:51.413346+0000","last_became_peered":"2024-05-08T06:05:51.413346+0000","last_unstale":"2024-05-08T06:08:02.544515+0000","last_undegraded":"2024-05-08T06:08:02.544515+0000","last_fullsized":"2024-05-08T06:08:02.544515+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:46:56.109755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.019506+0000","last_change":"2024-05-08T06:05:46.552604+0000","last_active":"2024-05-08T06:08:01.019506+0000","last_peered":"2024-05-08T06:08:01.019506+0000","last_clean":"2024-05-08T06:08:01.019506+0000","last_became_active":"2024-05-08T06:05:46.552248+0000","last_became_peered":"2024-05-08T06:05:46.552248+0000","last_unstale":"2024-05-08T06:08:01.019506+0000","last_undegraded":"2024-05-08T06:08:01.019506+0000","last_fullsized":"2024-05-08T06:08:01.019506+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:39:10.306692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.054186+0000","last_change":"2024-05-08T06:05:51.404988+0000","last_active":"2024-05-08T06:08:01.054186+0000","last_peered":"2024-05-08T06:08:01.054186+0000","last_clean":"2024-05-08T06:08:01.054186+0000","last_became_active":"2024-05-08T06:05:51.404625+0000","last_became_peered":"2024-05-08T06:05:51.404625+0000","last_unstale":"2024-05-08T06:08:01.054186+0000","last_undegraded":"2024-05-08T06:08:01.054186+0000","last_fullsized":"2024-05-08T06:08:01.054186+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:33:27.919929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.596178+0000","last_change":"2024-05-08T06:05:46.361466+0000","last_active":"2024-05-08T06:06:32.596178+0000","last_peered":"2024-05-08T06:06:32.596178+0000","last_clean":"2024-05-08T06:06:32.596178+0000","last_became_active":"2024-05-08T06:05:46.361107+0000","last_became_peered":"2024-05-08T06:05:46.361107+0000","last_unstale":"2024-05-08T06:06:32.596178+0000","last_undegraded":"2024-05-08T06:06:32.596178+0000","last_fullsized":"2024-05-08T06:06:32.596178+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:15:31.314226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.062827+0000","last_change":"2024-05-08T06:05:51.416437+0000","last_active":"2024-05-08T06:08:01.062827+0000","last_peered":"2024-05-08T06:08:01.062827+0000","last_clean":"2024-05-08T06:08:01.062827+0000","last_became_active":"2024-05-08T06:05:51.416070+0000","last_became_peered":"2024-05-08T06:05:51.416070+0000","last_unstale":"2024-05-08T06:08:01.062827+0000","last_undegraded":"2024-05-08T06:08:01.062827+0000","last_fullsized":"2024-05-08T06:08:01.062827+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:35:23.652373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:00.709889+0000","last_change":"2024-05-08T06:05:46.362475+0000","last_active":"2024-05-08T06:08:00.709889+0000","last_peered":"2024-05-08T06:08:00.709889+0000","last_clean":"2024-05-08T06:08:00.709889+0000","last_became_active":"2024-05-08T06:05:46.361852+0000","last_became_peered":"2024-05-08T06:05:46.361852+0000","last_unstale":"2024-05-08T06:08:00.709889+0000","last_undegraded":"2024-05-08T06:08:00.709889+0000","last_fullsized":"2024-05-08T06:08:00.709889+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:16:54.536330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.049399+0000","last_change":"2024-05-08T06:05:51.410718+0000","last_active":"2024-05-08T06:08:01.049399+0000","last_peered":"2024-05-08T06:08:01.049399+0000","last_clean":"2024-05-08T06:08:01.049399+0000","last_became_active":"2024-05-08T06:05:51.408902+0000","last_became_peered":"2024-05-08T06:05:51.408902+0000","last_unstale":"2024-05-08T06:08:01.049399+0000","last_undegraded":"2024-05-08T06:08:01.049399+0000","last_fullsized":"2024-05-08T06:08:01.049399+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:23:33.857677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.039067+0000","last_change":"2024-05-08T06:05:51.414362+0000","last_active":"2024-05-08T06:08:01.039067+0000","last_peered":"2024-05-08T06:08:01.039067+0000","last_clean":"2024-05-08T06:08:01.039067+0000","last_became_active":"2024-05-08T06:05:51.413998+0000","last_became_peered":"2024-05-08T06:05:51.413998+0000","last_unstale":"2024-05-08T06:08:01.039067+0000","last_undegraded":"2024-05-08T06:08:01.039067+0000","last_fullsized":"2024-05-08T06:08:01.039067+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:38:22.198219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.032578+0000","last_change":"2024-05-08T06:05:51.415508+0000","last_active":"2024-05-08T06:08:01.032578+0000","last_peered":"2024-05-08T06:08:01.032578+0000","last_clean":"2024-05-08T06:08:01.032578+0000","last_became_active":"2024-05-08T06:05:51.415137+0000","last_became_peered":"2024-05-08T06:05:51.415137+0000","last_unstale":"2024-05-08T06:08:01.032578+0000","last_undegraded":"2024-05-08T06:08:01.032578+0000","last_fullsized":"2024-05-08T06:08:01.032578+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:15.007996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.064401+0000","last_change":"2024-05-08T06:05:51.422093+0000","last_active":"2024-05-08T06:08:01.064401+0000","last_peered":"2024-05-08T06:08:01.064401+0000","last_clean":"2024-05-08T06:08:01.064401+0000","last_became_active":"2024-05-08T06:05:51.421733+0000","last_became_peered":"2024-05-08T06:05:51.421733+0000","last_unstale":"2024-05-08T06:08:01.064401+0000","last_undegraded":"2024-05-08T06:08:01.064401+0000","last_fullsized":"2024-05-08T06:08:01.064401+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:35:55.950229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.062534+0000","last_change":"2024-05-08T06:05:51.434705+0000","last_active":"2024-05-08T06:08:01.062534+0000","last_peered":"2024-05-08T06:08:01.062534+0000","last_clean":"2024-05-08T06:08:01.062534+0000","last_became_active":"2024-05-08T06:05:51.434324+0000","last_became_peered":"2024-05-08T06:05:51.434324+0000","last_unstale":"2024-05-08T06:08:01.062534+0000","last_undegraded":"2024-05-08T06:08:01.062534+0000","last_fullsized":"2024-05-08T06:08:01.062534+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:38:23.101247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.064059+0000","last_change":"2024-05-08T06:06:32.499837+0000","last_active":"2024-05-08T06:08:01.064059+0000","last_peered":"2024-05-08T06:08:01.064059+0000","last_clean":"2024-05-08T06:08:01.064059+0000","last_became_active":"2024-05-08T06:06:32.499026+0000","last_became_peered":"2024-05-08T06:06:32.499026+0000","last_unstale":"2024-05-08T06:08:01.064059+0000","last_undegraded":"2024-05-08T06:08:01.064059+0000","last_fullsized":"2024-05-08T06:08:01.064059+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:48:51.910384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:04.253787+0000","last_change":"2024-05-08T06:05:51.412370+0000","last_active":"2024-05-08T06:08:04.253787+0000","last_peered":"2024-05-08T06:08:04.253787+0000","last_clean":"2024-05-08T06:08:04.253787+0000","last_became_active":"2024-05-08T06:05:51.412057+0000","last_became_peered":"2024-05-08T06:05:51.412057+0000","last_unstale":"2024-05-08T06:08:04.253787+0000","last_undegraded":"2024-05-08T06:08:04.253787+0000","last_fullsized":"2024-05-08T06:08:04.253787+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:20:51.549926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.048132+0000","last_change":"2024-05-08T06:05:51.408393+0000","last_active":"2024-05-08T06:08:01.048132+0000","last_peered":"2024-05-08T06:08:01.048132+0000","last_clean":"2024-05-08T06:08:01.048132+0000","last_became_active":"2024-05-08T06:05:51.408023+0000","last_became_peered":"2024-05-08T06:05:51.408023+0000","last_unstale":"2024-05-08T06:08:01.048132+0000","last_undegraded":"2024-05-08T06:08:01.048132+0000","last_fullsized":"2024-05-08T06:08:01.048132+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:53.898929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:05.007852+0000","last_change":"2024-05-08T06:05:51.418055+0000","last_active":"2024-05-08T06:08:05.007852+0000","last_peered":"2024-05-08T06:08:05.007852+0000","last_clean":"2024-05-08T06:08:05.007852+0000","last_became_active":"2024-05-08T06:05:51.417067+0000","last_became_peered":"2024-05-08T06:05:51.417067+0000","last_unstale":"2024-05-08T06:08:05.007852+0000","last_undegraded":"2024-05-08T06:08:05.007852+0000","last_fullsized":"2024-05-08T06:08:05.007852+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:07:35.413672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:04.506147+0000","last_change":"2024-05-08T06:05:51.421025+0000","last_active":"2024-05-08T06:08:04.506147+0000","last_peered":"2024-05-08T06:08:04.506147+0000","last_clean":"2024-05-08T06:08:04.506147+0000","last_became_active":"2024-05-08T06:05:51.420794+0000","last_became_peered":"2024-05-08T06:05:51.420794+0000","last_unstale":"2024-05-08T06:08:04.506147+0000","last_undegraded":"2024-05-08T06:08:04.506147+0000","last_fullsized":"2024-05-08T06:08:04.506147+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:45:01.852405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.060497+0000","last_change":"2024-05-08T06:05:51.417709+0000","last_active":"2024-05-08T06:08:01.060497+0000","last_peered":"2024-05-08T06:08:01.060497+0000","last_clean":"2024-05-08T06:08:01.060497+0000","last_became_active":"2024-05-08T06:05:51.417337+0000","last_became_peered":"2024-05-08T06:05:51.417337+0000","last_unstale":"2024-05-08T06:08:01.060497+0000","last_undegraded":"2024-05-08T06:08:01.060497+0000","last_fullsized":"2024-05-08T06:08:01.060497+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:32:24.551768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.065666+0000","last_change":"2024-05-08T06:05:51.425318+0000","last_active":"2024-05-08T06:08:01.065666+0000","last_peered":"2024-05-08T06:08:01.065666+0000","last_clean":"2024-05-08T06:08:01.065666+0000","last_became_active":"2024-05-08T06:05:51.424931+0000","last_became_peered":"2024-05-08T06:05:51.424931+0000","last_unstale":"2024-05-08T06:08:01.065666+0000","last_undegraded":"2024-05-08T06:08:01.065666+0000","last_fullsized":"2024-05-08T06:08:01.065666+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:59:45.110201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.064188+0000","last_change":"2024-05-08T06:05:51.433529+0000","last_active":"2024-05-08T06:08:01.064188+0000","last_peered":"2024-05-08T06:08:01.064188+0000","last_clean":"2024-05-08T06:08:01.064188+0000","last_became_active":"2024-05-08T06:05:51.433173+0000","last_became_peered":"2024-05-08T06:05:51.433173+0000","last_unstale":"2024-05-08T06:08:01.064188+0000","last_undegraded":"2024-05-08T06:08:01.064188+0000","last_fullsized":"2024-05-08T06:08:01.064188+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:00:29.042188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.045237+0000","last_change":"2024-05-08T06:05:51.411418+0000","last_active":"2024-05-08T06:08:01.045237+0000","last_peered":"2024-05-08T06:08:01.045237+0000","last_clean":"2024-05-08T06:08:01.045237+0000","last_became_active":"2024-05-08T06:05:51.411084+0000","last_became_peered":"2024-05-08T06:05:51.411084+0000","last_unstale":"2024-05-08T06:08:01.045237+0000","last_undegraded":"2024-05-08T06:08:01.045237+0000","last_fullsized":"2024-05-08T06:08:01.045237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:08:11.522282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282383,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":78287,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1466,"num_read_kb":818,"num_write":190,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":234861,"data_compressed":5508,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645383,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":888832,"data_stored":431859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841613,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":1097260,"data_compressed":37744,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005139,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":897024,"data_stored":432039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201369,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":892928,"data_stored":431937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397598,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":1025122,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561123,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":884736,"data_stored":431428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574976,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":504274,"data_compressed":1836,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738502,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":432699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411451,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":974848,"data_stored":506291,"data_compressed":1836,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117109,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":431739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920879,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":888832,"data_stored":432788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757353,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":1023883,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34281,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34281,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34281,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":73053,"data_compressed":1836,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":75070,"data_compressed":1836,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":75554,"data_compressed":1836,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T06:10:20.412 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-05-08T06:10:20.412 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T06:10:20.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.658+0000 7fd2e6000640 1 -- 172.21.15.123:0/2072513384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2e0104c30 msgr2=0x7fd2e0105050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:20.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.658+0000 7fd2e6000640 1 --2- 172.21.15.123:0/2072513384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2e0104c30 0x7fd2e0105050 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0009900 tx=0x7fd2d002a3f0 comp rx=0 tx=0).stop 2024-05-08T06:10:20.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.658+0000 7fd2e6000640 1 -- 172.21.15.123:0/2072513384 shutdown_connections 2024-05-08T06:10:20.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.658+0000 7fd2e6000640 1 --2- 172.21.15.123:0/2072513384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2e01067e0 0x7fd2e010d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:20.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.658+0000 7fd2e6000640 1 --2- 172.21.15.123:0/2072513384 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2e0105e20 0x7fd2e01062a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:20.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.658+0000 7fd2e6000640 1 --2- 172.21.15.123:0/2072513384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2e0104c30 0x7fd2e0105050 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:20.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.658+0000 7fd2e6000640 1 -- 172.21.15.123:0/2072513384 >> 172.21.15.123:0/2072513384 conn(0x7fd2e01001a0 msgr2=0x7fd2e01025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:20.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.658+0000 7fd2e6000640 1 -- 172.21.15.123:0/2072513384 shutdown_connections 2024-05-08T06:10:20.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.658+0000 7fd2e6000640 1 -- 172.21.15.123:0/2072513384 wait complete. 2024-05-08T06:10:20.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.659+0000 7fd2e6000640 1 Processor -- start 2024-05-08T06:10:20.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.660+0000 7fd2e6000640 1 -- start start 2024-05-08T06:10:20.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.660+0000 7fd2e6000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2e0104c30 0x7fd2e019ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:20.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.660+0000 7fd2e6000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2e0105e20 0x7fd2e019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:20.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.660+0000 7fd2e6000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2e01067e0 0x7fd2e01a2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:20.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.660+0000 7fd2e6000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e01107a0 con 0x7fd2e0105e20 2024-05-08T06:10:20.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.660+0000 7fd2e6000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e0110620 con 0x7fd2e0104c30 2024-05-08T06:10:20.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.660+0000 7fd2e6000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e0110920 con 0x7fd2e01067e0 2024-05-08T06:10:20.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.660+0000 7fd2d6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2e0105e20 0x7fd2e019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:20.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.660+0000 7fd2d6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2e0105e20 0x7fd2e019b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41688/0 (socket says 172.21.15.123:41688) 2024-05-08T06:10:20.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.660+0000 7fd2d6a00640 1 -- 172.21.15.123:0/3730622971 learned_addr learned my addr 172.21.15.123:0/3730622971 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:10:20.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.660+0000 7fd2df400640 1 --2- 172.21.15.123:0/3730622971 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2e0104c30 0x7fd2e019ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:20.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.660+0000 7fd2dfe00640 1 --2- 172.21.15.123:0/3730622971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2e01067e0 0x7fd2e01a2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:20.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.661+0000 7fd2d6a00640 1 -- 172.21.15.123:0/3730622971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2e01067e0 msgr2=0x7fd2e01a2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:20.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.661+0000 7fd2d6a00640 1 --2- 172.21.15.123:0/3730622971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2e01067e0 0x7fd2e01a2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:20.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.661+0000 7fd2d6a00640 1 -- 172.21.15.123:0/3730622971 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2e0104c30 msgr2=0x7fd2e019ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:20.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.661+0000 7fd2d6a00640 1 --2- 172.21.15.123:0/3730622971 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2e0104c30 0x7fd2e019ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:20.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.661+0000 7fd2d6a00640 1 -- 172.21.15.123:0/3730622971 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2e01a2a90 con 0x7fd2e0105e20 2024-05-08T06:10:20.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.661+0000 7fd2df400640 1 --2- 172.21.15.123:0/3730622971 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2e0104c30 0x7fd2e019ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:10:20.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.661+0000 7fd2d6a00640 1 --2- 172.21.15.123:0/3730622971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2e0105e20 0x7fd2e019b3b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd2c800e9f0 tx=0x7fd2c800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:20.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.661+0000 7fd2dcc00640 1 -- 172.21.15.123:0/3730622971 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c800cd80 con 0x7fd2e0105e20 2024-05-08T06:10:20.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.661+0000 7fd2e6000640 1 -- 172.21.15.123:0/3730622971 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2e01a2d80 con 0x7fd2e0105e20 2024-05-08T06:10:20.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.661+0000 7fd2dcc00640 1 -- 172.21.15.123:0/3730622971 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2c80054d0 con 0x7fd2e0105e20 2024-05-08T06:10:20.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.661+0000 7fd2dcc00640 1 -- 172.21.15.123:0/3730622971 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8010690 con 0x7fd2e0105e20 2024-05-08T06:10:20.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.661+0000 7fd2e6000640 1 -- 172.21.15.123:0/3730622971 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2e01a3220 con 0x7fd2e0105e20 2024-05-08T06:10:20.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.663+0000 7fd2e6000640 1 -- 172.21.15.123:0/3730622971 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2e010c020 con 0x7fd2e0105e20 2024-05-08T06:10:20.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.665+0000 7fd2dcc00640 1 -- 172.21.15.123:0/3730622971 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 19) ==== 149704+0+0 (secure 0 0 0) 0x7fd2c8004e90 con 0x7fd2e0105e20 2024-05-08T06:10:20.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.666+0000 7fd2dcc00640 1 --2- 172.21.15.123:0/3730622971 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd2c40b2880 0x7fd2c40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:20.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.666+0000 7fd2dcc00640 1 -- 172.21.15.123:0/3730622971 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fd2c80e2890 con 0x7fd2e0105e20 2024-05-08T06:10:20.667 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.667+0000 7fd2df400640 1 --2- 172.21.15.123:0/3730622971 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd2c40b2880 0x7fd2c40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:20.668 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.667+0000 7fd2df400640 1 --2- 172.21.15.123:0/3730622971 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd2c40b2880 0x7fd2c40b4d40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0005ab0 tx=0x7fd2d0005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:20.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.671+0000 7fd2dcc00640 1 -- 172.21.15.123:0/3730622971 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd2c80aa400 con 0x7fd2e0105e20 2024-05-08T06:10:20.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.877+0000 7fd2e6000640 1 -- 172.21.15.123:0/3730622971 --> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd2e0062df0 con 0x7fd2c40b2880 2024-05-08T06:10:20.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.892+0000 7fd2dcc00640 1 -- 172.21.15.123:0/3730622971 <== mgr.14152 v2:172.21.15.123:6800/4169246006 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259055 (secure 0 0 0) 0x7fd2e0062df0 con 0x7fd2c40b2880 2024-05-08T06:10:20.896 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:10:20.898 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-05-08T06:10:20.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.896+0000 7fd2e6000640 1 -- 172.21.15.123:0/3730622971 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd2c40b2880 msgr2=0x7fd2c40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:20.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.896+0000 7fd2e6000640 1 --2- 172.21.15.123:0/3730622971 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd2c40b2880 0x7fd2c40b4d40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0005ab0 tx=0x7fd2d0005a20 comp rx=0 tx=0).stop 2024-05-08T06:10:20.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.897+0000 7fd2e6000640 1 -- 172.21.15.123:0/3730622971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2e0105e20 msgr2=0x7fd2e019b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:20.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.897+0000 7fd2e6000640 1 --2- 172.21.15.123:0/3730622971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2e0105e20 0x7fd2e019b3b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd2c800e9f0 tx=0x7fd2c800edf0 comp rx=0 tx=0).stop 2024-05-08T06:10:20.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.897+0000 7fd2e6000640 1 -- 172.21.15.123:0/3730622971 shutdown_connections 2024-05-08T06:10:20.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.897+0000 7fd2e6000640 1 --2- 172.21.15.123:0/3730622971 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd2c40b2880 0x7fd2c40b4d40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:20.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.897+0000 7fd2e6000640 1 --2- 172.21.15.123:0/3730622971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2e01067e0 0x7fd2e01a2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:20.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.897+0000 7fd2e6000640 1 --2- 172.21.15.123:0/3730622971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2e0105e20 0x7fd2e019b3b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:20.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.897+0000 7fd2e6000640 1 --2- 172.21.15.123:0/3730622971 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2e0104c30 0x7fd2e019ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:20.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.897+0000 7fd2e6000640 1 -- 172.21.15.123:0/3730622971 >> 172.21.15.123:0/3730622971 conn(0x7fd2e01001a0 msgr2=0x7fd2e01091f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:20.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.897+0000 7fd2e6000640 1 -- 172.21.15.123:0/3730622971 shutdown_connections 2024-05-08T06:10:20.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:20.897+0000 7fd2e6000640 1 -- 172.21.15.123:0/3730622971 wait complete. 2024-05-08T06:10:20.918 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":456,"stamp":"2024-05-08T06:10:19.154293+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668889,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1542,"num_read_kb":882,"num_write":315,"num_write_kb":2293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":2951896,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":755,"ondisk_log_size":755,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372504,"kb_used_data":10872,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569320,"statfs":{"total":1151940427776,"available":1151558983680,"internally_reserved":0,"allocated":11132928,"data_stored":7181319,"data_compressed":113232,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9686,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":81,"num_read_kb":53,"num_write":15,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005348"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.064237+0000","last_change":"2024-05-08T06:05:51.431744+0000","last_active":"2024-05-08T06:08:01.064237+0000","last_peered":"2024-05-08T06:08:01.064237+0000","last_clean":"2024-05-08T06:08:01.064237+0000","last_became_active":"2024-05-08T06:05:51.431405+0000","last_became_peered":"2024-05-08T06:05:51.431405+0000","last_unstale":"2024-05-08T06:08:01.064237+0000","last_undegraded":"2024-05-08T06:08:01.064237+0000","last_fullsized":"2024-05-08T06:08:01.064237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:56:40.503607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.062583+0000","last_change":"2024-05-08T06:05:51.426473+0000","last_active":"2024-05-08T06:08:01.062583+0000","last_peered":"2024-05-08T06:08:01.062583+0000","last_clean":"2024-05-08T06:08:01.062583+0000","last_became_active":"2024-05-08T06:05:51.424477+0000","last_became_peered":"2024-05-08T06:05:51.424477+0000","last_unstale":"2024-05-08T06:08:01.062583+0000","last_undegraded":"2024-05-08T06:08:01.062583+0000","last_fullsized":"2024-05-08T06:08:01.062583+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:26:55.747808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.060622+0000","last_change":"2024-05-08T06:05:51.423767+0000","last_active":"2024-05-08T06:08:01.060622+0000","last_peered":"2024-05-08T06:08:01.060622+0000","last_clean":"2024-05-08T06:08:01.060622+0000","last_became_active":"2024-05-08T06:05:51.423273+0000","last_became_peered":"2024-05-08T06:05:51.423273+0000","last_unstale":"2024-05-08T06:08:01.060622+0000","last_undegraded":"2024-05-08T06:08:01.060622+0000","last_fullsized":"2024-05-08T06:08:01.060622+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:24:44.099743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.047241+0000","last_change":"2024-05-08T06:05:51.416604+0000","last_active":"2024-05-08T06:08:01.047241+0000","last_peered":"2024-05-08T06:08:01.047241+0000","last_clean":"2024-05-08T06:08:01.047241+0000","last_became_active":"2024-05-08T06:05:51.416284+0000","last_became_peered":"2024-05-08T06:05:51.416284+0000","last_unstale":"2024-05-08T06:08:01.047241+0000","last_undegraded":"2024-05-08T06:08:01.047241+0000","last_fullsized":"2024-05-08T06:08:01.047241+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:05:26.014323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.059697+0000","last_change":"2024-05-08T06:05:51.423915+0000","last_active":"2024-05-08T06:08:01.059697+0000","last_peered":"2024-05-08T06:08:01.059697+0000","last_clean":"2024-05-08T06:08:01.059697+0000","last_became_active":"2024-05-08T06:05:51.423534+0000","last_became_peered":"2024-05-08T06:05:51.423534+0000","last_unstale":"2024-05-08T06:08:01.059697+0000","last_undegraded":"2024-05-08T06:08:01.059697+0000","last_fullsized":"2024-05-08T06:08:01.059697+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:06:46.197071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.041181+0000","last_change":"2024-05-08T06:05:51.411102+0000","last_active":"2024-05-08T06:08:01.041181+0000","last_peered":"2024-05-08T06:08:01.041181+0000","last_clean":"2024-05-08T06:08:01.041181+0000","last_became_active":"2024-05-08T06:05:51.410791+0000","last_became_peered":"2024-05-08T06:05:51.410791+0000","last_unstale":"2024-05-08T06:08:01.041181+0000","last_undegraded":"2024-05-08T06:08:01.041181+0000","last_fullsized":"2024-05-08T06:08:01.041181+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:31:47.861663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.041524+0000","last_change":"2024-05-08T06:05:51.417756+0000","last_active":"2024-05-08T06:08:01.041524+0000","last_peered":"2024-05-08T06:08:01.041524+0000","last_clean":"2024-05-08T06:08:01.041524+0000","last_became_active":"2024-05-08T06:05:51.417373+0000","last_became_peered":"2024-05-08T06:05:51.417373+0000","last_unstale":"2024-05-08T06:08:01.041524+0000","last_undegraded":"2024-05-08T06:08:01.041524+0000","last_fullsized":"2024-05-08T06:08:01.041524+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:07:48.330401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.063489+0000","last_change":"2024-05-08T06:05:51.413883+0000","last_active":"2024-05-08T06:08:01.063489+0000","last_peered":"2024-05-08T06:08:01.063489+0000","last_clean":"2024-05-08T06:08:01.063489+0000","last_became_active":"2024-05-08T06:05:51.413565+0000","last_became_peered":"2024-05-08T06:05:51.413565+0000","last_unstale":"2024-05-08T06:08:01.063489+0000","last_undegraded":"2024-05-08T06:08:01.063489+0000","last_fullsized":"2024-05-08T06:08:01.063489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:06:38.186225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.053054+0000","last_change":"2024-05-08T06:05:51.416300+0000","last_active":"2024-05-08T06:08:01.053054+0000","last_peered":"2024-05-08T06:08:01.053054+0000","last_clean":"2024-05-08T06:08:01.053054+0000","last_became_active":"2024-05-08T06:05:51.415970+0000","last_became_peered":"2024-05-08T06:05:51.415970+0000","last_unstale":"2024-05-08T06:08:01.053054+0000","last_undegraded":"2024-05-08T06:08:01.053054+0000","last_fullsized":"2024-05-08T06:08:01.053054+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:44:51.853339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.039789+0000","last_change":"2024-05-08T06:05:51.410396+0000","last_active":"2024-05-08T06:08:01.039789+0000","last_peered":"2024-05-08T06:08:01.039789+0000","last_clean":"2024-05-08T06:08:01.039789+0000","last_became_active":"2024-05-08T06:05:51.410067+0000","last_became_peered":"2024-05-08T06:05:51.410067+0000","last_unstale":"2024-05-08T06:08:01.039789+0000","last_undegraded":"2024-05-08T06:08:01.039789+0000","last_fullsized":"2024-05-08T06:08:01.039789+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:32:00.217217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.052539+0000","last_change":"2024-05-08T06:05:51.409276+0000","last_active":"2024-05-08T06:08:01.052539+0000","last_peered":"2024-05-08T06:08:01.052539+0000","last_clean":"2024-05-08T06:08:01.052539+0000","last_became_active":"2024-05-08T06:05:51.408768+0000","last_became_peered":"2024-05-08T06:05:51.408768+0000","last_unstale":"2024-05-08T06:08:01.052539+0000","last_undegraded":"2024-05-08T06:08:01.052539+0000","last_fullsized":"2024-05-08T06:08:01.052539+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:22:57.483944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.048049+0000","last_change":"2024-05-08T06:05:51.419083+0000","last_active":"2024-05-08T06:08:01.048049+0000","last_peered":"2024-05-08T06:08:01.048049+0000","last_clean":"2024-05-08T06:08:01.048049+0000","last_became_active":"2024-05-08T06:05:51.418685+0000","last_became_peered":"2024-05-08T06:05:51.418685+0000","last_unstale":"2024-05-08T06:08:01.048049+0000","last_undegraded":"2024-05-08T06:08:01.048049+0000","last_fullsized":"2024-05-08T06:08:01.048049+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:25:49.256242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.057514+0000","last_change":"2024-05-08T06:05:51.420399+0000","last_active":"2024-05-08T06:08:01.057514+0000","last_peered":"2024-05-08T06:08:01.057514+0000","last_clean":"2024-05-08T06:08:01.057514+0000","last_became_active":"2024-05-08T06:05:51.420076+0000","last_became_peered":"2024-05-08T06:05:51.420076+0000","last_unstale":"2024-05-08T06:08:01.057514+0000","last_undegraded":"2024-05-08T06:08:01.057514+0000","last_fullsized":"2024-05-08T06:08:01.057514+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:51:38.579280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.059465+0000","last_change":"2024-05-08T06:05:51.431518+0000","last_active":"2024-05-08T06:08:01.059465+0000","last_peered":"2024-05-08T06:08:01.059465+0000","last_clean":"2024-05-08T06:08:01.059465+0000","last_became_active":"2024-05-08T06:05:51.431029+0000","last_became_peered":"2024-05-08T06:05:51.431029+0000","last_unstale":"2024-05-08T06:08:01.059465+0000","last_undegraded":"2024-05-08T06:08:01.059465+0000","last_fullsized":"2024-05-08T06:08:01.059465+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:11:09.596221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.061333+0000","last_change":"2024-05-08T06:06:32.498002+0000","last_active":"2024-05-08T06:08:01.061333+0000","last_peered":"2024-05-08T06:08:01.061333+0000","last_clean":"2024-05-08T06:08:01.061333+0000","last_became_active":"2024-05-08T06:06:32.497367+0000","last_became_peered":"2024-05-08T06:06:32.497367+0000","last_unstale":"2024-05-08T06:08:01.061333+0000","last_undegraded":"2024-05-08T06:08:01.061333+0000","last_fullsized":"2024-05-08T06:08:01.061333+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:32:36.461777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.064064+0000","last_change":"2024-05-08T06:05:51.423857+0000","last_active":"2024-05-08T06:08:01.064064+0000","last_peered":"2024-05-08T06:08:01.064064+0000","last_clean":"2024-05-08T06:08:01.064064+0000","last_became_active":"2024-05-08T06:05:51.423487+0000","last_became_peered":"2024-05-08T06:05:51.423487+0000","last_unstale":"2024-05-08T06:08:01.064064+0000","last_undegraded":"2024-05-08T06:08:01.064064+0000","last_fullsized":"2024-05-08T06:08:01.064064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:17:33.269354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.064245+0000","last_change":"2024-05-08T06:05:51.431795+0000","last_active":"2024-05-08T06:08:01.064245+0000","last_peered":"2024-05-08T06:08:01.064245+0000","last_clean":"2024-05-08T06:08:01.064245+0000","last_became_active":"2024-05-08T06:05:51.431432+0000","last_became_peered":"2024-05-08T06:05:51.431432+0000","last_unstale":"2024-05-08T06:08:01.064245+0000","last_undegraded":"2024-05-08T06:08:01.064245+0000","last_fullsized":"2024-05-08T06:08:01.064245+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:11:16.561478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.048666+0000","last_change":"2024-05-08T06:05:51.414805+0000","last_active":"2024-05-08T06:08:01.048666+0000","last_peered":"2024-05-08T06:08:01.048666+0000","last_clean":"2024-05-08T06:08:01.048666+0000","last_became_active":"2024-05-08T06:05:51.414482+0000","last_became_peered":"2024-05-08T06:05:51.414482+0000","last_unstale":"2024-05-08T06:08:01.048666+0000","last_undegraded":"2024-05-08T06:08:01.048666+0000","last_fullsized":"2024-05-08T06:08:01.048666+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:08:13.444591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.064433+0000","last_change":"2024-05-08T06:05:51.418516+0000","last_active":"2024-05-08T06:08:01.064433+0000","last_peered":"2024-05-08T06:08:01.064433+0000","last_clean":"2024-05-08T06:08:01.064433+0000","last_became_active":"2024-05-08T06:05:51.418182+0000","last_became_peered":"2024-05-08T06:05:51.418182+0000","last_unstale":"2024-05-08T06:08:01.064433+0000","last_undegraded":"2024-05-08T06:08:01.064433+0000","last_fullsized":"2024-05-08T06:08:01.064433+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:32:33.338311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:07:51.948755+0000","last_change":"2024-05-08T06:05:46.350509+0000","last_active":"2024-05-08T06:07:51.948755+0000","last_peered":"2024-05-08T06:07:51.948755+0000","last_clean":"2024-05-08T06:07:51.948755+0000","last_became_active":"2024-05-08T06:05:46.349986+0000","last_became_peered":"2024-05-08T06:05:46.349986+0000","last_unstale":"2024-05-08T06:07:51.948755+0000","last_undegraded":"2024-05-08T06:07:51.948755+0000","last_fullsized":"2024-05-08T06:07:51.948755+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:53:00.928971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.061073+0000","last_change":"2024-05-08T06:05:51.413621+0000","last_active":"2024-05-08T06:08:01.061073+0000","last_peered":"2024-05-08T06:08:01.061073+0000","last_clean":"2024-05-08T06:08:01.061073+0000","last_became_active":"2024-05-08T06:05:51.413188+0000","last_became_peered":"2024-05-08T06:05:51.413188+0000","last_unstale":"2024-05-08T06:08:01.061073+0000","last_undegraded":"2024-05-08T06:08:01.061073+0000","last_fullsized":"2024-05-08T06:08:01.061073+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:13:21.413696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:07:50.005420+0000","last_change":"2024-05-08T06:05:46.552444+0000","last_active":"2024-05-08T06:07:50.005420+0000","last_peered":"2024-05-08T06:07:50.005420+0000","last_clean":"2024-05-08T06:07:50.005420+0000","last_became_active":"2024-05-08T06:05:46.551310+0000","last_became_peered":"2024-05-08T06:05:46.551310+0000","last_unstale":"2024-05-08T06:07:50.005420+0000","last_undegraded":"2024-05-08T06:07:50.005420+0000","last_fullsized":"2024-05-08T06:07:50.005420+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:58:17.632832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.062434+0000","last_change":"2024-05-08T06:05:51.425649+0000","last_active":"2024-05-08T06:08:01.062434+0000","last_peered":"2024-05-08T06:08:01.062434+0000","last_clean":"2024-05-08T06:08:01.062434+0000","last_became_active":"2024-05-08T06:05:51.425288+0000","last_became_peered":"2024-05-08T06:05:51.425288+0000","last_unstale":"2024-05-08T06:08:01.062434+0000","last_undegraded":"2024-05-08T06:08:01.062434+0000","last_fullsized":"2024-05-08T06:08:01.062434+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:42:40.580641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480655+0000","last_change":"2024-05-08T06:05:46.350300+0000","last_active":"2024-05-08T06:06:32.480655+0000","last_peered":"2024-05-08T06:06:32.480655+0000","last_clean":"2024-05-08T06:06:32.480655+0000","last_became_active":"2024-05-08T06:05:46.349918+0000","last_became_peered":"2024-05-08T06:05:46.349918+0000","last_unstale":"2024-05-08T06:06:32.480655+0000","last_undegraded":"2024-05-08T06:06:32.480655+0000","last_fullsized":"2024-05-08T06:06:32.480655+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:32:41.952772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.057546+0000","last_change":"2024-05-08T06:05:51.430130+0000","last_active":"2024-05-08T06:08:01.057546+0000","last_peered":"2024-05-08T06:08:01.057546+0000","last_clean":"2024-05-08T06:08:01.057546+0000","last_became_active":"2024-05-08T06:05:51.429794+0000","last_became_peered":"2024-05-08T06:05:51.429794+0000","last_unstale":"2024-05-08T06:08:01.057546+0000","last_undegraded":"2024-05-08T06:08:01.057546+0000","last_fullsized":"2024-05-08T06:08:01.057546+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:21:55.773309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480539+0000","last_change":"2024-05-08T06:05:46.351664+0000","last_active":"2024-05-08T06:06:32.480539+0000","last_peered":"2024-05-08T06:06:32.480539+0000","last_clean":"2024-05-08T06:06:32.480539+0000","last_became_active":"2024-05-08T06:05:46.351305+0000","last_became_peered":"2024-05-08T06:05:46.351305+0000","last_unstale":"2024-05-08T06:06:32.480539+0000","last_undegraded":"2024-05-08T06:06:32.480539+0000","last_fullsized":"2024-05-08T06:06:32.480539+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:24:36.164315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.043418+0000","last_change":"2024-05-08T06:05:51.414793+0000","last_active":"2024-05-08T06:08:01.043418+0000","last_peered":"2024-05-08T06:08:01.043418+0000","last_clean":"2024-05-08T06:08:01.043418+0000","last_became_active":"2024-05-08T06:05:51.414317+0000","last_became_peered":"2024-05-08T06:05:51.414317+0000","last_unstale":"2024-05-08T06:08:01.043418+0000","last_undegraded":"2024-05-08T06:08:01.043418+0000","last_fullsized":"2024-05-08T06:08:01.043418+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:57:56.152836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":274,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:10:13.723637+0000","last_change":"2024-05-08T06:05:46.352683+0000","last_active":"2024-05-08T06:10:13.723637+0000","last_peered":"2024-05-08T06:10:13.723637+0000","last_clean":"2024-05-08T06:10:13.723637+0000","last_became_active":"2024-05-08T06:05:46.352278+0000","last_became_peered":"2024-05-08T06:05:46.352278+0000","last_unstale":"2024-05-08T06:10:13.723637+0000","last_undegraded":"2024-05-08T06:10:13.723637+0000","last_fullsized":"2024-05-08T06:10:13.723637+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:39:22.655516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1323,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":240,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.037458+0000","last_change":"2024-05-08T06:05:51.415822+0000","last_active":"2024-05-08T06:08:01.037458+0000","last_peered":"2024-05-08T06:08:01.037458+0000","last_clean":"2024-05-08T06:08:01.037458+0000","last_became_active":"2024-05-08T06:05:51.415449+0000","last_became_peered":"2024-05-08T06:05:51.415449+0000","last_unstale":"2024-05-08T06:08:01.037458+0000","last_undegraded":"2024-05-08T06:08:01.037458+0000","last_fullsized":"2024-05-08T06:08:01.037458+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:44:49.127335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.478073+0000","last_change":"2024-05-08T06:05:46.361288+0000","last_active":"2024-05-08T06:06:32.478073+0000","last_peered":"2024-05-08T06:06:32.478073+0000","last_clean":"2024-05-08T06:06:32.478073+0000","last_became_active":"2024-05-08T06:05:46.360761+0000","last_became_peered":"2024-05-08T06:05:46.360761+0000","last_unstale":"2024-05-08T06:06:32.478073+0000","last_undegraded":"2024-05-08T06:06:32.478073+0000","last_fullsized":"2024-05-08T06:06:32.478073+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:14:59.858935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.063490+0000","last_change":"2024-05-08T06:05:51.418547+0000","last_active":"2024-05-08T06:08:01.063490+0000","last_peered":"2024-05-08T06:08:01.063490+0000","last_clean":"2024-05-08T06:08:01.063490+0000","last_became_active":"2024-05-08T06:05:51.417588+0000","last_became_peered":"2024-05-08T06:05:51.417588+0000","last_unstale":"2024-05-08T06:08:01.063490+0000","last_undegraded":"2024-05-08T06:08:01.063490+0000","last_fullsized":"2024-05-08T06:08:01.063490+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:16:41.451351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.492080+0000","last_change":"2024-05-08T06:05:46.362077+0000","last_active":"2024-05-08T06:06:32.492080+0000","last_peered":"2024-05-08T06:06:32.492080+0000","last_clean":"2024-05-08T06:06:32.492080+0000","last_became_active":"2024-05-08T06:05:46.360823+0000","last_became_peered":"2024-05-08T06:05:46.360823+0000","last_unstale":"2024-05-08T06:06:32.492080+0000","last_undegraded":"2024-05-08T06:06:32.492080+0000","last_fullsized":"2024-05-08T06:06:32.492080+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:14.169822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.063350+0000","last_change":"2024-05-08T06:05:51.423687+0000","last_active":"2024-05-08T06:08:01.063350+0000","last_peered":"2024-05-08T06:08:01.063350+0000","last_clean":"2024-05-08T06:08:01.063350+0000","last_became_active":"2024-05-08T06:05:51.417656+0000","last_became_peered":"2024-05-08T06:05:51.417656+0000","last_unstale":"2024-05-08T06:08:01.063350+0000","last_undegraded":"2024-05-08T06:08:01.063350+0000","last_fullsized":"2024-05-08T06:08:01.063350+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:59:31.319915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.595803+0000","last_change":"2024-05-08T06:05:46.351892+0000","last_active":"2024-05-08T06:06:32.595803+0000","last_peered":"2024-05-08T06:06:32.595803+0000","last_clean":"2024-05-08T06:06:32.595803+0000","last_became_active":"2024-05-08T06:05:46.351523+0000","last_became_peered":"2024-05-08T06:05:46.351523+0000","last_unstale":"2024-05-08T06:06:32.595803+0000","last_undegraded":"2024-05-08T06:06:32.595803+0000","last_fullsized":"2024-05-08T06:06:32.595803+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:11:23.794313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.063933+0000","last_change":"2024-05-08T06:05:51.407681+0000","last_active":"2024-05-08T06:08:01.063933+0000","last_peered":"2024-05-08T06:08:01.063933+0000","last_clean":"2024-05-08T06:08:01.063933+0000","last_became_active":"2024-05-08T06:05:51.407243+0000","last_became_peered":"2024-05-08T06:05:51.407243+0000","last_unstale":"2024-05-08T06:08:01.063933+0000","last_undegraded":"2024-05-08T06:08:01.063933+0000","last_fullsized":"2024-05-08T06:08:01.063933+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:06:30.490268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":271,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:10:16.728337+0000","last_change":"2024-05-08T06:05:46.356675+0000","last_active":"2024-05-08T06:10:16.728337+0000","last_peered":"2024-05-08T06:10:16.728337+0000","last_clean":"2024-05-08T06:10:16.728337+0000","last_became_active":"2024-05-08T06:05:46.356360+0000","last_became_peered":"2024-05-08T06:05:46.356360+0000","last_unstale":"2024-05-08T06:10:16.728337+0000","last_undegraded":"2024-05-08T06:10:16.728337+0000","last_fullsized":"2024-05-08T06:10:16.728337+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:46:48.131551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":240,"num_read_kb":240,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.047752+0000","last_change":"2024-05-08T06:05:51.421325+0000","last_active":"2024-05-08T06:08:01.047752+0000","last_peered":"2024-05-08T06:08:01.047752+0000","last_clean":"2024-05-08T06:08:01.047752+0000","last_became_active":"2024-05-08T06:05:51.420905+0000","last_became_peered":"2024-05-08T06:05:51.420905+0000","last_unstale":"2024-05-08T06:08:01.047752+0000","last_undegraded":"2024-05-08T06:08:01.047752+0000","last_fullsized":"2024-05-08T06:08:01.047752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:37:27.324920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.482723+0000","last_change":"2024-05-08T06:05:46.351958+0000","last_active":"2024-05-08T06:06:32.482723+0000","last_peered":"2024-05-08T06:06:32.482723+0000","last_clean":"2024-05-08T06:06:32.482723+0000","last_became_active":"2024-05-08T06:05:46.351561+0000","last_became_peered":"2024-05-08T06:05:46.351561+0000","last_unstale":"2024-05-08T06:06:32.482723+0000","last_undegraded":"2024-05-08T06:06:32.482723+0000","last_fullsized":"2024-05-08T06:06:32.482723+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:11:27.727029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.061468+0000","last_change":"2024-05-08T06:05:51.431809+0000","last_active":"2024-05-08T06:08:01.061468+0000","last_peered":"2024-05-08T06:08:01.061468+0000","last_clean":"2024-05-08T06:08:01.061468+0000","last_became_active":"2024-05-08T06:05:51.431416+0000","last_became_peered":"2024-05-08T06:05:51.431416+0000","last_unstale":"2024-05-08T06:08:01.061468+0000","last_undegraded":"2024-05-08T06:08:01.061468+0000","last_fullsized":"2024-05-08T06:08:01.061468+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:48:16.704134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.596352+0000","last_change":"2024-05-08T06:05:46.550763+0000","last_active":"2024-05-08T06:06:32.596352+0000","last_peered":"2024-05-08T06:06:32.596352+0000","last_clean":"2024-05-08T06:06:32.596352+0000","last_became_active":"2024-05-08T06:05:46.550297+0000","last_became_peered":"2024-05-08T06:05:46.550297+0000","last_unstale":"2024-05-08T06:06:32.596352+0000","last_undegraded":"2024-05-08T06:06:32.596352+0000","last_fullsized":"2024-05-08T06:06:32.596352+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:32:19.907220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.051960+0000","last_change":"2024-05-08T06:05:51.414738+0000","last_active":"2024-05-08T06:08:01.051960+0000","last_peered":"2024-05-08T06:08:01.051960+0000","last_clean":"2024-05-08T06:08:01.051960+0000","last_became_active":"2024-05-08T06:05:51.414369+0000","last_became_peered":"2024-05-08T06:05:51.414369+0000","last_unstale":"2024-05-08T06:08:01.051960+0000","last_undegraded":"2024-05-08T06:08:01.051960+0000","last_fullsized":"2024-05-08T06:08:01.051960+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:57:44.215343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'39","reported_seq":560,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:10:17.729921+0000","last_change":"2024-05-08T06:05:46.349825+0000","last_active":"2024-05-08T06:10:17.729921+0000","last_peered":"2024-05-08T06:10:17.729921+0000","last_clean":"2024-05-08T06:10:17.729921+0000","last_became_active":"2024-05-08T06:05:46.349425+0000","last_became_peered":"2024-05-08T06:05:46.349425+0000","last_unstale":"2024-05-08T06:10:17.729921+0000","last_undegraded":"2024-05-08T06:10:17.729921+0000","last_fullsized":"2024-05-08T06:10:17.729921+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:29:39.123817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73053,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":494,"num_read_kb":319,"num_write":40,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.022668+0000","last_change":"2024-05-08T06:05:51.411341+0000","last_active":"2024-05-08T06:08:01.022668+0000","last_peered":"2024-05-08T06:08:01.022668+0000","last_clean":"2024-05-08T06:08:01.022668+0000","last_became_active":"2024-05-08T06:05:51.411002+0000","last_became_peered":"2024-05-08T06:05:51.411002+0000","last_unstale":"2024-05-08T06:08:01.022668+0000","last_undegraded":"2024-05-08T06:08:01.022668+0000","last_fullsized":"2024-05-08T06:08:01.022668+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:27:07.013809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:34.244232+0000","last_change":"2024-05-08T06:06:32.508904+0000","last_active":"2024-05-08T06:06:34.244232+0000","last_peered":"2024-05-08T06:06:34.244232+0000","last_clean":"2024-05-08T06:06:34.244232+0000","last_became_active":"2024-05-08T06:06:32.496001+0000","last_became_peered":"2024-05-08T06:06:32.496001+0000","last_unstale":"2024-05-08T06:06:34.244232+0000","last_undegraded":"2024-05-08T06:06:34.244232+0000","last_fullsized":"2024-05-08T06:06:34.244232+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:34:32.417105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.065212+0000","last_change":"2024-05-08T06:05:51.418395+0000","last_active":"2024-05-08T06:08:01.065212+0000","last_peered":"2024-05-08T06:08:01.065212+0000","last_clean":"2024-05-08T06:08:01.065212+0000","last_became_active":"2024-05-08T06:05:51.418035+0000","last_became_peered":"2024-05-08T06:05:51.418035+0000","last_unstale":"2024-05-08T06:08:01.065212+0000","last_undegraded":"2024-05-08T06:08:01.065212+0000","last_fullsized":"2024-05-08T06:08:01.065212+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:58:16.529144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.595996+0000","last_change":"2024-05-08T06:05:46.363088+0000","last_active":"2024-05-08T06:06:32.595996+0000","last_peered":"2024-05-08T06:06:32.595996+0000","last_clean":"2024-05-08T06:06:32.595996+0000","last_became_active":"2024-05-08T06:05:46.362488+0000","last_became_peered":"2024-05-08T06:05:46.362488+0000","last_unstale":"2024-05-08T06:06:32.595996+0000","last_undegraded":"2024-05-08T06:06:32.595996+0000","last_fullsized":"2024-05-08T06:06:32.595996+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:28:10.039647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.059203+0000","last_change":"2024-05-08T06:05:51.415567+0000","last_active":"2024-05-08T06:08:01.059203+0000","last_peered":"2024-05-08T06:08:01.059203+0000","last_clean":"2024-05-08T06:08:01.059203+0000","last_became_active":"2024-05-08T06:05:51.415148+0000","last_became_peered":"2024-05-08T06:05:51.415148+0000","last_unstale":"2024-05-08T06:08:01.059203+0000","last_undegraded":"2024-05-08T06:08:01.059203+0000","last_fullsized":"2024-05-08T06:08:01.059203+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:51:48.871149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'6","reported_seq":275,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:10:13.925119+0000","last_change":"2024-05-08T06:05:46.348769+0000","last_active":"2024-05-08T06:10:13.925119+0000","last_peered":"2024-05-08T06:10:13.925119+0000","last_clean":"2024-05-08T06:10:13.925119+0000","last_became_active":"2024-05-08T06:05:46.348072+0000","last_became_peered":"2024-05-08T06:05:46.348072+0000","last_unstale":"2024-05-08T06:10:13.925119+0000","last_undegraded":"2024-05-08T06:10:13.925119+0000","last_fullsized":"2024-05-08T06:10:13.925119+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:45.292436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":243,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.049147+0000","last_change":"2024-05-08T06:05:51.414623+0000","last_active":"2024-05-08T06:08:01.049147+0000","last_peered":"2024-05-08T06:08:01.049147+0000","last_clean":"2024-05-08T06:08:01.049147+0000","last_became_active":"2024-05-08T06:05:51.414258+0000","last_became_peered":"2024-05-08T06:05:51.414258+0000","last_unstale":"2024-05-08T06:08:01.049147+0000","last_undegraded":"2024-05-08T06:08:01.049147+0000","last_fullsized":"2024-05-08T06:08:01.049147+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:38:48.297900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":43,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.480030+0000","last_change":"2024-05-08T06:05:46.546936+0000","last_active":"2024-05-08T06:06:32.480030+0000","last_peered":"2024-05-08T06:06:32.480030+0000","last_clean":"2024-05-08T06:06:32.480030+0000","last_became_active":"2024-05-08T06:05:46.546574+0000","last_became_peered":"2024-05-08T06:05:46.546574+0000","last_unstale":"2024-05-08T06:06:32.480030+0000","last_undegraded":"2024-05-08T06:06:32.480030+0000","last_fullsized":"2024-05-08T06:06:32.480030+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:16:29.720304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.029792+0000","last_change":"2024-05-08T06:05:51.418012+0000","last_active":"2024-05-08T06:08:01.029792+0000","last_peered":"2024-05-08T06:08:01.029792+0000","last_clean":"2024-05-08T06:08:01.029792+0000","last_became_active":"2024-05-08T06:05:51.417646+0000","last_became_peered":"2024-05-08T06:05:51.417646+0000","last_unstale":"2024-05-08T06:08:01.029792+0000","last_undegraded":"2024-05-08T06:08:01.029792+0000","last_fullsized":"2024-05-08T06:08:01.029792+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:43:02.733967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:00.689473+0000","last_change":"2024-05-08T06:05:46.347748+0000","last_active":"2024-05-08T06:08:00.689473+0000","last_peered":"2024-05-08T06:08:00.689473+0000","last_clean":"2024-05-08T06:08:00.689473+0000","last_became_active":"2024-05-08T06:05:46.347367+0000","last_became_peered":"2024-05-08T06:05:46.347367+0000","last_unstale":"2024-05-08T06:08:00.689473+0000","last_undegraded":"2024-05-08T06:08:00.689473+0000","last_fullsized":"2024-05-08T06:08:00.689473+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:51:50.973855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.038216+0000","last_change":"2024-05-08T06:06:32.597018+0000","last_active":"2024-05-08T06:08:01.038216+0000","last_peered":"2024-05-08T06:08:01.038216+0000","last_clean":"2024-05-08T06:08:01.038216+0000","last_became_active":"2024-05-08T06:06:32.596677+0000","last_became_peered":"2024-05-08T06:06:32.596677+0000","last_unstale":"2024-05-08T06:08:01.038216+0000","last_undegraded":"2024-05-08T06:08:01.038216+0000","last_fullsized":"2024-05-08T06:08:01.038216+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:02:46.223949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:07:51.948701+0000","last_change":"2024-05-08T06:05:46.554443+0000","last_active":"2024-05-08T06:07:51.948701+0000","last_peered":"2024-05-08T06:07:51.948701+0000","last_clean":"2024-05-08T06:07:51.948701+0000","last_became_active":"2024-05-08T06:05:46.553226+0000","last_became_peered":"2024-05-08T06:05:46.553226+0000","last_unstale":"2024-05-08T06:07:51.948701+0000","last_undegraded":"2024-05-08T06:07:51.948701+0000","last_fullsized":"2024-05-08T06:07:51.948701+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:56:39.784652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.060932+0000","last_change":"2024-05-08T06:05:51.413121+0000","last_active":"2024-05-08T06:08:01.060932+0000","last_peered":"2024-05-08T06:08:01.060932+0000","last_clean":"2024-05-08T06:08:01.060932+0000","last_became_active":"2024-05-08T06:05:51.412767+0000","last_became_peered":"2024-05-08T06:05:51.412767+0000","last_unstale":"2024-05-08T06:08:01.060932+0000","last_undegraded":"2024-05-08T06:08:01.060932+0000","last_fullsized":"2024-05-08T06:08:01.060932+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:59:13.416981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:07:51.431460+0000","last_change":"2024-05-08T06:05:46.361349+0000","last_active":"2024-05-08T06:07:51.431460+0000","last_peered":"2024-05-08T06:07:51.431460+0000","last_clean":"2024-05-08T06:07:51.431460+0000","last_became_active":"2024-05-08T06:05:46.360654+0000","last_became_peered":"2024-05-08T06:05:46.360654+0000","last_unstale":"2024-05-08T06:07:51.431460+0000","last_undegraded":"2024-05-08T06:07:51.431460+0000","last_fullsized":"2024-05-08T06:07:51.431460+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:19:22.144874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.062214+0000","last_change":"2024-05-08T06:05:51.423836+0000","last_active":"2024-05-08T06:08:01.062214+0000","last_peered":"2024-05-08T06:08:01.062214+0000","last_clean":"2024-05-08T06:08:01.062214+0000","last_became_active":"2024-05-08T06:05:51.423424+0000","last_became_peered":"2024-05-08T06:05:51.423424+0000","last_unstale":"2024-05-08T06:08:01.062214+0000","last_undegraded":"2024-05-08T06:08:01.062214+0000","last_fullsized":"2024-05-08T06:08:01.062214+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:56:14.779765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.048549+0000","last_change":"2024-05-08T06:06:32.598978+0000","last_active":"2024-05-08T06:08:01.048549+0000","last_peered":"2024-05-08T06:08:01.048549+0000","last_clean":"2024-05-08T06:08:01.048549+0000","last_became_active":"2024-05-08T06:06:32.598158+0000","last_became_peered":"2024-05-08T06:06:32.598158+0000","last_unstale":"2024-05-08T06:08:01.048549+0000","last_undegraded":"2024-05-08T06:08:01.048549+0000","last_fullsized":"2024-05-08T06:08:01.048549+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:51:09.270066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":271,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:10:13.923393+0000","last_change":"2024-05-08T06:05:46.547450+0000","last_active":"2024-05-08T06:10:13.923393+0000","last_peered":"2024-05-08T06:10:13.923393+0000","last_clean":"2024-05-08T06:10:13.923393+0000","last_became_active":"2024-05-08T06:05:46.546444+0000","last_became_peered":"2024-05-08T06:05:46.546444+0000","last_unstale":"2024-05-08T06:10:13.923393+0000","last_undegraded":"2024-05-08T06:10:13.923393+0000","last_fullsized":"2024-05-08T06:10:13.923393+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:17:14.124224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":237,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.065294+0000","last_change":"2024-05-08T06:05:51.423191+0000","last_active":"2024-05-08T06:08:01.065294+0000","last_peered":"2024-05-08T06:08:01.065294+0000","last_clean":"2024-05-08T06:08:01.065294+0000","last_became_active":"2024-05-08T06:05:51.420912+0000","last_became_peered":"2024-05-08T06:05:51.420912+0000","last_unstale":"2024-05-08T06:08:01.065294+0000","last_undegraded":"2024-05-08T06:08:01.065294+0000","last_fullsized":"2024-05-08T06:08:01.065294+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:27:57.678527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:07:50.743006+0000","last_change":"2024-05-08T06:05:46.348650+0000","last_active":"2024-05-08T06:07:50.743006+0000","last_peered":"2024-05-08T06:07:50.743006+0000","last_clean":"2024-05-08T06:07:50.743006+0000","last_became_active":"2024-05-08T06:05:46.347266+0000","last_became_peered":"2024-05-08T06:05:46.347266+0000","last_unstale":"2024-05-08T06:07:50.743006+0000","last_undegraded":"2024-05-08T06:07:50.743006+0000","last_fullsized":"2024-05-08T06:07:50.743006+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:37:30.623580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":101,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:47.536769+0000","last_change":"2024-05-08T06:04:09.404663+0000","last_active":"2024-05-08T06:06:47.536769+0000","last_peered":"2024-05-08T06:06:47.536769+0000","last_clean":"2024-05-08T06:06:47.536769+0000","last_became_active":"2024-05-08T06:04:09.400115+0000","last_became_peered":"2024-05-08T06:04:09.400115+0000","last_unstale":"2024-05-08T06:06:47.536769+0000","last_undegraded":"2024-05-08T06:06:47.536769+0000","last_fullsized":"2024-05-08T06:06:47.536769+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:00:17.419712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:00:17.419712+0000","last_clean_scrub_stamp":"2024-05-08T06:00:17.419712+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:27:39.154694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.053609+0000","last_change":"2024-05-08T06:05:51.421826+0000","last_active":"2024-05-08T06:08:01.053609+0000","last_peered":"2024-05-08T06:08:01.053609+0000","last_clean":"2024-05-08T06:08:01.053609+0000","last_became_active":"2024-05-08T06:05:51.420706+0000","last_became_peered":"2024-05-08T06:05:51.420706+0000","last_unstale":"2024-05-08T06:08:01.053609+0000","last_undegraded":"2024-05-08T06:08:01.053609+0000","last_fullsized":"2024-05-08T06:08:01.053609+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:06:16.242332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:07:47.542941+0000","last_change":"2024-05-08T06:05:46.351365+0000","last_active":"2024-05-08T06:07:47.542941+0000","last_peered":"2024-05-08T06:07:47.542941+0000","last_clean":"2024-05-08T06:07:47.542941+0000","last_became_active":"2024-05-08T06:05:46.351022+0000","last_became_peered":"2024-05-08T06:05:46.351022+0000","last_unstale":"2024-05-08T06:07:47.542941+0000","last_undegraded":"2024-05-08T06:07:47.542941+0000","last_fullsized":"2024-05-08T06:07:47.542941+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:32:44.134994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:34.244260+0000","last_change":"2024-05-08T06:06:32.501263+0000","last_active":"2024-05-08T06:06:34.244260+0000","last_peered":"2024-05-08T06:06:34.244260+0000","last_clean":"2024-05-08T06:06:34.244260+0000","last_became_active":"2024-05-08T06:06:32.493795+0000","last_became_peered":"2024-05-08T06:06:32.493795+0000","last_unstale":"2024-05-08T06:06:34.244260+0000","last_undegraded":"2024-05-08T06:06:34.244260+0000","last_fullsized":"2024-05-08T06:06:34.244260+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:49:03.898162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.056756+0000","last_change":"2024-05-08T06:05:51.428382+0000","last_active":"2024-05-08T06:08:01.056756+0000","last_peered":"2024-05-08T06:08:01.056756+0000","last_clean":"2024-05-08T06:08:01.056756+0000","last_became_active":"2024-05-08T06:05:51.421443+0000","last_became_peered":"2024-05-08T06:05:51.421443+0000","last_unstale":"2024-05-08T06:08:01.056756+0000","last_undegraded":"2024-05-08T06:08:01.056756+0000","last_fullsized":"2024-05-08T06:08:01.056756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:26:10.215020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'15","reported_seq":43,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:10:11.475242+0000","last_change":"2024-05-08T06:05:46.364306+0000","last_active":"2024-05-08T06:10:11.475242+0000","last_peered":"2024-05-08T06:10:11.475242+0000","last_clean":"2024-05-08T06:10:11.475242+0000","last_became_active":"2024-05-08T06:05:46.363871+0000","last_became_peered":"2024-05-08T06:05:46.363871+0000","last_unstale":"2024-05-08T06:10:11.475242+0000","last_undegraded":"2024-05-08T06:10:11.475242+0000","last_fullsized":"2024-05-08T06:10:11.475242+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:38:34.531162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:05.007928+0000","last_change":"2024-05-08T06:05:51.415020+0000","last_active":"2024-05-08T06:08:05.007928+0000","last_peered":"2024-05-08T06:08:05.007928+0000","last_clean":"2024-05-08T06:08:05.007928+0000","last_became_active":"2024-05-08T06:05:51.413690+0000","last_became_peered":"2024-05-08T06:05:51.413690+0000","last_unstale":"2024-05-08T06:08:05.007928+0000","last_undegraded":"2024-05-08T06:08:05.007928+0000","last_fullsized":"2024-05-08T06:08:05.007928+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:55:38.121492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:07:55.743454+0000","last_change":"2024-05-08T06:05:46.359100+0000","last_active":"2024-05-08T06:07:55.743454+0000","last_peered":"2024-05-08T06:07:55.743454+0000","last_clean":"2024-05-08T06:07:55.743454+0000","last_became_active":"2024-05-08T06:05:46.358815+0000","last_became_peered":"2024-05-08T06:05:46.358815+0000","last_unstale":"2024-05-08T06:07:55.743454+0000","last_undegraded":"2024-05-08T06:07:55.743454+0000","last_fullsized":"2024-05-08T06:07:55.743454+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:22:58.532358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:02.879400+0000","last_change":"2024-05-08T06:05:51.429222+0000","last_active":"2024-05-08T06:08:02.879400+0000","last_peered":"2024-05-08T06:08:02.879400+0000","last_clean":"2024-05-08T06:08:02.879400+0000","last_became_active":"2024-05-08T06:05:51.428845+0000","last_became_peered":"2024-05-08T06:05:51.428845+0000","last_unstale":"2024-05-08T06:08:02.879400+0000","last_undegraded":"2024-05-08T06:08:02.879400+0000","last_fullsized":"2024-05-08T06:08:02.879400+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:36:41.435666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:07:57.210823+0000","last_change":"2024-05-08T06:05:46.547931+0000","last_active":"2024-05-08T06:07:57.210823+0000","last_peered":"2024-05-08T06:07:57.210823+0000","last_clean":"2024-05-08T06:07:57.210823+0000","last_became_active":"2024-05-08T06:05:46.547547+0000","last_became_peered":"2024-05-08T06:05:46.547547+0000","last_unstale":"2024-05-08T06:07:57.210823+0000","last_undegraded":"2024-05-08T06:07:57.210823+0000","last_fullsized":"2024-05-08T06:07:57.210823+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:27:03.502153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.020367+0000","last_change":"2024-05-08T06:05:51.413259+0000","last_active":"2024-05-08T06:08:01.020367+0000","last_peered":"2024-05-08T06:08:01.020367+0000","last_clean":"2024-05-08T06:08:01.020367+0000","last_became_active":"2024-05-08T06:05:51.412890+0000","last_became_peered":"2024-05-08T06:05:51.412890+0000","last_unstale":"2024-05-08T06:08:01.020367+0000","last_undegraded":"2024-05-08T06:08:01.020367+0000","last_fullsized":"2024-05-08T06:08:01.020367+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:04:56.690098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:00.007037+0000","last_change":"2024-05-08T06:05:46.365409+0000","last_active":"2024-05-08T06:08:00.007037+0000","last_peered":"2024-05-08T06:08:00.007037+0000","last_clean":"2024-05-08T06:08:00.007037+0000","last_became_active":"2024-05-08T06:05:46.361277+0000","last_became_peered":"2024-05-08T06:05:46.361277+0000","last_unstale":"2024-05-08T06:08:00.007037+0000","last_undegraded":"2024-05-08T06:08:00.007037+0000","last_fullsized":"2024-05-08T06:08:00.007037+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:06.927385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:04.253756+0000","last_change":"2024-05-08T06:05:51.423749+0000","last_active":"2024-05-08T06:08:04.253756+0000","last_peered":"2024-05-08T06:08:04.253756+0000","last_clean":"2024-05-08T06:08:04.253756+0000","last_became_active":"2024-05-08T06:05:51.423386+0000","last_became_peered":"2024-05-08T06:05:51.423386+0000","last_unstale":"2024-05-08T06:08:04.253756+0000","last_undegraded":"2024-05-08T06:08:04.253756+0000","last_fullsized":"2024-05-08T06:08:04.253756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:41:26.261683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'2","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:10:06.033061+0000","last_change":"2024-05-08T06:05:46.362497+0000","last_active":"2024-05-08T06:10:06.033061+0000","last_peered":"2024-05-08T06:10:06.033061+0000","last_clean":"2024-05-08T06:10:06.033061+0000","last_became_active":"2024-05-08T06:05:46.362110+0000","last_became_peered":"2024-05-08T06:05:46.362110+0000","last_unstale":"2024-05-08T06:10:06.033061+0000","last_undegraded":"2024-05-08T06:10:06.033061+0000","last_fullsized":"2024-05-08T06:10:06.033061+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:25:06.205380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.060826+0000","last_change":"2024-05-08T06:06:32.495078+0000","last_active":"2024-05-08T06:08:01.060826+0000","last_peered":"2024-05-08T06:08:01.060826+0000","last_clean":"2024-05-08T06:08:01.060826+0000","last_became_active":"2024-05-08T06:06:32.493855+0000","last_became_peered":"2024-05-08T06:06:32.493855+0000","last_unstale":"2024-05-08T06:08:01.060826+0000","last_undegraded":"2024-05-08T06:08:01.060826+0000","last_fullsized":"2024-05-08T06:08:01.060826+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:56:54.143080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:00.689448+0000","last_change":"2024-05-08T06:06:32.601826+0000","last_active":"2024-05-08T06:08:00.689448+0000","last_peered":"2024-05-08T06:08:00.689448+0000","last_clean":"2024-05-08T06:08:00.689448+0000","last_became_active":"2024-05-08T06:06:32.597823+0000","last_became_peered":"2024-05-08T06:06:32.597823+0000","last_unstale":"2024-05-08T06:08:00.689448+0000","last_undegraded":"2024-05-08T06:08:00.689448+0000","last_fullsized":"2024-05-08T06:08:00.689448+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:59:09.741096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:02.544515+0000","last_change":"2024-05-08T06:05:51.413744+0000","last_active":"2024-05-08T06:08:02.544515+0000","last_peered":"2024-05-08T06:08:02.544515+0000","last_clean":"2024-05-08T06:08:02.544515+0000","last_became_active":"2024-05-08T06:05:51.413346+0000","last_became_peered":"2024-05-08T06:05:51.413346+0000","last_unstale":"2024-05-08T06:08:02.544515+0000","last_undegraded":"2024-05-08T06:08:02.544515+0000","last_fullsized":"2024-05-08T06:08:02.544515+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:46:56.109755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.019506+0000","last_change":"2024-05-08T06:05:46.552604+0000","last_active":"2024-05-08T06:08:01.019506+0000","last_peered":"2024-05-08T06:08:01.019506+0000","last_clean":"2024-05-08T06:08:01.019506+0000","last_became_active":"2024-05-08T06:05:46.552248+0000","last_became_peered":"2024-05-08T06:05:46.552248+0000","last_unstale":"2024-05-08T06:08:01.019506+0000","last_undegraded":"2024-05-08T06:08:01.019506+0000","last_fullsized":"2024-05-08T06:08:01.019506+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:39:10.306692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.054186+0000","last_change":"2024-05-08T06:05:51.404988+0000","last_active":"2024-05-08T06:08:01.054186+0000","last_peered":"2024-05-08T06:08:01.054186+0000","last_clean":"2024-05-08T06:08:01.054186+0000","last_became_active":"2024-05-08T06:05:51.404625+0000","last_became_peered":"2024-05-08T06:05:51.404625+0000","last_unstale":"2024-05-08T06:08:01.054186+0000","last_undegraded":"2024-05-08T06:08:01.054186+0000","last_fullsized":"2024-05-08T06:08:01.054186+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:33:27.919929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:06:32.596178+0000","last_change":"2024-05-08T06:05:46.361466+0000","last_active":"2024-05-08T06:06:32.596178+0000","last_peered":"2024-05-08T06:06:32.596178+0000","last_clean":"2024-05-08T06:06:32.596178+0000","last_became_active":"2024-05-08T06:05:46.361107+0000","last_became_peered":"2024-05-08T06:05:46.361107+0000","last_unstale":"2024-05-08T06:06:32.596178+0000","last_undegraded":"2024-05-08T06:06:32.596178+0000","last_fullsized":"2024-05-08T06:06:32.596178+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:15:31.314226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.062827+0000","last_change":"2024-05-08T06:05:51.416437+0000","last_active":"2024-05-08T06:08:01.062827+0000","last_peered":"2024-05-08T06:08:01.062827+0000","last_clean":"2024-05-08T06:08:01.062827+0000","last_became_active":"2024-05-08T06:05:51.416070+0000","last_became_peered":"2024-05-08T06:05:51.416070+0000","last_unstale":"2024-05-08T06:08:01.062827+0000","last_undegraded":"2024-05-08T06:08:01.062827+0000","last_fullsized":"2024-05-08T06:08:01.062827+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:35:23.652373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:00.709889+0000","last_change":"2024-05-08T06:05:46.362475+0000","last_active":"2024-05-08T06:08:00.709889+0000","last_peered":"2024-05-08T06:08:00.709889+0000","last_clean":"2024-05-08T06:08:00.709889+0000","last_became_active":"2024-05-08T06:05:46.361852+0000","last_became_peered":"2024-05-08T06:05:46.361852+0000","last_unstale":"2024-05-08T06:08:00.709889+0000","last_undegraded":"2024-05-08T06:08:00.709889+0000","last_fullsized":"2024-05-08T06:08:00.709889+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:45.302256+0000","last_clean_scrub_stamp":"2024-05-08T06:05:45.302256+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:16:54.536330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.049399+0000","last_change":"2024-05-08T06:05:51.410718+0000","last_active":"2024-05-08T06:08:01.049399+0000","last_peered":"2024-05-08T06:08:01.049399+0000","last_clean":"2024-05-08T06:08:01.049399+0000","last_became_active":"2024-05-08T06:05:51.408902+0000","last_became_peered":"2024-05-08T06:05:51.408902+0000","last_unstale":"2024-05-08T06:08:01.049399+0000","last_undegraded":"2024-05-08T06:08:01.049399+0000","last_fullsized":"2024-05-08T06:08:01.049399+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:23:33.857677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.039067+0000","last_change":"2024-05-08T06:05:51.414362+0000","last_active":"2024-05-08T06:08:01.039067+0000","last_peered":"2024-05-08T06:08:01.039067+0000","last_clean":"2024-05-08T06:08:01.039067+0000","last_became_active":"2024-05-08T06:05:51.413998+0000","last_became_peered":"2024-05-08T06:05:51.413998+0000","last_unstale":"2024-05-08T06:08:01.039067+0000","last_undegraded":"2024-05-08T06:08:01.039067+0000","last_fullsized":"2024-05-08T06:08:01.039067+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:38:22.198219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.032578+0000","last_change":"2024-05-08T06:05:51.415508+0000","last_active":"2024-05-08T06:08:01.032578+0000","last_peered":"2024-05-08T06:08:01.032578+0000","last_clean":"2024-05-08T06:08:01.032578+0000","last_became_active":"2024-05-08T06:05:51.415137+0000","last_became_peered":"2024-05-08T06:05:51.415137+0000","last_unstale":"2024-05-08T06:08:01.032578+0000","last_undegraded":"2024-05-08T06:08:01.032578+0000","last_fullsized":"2024-05-08T06:08:01.032578+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:15.007996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.064401+0000","last_change":"2024-05-08T06:05:51.422093+0000","last_active":"2024-05-08T06:08:01.064401+0000","last_peered":"2024-05-08T06:08:01.064401+0000","last_clean":"2024-05-08T06:08:01.064401+0000","last_became_active":"2024-05-08T06:05:51.421733+0000","last_became_peered":"2024-05-08T06:05:51.421733+0000","last_unstale":"2024-05-08T06:08:01.064401+0000","last_undegraded":"2024-05-08T06:08:01.064401+0000","last_fullsized":"2024-05-08T06:08:01.064401+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:35:55.950229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.062534+0000","last_change":"2024-05-08T06:05:51.434705+0000","last_active":"2024-05-08T06:08:01.062534+0000","last_peered":"2024-05-08T06:08:01.062534+0000","last_clean":"2024-05-08T06:08:01.062534+0000","last_became_active":"2024-05-08T06:05:51.434324+0000","last_became_peered":"2024-05-08T06:05:51.434324+0000","last_unstale":"2024-05-08T06:08:01.062534+0000","last_undegraded":"2024-05-08T06:08:01.062534+0000","last_fullsized":"2024-05-08T06:08:01.062534+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:38:23.101247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.064059+0000","last_change":"2024-05-08T06:06:32.499837+0000","last_active":"2024-05-08T06:08:01.064059+0000","last_peered":"2024-05-08T06:08:01.064059+0000","last_clean":"2024-05-08T06:08:01.064059+0000","last_became_active":"2024-05-08T06:06:32.499026+0000","last_became_peered":"2024-05-08T06:06:32.499026+0000","last_unstale":"2024-05-08T06:08:01.064059+0000","last_undegraded":"2024-05-08T06:08:01.064059+0000","last_fullsized":"2024-05-08T06:08:01.064059+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:48:51.910384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:04.253787+0000","last_change":"2024-05-08T06:05:51.412370+0000","last_active":"2024-05-08T06:08:04.253787+0000","last_peered":"2024-05-08T06:08:04.253787+0000","last_clean":"2024-05-08T06:08:04.253787+0000","last_became_active":"2024-05-08T06:05:51.412057+0000","last_became_peered":"2024-05-08T06:05:51.412057+0000","last_unstale":"2024-05-08T06:08:04.253787+0000","last_undegraded":"2024-05-08T06:08:04.253787+0000","last_fullsized":"2024-05-08T06:08:04.253787+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:20:51.549926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.048132+0000","last_change":"2024-05-08T06:05:51.408393+0000","last_active":"2024-05-08T06:08:01.048132+0000","last_peered":"2024-05-08T06:08:01.048132+0000","last_clean":"2024-05-08T06:08:01.048132+0000","last_became_active":"2024-05-08T06:05:51.408023+0000","last_became_peered":"2024-05-08T06:05:51.408023+0000","last_unstale":"2024-05-08T06:08:01.048132+0000","last_undegraded":"2024-05-08T06:08:01.048132+0000","last_fullsized":"2024-05-08T06:08:01.048132+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:53.898929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:05.007852+0000","last_change":"2024-05-08T06:05:51.418055+0000","last_active":"2024-05-08T06:08:05.007852+0000","last_peered":"2024-05-08T06:08:05.007852+0000","last_clean":"2024-05-08T06:08:05.007852+0000","last_became_active":"2024-05-08T06:05:51.417067+0000","last_became_peered":"2024-05-08T06:05:51.417067+0000","last_unstale":"2024-05-08T06:08:05.007852+0000","last_undegraded":"2024-05-08T06:08:05.007852+0000","last_fullsized":"2024-05-08T06:08:05.007852+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:07:35.413672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:04.506147+0000","last_change":"2024-05-08T06:05:51.421025+0000","last_active":"2024-05-08T06:08:04.506147+0000","last_peered":"2024-05-08T06:08:04.506147+0000","last_clean":"2024-05-08T06:08:04.506147+0000","last_became_active":"2024-05-08T06:05:51.420794+0000","last_became_peered":"2024-05-08T06:05:51.420794+0000","last_unstale":"2024-05-08T06:08:04.506147+0000","last_undegraded":"2024-05-08T06:08:04.506147+0000","last_fullsized":"2024-05-08T06:08:04.506147+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:45:01.852405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.060497+0000","last_change":"2024-05-08T06:05:51.417709+0000","last_active":"2024-05-08T06:08:01.060497+0000","last_peered":"2024-05-08T06:08:01.060497+0000","last_clean":"2024-05-08T06:08:01.060497+0000","last_became_active":"2024-05-08T06:05:51.417337+0000","last_became_peered":"2024-05-08T06:05:51.417337+0000","last_unstale":"2024-05-08T06:08:01.060497+0000","last_undegraded":"2024-05-08T06:08:01.060497+0000","last_fullsized":"2024-05-08T06:08:01.060497+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:32:24.551768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.065666+0000","last_change":"2024-05-08T06:05:51.425318+0000","last_active":"2024-05-08T06:08:01.065666+0000","last_peered":"2024-05-08T06:08:01.065666+0000","last_clean":"2024-05-08T06:08:01.065666+0000","last_became_active":"2024-05-08T06:05:51.424931+0000","last_became_peered":"2024-05-08T06:05:51.424931+0000","last_unstale":"2024-05-08T06:08:01.065666+0000","last_undegraded":"2024-05-08T06:08:01.065666+0000","last_fullsized":"2024-05-08T06:08:01.065666+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:59:45.110201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.064188+0000","last_change":"2024-05-08T06:05:51.433529+0000","last_active":"2024-05-08T06:08:01.064188+0000","last_peered":"2024-05-08T06:08:01.064188+0000","last_clean":"2024-05-08T06:08:01.064188+0000","last_became_active":"2024-05-08T06:05:51.433173+0000","last_became_peered":"2024-05-08T06:05:51.433173+0000","last_unstale":"2024-05-08T06:08:01.064188+0000","last_undegraded":"2024-05-08T06:08:01.064188+0000","last_fullsized":"2024-05-08T06:08:01.064188+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:00:29.042188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T06:08:01.045237+0000","last_change":"2024-05-08T06:05:51.411418+0000","last_active":"2024-05-08T06:08:01.045237+0000","last_peered":"2024-05-08T06:08:01.045237+0000","last_clean":"2024-05-08T06:08:01.045237+0000","last_became_active":"2024-05-08T06:05:51.411084+0000","last_became_peered":"2024-05-08T06:05:51.411084+0000","last_unstale":"2024-05-08T06:08:01.045237+0000","last_undegraded":"2024-05-08T06:08:01.045237+0000","last_fullsized":"2024-05-08T06:08:01.045237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T06:05:50.356577+0000","last_clean_scrub_stamp":"2024-05-08T06:05:50.356577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:08:11.522282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282383,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":78287,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1466,"num_read_kb":818,"num_write":190,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":234861,"data_compressed":5508,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645383,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":888832,"data_stored":431859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841613,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":1097260,"data_compressed":37744,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005139,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":897024,"data_stored":432039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201369,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":892928,"data_stored":431937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397598,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":1025122,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561123,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":884736,"data_stored":431428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574976,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":504274,"data_compressed":1836,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738502,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":432699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411451,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":974848,"data_stored":506291,"data_compressed":1836,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117109,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":431739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920879,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":888832,"data_stored":432788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757353,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":1023883,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34281,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34281,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34281,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35908,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":73053,"data_compressed":1836,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":75070,"data_compressed":1836,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":75554,"data_compressed":1836,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T06:10:20.923 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T06:10:21.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:20 smithi123 ceph-mon[29787]: pgmap v456: 97 pgs: 97 active+clean; 653 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-05-08T06:10:21.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:20 smithi123 ceph-mon[29787]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T06:10:21.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:20 smithi163 ceph-mon[34520]: pgmap v456: 97 pgs: 97 active+clean; 653 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-05-08T06:10:21.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:20 smithi163 ceph-mon[34520]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T06:10:21.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:20 smithi192 ceph-mon[34664]: pgmap v456: 97 pgs: 97 active+clean; 653 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-05-08T06:10:21.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:20 smithi192 ceph-mon[34664]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T06:10:21.427 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:10:22.142 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:21 smithi123 ceph-mon[29787]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T06:10:22.189 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:21 smithi163 ceph-mon[34520]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T06:10:22.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:21 smithi192 ceph-mon[34664]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T06:10:23.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:22 smithi123 ceph-mon[29787]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T06:10:23.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:22 smithi123 ceph-mon[29787]: pgmap v457: 97 pgs: 97 active+clean; 653 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-05-08T06:10:23.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:22 smithi163 ceph-mon[34520]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T06:10:23.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:22 smithi163 ceph-mon[34520]: pgmap v457: 97 pgs: 97 active+clean; 653 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-05-08T06:10:23.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:22 smithi192 ceph-mon[34664]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T06:10:23.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:22 smithi192 ceph-mon[34664]: pgmap v457: 97 pgs: 97 active+clean; 653 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-05-08T06:10:24.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.026+0000 7ff792e00700 1 -- 172.21.15.123:0/1260224002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff78c10fb60 msgr2=0x7ff78c10ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:24.026 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.026+0000 7ff792e00700 1 --2- 172.21.15.123:0/1260224002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff78c10fb60 0x7ff78c10ff40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff77c009990 tx=0x7ff77c009ca0 comp rx=0 tx=0).stop 2024-05-08T06:10:24.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.026+0000 7ff792e00700 1 -- 172.21.15.123:0/1260224002 shutdown_connections 2024-05-08T06:10:24.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.026+0000 7ff792e00700 1 --2- 172.21.15.123:0/1260224002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff78c10fb60 0x7ff78c10ff40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:24.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.026+0000 7ff792e00700 1 --2- 172.21.15.123:0/1260224002 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff78c10a490 0x7ff78c1169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:24.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.026+0000 7ff792e00700 1 --2- 172.21.15.123:0/1260224002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff78c109ad0 0x7ff78c109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:24.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.026+0000 7ff792e00700 1 -- 172.21.15.123:0/1260224002 >> 172.21.15.123:0/1260224002 conn(0x7ff78c076c70 msgr2=0x7ff78c1082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:24.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.026+0000 7ff792e00700 1 -- 172.21.15.123:0/1260224002 shutdown_connections 2024-05-08T06:10:24.027 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.026+0000 7ff792e00700 1 -- 172.21.15.123:0/1260224002 wait complete. 2024-05-08T06:10:24.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.027+0000 7ff792e00700 1 Processor -- start 2024-05-08T06:10:24.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.027+0000 7ff792e00700 1 -- start start 2024-05-08T06:10:24.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.028+0000 7ff792e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff78c109ad0 0x7ff78c1a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:24.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.028+0000 7ff792e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff78c10a490 0x7ff78c1a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:24.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.028+0000 7ff792e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff78c10fb60 0x7ff78c1a4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:24.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.028+0000 7ff792e00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff78c119cd0 con 0x7ff78c109ad0 2024-05-08T06:10:24.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.028+0000 7ff792e00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff78c119b50 con 0x7ff78c10fb60 2024-05-08T06:10:24.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.028+0000 7ff792e00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff78c119e50 con 0x7ff78c10a490 2024-05-08T06:10:24.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.028+0000 7ff78b400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff78c10a490 0x7ff78c1a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:24.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.028+0000 7ff78b400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff78c10a490 0x7ff78c1a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:46936/0 (socket says 172.21.15.123:46936) 2024-05-08T06:10:24.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.028+0000 7ff78b400700 1 -- 172.21.15.123:0/1722943822 learned_addr learned my addr 172.21.15.123:0/1722943822 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:10:24.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.028+0000 7ff78be00700 1 --2- 172.21.15.123:0/1722943822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff78c109ad0 0x7ff78c1a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:24.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.028+0000 7ff790e00700 1 --2- 172.21.15.123:0/1722943822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff78c10fb60 0x7ff78c1a4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:24.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.029+0000 7ff78b400700 1 -- 172.21.15.123:0/1722943822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff78c10fb60 msgr2=0x7ff78c1a4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:24.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.029+0000 7ff78b400700 1 --2- 172.21.15.123:0/1722943822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff78c10fb60 0x7ff78c1a4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:24.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.029+0000 7ff78b400700 1 -- 172.21.15.123:0/1722943822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff78c109ad0 msgr2=0x7ff78c1a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:24.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.029+0000 7ff78b400700 1 --2- 172.21.15.123:0/1722943822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff78c109ad0 0x7ff78c1a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:24.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.029+0000 7ff78b400700 1 -- 172.21.15.123:0/1722943822 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff78c1a5160 con 0x7ff78c10a490 2024-05-08T06:10:24.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.029+0000 7ff78b400700 1 --2- 172.21.15.123:0/1722943822 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff78c10a490 0x7ff78c1a06e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff77800e9a0 tx=0x7ff77800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:24.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.029+0000 7ff788c00700 1 -- 172.21.15.123:0/1722943822 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77800ca10 con 0x7ff78c10a490 2024-05-08T06:10:24.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.029+0000 7ff792e00700 1 -- 172.21.15.123:0/1722943822 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff78c1a5450 con 0x7ff78c10a490 2024-05-08T06:10:24.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.029+0000 7ff788c00700 1 -- 172.21.15.123:0/1722943822 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff77800cbb0 con 0x7ff78c10a490 2024-05-08T06:10:24.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.029+0000 7ff788c00700 1 -- 172.21.15.123:0/1722943822 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff778018680 con 0x7ff78c10a490 2024-05-08T06:10:24.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.029+0000 7ff792e00700 1 -- 172.21.15.123:0/1722943822 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff78c1accd0 con 0x7ff78c10a490 2024-05-08T06:10:24.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.030+0000 7ff792e00700 1 -- 172.21.15.123:0/1722943822 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff78c06a190 con 0x7ff78c10a490 2024-05-08T06:10:24.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.034+0000 7ff788c00700 1 -- 172.21.15.123:0/1722943822 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) ==== 149704+0+0 (secure 0 0 0) 0x7ff778010430 con 0x7ff78c10a490 2024-05-08T06:10:24.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.036+0000 7ff788c00700 1 --2- 172.21.15.123:0/1722943822 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7ff7700b2990 0x7ff7700b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:24.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.036+0000 7ff788c00700 1 -- 172.21.15.123:0/1722943822 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7ff7780e2820 con 0x7ff78c10a490 2024-05-08T06:10:24.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.036+0000 7ff78be00700 1 --2- 172.21.15.123:0/1722943822 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7ff7700b2990 0x7ff7700b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:24.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.037+0000 7ff788c00700 1 -- 172.21.15.123:0/1722943822 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff7780aa390 con 0x7ff78c10a490 2024-05-08T06:10:24.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.037+0000 7ff78be00700 1 --2- 172.21.15.123:0/1722943822 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7ff7700b2990 0x7ff7700b4e50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0098e0 tx=0x7ff77c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:24.319 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.319+0000 7ff792e00700 1 -- 172.21.15.123:0/1722943822 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff78c02e510 con 0x7ff78c10a490 2024-05-08T06:10:24.320 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.320+0000 7ff788c00700 1 -- 172.21.15.123:0/1722943822 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11885 (secure 0 0 0) 0x7ff7780b2410 con 0x7ff78c10a490 2024-05-08T06:10:24.321 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:10:24.321 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3646476522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3646476522},{"type":"v1","addr":"172.21.15.123:6835","nonce":3646476522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2763706497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2763706497},{"type":"v1","addr":"172.21.15.123:6837","nonce":2763706497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/274635265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":274635265},{"type":"v1","addr":"172.21.15.123:6839","nonce":274635265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/175423115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":175423115},{"type":"v1","addr":"172.21.15.192:6833","nonce":175423115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/3263009472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3263009472},{"type":"v1","addr":"172.21.15.163:6833","nonce":3263009472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3308612826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3308612826},{"type":"v1","addr":"172.21.15.163:6835","nonce":3308612826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T06:05:54.755550+0000","modified":"2024-05-08T06:10:12.600417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24512,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/202513680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":202513680},{"type":"v1","addr":"172.21.15.192:6835","nonce":202513680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/2483545969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2483545969},{"type":"v1","addr":"172.21.15.192:6837","nonce":2483545969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/660511838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":660511838},{"type":"v1","addr":"172.21.15.163:6837","nonce":660511838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.192:6839/3598682360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":3598682360},{"type":"v1","addr":"172.21.15.192:6839","nonce":3598682360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6841/1571726231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":1571726231},{"type":"v1","addr":"172.21.15.123:6841","nonce":1571726231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.163:6839/1568161515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":1568161515},{"type":"v1","addr":"172.21.15.163:6839","nonce":1568161515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24512,24496,24524]},"id":1}]} 2024-05-08T06:10:24.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.324+0000 7ff792e00700 1 -- 172.21.15.123:0/1722943822 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7ff7700b2990 msgr2=0x7ff7700b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:24.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.324+0000 7ff792e00700 1 --2- 172.21.15.123:0/1722943822 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7ff7700b2990 0x7ff7700b4e50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0098e0 tx=0x7ff77c005960 comp rx=0 tx=0).stop 2024-05-08T06:10:24.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.324+0000 7ff792e00700 1 -- 172.21.15.123:0/1722943822 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff78c10a490 msgr2=0x7ff78c1a06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:24.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.324+0000 7ff792e00700 1 --2- 172.21.15.123:0/1722943822 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff78c10a490 0x7ff78c1a06e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff77800e9a0 tx=0x7ff77800ecb0 comp rx=0 tx=0).stop 2024-05-08T06:10:24.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.325+0000 7ff792e00700 1 -- 172.21.15.123:0/1722943822 shutdown_connections 2024-05-08T06:10:24.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.325+0000 7ff792e00700 1 --2- 172.21.15.123:0/1722943822 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7ff7700b2990 0x7ff7700b4e50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:24.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.325+0000 7ff792e00700 1 --2- 172.21.15.123:0/1722943822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff78c109ad0 0x7ff78c1a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:24.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.325+0000 7ff792e00700 1 --2- 172.21.15.123:0/1722943822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff78c10fb60 0x7ff78c1a4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:24.325 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.325+0000 7ff792e00700 1 --2- 172.21.15.123:0/1722943822 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff78c10a490 0x7ff78c1a06e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:24.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.325+0000 7ff792e00700 1 -- 172.21.15.123:0/1722943822 >> 172.21.15.123:0/1722943822 conn(0x7ff78c076c70 msgr2=0x7ff78c077270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:24.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.325+0000 7ff792e00700 1 -- 172.21.15.123:0/1722943822 shutdown_connections 2024-05-08T06:10:24.326 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:24.325+0000 7ff792e00700 1 -- 172.21.15.123:0/1722943822 wait complete. 2024-05-08T06:10:24.326 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 24 2024-05-08T06:10:25.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:24 smithi123 ceph-mon[29787]: pgmap v458: 97 pgs: 97 active+clean; 653 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-08T06:10:25.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:24 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/1722943822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:10:25.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:24 smithi163 ceph-mon[34520]: pgmap v458: 97 pgs: 97 active+clean; 653 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-08T06:10:25.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:24 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/1722943822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:10:25.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:24 smithi192 ceph-mon[34664]: pgmap v458: 97 pgs: 97 active+clean; 653 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-08T06:10:25.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:24 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/1722943822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:10:25.381 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T06:10:25.822 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:10:27.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:26 smithi123 ceph-mon[29787]: pgmap v459: 97 pgs: 97 active+clean; 653 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-05-08T06:10:27.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:26 smithi163 ceph-mon[34520]: pgmap v459: 97 pgs: 97 active+clean; 653 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-05-08T06:10:27.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:26 smithi192 ceph-mon[34664]: pgmap v459: 97 pgs: 97 active+clean; 653 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-05-08T06:10:28.111 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:27 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:10:28.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.269+0000 7fa429600700 1 -- 172.21.15.123:0/547193501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa424108850 msgr2=0x7fa4241103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:28.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.269+0000 7fa429600700 1 --2- 172.21.15.123:0/547193501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa424108850 0x7fa4241103d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa41c00b0d0 tx=0x7fa41c00b3e0 comp rx=0 tx=0).stop 2024-05-08T06:10:28.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.270+0000 7fa429600700 1 -- 172.21.15.123:0/547193501 shutdown_connections 2024-05-08T06:10:28.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.270+0000 7fa429600700 1 --2- 172.21.15.123:0/547193501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa424108850 0x7fa4241103d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:28.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.270+0000 7fa429600700 1 --2- 172.21.15.123:0/547193501 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa424103750 0x7fa424103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:28.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.270+0000 7fa429600700 1 --2- 172.21.15.123:0/547193501 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa424104100 0x7fa424108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:28.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.270+0000 7fa429600700 1 -- 172.21.15.123:0/547193501 >> 172.21.15.123:0/547193501 conn(0x7fa42407a600 msgr2=0x7fa42407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:28.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.270+0000 7fa429600700 1 -- 172.21.15.123:0/547193501 shutdown_connections 2024-05-08T06:10:28.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.270+0000 7fa429600700 1 -- 172.21.15.123:0/547193501 wait complete. 2024-05-08T06:10:28.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.271+0000 7fa429600700 1 Processor -- start 2024-05-08T06:10:28.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.271+0000 7fa429600700 1 -- start start 2024-05-08T06:10:28.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.271+0000 7fa429600700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa424103750 0x7fa42419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:28.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.271+0000 7fa429600700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa424104100 0x7fa42419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:28.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.271+0000 7fa429600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa424108850 0x7fa4241998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:28.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.271+0000 7fa429600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa424113670 con 0x7fa424108850 2024-05-08T06:10:28.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.271+0000 7fa429600700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa4241134f0 con 0x7fa424103750 2024-05-08T06:10:28.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.271+0000 7fa429600700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa4241137f0 con 0x7fa424104100 2024-05-08T06:10:28.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.272+0000 7fa422a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa424103750 0x7fa42419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:28.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.272+0000 7fa423400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa424108850 0x7fa4241998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:28.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.272+0000 7fa422a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa424103750 0x7fa42419f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:46146/0 (socket says 172.21.15.123:46146) 2024-05-08T06:10:28.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.272+0000 7fa422a00700 1 -- 172.21.15.123:0/102346344 learned_addr learned my addr 172.21.15.123:0/102346344 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:10:28.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.272+0000 7fa422000700 1 --2- 172.21.15.123:0/102346344 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa424104100 0x7fa42419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:28.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.272+0000 7fa422a00700 1 -- 172.21.15.123:0/102346344 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa424104100 msgr2=0x7fa42419fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:28.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.272+0000 7fa422a00700 1 --2- 172.21.15.123:0/102346344 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa424104100 0x7fa42419fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:28.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.272+0000 7fa422a00700 1 -- 172.21.15.123:0/102346344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa424108850 msgr2=0x7fa4241998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:28.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.272+0000 7fa422a00700 1 --2- 172.21.15.123:0/102346344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa424108850 0x7fa4241998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:28.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.272+0000 7fa422a00700 1 -- 172.21.15.123:0/102346344 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa42419a010 con 0x7fa424103750 2024-05-08T06:10:28.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.272+0000 7fa422a00700 1 --2- 172.21.15.123:0/102346344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa424103750 0x7fa42419f6f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa41400e9a0 tx=0x7fa41400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:28.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.273+0000 7fa41b400700 1 -- 172.21.15.123:0/102346344 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41400ca10 con 0x7fa424103750 2024-05-08T06:10:28.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.273+0000 7fa41b400700 1 -- 172.21.15.123:0/102346344 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa41400cbb0 con 0x7fa424103750 2024-05-08T06:10:28.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.273+0000 7fa41b400700 1 -- 172.21.15.123:0/102346344 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa414018680 con 0x7fa424103750 2024-05-08T06:10:28.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.273+0000 7fa429600700 1 -- 172.21.15.123:0/102346344 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa42419a300 con 0x7fa424103750 2024-05-08T06:10:28.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.273+0000 7fa429600700 1 -- 172.21.15.123:0/102346344 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4241a65c0 con 0x7fa424103750 2024-05-08T06:10:28.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.274+0000 7fa429600700 1 -- 172.21.15.123:0/102346344 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa42406a190 con 0x7fa424103750 2024-05-08T06:10:28.276 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.276+0000 7fa41b400700 1 -- 172.21.15.123:0/102346344 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 19) ==== 149704+0+0 (secure 0 0 0) 0x7fa414010430 con 0x7fa424103750 2024-05-08T06:10:28.277 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.277+0000 7fa41b400700 1 --2- 172.21.15.123:0/102346344 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa4080b2990 0x7fa4080b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:28.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.278+0000 7fa41b400700 1 -- 172.21.15.123:0/102346344 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fa4140e24e0 con 0x7fa424103750 2024-05-08T06:10:28.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.278+0000 7fa422000700 1 --2- 172.21.15.123:0/102346344 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa4080b2990 0x7fa4080b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:28.278 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.278+0000 7fa422000700 1 --2- 172.21.15.123:0/102346344 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa4080b2990 0x7fa4080b4e50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa424103000 tx=0x7fa410009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:28.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.282+0000 7fa41b400700 1 -- 172.21.15.123:0/102346344 <== mon.2 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa4140aa050 con 0x7fa424103750 2024-05-08T06:10:28.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:27 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:10:28.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:27 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:10:28.516 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.516+0000 7fa429600700 1 -- 172.21.15.123:0/102346344 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa42402e6d0 con 0x7fa424103750 2024-05-08T06:10:28.518 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.518+0000 7fa41b400700 1 -- 172.21.15.123:0/102346344 <== mon.2 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22719 (secure 0 0 0) 0x7fa4140b20d0 con 0x7fa424103750 2024-05-08T06:10:28.519 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:10:28.519 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":82,"fsid":"85cfb2e6-0cff-11ef-bc97-c7b262605968","created":"2024-05-08T05:56:12.544750+0000","modified":"2024-05-08T06:06:32.471993+0000","last_up_change":"2024-05-08T06:04:33.654727+0000","last_in_change":"2024-05-08T06:04:13.809050+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T06:00:17.050518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T06:05:44.815334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T06:05:49.604457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a14fde9b-bfb1-40a7-a105-305e8981a778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6803","nonce":3608618058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6805","nonce":3608618058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6809","nonce":3608618058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6807","nonce":3608618058}]},"public_addr":"172.21.15.123:6803/3608618058","cluster_addr":"172.21.15.123:6805/3608618058","heartbeat_back_addr":"172.21.15.123:6809/3608618058","heartbeat_front_addr":"172.21.15.123:6807/3608618058","state":["exists","up"]},{"osd":1,"uuid":"e023ae3d-a82e-4bb6-8fef-889dd07fe183","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6801","nonce":1884753449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6803","nonce":1884753449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6807","nonce":1884753449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6805","nonce":1884753449}]},"public_addr":"172.21.15.163:6801/1884753449","cluster_addr":"172.21.15.163:6803/1884753449","heartbeat_back_addr":"172.21.15.163:6807/1884753449","heartbeat_front_addr":"172.21.15.163:6805/1884753449","state":["exists","up"]},{"osd":2,"uuid":"69f4cfca-63cb-4862-9480-d79891b0f338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6801","nonce":4161570404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6803","nonce":4161570404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6807","nonce":4161570404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6805","nonce":4161570404}]},"public_addr":"172.21.15.192:6801/4161570404","cluster_addr":"172.21.15.192:6803/4161570404","heartbeat_back_addr":"172.21.15.192:6807/4161570404","heartbeat_front_addr":"172.21.15.192:6805/4161570404","state":["exists","up"]},{"osd":3,"uuid":"71bb5ecd-c88c-45fb-8b51-0eefb53941c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6811","nonce":1243003258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6813","nonce":1243003258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6817","nonce":1243003258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6815","nonce":1243003258}]},"public_addr":"172.21.15.123:6811/1243003258","cluster_addr":"172.21.15.123:6813/1243003258","heartbeat_back_addr":"172.21.15.123:6817/1243003258","heartbeat_front_addr":"172.21.15.123:6815/1243003258","state":["exists","up"]},{"osd":4,"uuid":"5a7a3462-e74f-4d15-a85c-709b5d423745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6809","nonce":591680273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6811","nonce":591680273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6815","nonce":591680273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6813","nonce":591680273}]},"public_addr":"172.21.15.163:6809/591680273","cluster_addr":"172.21.15.163:6811/591680273","heartbeat_back_addr":"172.21.15.163:6815/591680273","heartbeat_front_addr":"172.21.15.163:6813/591680273","state":["exists","up"]},{"osd":5,"uuid":"85e3f1ab-f8c6-4025-bab9-9ddb6bd3e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6809","nonce":1734821180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6811","nonce":1734821180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6815","nonce":1734821180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6813","nonce":1734821180}]},"public_addr":"172.21.15.192:6809/1734821180","cluster_addr":"172.21.15.192:6811/1734821180","heartbeat_back_addr":"172.21.15.192:6815/1734821180","heartbeat_front_addr":"172.21.15.192:6813/1734821180","state":["exists","up"]},{"osd":6,"uuid":"f21a5624-6f8f-48be-bc6d-0a883d6b7b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6819","nonce":1002138878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6821","nonce":1002138878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6825","nonce":1002138878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6823","nonce":1002138878}]},"public_addr":"172.21.15.123:6819/1002138878","cluster_addr":"172.21.15.123:6821/1002138878","heartbeat_back_addr":"172.21.15.123:6825/1002138878","heartbeat_front_addr":"172.21.15.123:6823/1002138878","state":["exists","up"]},{"osd":7,"uuid":"6eb16729-d260-4041-9829-3e868795b111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6817","nonce":354411978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6819","nonce":354411978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6823","nonce":354411978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6821","nonce":354411978}]},"public_addr":"172.21.15.163:6817/354411978","cluster_addr":"172.21.15.163:6819/354411978","heartbeat_back_addr":"172.21.15.163:6823/354411978","heartbeat_front_addr":"172.21.15.163:6821/354411978","state":["exists","up"]},{"osd":8,"uuid":"122a281e-18b8-4728-97db-458e7432b2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6817","nonce":3669753689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6819","nonce":3669753689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6823","nonce":3669753689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6821","nonce":3669753689}]},"public_addr":"172.21.15.192:6817/3669753689","cluster_addr":"172.21.15.192:6819/3669753689","heartbeat_back_addr":"172.21.15.192:6823/3669753689","heartbeat_front_addr":"172.21.15.192:6821/3669753689","state":["exists","up"]},{"osd":9,"uuid":"9fa022a9-b873-4e07-806c-641a706a150f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6827","nonce":3104436473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6829","nonce":3104436473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6833","nonce":3104436473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6831","nonce":3104436473}]},"public_addr":"172.21.15.123:6827/3104436473","cluster_addr":"172.21.15.123:6829/3104436473","heartbeat_back_addr":"172.21.15.123:6833/3104436473","heartbeat_front_addr":"172.21.15.123:6831/3104436473","state":["exists","up"]},{"osd":10,"uuid":"fb59339b-e11b-4ffc-b202-7f67cb21da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6825","nonce":2628487883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6827","nonce":2628487883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6831","nonce":2628487883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6829","nonce":2628487883}]},"public_addr":"172.21.15.163:6825/2628487883","cluster_addr":"172.21.15.163:6827/2628487883","heartbeat_back_addr":"172.21.15.163:6831/2628487883","heartbeat_front_addr":"172.21.15.163:6829/2628487883","state":["exists","up"]},{"osd":11,"uuid":"be7a5f36-c1be-41d7-80ee-004d0546eb9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6825","nonce":1339159347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6827","nonce":1339159347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6831","nonce":1339159347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6829","nonce":1339159347}]},"public_addr":"172.21.15.192:6825/1339159347","cluster_addr":"172.21.15.192:6827/1339159347","heartbeat_back_addr":"172.21.15.192:6831/1339159347","heartbeat_front_addr":"172.21.15.192:6829/1339159347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:59:24.923234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:59:49.867071+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:00:13.454062+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:00:46.983594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:01:12.820452+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:01:37.451382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:02:13.325181+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:02:39.453436+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:03:03.837509+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:03:39.460579+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:04:06.314082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:04:31.723154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6801/3057688649":"2024-05-09T05:57:30.887936+0000","172.21.15.123:6800/3057688649":"2024-05-09T05:57:30.887936+0000","172.21.15.123:6800/2769724312":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/399884043":"2024-05-09T05:57:30.887936+0000","172.21.15.123:0/3406061897":"2024-05-09T05:57:30.887936+0000","172.21.15.123:0/644457268":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/3055092197":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/343390357":"2024-05-09T05:57:30.887936+0000","172.21.15.123:6801/2769724312":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/2263171562":"2024-05-09T05:56:55.849036+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T06:10:28.521 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.521+0000 7fa429600700 1 -- 172.21.15.123:0/102346344 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa4080b2990 msgr2=0x7fa4080b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:28.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.521+0000 7fa429600700 1 --2- 172.21.15.123:0/102346344 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa4080b2990 0x7fa4080b4e50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa424103000 tx=0x7fa410009260 comp rx=0 tx=0).stop 2024-05-08T06:10:28.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.521+0000 7fa429600700 1 -- 172.21.15.123:0/102346344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa424103750 msgr2=0x7fa42419f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:28.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.521+0000 7fa429600700 1 --2- 172.21.15.123:0/102346344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa424103750 0x7fa42419f6f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa41400e9a0 tx=0x7fa41400ecb0 comp rx=0 tx=0).stop 2024-05-08T06:10:28.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.522+0000 7fa429600700 1 -- 172.21.15.123:0/102346344 shutdown_connections 2024-05-08T06:10:28.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.522+0000 7fa429600700 1 --2- 172.21.15.123:0/102346344 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa4080b2990 0x7fa4080b4e50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:28.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.522+0000 7fa429600700 1 --2- 172.21.15.123:0/102346344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa424108850 0x7fa4241998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:28.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.522+0000 7fa429600700 1 --2- 172.21.15.123:0/102346344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa424103750 0x7fa42419f6f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:28.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.522+0000 7fa429600700 1 --2- 172.21.15.123:0/102346344 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa424104100 0x7fa42419fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:28.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.522+0000 7fa429600700 1 -- 172.21.15.123:0/102346344 >> 172.21.15.123:0/102346344 conn(0x7fa42407a600 msgr2=0x7fa4241015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:28.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.522+0000 7fa429600700 1 -- 172.21.15.123:0/102346344 shutdown_connections 2024-05-08T06:10:28.522 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:28.522+0000 7fa429600700 1 -- 172.21.15.123:0/102346344 wait complete. 2024-05-08T06:10:29.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:28 smithi123 ceph-mon[29787]: pgmap v460: 97 pgs: 97 active+clean; 653 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-05-08T06:10:29.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:28 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:10:29.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:28 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:10:29.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:28 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:10:29.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:28 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/102346344' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:10:29.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:28 smithi163 ceph-mon[34520]: pgmap v460: 97 pgs: 97 active+clean; 653 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-05-08T06:10:29.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:28 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:10:29.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:28 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:10:29.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:28 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:10:29.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:28 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/102346344' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:10:29.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:28 smithi192 ceph-mon[34664]: pgmap v460: 97 pgs: 97 active+clean; 653 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-05-08T06:10:29.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:28 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:10:29.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:28 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:10:29.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:28 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:10:29.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:28 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/102346344' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:10:29.549 DEBUG:tasks.fwd_scrub:Yielding 2024-05-08T06:10:29.549 INFO:teuthology.run_tasks:Running task background_exec... 2024-05-08T06:10:29.559 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi123.front.sepia.ceph.com 2024-05-08T06:10:29.560 DEBUG:teuthology.orchestra.run.smithi123:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-05-08T06:10:29.560 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-05-08T06:10:29.560 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T06:10:29.561 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T06:10:29.566 INFO:teuthology.run_tasks:Running task workunit... 2024-05-08T06:10:29.584 INFO:tasks.workunit:Pulling workunits from ref bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T06:10:29.585 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-08T06:10:29.585 INFO:tasks.workunit:timeout=3h 2024-05-08T06:10:29.585 INFO:tasks.workunit:cleanup=True 2024-05-08T06:10:29.585 DEBUG:teuthology.orchestra.run.smithi123:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-08T06:10:29.628 INFO:teuthology.orchestra.run.smithi123.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-08T06:10:29.628 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-08T06:10:29.628 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 72h/114d Inode: 1099511628283 Links: 2 2024-05-08T06:10:29.628 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T06:10:29.628 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-05-08T06:10:29.628 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-05-08 06:06:39.899647220 +0000 2024-05-08T06:10:29.629 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-05-08 06:06:39.899647220 +0000 2024-05-08T06:10:29.629 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-05-08 06:10:11.544123312 +0000 2024-05-08T06:10:29.629 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-05-08 06:06:39.899647220 +0000 2024-05-08T06:10:29.629 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-08T06:10:29.629 DEBUG:teuthology.orchestra.run.smithi123:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-08T06:10:29.746 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T06:10:29.795 INFO:tasks.workunit.client.0.smithi123.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-08T06:10:29.992 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:10:30.139 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:29 smithi123 ceph-mon[29787]: pgmap v461: 97 pgs: 97 active+clean; 653 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-08T06:10:30.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:29 smithi163 ceph-mon[34520]: pgmap v461: 97 pgs: 97 active+clean; 653 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-08T06:10:30.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:29 smithi192 ceph-mon[34664]: pgmap v461: 97 pgs: 97 active+clean; 653 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-08T06:10:32.354 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:32 smithi123 ceph-mon[29787]: pgmap v462: 97 pgs: 97 active+clean; 653 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-08T06:10:32.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.418+0000 7fce8b000700 1 -- 172.21.15.123:0/901136799 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce841055d0 msgr2=0x7fce84105a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:32.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.418+0000 7fce8b000700 1 --2- 172.21.15.123:0/901136799 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce841055d0 0x7fce84105a50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fce70009990 tx=0x7fce70009ca0 comp rx=0 tx=0).stop 2024-05-08T06:10:32.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.419+0000 7fce8b000700 1 -- 172.21.15.123:0/901136799 shutdown_connections 2024-05-08T06:10:32.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.419+0000 7fce8b000700 1 --2- 172.21.15.123:0/901136799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce8410b660 0x7fce8410ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:32.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.419+0000 7fce8b000700 1 --2- 172.21.15.123:0/901136799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce84105f90 0x7fce841124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:32.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.419+0000 7fce8b000700 1 --2- 172.21.15.123:0/901136799 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce841055d0 0x7fce84105a50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:32.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.419+0000 7fce8b000700 1 -- 172.21.15.123:0/901136799 >> 172.21.15.123:0/901136799 conn(0x7fce841010c0 msgr2=0x7fce841034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:32.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.419+0000 7fce8b000700 1 -- 172.21.15.123:0/901136799 shutdown_connections 2024-05-08T06:10:32.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.419+0000 7fce8b000700 1 -- 172.21.15.123:0/901136799 wait complete. 2024-05-08T06:10:32.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.420+0000 7fce8b000700 1 Processor -- start 2024-05-08T06:10:32.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.420+0000 7fce8b000700 1 -- start start 2024-05-08T06:10:32.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.421+0000 7fce8b000700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce841055d0 0x7fce841a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:32.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.421+0000 7fce8b000700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce84105f90 0x7fce841a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:32.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.421+0000 7fce8b000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce8410b660 0x7fce8419ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:32.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.421+0000 7fce8b000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fce84115800 con 0x7fce8410b660 2024-05-08T06:10:32.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.421+0000 7fce8b000700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fce84115680 con 0x7fce841055d0 2024-05-08T06:10:32.421 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.421+0000 7fce8b000700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fce84115980 con 0x7fce84105f90 2024-05-08T06:10:32.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.421+0000 7fce89c00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce841055d0 0x7fce841a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:32.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.421+0000 7fce8a600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce8410b660 0x7fce8419ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:32.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.421+0000 7fce89c00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce841055d0 0x7fce841a18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:46166/0 (socket says 172.21.15.123:46166) 2024-05-08T06:10:32.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.421+0000 7fce89c00700 1 -- 172.21.15.123:0/2996296673 learned_addr learned my addr 172.21.15.123:0/2996296673 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:10:32.422 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.421+0000 7fce89200700 1 --2- 172.21.15.123:0/2996296673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce84105f90 0x7fce841a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:32.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.422+0000 7fce89c00700 1 -- 172.21.15.123:0/2996296673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce84105f90 msgr2=0x7fce841a1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:32.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.422+0000 7fce89c00700 1 --2- 172.21.15.123:0/2996296673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce84105f90 0x7fce841a1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:32.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.422+0000 7fce89c00700 1 -- 172.21.15.123:0/2996296673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce8410b660 msgr2=0x7fce8419ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:32.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.422+0000 7fce89c00700 1 --2- 172.21.15.123:0/2996296673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce8410b660 0x7fce8419ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:32.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.422+0000 7fce89c00700 1 -- 172.21.15.123:0/2996296673 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce8419c1e0 con 0x7fce841055d0 2024-05-08T06:10:32.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.422+0000 7fce89c00700 1 --2- 172.21.15.123:0/2996296673 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce841055d0 0x7fce841a18c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fce7800e970 tx=0x7fce7800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:32.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.422+0000 7fce82000700 1 -- 172.21.15.123:0/2996296673 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce78004220 con 0x7fce841055d0 2024-05-08T06:10:32.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.423+0000 7fce82000700 1 -- 172.21.15.123:0/2996296673 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce780043c0 con 0x7fce841055d0 2024-05-08T06:10:32.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.423+0000 7fce82000700 1 -- 172.21.15.123:0/2996296673 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce78003ce0 con 0x7fce841055d0 2024-05-08T06:10:32.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.423+0000 7fce8b000700 1 -- 172.21.15.123:0/2996296673 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce8419c4d0 con 0x7fce841055d0 2024-05-08T06:10:32.424 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.423+0000 7fce8b000700 1 -- 172.21.15.123:0/2996296673 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce841a8790 con 0x7fce841055d0 2024-05-08T06:10:32.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.426+0000 7fce82000700 1 -- 172.21.15.123:0/2996296673 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 19) ==== 149704+0+0 (secure 0 0 0) 0x7fce7800c7c0 con 0x7fce841055d0 2024-05-08T06:10:32.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.427+0000 7fce96f37b80 1 Processor -- start 2024-05-08T06:10:32.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.428+0000 7fce96f37b80 1 -- start start 2024-05-08T06:10:32.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.428+0000 7fce82000700 1 --2- 172.21.15.123:0/2996296673 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fce6c0b2990 0x7fce6c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:32.428 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.428+0000 7fce82000700 1 -- 172.21.15.123:0/2996296673 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fce780e2570 con 0x7fce841055d0 2024-05-08T06:10:32.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.428+0000 7fce8a600700 1 --2- 172.21.15.123:0/2996296673 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fce6c0b2990 0x7fce6c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:32.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.428+0000 7fce96f37b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b7a376fa60 0x55b7a376fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:32.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.428+0000 7fce96f37b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7a3770380 0x55b7a376a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:32.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.428+0000 7fce96f37b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7a376a680 0x55b7a376ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:32.431 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.428+0000 7fce96f37b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b7a36bcaa0 con 0x55b7a376a680 2024-05-08T06:10:32.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.428+0000 7fce96f37b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b7a365a7e0 con 0x55b7a376fa60 2024-05-08T06:10:32.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.428+0000 7fce96f37b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b7a36b8a60 con 0x55b7a3770380 2024-05-08T06:10:32.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.428+0000 7fce89c00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b7a376fa60 0x55b7a376fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:32.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.428+0000 7fce89c00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b7a376fa60 0x55b7a376fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:46176/0 (socket says 172.21.15.123:46176) 2024-05-08T06:10:32.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.428+0000 7fce89c00700 1 -- 172.21.15.123:0/1178885564 learned_addr learned my addr 172.21.15.123:0/1178885564 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:10:32.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.429+0000 7fce89200700 1 --2- 172.21.15.123:0/1178885564 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7a3770380 0x55b7a376a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:32.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.429+0000 7fce8a600700 1 --2- 172.21.15.123:0/1178885564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7a376a680 0x55b7a376ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:32.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.429+0000 7fce89c00700 1 -- 172.21.15.123:0/1178885564 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7a3770380 msgr2=0x55b7a376a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:32.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.429+0000 7fce89c00700 1 --2- 172.21.15.123:0/1178885564 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7a3770380 0x55b7a376a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:32.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.429+0000 7fce89c00700 1 -- 172.21.15.123:0/1178885564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7a376a680 msgr2=0x55b7a376ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:32.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.429+0000 7fce89c00700 1 --2- 172.21.15.123:0/1178885564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7a376a680 0x55b7a376ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:32.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.429+0000 7fce89c00700 1 -- 172.21.15.123:0/1178885564 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7a3609950 con 0x55b7a376fa60 2024-05-08T06:10:32.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.429+0000 7fce89200700 1 --2- 172.21.15.123:0/1178885564 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7a3770380 0x55b7a376a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:10:32.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.429+0000 7fce8a600700 1 --2- 172.21.15.123:0/2996296673 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fce6c0b2990 0x7fce6c0b4e50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fce7c009fd0 tx=0x7fce7c00fc40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:32.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.429+0000 7fce89c00700 1 --2- 172.21.15.123:0/1178885564 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b7a376fa60 0x55b7a376fe40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fce780a9970 tx=0x7fce780a9150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4896ec0236668fa2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:32.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.430+0000 7fce76a00700 1 -- 172.21.15.123:0/1178885564 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce780a5170 con 0x55b7a376fa60 2024-05-08T06:10:32.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.430+0000 7fce76a00700 1 -- 172.21.15.123:0/1178885564 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce780b2190 con 0x55b7a376fa60 2024-05-08T06:10:32.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.430+0000 7fce76a00700 1 -- 172.21.15.123:0/1178885564 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce780e0120 con 0x55b7a376fa60 2024-05-08T06:10:32.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.430+0000 7fce96f37b80 1 -- 172.21.15.123:0/1178885564 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b7a376fa60 msgr2=0x55b7a376fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:32.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.430+0000 7fce96f37b80 1 --2- 172.21.15.123:0/1178885564 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b7a376fa60 0x55b7a376fe40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fce780a9970 tx=0x7fce780a9150 comp rx=0 tx=0).stop 2024-05-08T06:10:32.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.431+0000 7fce96f37b80 1 -- 172.21.15.123:0/1178885564 shutdown_connections 2024-05-08T06:10:32.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.431+0000 7fce96f37b80 1 --2- 172.21.15.123:0/1178885564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7a376a680 0x55b7a376ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:32.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.431+0000 7fce96f37b80 1 --2- 172.21.15.123:0/1178885564 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b7a376fa60 0x55b7a376fe40 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:32.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.431+0000 7fce96f37b80 1 --2- 172.21.15.123:0/1178885564 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7a3770380 0x55b7a376a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:32.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.431+0000 7fce96f37b80 1 -- 172.21.15.123:0/1178885564 >> 172.21.15.123:0/1178885564 conn(0x55b7a3698be0 msgr2=0x55b7a3764a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:32.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.431+0000 7fce96f37b80 1 -- 172.21.15.123:0/1178885564 shutdown_connections 2024-05-08T06:10:32.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.431+0000 7fce96f37b80 1 -- 172.21.15.123:0/1178885564 wait complete. 2024-05-08T06:10:32.434 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.431+0000 7fce96f37b80 1 -- start start 2024-05-08T06:10:32.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.432+0000 7fce96f37b80 1 Processor -- start 2024-05-08T06:10:32.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.432+0000 7fce96f37b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b7a376a680 0x55b7a3776050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:32.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.432+0000 7fce96f37b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7a376fa60 0x55b7a37733f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:32.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.432+0000 7fce96f37b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7a3770380 0x55b7a3773930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:32.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.432+0000 7fce96f37b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b7a3776e30 con 0x55b7a3770380 2024-05-08T06:10:32.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.432+0000 7fce96f37b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b7a3622ba0 con 0x55b7a376a680 2024-05-08T06:10:32.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.432+0000 7fce96f37b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b7a3776fb0 con 0x55b7a376fa60 2024-05-08T06:10:32.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.432+0000 7fce96f37b80 10 client.0 _ll_register_callbacks cb 0x55b7a3663cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:10:32.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.433+0000 7fce89c00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b7a376a680 0x55b7a3776050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:32.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.433+0000 7fce8a600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7a3770380 0x55b7a3773930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:32.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.433+0000 7fce89c00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b7a376a680 0x55b7a3776050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:46190/0 (socket says 172.21.15.123:46190) 2024-05-08T06:10:32.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.433+0000 7fce89c00700 1 -- 172.21.15.123:0/3472149425 learned_addr learned my addr 172.21.15.123:0/3472149425 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:10:32.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.433+0000 7fce89200700 1 --2- 172.21.15.123:0/3472149425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7a376fa60 0x55b7a37733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:32.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.433+0000 7fce89c00700 1 -- 172.21.15.123:0/3472149425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7a376fa60 msgr2=0x55b7a37733f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:32.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.433+0000 7fce89c00700 1 --2- 172.21.15.123:0/3472149425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7a376fa60 0x55b7a37733f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:32.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.433+0000 7fce89c00700 1 -- 172.21.15.123:0/3472149425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7a3770380 msgr2=0x55b7a3773930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:32.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.433+0000 7fce89c00700 1 --2- 172.21.15.123:0/3472149425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7a3770380 0x55b7a3773930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:32.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.433+0000 7fce89c00700 1 -- 172.21.15.123:0/3472149425 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7a36824a0 con 0x55b7a376a680 2024-05-08T06:10:32.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.433+0000 7fce8a600700 1 --2- 172.21.15.123:0/3472149425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7a3770380 0x55b7a3773930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:10:32.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.433+0000 7fce89c00700 1 --2- 172.21.15.123:0/3472149425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b7a376a680 0x55b7a3776050 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fce78004800 tx=0x7fce7800cf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82e0b1e24ffc631e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:32.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.433+0000 7fce6b400700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-05-08T06:10:32.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.434+0000 7fce6b400700 1 -- 172.21.15.123:0/3472149425 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce780a5170 con 0x55b7a376a680 2024-05-08T06:10:32.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.434+0000 7fce6b400700 1 -- 172.21.15.123:0/3472149425 <== mon.2 v2:172.21.15.163:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fce780cc0d0 con 0x55b7a376a680 2024-05-08T06:10:32.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.434+0000 7fce96f37b80 1 -- 172.21.15.123:0/3472149425 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce780b2190 con 0x55b7a376a680 2024-05-08T06:10:32.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.434+0000 7fce6b400700 1 -- 172.21.15.123:0/3472149425 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fce780a99e0 con 0x55b7a376a680 2024-05-08T06:10:32.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.434+0000 7fce96f37b80 10 client.25121 fetch_fsmap learned FSMap version 24 2024-05-08T06:10:32.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.434+0000 7fce96f37b80 1 -- 172.21.15.123:0/3472149425 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b7a3781630 con 0x55b7a376a680 2024-05-08T06:10:32.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.434+0000 7fce6b400700 1 -- 172.21.15.123:0/3472149425 <== mon.2 v2:172.21.15.163:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce780b2190 con 0x55b7a376a680 2024-05-08T06:10:32.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.434+0000 7fce6b400700 1 -- 172.21.15.123:0/3472149425 <== mon.2 v2:172.21.15.163:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fce7808b020 con 0x55b7a376a680 2024-05-08T06:10:32.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.435+0000 7fce96f37b80 10 client.25121 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:10:32.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.435+0000 7fce96f37b80 10 client.25121 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T06:10:32.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.435+0000 7fce96f37b80 20 client.25121 populate_metadata read hostname 'smithi123' 2024-05-08T06:10:32.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.435+0000 7fce96f37b80 1 --2- 172.21.15.123:0/3472149425 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55b7a3783bf0 0x55b7a3785fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:32.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.435+0000 7fce96f37b80 4 client.25121 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:10:32.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.435+0000 7fce96f37b80 1 -- 172.21.15.123:0/3472149425 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7a3737120 con 0x55b7a3783bf0 2024-05-08T06:10:32.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.435+0000 7fce89200700 1 --2- 172.21.15.123:0/3472149425 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55b7a3783bf0 0x55b7a3785fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:32.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.436+0000 7fce6b400700 10 client.25121 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:10:32.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.436+0000 7fce89200700 1 --2- 172.21.15.123:0/3472149425 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55b7a3783bf0 0x55b7a3785fe0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bd8e5508a0bdf94 server_cookie=98fe9de104662e84 in_seq=0 out_seq=0 2024-05-08T06:10:32.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.436+0000 7fce6b400700 1 -- 172.21.15.123:0/3472149425 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b7a3737120 con 0x55b7a3783bf0 2024-05-08T06:10:32.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.436+0000 7fce6b400700 10 client.25121 handle_command_reply: tid=0 2024-05-08T06:10:32.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.437+0000 7fce96f37b80 1 client.25121 shutdown 2024-05-08T06:10:32.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.437+0000 7fce96f37b80 1 -- 172.21.15.123:0/3472149425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b7a376a680 msgr2=0x55b7a3776050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:32.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.437+0000 7fce96f37b80 1 --2- 172.21.15.123:0/3472149425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b7a376a680 0x55b7a3776050 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fce78004800 tx=0x7fce7800cf30 comp rx=0 tx=0).stop 2024-05-08T06:10:32.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.437+0000 7fce96f37b80 1 -- 172.21.15.123:0/3472149425 shutdown_connections 2024-05-08T06:10:32.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.437+0000 7fce96f37b80 1 --2- 172.21.15.123:0/3472149425 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55b7a3783bf0 0x55b7a3785fe0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:32.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.437+0000 7fce96f37b80 1 --2- 172.21.15.123:0/3472149425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7a3770380 0x55b7a3773930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:32.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.437+0000 7fce96f37b80 1 --2- 172.21.15.123:0/3472149425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b7a376a680 0x55b7a3776050 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:32.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.437+0000 7fce96f37b80 1 --2- 172.21.15.123:0/3472149425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7a376fa60 0x55b7a37733f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:32.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.437+0000 7fce6b400700 0 client.25121 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:10:32.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.437+0000 7fce96f37b80 1 -- 172.21.15.123:0/3472149425 >> 172.21.15.123:0/3472149425 conn(0x55b7a3698be0 msgr2=0x55b7a375f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:32.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.437+0000 7fce96f37b80 1 -- 172.21.15.123:0/3472149425 shutdown_connections 2024-05-08T06:10:32.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.437+0000 7fce96f37b80 1 -- 172.21.15.123:0/3472149425 wait complete. 2024-05-08T06:10:32.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.437+0000 7fce96f37b80 20 client.25121 trim_cache size 0 max 16384 2024-05-08T06:10:32.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.460+0000 7fce96f37b80 1 Processor -- start 2024-05-08T06:10:32.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.460+0000 7fce96f37b80 1 -- start start 2024-05-08T06:10:32.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.460+0000 7fce96f37b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7a376a680 0x55b7a376a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:32.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.460+0000 7fce96f37b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7a376fa60 0x55b7a3770eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:32.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.460+0000 7fce96f37b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b7a3770380 0x55b7a3789000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:32.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.460+0000 7fce96f37b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fce841b25a0 con 0x55b7a376fa60 2024-05-08T06:10:32.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.460+0000 7fce96f37b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fce841afe30 con 0x55b7a3770380 2024-05-08T06:10:32.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.460+0000 7fce96f37b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fce841adc80 con 0x55b7a376a680 2024-05-08T06:10:32.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.460+0000 7fce89200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7a376fa60 0x55b7a3770eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:32.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.461+0000 7fce89200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7a376fa60 0x55b7a3770eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53420/0 (socket says 172.21.15.123:53420) 2024-05-08T06:10:32.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.461+0000 7fce89200700 1 -- 172.21.15.123:0/4073216428 learned_addr learned my addr 172.21.15.123:0/4073216428 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:10:32.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.461+0000 7fce8a600700 1 --2- 172.21.15.123:0/4073216428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b7a3770380 0x55b7a3789000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:32.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.461+0000 7fce89c00700 1 --2- 172.21.15.123:0/4073216428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7a376a680 0x55b7a376a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:32.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.461+0000 7fce89200700 1 -- 172.21.15.123:0/4073216428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7a376a680 msgr2=0x55b7a376a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:32.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.461+0000 7fce89200700 1 --2- 172.21.15.123:0/4073216428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7a376a680 0x55b7a376a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:32.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.461+0000 7fce89200700 1 -- 172.21.15.123:0/4073216428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b7a3770380 msgr2=0x55b7a3789000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:32.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.461+0000 7fce89200700 1 --2- 172.21.15.123:0/4073216428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b7a3770380 0x55b7a3789000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:32.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.461+0000 7fce89200700 1 -- 172.21.15.123:0/4073216428 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7a37723f0 con 0x55b7a376fa60 2024-05-08T06:10:32.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.461+0000 7fce89200700 1 --2- 172.21.15.123:0/4073216428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7a376fa60 0x55b7a3770eb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fce70009990 tx=0x7fce7000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec6c6bebd2cb5a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:32.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.462+0000 7fce76000700 1 -- 172.21.15.123:0/4073216428 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7000b920 con 0x55b7a376fa60 2024-05-08T06:10:32.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.462+0000 7fce76000700 1 -- 172.21.15.123:0/4073216428 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce70003680 con 0x55b7a376fa60 2024-05-08T06:10:32.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.462+0000 7fce76000700 1 -- 172.21.15.123:0/4073216428 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce70029430 con 0x55b7a376fa60 2024-05-08T06:10:32.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.462+0000 7fce96f37b80 1 -- 172.21.15.123:0/4073216428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7a376fa60 msgr2=0x55b7a3770eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:32.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.462+0000 7fce96f37b80 1 --2- 172.21.15.123:0/4073216428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7a376fa60 0x55b7a3770eb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fce70009990 tx=0x7fce7000b260 comp rx=0 tx=0).stop 2024-05-08T06:10:32.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.462+0000 7fce96f37b80 1 -- 172.21.15.123:0/4073216428 shutdown_connections 2024-05-08T06:10:32.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.463+0000 7fce96f37b80 1 --2- 172.21.15.123:0/4073216428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b7a3770380 0x55b7a3789000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:32.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.463+0000 7fce96f37b80 1 --2- 172.21.15.123:0/4073216428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7a376a680 0x55b7a376a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:32.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.463+0000 7fce96f37b80 1 --2- 172.21.15.123:0/4073216428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7a376fa60 0x55b7a3770eb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:32.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.463+0000 7fce96f37b80 1 -- 172.21.15.123:0/4073216428 >> 172.21.15.123:0/4073216428 conn(0x55b7a3698be0 msgr2=0x55b7a375f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:32.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.463+0000 7fce96f37b80 1 -- 172.21.15.123:0/4073216428 shutdown_connections 2024-05-08T06:10:32.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.463+0000 7fce96f37b80 1 -- 172.21.15.123:0/4073216428 wait complete. 2024-05-08T06:10:32.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.463+0000 7fce96f37b80 1 -- start start 2024-05-08T06:10:32.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.464+0000 7fce96f37b80 1 Processor -- start 2024-05-08T06:10:32.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.464+0000 7fce96f37b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b7a376a680 0x55b7a3787d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:32.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.464+0000 7fce96f37b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7a376fa60 0x55b7a3788250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:32.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.464+0000 7fce96f37b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7a3770380 0x55b7a3788790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:32.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.464+0000 7fce96f37b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fce841ae2f0 con 0x55b7a376fa60 2024-05-08T06:10:32.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.464+0000 7fce96f37b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fce841af760 con 0x55b7a376a680 2024-05-08T06:10:32.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.464+0000 7fce96f37b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fce841ae9c0 con 0x55b7a3770380 2024-05-08T06:10:32.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.464+0000 7fce96f37b80 10 client.0 _ll_register_callbacks cb 0x55b7a3663cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:10:32.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.465+0000 7fce89200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7a376fa60 0x55b7a3788250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:32.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.465+0000 7fce89c00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b7a376a680 0x55b7a3787d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:32.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.465+0000 7fce89200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7a376fa60 0x55b7a3788250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53426/0 (socket says 172.21.15.123:53426) 2024-05-08T06:10:32.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.465+0000 7fce89200700 1 -- 172.21.15.123:0/3015496080 learned_addr learned my addr 172.21.15.123:0/3015496080 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:10:32.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.465+0000 7fce8a600700 1 --2- 172.21.15.123:0/3015496080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7a3770380 0x55b7a3788790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:32.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.465+0000 7fce89200700 1 -- 172.21.15.123:0/3015496080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7a3770380 msgr2=0x55b7a3788790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:32.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.465+0000 7fce89200700 1 --2- 172.21.15.123:0/3015496080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7a3770380 0x55b7a3788790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:32.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.465+0000 7fce89200700 1 -- 172.21.15.123:0/3015496080 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b7a376a680 msgr2=0x55b7a3787d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:32.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.465+0000 7fce89200700 1 --2- 172.21.15.123:0/3015496080 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b7a376a680 0x55b7a3787d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:32.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.465+0000 7fce89200700 1 -- 172.21.15.123:0/3015496080 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7a37ee260 con 0x55b7a376fa60 2024-05-08T06:10:32.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.465+0000 7fce89c00700 1 --2- 172.21.15.123:0/3015496080 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b7a376a680 0x55b7a3787d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:10:32.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.465+0000 7fce89200700 1 --2- 172.21.15.123:0/3015496080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7a376fa60 0x55b7a3788250 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fce700098e0 tx=0x7fce70005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda39c6e8a5ad4cd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:32.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.465+0000 7fce6b400700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-08T06:10:32.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.466+0000 7fce6b400700 1 -- 172.21.15.123:0/3015496080 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce70039020 con 0x55b7a376fa60 2024-05-08T06:10:32.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.466+0000 7fce6b400700 1 -- 172.21.15.123:0/3015496080 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fce70003a30 con 0x55b7a376fa60 2024-05-08T06:10:32.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.466+0000 7fce96f37b80 1 -- 172.21.15.123:0/3015496080 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce70003680 con 0x55b7a376fa60 2024-05-08T06:10:32.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.466+0000 7fce6b400700 1 -- 172.21.15.123:0/3015496080 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fce7000b720 con 0x55b7a376fa60 2024-05-08T06:10:32.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.466+0000 7fce96f37b80 10 client.15012 fetch_fsmap learned FSMap version 24 2024-05-08T06:10:32.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.466+0000 7fce96f37b80 1 -- 172.21.15.123:0/3015496080 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b7a37ef580 con 0x55b7a376fa60 2024-05-08T06:10:32.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.466+0000 7fce6b400700 1 -- 172.21.15.123:0/3015496080 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce70003680 con 0x55b7a376fa60 2024-05-08T06:10:32.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.467+0000 7fce6b400700 1 -- 172.21.15.123:0/3015496080 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fce70029490 con 0x55b7a376fa60 2024-05-08T06:10:32.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.467+0000 7fce96f37b80 10 client.15012 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:10:32.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.467+0000 7fce96f37b80 10 client.15012 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T06:10:32.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.467+0000 7fce96f37b80 20 client.15012 populate_metadata read hostname 'smithi123' 2024-05-08T06:10:32.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.467+0000 7fce96f37b80 1 --2- 172.21.15.123:0/3015496080 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55b7a37efb80 0x55b7a37f1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:32.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.467+0000 7fce96f37b80 4 client.15012 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T06:10:32.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.467+0000 7fce96f37b80 1 -- 172.21.15.123:0/3015496080 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b7a36ca480 con 0x55b7a37efb80 2024-05-08T06:10:32.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.467+0000 7fce89c00700 1 --2- 172.21.15.123:0/3015496080 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55b7a37efb80 0x55b7a37f1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:32.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.468+0000 7fce89c00700 1 --2- 172.21.15.123:0/3015496080 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55b7a37efb80 0x55b7a37f1fa0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8740302cf13708b server_cookie=43db5db9de605052 in_seq=0 out_seq=0 2024-05-08T06:10:32.471 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.468+0000 7fce6b400700 10 client.15012 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:10:32.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.469+0000 7fce6b400700 1 -- 172.21.15.123:0/3015496080 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b7a36ca480 con 0x55b7a37efb80 2024-05-08T06:10:32.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.469+0000 7fce6b400700 10 client.15012 handle_command_reply: tid=0 2024-05-08T06:10:32.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.469+0000 7fce96f37b80 1 client.15012 shutdown 2024-05-08T06:10:32.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.469+0000 7fce96f37b80 1 -- 172.21.15.123:0/3015496080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7a376fa60 msgr2=0x55b7a3788250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:32.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.469+0000 7fce96f37b80 1 --2- 172.21.15.123:0/3015496080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7a376fa60 0x55b7a3788250 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fce700098e0 tx=0x7fce70005c40 comp rx=0 tx=0).stop 2024-05-08T06:10:32.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.469+0000 7fce96f37b80 1 -- 172.21.15.123:0/3015496080 shutdown_connections 2024-05-08T06:10:32.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.469+0000 7fce96f37b80 1 --2- 172.21.15.123:0/3015496080 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55b7a37efb80 0x55b7a37f1fa0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:32.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.469+0000 7fce96f37b80 1 --2- 172.21.15.123:0/3015496080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b7a3770380 0x55b7a3788790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:32.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.469+0000 7fce96f37b80 1 --2- 172.21.15.123:0/3015496080 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b7a376a680 0x55b7a3787d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:32.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.469+0000 7fce96f37b80 1 --2- 172.21.15.123:0/3015496080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7a376fa60 0x55b7a3788250 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:32.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.469+0000 7fce6b400700 0 client.15012 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:10:32.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.469+0000 7fce96f37b80 1 -- 172.21.15.123:0/3015496080 >> 172.21.15.123:0/3015496080 conn(0x55b7a3698be0 msgr2=0x55b7a375f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:32.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.469+0000 7fce96f37b80 1 -- 172.21.15.123:0/3015496080 shutdown_connections 2024-05-08T06:10:32.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.469+0000 7fce96f37b80 1 -- 172.21.15.123:0/3015496080 wait complete. 2024-05-08T06:10:32.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.470+0000 7fce96f37b80 20 client.15012 trim_cache size 0 max 16384 2024-05-08T06:10:32.473 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-05-08T06:10:32.473 INFO:teuthology.orchestra.run.smithi123.stdout: "return_code": 0, 2024-05-08T06:10:32.473 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_tag": "5441d9f6-055e-4110-a4e7-ac246dc797cd", 2024-05-08T06:10:32.473 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": "asynchronous" 2024-05-08T06:10:32.473 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-05-08T06:10:32.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.470+0000 7fce8b000700 1 -- 172.21.15.123:0/2996296673 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fce6c0b2990 msgr2=0x7fce6c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:32.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.470+0000 7fce8b000700 1 --2- 172.21.15.123:0/2996296673 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fce6c0b2990 0x7fce6c0b4e50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fce7c009fd0 tx=0x7fce7c00fc40 comp rx=0 tx=0).stop 2024-05-08T06:10:32.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.470+0000 7fce8b000700 1 -- 172.21.15.123:0/2996296673 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce841055d0 msgr2=0x7fce841a18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:32.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.470+0000 7fce8b000700 1 --2- 172.21.15.123:0/2996296673 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce841055d0 0x7fce841a18c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fce7800e970 tx=0x7fce7800ec80 comp rx=0 tx=0).stop 2024-05-08T06:10:32.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.470+0000 7fce8b000700 1 -- 172.21.15.123:0/2996296673 shutdown_connections 2024-05-08T06:10:32.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.470+0000 7fce8b000700 1 --2- 172.21.15.123:0/2996296673 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fce6c0b2990 0x7fce6c0b4e50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:32.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.470+0000 7fce8b000700 1 --2- 172.21.15.123:0/2996296673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fce8410b660 0x7fce8419ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:32.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.470+0000 7fce8b000700 1 --2- 172.21.15.123:0/2996296673 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce841055d0 0x7fce841a18c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:32.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.470+0000 7fce8b000700 1 --2- 172.21.15.123:0/2996296673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce84105f90 0x7fce841a1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:32.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.470+0000 7fce8b000700 1 -- 172.21.15.123:0/2996296673 >> 172.21.15.123:0/2996296673 conn(0x7fce841010c0 msgr2=0x7fce841027e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:32.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.471+0000 7fce8b000700 1 -- 172.21.15.123:0/2996296673 shutdown_connections 2024-05-08T06:10:32.475 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:32.471+0000 7fce8b000700 1 -- 172.21.15.123:0/2996296673 wait complete. 2024-05-08T06:10:32.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:32 smithi163 ceph-mon[34520]: pgmap v462: 97 pgs: 97 active+clean; 653 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-08T06:10:32.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:32 smithi192 ceph-mon[34664]: pgmap v462: 97 pgs: 97 active+clean; 653 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-08T06:10:33.332 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-08T06:10:33.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:33 smithi163 ceph-mon[34520]: scrub queued for path: / 2024-05-08T06:10:33.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:33 smithi163 ceph-mon[34520]: scrub summary: idle+waiting paths [/] 2024-05-08T06:10:33.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:33 smithi163 ceph-mon[34520]: scrub summary: active paths [/] 2024-05-08T06:10:33.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:33 smithi163 ceph-mon[34520]: scrub summary: idle+waiting paths [/] 2024-05-08T06:10:33.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:33 smithi192 ceph-mon[34664]: scrub queued for path: / 2024-05-08T06:10:33.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:33 smithi192 ceph-mon[34664]: scrub summary: idle+waiting paths [/] 2024-05-08T06:10:33.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:33 smithi192 ceph-mon[34664]: scrub summary: active paths [/] 2024-05-08T06:10:33.551 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:33 smithi192 ceph-mon[34664]: scrub summary: idle+waiting paths [/] 2024-05-08T06:10:33.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:33 smithi123 ceph-mon[29787]: scrub queued for path: / 2024-05-08T06:10:33.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:33 smithi123 ceph-mon[29787]: scrub summary: idle+waiting paths [/] 2024-05-08T06:10:33.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:33 smithi123 ceph-mon[29787]: scrub summary: active paths [/] 2024-05-08T06:10:33.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:33 smithi123 ceph-mon[29787]: scrub summary: idle+waiting paths [/] 2024-05-08T06:10:33.778 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:10:34.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:34 smithi123 ceph-mon[29787]: pgmap v463: 97 pgs: 97 active+clean; 653 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-08T06:10:34.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:34 smithi163 ceph-mon[34520]: pgmap v463: 97 pgs: 97 active+clean; 653 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-08T06:10:34.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:34 smithi192 ceph-mon[34664]: pgmap v463: 97 pgs: 97 active+clean; 653 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-08T06:10:36.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.446+0000 7f43ab000700 1 -- 172.21.15.123:0/2445746969 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43a410b6c0 msgr2=0x7f43a410baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:36.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.446+0000 7f43ab000700 1 --2- 172.21.15.123:0/2445746969 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43a410b6c0 0x7f43a410baa0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4394009990 tx=0x7f4394009ca0 comp rx=0 tx=0).stop 2024-05-08T06:10:36.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.446+0000 7f43ab000700 1 -- 172.21.15.123:0/2445746969 shutdown_connections 2024-05-08T06:10:36.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.446+0000 7f43ab000700 1 --2- 172.21.15.123:0/2445746969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43a4106080 0x7f43a4112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:36.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.446+0000 7f43ab000700 1 --2- 172.21.15.123:0/2445746969 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43a410b6c0 0x7f43a410baa0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:36.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.446+0000 7f43ab000700 1 --2- 172.21.15.123:0/2445746969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43a41056c0 0x7f43a4105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:36.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.446+0000 7f43ab000700 1 -- 172.21.15.123:0/2445746969 >> 172.21.15.123:0/2445746969 conn(0x7f43a41011b0 msgr2=0x7f43a41035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:36.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.446+0000 7f43ab000700 1 -- 172.21.15.123:0/2445746969 shutdown_connections 2024-05-08T06:10:36.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.446+0000 7f43ab000700 1 -- 172.21.15.123:0/2445746969 wait complete. 2024-05-08T06:10:36.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.447+0000 7f43ab000700 1 Processor -- start 2024-05-08T06:10:36.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.447+0000 7f43ab000700 1 -- start start 2024-05-08T06:10:36.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.448+0000 7f43ab000700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43a41056c0 0x7f43a40779b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:36.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.448+0000 7f43ab000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43a4106080 0x7f43a4077ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:36.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.448+0000 7f43ab000700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43a410b6c0 0x7f43a4078430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:36.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.448+0000 7f43ab000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f43a41158a0 con 0x7f43a4106080 2024-05-08T06:10:36.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.448+0000 7f43ab000700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f43a4115720 con 0x7f43a410b6c0 2024-05-08T06:10:36.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.448+0000 7f43ab000700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f43a4115a20 con 0x7f43a41056c0 2024-05-08T06:10:36.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.448+0000 7f43a3400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43a4106080 0x7f43a4077ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:36.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.448+0000 7f43a3e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43a41056c0 0x7f43a40779b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:36.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.448+0000 7f43a3400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43a4106080 0x7f43a4077ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54094/0 (socket says 172.21.15.123:54094) 2024-05-08T06:10:36.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.448+0000 7f43a3400700 1 -- 172.21.15.123:0/2592254138 learned_addr learned my addr 172.21.15.123:0/2592254138 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:10:36.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.448+0000 7f43a9000700 1 --2- 172.21.15.123:0/2592254138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43a410b6c0 0x7f43a4078430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:36.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.449+0000 7f43a3400700 1 -- 172.21.15.123:0/2592254138 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43a41056c0 msgr2=0x7f43a40779b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:36.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.449+0000 7f43a3400700 1 --2- 172.21.15.123:0/2592254138 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43a41056c0 0x7f43a40779b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:36.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.449+0000 7f43a3400700 1 -- 172.21.15.123:0/2592254138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43a410b6c0 msgr2=0x7f43a4078430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:36.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.449+0000 7f43a3400700 1 --2- 172.21.15.123:0/2592254138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43a410b6c0 0x7f43a4078430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:36.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.449+0000 7f43a3400700 1 -- 172.21.15.123:0/2592254138 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a41ac420 con 0x7f43a4106080 2024-05-08T06:10:36.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.449+0000 7f43a3400700 1 --2- 172.21.15.123:0/2592254138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43a4106080 0x7f43a4077ef0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f439000d950 tx=0x7f439000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:36.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.449+0000 7f43a0c00700 1 -- 172.21.15.123:0/2592254138 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4390016070 con 0x7f43a4106080 2024-05-08T06:10:36.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.449+0000 7f43ab000700 1 -- 172.21.15.123:0/2592254138 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43a41ac710 con 0x7f43a4106080 2024-05-08T06:10:36.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.449+0000 7f43a0c00700 1 -- 172.21.15.123:0/2592254138 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4390009df0 con 0x7f43a4106080 2024-05-08T06:10:36.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.449+0000 7f43a0c00700 1 -- 172.21.15.123:0/2592254138 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439000f620 con 0x7f43a4106080 2024-05-08T06:10:36.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.450+0000 7f43ab000700 1 -- 172.21.15.123:0/2592254138 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43a41acc50 con 0x7f43a4106080 2024-05-08T06:10:36.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.454+0000 7f43a0c00700 1 -- 172.21.15.123:0/2592254138 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 19) ==== 149704+0+0 (secure 0 0 0) 0x7f4390019070 con 0x7f43a4106080 2024-05-08T06:10:36.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.455+0000 7f43b59adb80 1 Processor -- start 2024-05-08T06:10:36.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.455+0000 7f43b59adb80 1 -- start start 2024-05-08T06:10:36.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.455+0000 7f43b59adb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586d8518bc0 0x5586d8518fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:36.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.455+0000 7f43b59adb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586d85194e0 0x5586d85132d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:36.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.455+0000 7f43b59adb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586d85138f0 0x5586d8513d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:36.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.455+0000 7f43b59adb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5586d8465a20 con 0x5586d85138f0 2024-05-08T06:10:36.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.455+0000 7f43b59adb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5586d8403760 con 0x5586d85194e0 2024-05-08T06:10:36.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.455+0000 7f43b59adb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5586d84619e0 con 0x5586d8518bc0 2024-05-08T06:10:36.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.455+0000 7f43a3400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586d85138f0 0x5586d8513d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:36.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.456+0000 7f43a3400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586d85138f0 0x5586d8513d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54110/0 (socket says 172.21.15.123:54110) 2024-05-08T06:10:36.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.456+0000 7f43a3e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586d85194e0 0x5586d85132d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:36.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.456+0000 7f43a3400700 1 -- 172.21.15.123:0/1152227005 learned_addr learned my addr 172.21.15.123:0/1152227005 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:10:36.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.456+0000 7f43a9000700 1 --2- 172.21.15.123:0/1152227005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586d8518bc0 0x5586d8518fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:36.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.456+0000 7f43a0c00700 1 --2- 172.21.15.123:0/2592254138 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f43880b2910 0x7f43880b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:36.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.456+0000 7f43a3e00700 1 -- 172.21.15.123:0/1152227005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586d8518bc0 msgr2=0x5586d8518fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:36.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.456+0000 7f43a3e00700 1 --2- 172.21.15.123:0/1152227005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586d8518bc0 0x5586d8518fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:36.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.456+0000 7f43a3e00700 1 -- 172.21.15.123:0/1152227005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586d85138f0 msgr2=0x5586d8513d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:36.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.456+0000 7f43a3e00700 1 --2- 172.21.15.123:0/1152227005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586d85138f0 0x5586d8513d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:36.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.456+0000 7f43a3e00700 1 -- 172.21.15.123:0/1152227005 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586d83b28d0 con 0x5586d85194e0 2024-05-08T06:10:36.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.456+0000 7f43a0c00700 1 -- 172.21.15.123:0/2592254138 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f43900e1fe0 con 0x7f43a4106080 2024-05-08T06:10:36.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.456+0000 7f43a3e00700 1 --2- 172.21.15.123:0/1152227005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586d85194e0 0x5586d85132d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4394006010 tx=0x7f4394005bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36653fe088f5137d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:36.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.456+0000 7f43a9000700 1 --2- 172.21.15.123:0/2592254138 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f43880b2910 0x7f43880b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:36.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.457+0000 7f4399600700 1 -- 172.21.15.123:0/1152227005 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4394027070 con 0x5586d85194e0 2024-05-08T06:10:36.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.457+0000 7f4399600700 1 -- 172.21.15.123:0/1152227005 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43940056a0 con 0x5586d85194e0 2024-05-08T06:10:36.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.457+0000 7f4399600700 1 -- 172.21.15.123:0/1152227005 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4394003b50 con 0x5586d85194e0 2024-05-08T06:10:36.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.458+0000 7f43b59adb80 1 -- 172.21.15.123:0/1152227005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586d85194e0 msgr2=0x5586d85132d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:36.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.458+0000 7f43a9000700 1 --2- 172.21.15.123:0/2592254138 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f43880b2910 0x7f43880b4dd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f439c0075c0 tx=0x7f439c009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:36.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.458+0000 7f43b59adb80 1 --2- 172.21.15.123:0/1152227005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586d85194e0 0x5586d85132d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4394006010 tx=0x7f4394005bc0 comp rx=0 tx=0).stop 2024-05-08T06:10:36.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.458+0000 7f43b59adb80 1 -- 172.21.15.123:0/1152227005 shutdown_connections 2024-05-08T06:10:36.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.458+0000 7f43b59adb80 1 --2- 172.21.15.123:0/1152227005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586d85138f0 0x5586d8513d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:36.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.458+0000 7f43b59adb80 1 --2- 172.21.15.123:0/1152227005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586d8518bc0 0x5586d8518fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:36.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.458+0000 7f43b59adb80 1 --2- 172.21.15.123:0/1152227005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586d85194e0 0x5586d85132d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:36.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.458+0000 7f43b59adb80 1 -- 172.21.15.123:0/1152227005 >> 172.21.15.123:0/1152227005 conn(0x5586d848b6d0 msgr2=0x5586d850db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:36.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.458+0000 7f43b59adb80 1 -- 172.21.15.123:0/1152227005 shutdown_connections 2024-05-08T06:10:36.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.458+0000 7f43b59adb80 1 -- 172.21.15.123:0/1152227005 wait complete. 2024-05-08T06:10:36.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.458+0000 7f43b59adb80 1 -- start start 2024-05-08T06:10:36.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.459+0000 7f43b59adb80 1 Processor -- start 2024-05-08T06:10:36.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.459+0000 7f43b59adb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586d85138f0 0x5586d851f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:36.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.459+0000 7f43b59adb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586d8518bc0 0x5586d851c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:36.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.459+0000 7f43b59adb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586d85194e0 0x5586d851ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:36.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.459+0000 7f43b59adb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5586d851fec0 con 0x5586d85194e0 2024-05-08T06:10:36.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.459+0000 7f43b59adb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5586d83cbb20 con 0x5586d8518bc0 2024-05-08T06:10:36.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.459+0000 7f43b59adb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5586d8520040 con 0x5586d85138f0 2024-05-08T06:10:36.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.459+0000 7f43b59adb80 10 client.0 _ll_register_callbacks cb 0x5586d840cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:10:36.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.459+0000 7f43a3400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586d8518bc0 0x5586d851c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:36.464 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.460+0000 7f43a3400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586d8518bc0 0x5586d851c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:53102/0 (socket says 172.21.15.123:53102) 2024-05-08T06:10:36.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.460+0000 7f43a3400700 1 -- 172.21.15.123:0/2192072561 learned_addr learned my addr 172.21.15.123:0/2192072561 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:10:36.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.460+0000 7f43a3e00700 1 --2- 172.21.15.123:0/2192072561 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586d85138f0 0x5586d851f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:36.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.460+0000 7f43a9000700 1 --2- 172.21.15.123:0/2192072561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586d85194e0 0x5586d851ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:36.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.460+0000 7f43a3e00700 1 -- 172.21.15.123:0/2192072561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586d8518bc0 msgr2=0x5586d851c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:36.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.460+0000 7f43a3e00700 1 --2- 172.21.15.123:0/2192072561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586d8518bc0 0x5586d851c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:36.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.460+0000 7f43a3e00700 1 -- 172.21.15.123:0/2192072561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586d85194e0 msgr2=0x5586d851ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:36.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.460+0000 7f43a3e00700 1 --2- 172.21.15.123:0/2192072561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586d85194e0 0x5586d851ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:36.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.460+0000 7f43a3e00700 1 -- 172.21.15.123:0/2192072561 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586d842b420 con 0x5586d85138f0 2024-05-08T06:10:36.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.460+0000 7f43a3400700 1 --2- 172.21.15.123:0/2192072561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586d8518bc0 0x5586d851c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:10:36.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.460+0000 7f43a3e00700 1 --2- 172.21.15.123:0/2192072561 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586d85138f0 0x5586d851f1b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f43940098e0 tx=0x7f439400bb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92592b876f0f4d54 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:36.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.460+0000 7f438e000700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-05-08T06:10:36.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.461+0000 7f438e000700 1 -- 172.21.15.123:0/2192072561 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439402f020 con 0x5586d85138f0 2024-05-08T06:10:36.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.461+0000 7f438e000700 1 -- 172.21.15.123:0/2192072561 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f4394003680 con 0x5586d85138f0 2024-05-08T06:10:36.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.461+0000 7f43b59adb80 1 -- 172.21.15.123:0/2192072561 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43940056a0 con 0x5586d85138f0 2024-05-08T06:10:36.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.461+0000 7f438e000700 1 -- 172.21.15.123:0/2192072561 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4394027040 con 0x5586d85138f0 2024-05-08T06:10:36.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.461+0000 7f43b59adb80 10 client.25135 fetch_fsmap learned FSMap version 24 2024-05-08T06:10:36.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.461+0000 7f43b59adb80 1 -- 172.21.15.123:0/2192072561 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5586d851fec0 con 0x5586d85138f0 2024-05-08T06:10:36.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.461+0000 7f438e000700 1 -- 172.21.15.123:0/2192072561 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43940056a0 con 0x5586d85138f0 2024-05-08T06:10:36.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.461+0000 7f438e000700 1 -- 172.21.15.123:0/2192072561 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f4394021070 con 0x5586d85138f0 2024-05-08T06:10:36.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.461+0000 7f43b59adb80 10 client.25135 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:10:36.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.461+0000 7f43b59adb80 10 client.25135 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T06:10:36.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.462+0000 7f43b59adb80 20 client.25135 populate_metadata read hostname 'smithi123' 2024-05-08T06:10:36.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.462+0000 7f43b59adb80 1 --2- 172.21.15.123:0/2192072561 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5586d852ca20 0x5586d852ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:36.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.462+0000 7f43b59adb80 4 client.25135 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:10:36.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.462+0000 7f43b59adb80 1 -- 172.21.15.123:0/2192072561 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586d84e0140 con 0x5586d852ca20 2024-05-08T06:10:36.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.462+0000 7f43a3400700 1 --2- 172.21.15.123:0/2192072561 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5586d852ca20 0x5586d852ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:36.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.462+0000 7f43a3400700 1 --2- 172.21.15.123:0/2192072561 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5586d852ca20 0x5586d852ee10 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=329e79a7ba10d9c9 server_cookie=ec3ad376e33ac9d9 in_seq=0 out_seq=0 2024-05-08T06:10:36.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.462+0000 7f438e000700 10 client.25135 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:10:36.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.463+0000 7f438e000700 1 -- 172.21.15.123:0/2192072561 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5586d84e0140 con 0x5586d852ca20 2024-05-08T06:10:36.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.463+0000 7f438e000700 10 client.25135 handle_command_reply: tid=0 2024-05-08T06:10:36.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.463+0000 7f43b59adb80 1 client.25135 shutdown 2024-05-08T06:10:36.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.463+0000 7f43b59adb80 1 -- 172.21.15.123:0/2192072561 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586d85138f0 msgr2=0x5586d851f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:36.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.463+0000 7f43b59adb80 1 --2- 172.21.15.123:0/2192072561 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586d85138f0 0x5586d851f1b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f43940098e0 tx=0x7f439400bb70 comp rx=0 tx=0).stop 2024-05-08T06:10:36.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.464+0000 7f43b59adb80 1 -- 172.21.15.123:0/2192072561 shutdown_connections 2024-05-08T06:10:36.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.464+0000 7f43b59adb80 1 --2- 172.21.15.123:0/2192072561 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5586d852ca20 0x5586d852ee10 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:36.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.464+0000 7f43b59adb80 1 --2- 172.21.15.123:0/2192072561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586d85194e0 0x5586d851ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:36.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.464+0000 7f438e000700 0 client.25135 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:10:36.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.464+0000 7f43b59adb80 1 --2- 172.21.15.123:0/2192072561 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586d85138f0 0x5586d851f1b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:36.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.464+0000 7f43b59adb80 1 --2- 172.21.15.123:0/2192072561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586d8518bc0 0x5586d851c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:36.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.464+0000 7f43b59adb80 1 -- 172.21.15.123:0/2192072561 >> 172.21.15.123:0/2192072561 conn(0x5586d848b6d0 msgr2=0x5586d850d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:36.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.464+0000 7f43b59adb80 1 -- 172.21.15.123:0/2192072561 shutdown_connections 2024-05-08T06:10:36.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.464+0000 7f43b59adb80 1 -- 172.21.15.123:0/2192072561 wait complete. 2024-05-08T06:10:36.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.464+0000 7f43b59adb80 20 client.25135 trim_cache size 0 max 16384 2024-05-08T06:10:36.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.486+0000 7f43b59adb80 1 Processor -- start 2024-05-08T06:10:36.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.486+0000 7f43b59adb80 1 -- start start 2024-05-08T06:10:36.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.486+0000 7f43b59adb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586d85138f0 0x5586d85132d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:36.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.486+0000 7f43b59adb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586d8518bc0 0x5586d8528f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:36.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.486+0000 7f43b59adb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586d85194e0 0x5586d850d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:36.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.486+0000 7f43b59adb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f43a41b6970 con 0x5586d85138f0 2024-05-08T06:10:36.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.486+0000 7f43b59adb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f43a41b4200 con 0x5586d85194e0 2024-05-08T06:10:36.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.486+0000 7f43b59adb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f43a41b2050 con 0x5586d8518bc0 2024-05-08T06:10:36.487 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:36 smithi123 ceph-mon[29787]: pgmap v464: 97 pgs: 97 active+clean; 659 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-08T06:10:36.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.486+0000 7f43a9000700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586d85194e0 0x5586d850d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:36.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.487+0000 7f43a3400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586d8518bc0 0x5586d8528f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:36.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.487+0000 7f43a9000700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586d85194e0 0x5586d850d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:53110/0 (socket says 172.21.15.123:53110) 2024-05-08T06:10:36.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.487+0000 7f43a9000700 1 -- 172.21.15.123:0/2567670390 learned_addr learned my addr 172.21.15.123:0/2567670390 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:10:36.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.487+0000 7f43a3e00700 1 --2- 172.21.15.123:0/2567670390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586d85138f0 0x5586d85132d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:36.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.487+0000 7f43a9000700 1 -- 172.21.15.123:0/2567670390 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586d8518bc0 msgr2=0x5586d8528f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:36.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.487+0000 7f43a9000700 1 --2- 172.21.15.123:0/2567670390 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586d8518bc0 0x5586d8528f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:36.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.487+0000 7f43a9000700 1 -- 172.21.15.123:0/2567670390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586d85138f0 msgr2=0x5586d85132d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:36.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.487+0000 7f43a9000700 1 --2- 172.21.15.123:0/2567670390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586d85138f0 0x5586d85132d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:36.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.487+0000 7f43a9000700 1 -- 172.21.15.123:0/2567670390 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586d8513f20 con 0x5586d85194e0 2024-05-08T06:10:36.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.487+0000 7f43a9000700 1 --2- 172.21.15.123:0/2567670390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586d85194e0 0x5586d850d910 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f439c0177a0 tx=0x7f439c004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51e70fd569e62ed3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:36.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.488+0000 7f4398c00700 1 -- 172.21.15.123:0/2567670390 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439c0238f0 con 0x5586d85194e0 2024-05-08T06:10:36.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.488+0000 7f4398c00700 1 -- 172.21.15.123:0/2567670390 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f439c028070 con 0x5586d85194e0 2024-05-08T06:10:36.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.488+0000 7f4398c00700 1 -- 172.21.15.123:0/2567670390 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439c022380 con 0x5586d85194e0 2024-05-08T06:10:36.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.488+0000 7f43b59adb80 1 -- 172.21.15.123:0/2567670390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586d85194e0 msgr2=0x5586d850d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:36.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.488+0000 7f43b59adb80 1 --2- 172.21.15.123:0/2567670390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586d85194e0 0x5586d850d910 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f439c0177a0 tx=0x7f439c004b40 comp rx=0 tx=0).stop 2024-05-08T06:10:36.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.488+0000 7f43b59adb80 1 -- 172.21.15.123:0/2567670390 shutdown_connections 2024-05-08T06:10:36.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.488+0000 7f43b59adb80 1 --2- 172.21.15.123:0/2567670390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586d85138f0 0x5586d85132d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:36.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.488+0000 7f43b59adb80 1 --2- 172.21.15.123:0/2567670390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586d85194e0 0x5586d850d910 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:36.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.488+0000 7f43b59adb80 1 --2- 172.21.15.123:0/2567670390 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586d8518bc0 0x5586d8528f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:36.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.488+0000 7f43b59adb80 1 -- 172.21.15.123:0/2567670390 >> 172.21.15.123:0/2567670390 conn(0x5586d848b6d0 msgr2=0x5586d851b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:36.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.489+0000 7f43b59adb80 1 -- 172.21.15.123:0/2567670390 shutdown_connections 2024-05-08T06:10:36.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.489+0000 7f43b59adb80 1 -- 172.21.15.123:0/2567670390 wait complete. 2024-05-08T06:10:36.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.489+0000 7f43b59adb80 1 -- start start 2024-05-08T06:10:36.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.489+0000 7f43b59adb80 1 Processor -- start 2024-05-08T06:10:36.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.490+0000 7f43b59adb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586d85138f0 0x5586d8521640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:36.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.490+0000 7f43b59adb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586d8518bc0 0x5586d8521b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:36.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.490+0000 7f43b59adb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586d85194e0 0x5586d8596070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:36.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.490+0000 7f43b59adb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f43a41b26c0 con 0x5586d85138f0 2024-05-08T06:10:36.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.490+0000 7f43b59adb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f43a41b3b30 con 0x5586d8518bc0 2024-05-08T06:10:36.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.490+0000 7f43b59adb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f43a41b2d90 con 0x5586d85194e0 2024-05-08T06:10:36.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.490+0000 7f43b59adb80 10 client.0 _ll_register_callbacks cb 0x5586d840cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:10:36.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.490+0000 7f43a3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586d85138f0 0x5586d8521640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:36.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.490+0000 7f43a3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586d85138f0 0x5586d8521640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54150/0 (socket says 172.21.15.123:54150) 2024-05-08T06:10:36.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.490+0000 7f43a3e00700 1 -- 172.21.15.123:0/32079293 learned_addr learned my addr 172.21.15.123:0/32079293 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:10:36.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.490+0000 7f43a3400700 1 --2- 172.21.15.123:0/32079293 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586d8518bc0 0x5586d8521b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:36.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.491+0000 7f43a9000700 1 --2- 172.21.15.123:0/32079293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586d85194e0 0x5586d8596070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:36.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.491+0000 7f43a3e00700 1 -- 172.21.15.123:0/32079293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586d85194e0 msgr2=0x5586d8596070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:36.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.491+0000 7f43a3e00700 1 --2- 172.21.15.123:0/32079293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586d85194e0 0x5586d8596070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:36.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.491+0000 7f43a3e00700 1 -- 172.21.15.123:0/32079293 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586d8518bc0 msgr2=0x5586d8521b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:36.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.491+0000 7f43a3e00700 1 --2- 172.21.15.123:0/32079293 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586d8518bc0 0x5586d8521b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:36.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.491+0000 7f43a3e00700 1 -- 172.21.15.123:0/32079293 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586d8596680 con 0x5586d85138f0 2024-05-08T06:10:36.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.491+0000 7f43a3e00700 1 --2- 172.21.15.123:0/32079293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586d85138f0 0x5586d8521640 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4394005c00 tx=0x7f439400f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0bba6c2fcbd686 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:36.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.491+0000 7f438e000700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-08T06:10:36.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.491+0000 7f438e000700 1 -- 172.21.15.123:0/32079293 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439402f070 con 0x5586d85138f0 2024-05-08T06:10:36.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.491+0000 7f438e000700 1 -- 172.21.15.123:0/32079293 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f4394003680 con 0x5586d85138f0 2024-05-08T06:10:36.495 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.491+0000 7f43b59adb80 1 -- 172.21.15.123:0/32079293 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f439c028070 con 0x5586d85138f0 2024-05-08T06:10:36.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.492+0000 7f438e000700 1 -- 172.21.15.123:0/32079293 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4394003880 con 0x5586d85138f0 2024-05-08T06:10:36.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.492+0000 7f43b59adb80 10 client.15030 fetch_fsmap learned FSMap version 24 2024-05-08T06:10:36.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.492+0000 7f43b59adb80 1 -- 172.21.15.123:0/32079293 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5586d8597990 con 0x5586d85138f0 2024-05-08T06:10:36.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.492+0000 7f438e000700 1 -- 172.21.15.123:0/32079293 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f439c028070 con 0x5586d85138f0 2024-05-08T06:10:36.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.492+0000 7f438e000700 1 -- 172.21.15.123:0/32079293 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f4394021040 con 0x5586d85138f0 2024-05-08T06:10:36.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.492+0000 7f43b59adb80 10 client.15030 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:10:36.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.492+0000 7f43b59adb80 10 client.15030 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T06:10:36.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.492+0000 7f43b59adb80 20 client.15030 populate_metadata read hostname 'smithi123' 2024-05-08T06:10:36.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.492+0000 7f43b59adb80 1 --2- 172.21.15.123:0/32079293 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5586d8598090 0x5586d859a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:36.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.493+0000 7f43b59adb80 4 client.15030 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T06:10:36.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.493+0000 7f43b59adb80 1 -- 172.21.15.123:0/32079293 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5586d8473400 con 0x5586d8598090 2024-05-08T06:10:36.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.493+0000 7f43a3400700 1 --2- 172.21.15.123:0/32079293 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5586d8598090 0x5586d859a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:36.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.493+0000 7f43a3400700 1 --2- 172.21.15.123:0/32079293 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5586d8598090 0x5586d859a4b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b9d8382d5d4cd51 server_cookie=d473843a861192d2 in_seq=0 out_seq=0 2024-05-08T06:10:36.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.493+0000 7f438e000700 10 client.15030 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:10:36.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.494+0000 7f438e000700 1 -- 172.21.15.123:0/32079293 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5586d8473400 con 0x5586d8598090 2024-05-08T06:10:36.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.494+0000 7f438e000700 10 client.15030 handle_command_reply: tid=0 2024-05-08T06:10:36.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.494+0000 7f43b59adb80 1 client.15030 shutdown 2024-05-08T06:10:36.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.494+0000 7f43b59adb80 1 -- 172.21.15.123:0/32079293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586d85138f0 msgr2=0x5586d8521640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:36.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.494+0000 7f43b59adb80 1 --2- 172.21.15.123:0/32079293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586d85138f0 0x5586d8521640 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4394005c00 tx=0x7f439400f6f0 comp rx=0 tx=0).stop 2024-05-08T06:10:36.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.494+0000 7f43b59adb80 1 -- 172.21.15.123:0/32079293 shutdown_connections 2024-05-08T06:10:36.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.494+0000 7f43b59adb80 1 --2- 172.21.15.123:0/32079293 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5586d8598090 0x5586d859a4b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:36.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.494+0000 7f43b59adb80 1 --2- 172.21.15.123:0/32079293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586d85138f0 0x5586d8521640 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:36.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.494+0000 7f43b59adb80 1 --2- 172.21.15.123:0/32079293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586d85194e0 0x5586d8596070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:36.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.494+0000 7f43b59adb80 1 --2- 172.21.15.123:0/32079293 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586d8518bc0 0x5586d8521b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:36.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.494+0000 7f43b59adb80 1 -- 172.21.15.123:0/32079293 >> 172.21.15.123:0/32079293 conn(0x5586d848b6d0 msgr2=0x5586d851da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:36.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.494+0000 7f438e000700 0 client.15030 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:10:36.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.494+0000 7f43b59adb80 1 -- 172.21.15.123:0/32079293 shutdown_connections 2024-05-08T06:10:36.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.494+0000 7f43b59adb80 1 -- 172.21.15.123:0/32079293 wait complete. 2024-05-08T06:10:36.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.494+0000 7f43b59adb80 20 client.15030 trim_cache size 0 max 16384 2024-05-08T06:10:36.500 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-05-08T06:10:36.500 INFO:teuthology.orchestra.run.smithi123.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T06:10:36.500 INFO:teuthology.orchestra.run.smithi123.stdout: "scrubs": { 2024-05-08T06:10:36.500 INFO:teuthology.orchestra.run.smithi123.stdout: "5441d9f6-055e-4110-a4e7-ac246dc797cd": { 2024-05-08T06:10:36.500 INFO:teuthology.orchestra.run.smithi123.stdout: "path": "/", 2024-05-08T06:10:36.500 INFO:teuthology.orchestra.run.smithi123.stdout: "tag": "5441d9f6-055e-4110-a4e7-ac246dc797cd", 2024-05-08T06:10:36.500 INFO:teuthology.orchestra.run.smithi123.stdout: "options": "recursive,force" 2024-05-08T06:10:36.500 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:10:36.500 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:10:36.501 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-05-08T06:10:36.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.495+0000 7f43ab000700 1 -- 172.21.15.123:0/2592254138 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f43880b2910 msgr2=0x7f43880b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:36.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.495+0000 7f43ab000700 1 --2- 172.21.15.123:0/2592254138 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f43880b2910 0x7f43880b4dd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f439c0075c0 tx=0x7f439c009040 comp rx=0 tx=0).stop 2024-05-08T06:10:36.501 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.495+0000 7f43ab000700 1 -- 172.21.15.123:0/2592254138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43a4106080 msgr2=0x7f43a4077ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:36.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.495+0000 7f43ab000700 1 --2- 172.21.15.123:0/2592254138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43a4106080 0x7f43a4077ef0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f439000d950 tx=0x7f439000dc60 comp rx=0 tx=0).stop 2024-05-08T06:10:36.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.495+0000 7f43ab000700 1 -- 172.21.15.123:0/2592254138 shutdown_connections 2024-05-08T06:10:36.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.495+0000 7f43ab000700 1 --2- 172.21.15.123:0/2592254138 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f43880b2910 0x7f43880b4dd0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:36.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.495+0000 7f43ab000700 1 --2- 172.21.15.123:0/2592254138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43a410b6c0 0x7f43a4078430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:36.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.495+0000 7f43ab000700 1 --2- 172.21.15.123:0/2592254138 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43a41056c0 0x7f43a40779b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:36.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.495+0000 7f43ab000700 1 --2- 172.21.15.123:0/2592254138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43a4106080 0x7f43a4077ef0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:36.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.495+0000 7f43ab000700 1 -- 172.21.15.123:0/2592254138 >> 172.21.15.123:0/2592254138 conn(0x7f43a41011b0 msgr2=0x7f43a4101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:36.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.495+0000 7f43ab000700 1 -- 172.21.15.123:0/2592254138 shutdown_connections 2024-05-08T06:10:36.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:10:36.495+0000 7f43ab000700 1 -- 172.21.15.123:0/2592254138 wait complete. 2024-05-08T06:10:36.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:36 smithi163 ceph-mon[34520]: pgmap v464: 97 pgs: 97 active+clean; 659 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-08T06:10:36.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:36 smithi192 ceph-mon[34664]: pgmap v464: 97 pgs: 97 active+clean; 659 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-08T06:10:37.466 INFO:tasks.cephfs.filesystem:scrub status for tag:5441d9f6-055e-4110-a4e7-ac246dc797cd - {'path': '/', 'tag': '5441d9f6-055e-4110-a4e7-ac246dc797cd', 'options': 'recursive,force'} 2024-05-08T06:10:38.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:38 smithi163 ceph-mon[34520]: pgmap v465: 97 pgs: 97 active+clean; 659 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-08T06:10:38.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:38 smithi192 ceph-mon[34664]: pgmap v465: 97 pgs: 97 active+clean; 659 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-08T06:10:38.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:38 smithi123 ceph-mon[29787]: pgmap v465: 97 pgs: 97 active+clean; 659 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-08T06:10:40.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:40 smithi163 ceph-mon[34520]: pgmap v466: 97 pgs: 97 active+clean; 659 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-08T06:10:40.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:40 smithi192 ceph-mon[34664]: pgmap v466: 97 pgs: 97 active+clean; 659 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-08T06:10:40.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:40 smithi123 ceph-mon[29787]: pgmap v466: 97 pgs: 97 active+clean; 659 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-08T06:10:42.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:42 smithi163 ceph-mon[34520]: pgmap v467: 97 pgs: 97 active+clean; 659 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-08T06:10:42.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:42 smithi192 ceph-mon[34664]: pgmap v467: 97 pgs: 97 active+clean; 659 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-08T06:10:42.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:42 smithi123 ceph-mon[29787]: pgmap v467: 97 pgs: 97 active+clean; 659 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-08T06:10:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.808+0000 7f37ab200640 1 -- 172.21.15.123:0/1827367707 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37a4108e60 msgr2=0x7f37a4109280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.808+0000 7f37ab200640 1 --2- 172.21.15.123:0/1827367707 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37a4108e60 0x7f37a4109280 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3798009860 tx=0x7f379802a340 comp rx=0 tx=0).stop 2024-05-08T06:10:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.809+0000 7f37ab200640 1 -- 172.21.15.123:0/1827367707 shutdown_connections 2024-05-08T06:10:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.809+0000 7f37ab200640 1 --2- 172.21.15.123:0/1827367707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37a410a970 0x7f37a4111290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.809+0000 7f37ab200640 1 --2- 172.21.15.123:0/1827367707 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37a4109fb0 0x7f37a410a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.809+0000 7f37ab200640 1 --2- 172.21.15.123:0/1827367707 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37a4108e60 0x7f37a4109280 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.809+0000 7f37ab200640 1 -- 172.21.15.123:0/1827367707 >> 172.21.15.123:0/1827367707 conn(0x7f37a4077e40 msgr2=0x7f37a407a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.809+0000 7f37ab200640 1 -- 172.21.15.123:0/1827367707 shutdown_connections 2024-05-08T06:10:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.809+0000 7f37ab200640 1 -- 172.21.15.123:0/1827367707 wait complete. 2024-05-08T06:10:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.810+0000 7f37ab200640 1 Processor -- start 2024-05-08T06:10:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.810+0000 7f37ab200640 1 -- start start 2024-05-08T06:10:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.810+0000 7f37ab200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37a4108e60 0x7f37a419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.810+0000 7f37ab200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37a4109fb0 0x7f37a419f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.810+0000 7f37ab200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37a410a970 0x7f37a41a6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.810+0000 7f37ab200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f37a41149b0 con 0x7f37a4109fb0 2024-05-08T06:10:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.810+0000 7f37ab200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f37a4114830 con 0x7f37a410a970 2024-05-08T06:10:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.810+0000 7f37ab200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f37a4114b30 con 0x7f37a4108e60 2024-05-08T06:10:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.811+0000 7f37a9400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37a4109fb0 0x7f37a419f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.811+0000 7f37aa800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37a410a970 0x7f37a41a6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.811+0000 7f37a9400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37a4109fb0 0x7f37a419f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54158/0 (socket says 172.21.15.123:54158) 2024-05-08T06:10:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.811+0000 7f37a9400640 1 -- 172.21.15.123:0/1150748361 learned_addr learned my addr 172.21.15.123:0/1150748361 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:10:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.811+0000 7f37a9e00640 1 --2- 172.21.15.123:0/1150748361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37a4108e60 0x7f37a419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.811+0000 7f37aa800640 1 -- 172.21.15.123:0/1150748361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37a4108e60 msgr2=0x7f37a419f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.811+0000 7f37aa800640 1 --2- 172.21.15.123:0/1150748361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37a4108e60 0x7f37a419f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.811+0000 7f37aa800640 1 -- 172.21.15.123:0/1150748361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37a4109fb0 msgr2=0x7f37a419f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.811+0000 7f37aa800640 1 --2- 172.21.15.123:0/1150748361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37a4109fb0 0x7f37a419f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.811+0000 7f37aa800640 1 -- 172.21.15.123:0/1150748361 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37a41a6c80 con 0x7f37a410a970 2024-05-08T06:10:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.811+0000 7f37a9400640 1 --2- 172.21.15.123:0/1150748361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37a4109fb0 0x7f37a419f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:10:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.811+0000 7f37aa800640 1 --2- 172.21.15.123:0/1150748361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37a410a970 0x7f37a41a6610 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f37a000ef00 tx=0x7f37a000c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.811+0000 7f379ea00640 1 -- 172.21.15.123:0/1150748361 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a0004b20 con 0x7f37a410a970 2024-05-08T06:10:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.812+0000 7f379ea00640 1 -- 172.21.15.123:0/1150748361 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37a0003e20 con 0x7f37a410a970 2024-05-08T06:10:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.812+0000 7f37ab200640 1 -- 172.21.15.123:0/1150748361 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37a41a6f70 con 0x7f37a410a970 2024-05-08T06:10:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.812+0000 7f379ea00640 1 -- 172.21.15.123:0/1150748361 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a000b570 con 0x7f37a410a970 2024-05-08T06:10:42.819 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.812+0000 7f37ab200640 1 -- 172.21.15.123:0/1150748361 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37a41a7430 con 0x7f37a410a970 2024-05-08T06:10:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.815+0000 7f37af5bc740 1 Processor -- start 2024-05-08T06:10:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.815+0000 7f379ea00640 1 -- 172.21.15.123:0/1150748361 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 19) ==== 149704+0+0 (secure 0 0 0) 0x7f37a001d020 con 0x7f37a410a970 2024-05-08T06:10:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.816+0000 7f37af5bc740 1 -- start start 2024-05-08T06:10:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.816+0000 7f37af5bc740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557866518320 0x557866518740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.816+0000 7f37af5bc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557866518d10 0x55786651d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.816+0000 7f37af5bc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55786651de80 0x557866520310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.816+0000 7f37af5bc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55786647c550 con 0x55786651de80 2024-05-08T06:10:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.816+0000 7f37af5bc740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x557866421bc0 con 0x557866518320 2024-05-08T06:10:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.816+0000 7f37af5bc740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557866478510 con 0x557866518d10 2024-05-08T06:10:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.816+0000 7f37a9400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55786651de80 0x557866520310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.816+0000 7f37a9e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557866518d10 0x55786651d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.816+0000 7f37a9400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55786651de80 0x557866520310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54160/0 (socket says 172.21.15.123:54160) 2024-05-08T06:10:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.816+0000 7f37a9400640 1 -- 172.21.15.123:0/1150218936 learned_addr learned my addr 172.21.15.123:0/1150218936 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:10:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.816+0000 7f37a9400640 1 --2- 172.21.15.123:0/1150218936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557866518320 0x557866518740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.817+0000 7f37a9e00640 1 -- 172.21.15.123:0/1150218936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557866518320 msgr2=0x557866518740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.817+0000 7f37a9e00640 1 --2- 172.21.15.123:0/1150218936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557866518320 0x557866518740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.817+0000 7f37a9e00640 1 -- 172.21.15.123:0/1150218936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55786651de80 msgr2=0x557866520310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.817+0000 7f37a9e00640 1 --2- 172.21.15.123:0/1150218936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55786651de80 0x557866520310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.817+0000 7f37a9e00640 1 -- 172.21.15.123:0/1150218936 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55786618e210 con 0x557866518d10 2024-05-08T06:10:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.817+0000 7f37a9400640 1 --2- 172.21.15.123:0/1150218936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55786651de80 0x557866520310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:10:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.817+0000 7f37a9e00640 1 --2- 172.21.15.123:0/1150218936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557866518d10 0x55786651d810 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3798035040 tx=0x7f3798009490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=331ccd4d8362b8a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.817+0000 7f379ea00640 1 --2- 172.21.15.123:0/1150748361 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f378c0b28f0 0x7f378c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.817+0000 7f3793000640 1 -- 172.21.15.123:0/1150218936 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3798037040 con 0x557866518d10 2024-05-08T06:10:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.818+0000 7f3793000640 1 -- 172.21.15.123:0/1150218936 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3798042070 con 0x557866518d10 2024-05-08T06:10:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.818+0000 7f3793000640 1 -- 172.21.15.123:0/1150218936 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3798003850 con 0x557866518d10 2024-05-08T06:10:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.818+0000 7f379ea00640 1 -- 172.21.15.123:0/1150748361 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f37a00e21b0 con 0x7f37a410a970 2024-05-08T06:10:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.818+0000 7f37a9400640 1 --2- 172.21.15.123:0/1150748361 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f378c0b28f0 0x7f378c0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.818+0000 7f37af5bc740 1 -- 172.21.15.123:0/1150218936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557866518d10 msgr2=0x55786651d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.818+0000 7f37af5bc740 1 --2- 172.21.15.123:0/1150218936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557866518d10 0x55786651d810 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3798035040 tx=0x7f3798009490 comp rx=0 tx=0).stop 2024-05-08T06:10:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.818+0000 7f37a9400640 1 --2- 172.21.15.123:0/1150748361 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f378c0b28f0 0x7f378c0b4db0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x557866519570 tx=0x7f3794009c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.819+0000 7f37af5bc740 1 -- 172.21.15.123:0/1150218936 shutdown_connections 2024-05-08T06:10:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.819+0000 7f37af5bc740 1 --2- 172.21.15.123:0/1150218936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55786651de80 0x557866520310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.819+0000 7f37af5bc740 1 --2- 172.21.15.123:0/1150218936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557866518d10 0x55786651d810 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.819+0000 7f37af5bc740 1 --2- 172.21.15.123:0/1150218936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557866518320 0x557866518740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.819+0000 7f37af5bc740 1 -- 172.21.15.123:0/1150218936 >> 172.21.15.123:0/1150218936 conn(0x5578664e0230 msgr2=0x557866513da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.819+0000 7f37af5bc740 1 -- 172.21.15.123:0/1150218936 shutdown_connections 2024-05-08T06:10:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.819+0000 7f37af5bc740 1 -- 172.21.15.123:0/1150218936 wait complete. 2024-05-08T06:10:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.819+0000 7f37af5bc740 1 -- start start 2024-05-08T06:10:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.819+0000 7f37af5bc740 1 Processor -- start 2024-05-08T06:10:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.820+0000 7f37af5bc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557866518320 0x5578665222c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.820+0000 7f37af5bc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557866518d10 0x557866522800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.820+0000 7f37af5bc740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55786651de80 0x557866529840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.820+0000 7f37af5bc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557866520dd0 con 0x557866518320 2024-05-08T06:10:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.820+0000 7f37af5bc740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x557866435660 con 0x55786651de80 2024-05-08T06:10:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.820+0000 7f37af5bc740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557866520f50 con 0x557866518d10 2024-05-08T06:10:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.820+0000 7f37af5bc740 10 client.0 _ll_register_callbacks cb 0x5578663ff400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:10:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.820+0000 7f37a9e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557866518320 0x5578665222c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.820+0000 7f37a9e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557866518320 0x5578665222c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54174/0 (socket says 172.21.15.123:54174) 2024-05-08T06:10:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.820+0000 7f37a9e00640 1 -- 172.21.15.123:0/2046727355 learned_addr learned my addr 172.21.15.123:0/2046727355 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:10:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.820+0000 7f37a9400640 1 --2- 172.21.15.123:0/2046727355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557866518d10 0x557866522800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.820+0000 7f37aa800640 1 --2- 172.21.15.123:0/2046727355 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55786651de80 0x557866529840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.820+0000 7f37a9e00640 1 -- 172.21.15.123:0/2046727355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557866518d10 msgr2=0x557866522800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.820+0000 7f37a9e00640 1 --2- 172.21.15.123:0/2046727355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557866518d10 0x557866522800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.820+0000 7f37a9e00640 1 -- 172.21.15.123:0/2046727355 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55786651de80 msgr2=0x557866529840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.820+0000 7f37a9e00640 1 --2- 172.21.15.123:0/2046727355 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55786651de80 0x557866529840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.820+0000 7f37a9e00640 1 -- 172.21.15.123:0/2046727355 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55786625a620 con 0x557866518320 2024-05-08T06:10:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.820+0000 7f37a9400640 1 --2- 172.21.15.123:0/2046727355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557866518d10 0x557866522800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:10:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.821+0000 7f37a9e00640 1 --2- 172.21.15.123:0/2046727355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557866518320 0x5578665222c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f379802a860 tx=0x7f37980058f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265d47f2d2c88e3f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.821+0000 7f3787400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-08T06:10:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.821+0000 7f3787400640 1 -- 172.21.15.123:0/2046727355 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379804c020 con 0x557866518320 2024-05-08T06:10:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.821+0000 7f3787400640 1 -- 172.21.15.123:0/2046727355 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f3798030750 con 0x557866518320 2024-05-08T06:10:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.821+0000 7f37af5bc740 1 -- 172.21.15.123:0/2046727355 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3798042070 con 0x557866518320 2024-05-08T06:10:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.821+0000 7f3787400640 1 -- 172.21.15.123:0/2046727355 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3798037040 con 0x557866518320 2024-05-08T06:10:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.821+0000 7f37af5bc740 10 client.15042 fetch_fsmap learned FSMap version 24 2024-05-08T06:10:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.821+0000 7f37af5bc740 1 -- 172.21.15.123:0/2046727355 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55786652c820 con 0x557866518320 2024-05-08T06:10:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.822+0000 7f3787400640 1 -- 172.21.15.123:0/2046727355 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3798042070 con 0x557866518320 2024-05-08T06:10:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.822+0000 7f3787400640 1 -- 172.21.15.123:0/2046727355 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f379803d4f0 con 0x557866518320 2024-05-08T06:10:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.822+0000 7f37af5bc740 10 client.15042 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:10:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.822+0000 7f37af5bc740 10 client.15042 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T06:10:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.822+0000 7f37af5bc740 20 client.15042 populate_metadata read hostname 'smithi123' 2024-05-08T06:10:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.822+0000 7f37af5bc740 1 --2- 172.21.15.123:0/2046727355 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55786652db70 0x55786652ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.822+0000 7f37af5bc740 4 client.15042 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:10:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.822+0000 7f37af5bc740 1 -- 172.21.15.123:0/2046727355 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55786648a160 con 0x55786652db70 2024-05-08T06:10:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.823+0000 7f37a9400640 1 --2- 172.21.15.123:0/2046727355 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55786652db70 0x55786652ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.823+0000 7f3787400640 10 client.15042 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:10:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.823+0000 7f37a9400640 1 --2- 172.21.15.123:0/2046727355 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55786652db70 0x55786652ff60 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea8b2e9b05188c86 server_cookie=6cb1c30b6232fb90 in_seq=0 out_seq=0 2024-05-08T06:10:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.824+0000 7f3787400640 1 -- 172.21.15.123:0/2046727355 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55786648a160 con 0x55786652db70 2024-05-08T06:10:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.824+0000 7f3787400640 10 client.15042 handle_command_reply: tid=0 2024-05-08T06:10:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.824+0000 7f37af5bc740 1 client.15042 shutdown 2024-05-08T06:10:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.825+0000 7f37af5bc740 1 -- 172.21.15.123:0/2046727355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557866518320 msgr2=0x5578665222c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.825+0000 7f37af5bc740 1 --2- 172.21.15.123:0/2046727355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557866518320 0x5578665222c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f379802a860 tx=0x7f37980058f0 comp rx=0 tx=0).stop 2024-05-08T06:10:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.825+0000 7f37af5bc740 1 -- 172.21.15.123:0/2046727355 shutdown_connections 2024-05-08T06:10:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.825+0000 7f37af5bc740 1 --2- 172.21.15.123:0/2046727355 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55786651de80 0x557866529840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.825+0000 7f37af5bc740 1 --2- 172.21.15.123:0/2046727355 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55786652db70 0x55786652ff60 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.825+0000 7f37af5bc740 1 --2- 172.21.15.123:0/2046727355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557866518d10 0x557866522800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.825+0000 7f37af5bc740 1 --2- 172.21.15.123:0/2046727355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557866518320 0x5578665222c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.825+0000 7f37af5bc740 1 -- 172.21.15.123:0/2046727355 >> 172.21.15.123:0/2046727355 conn(0x5578664e0230 msgr2=0x557866512e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.825+0000 7f3787400640 0 client.15042 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:10:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.825+0000 7f37af5bc740 1 -- 172.21.15.123:0/2046727355 shutdown_connections 2024-05-08T06:10:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.825+0000 7f37af5bc740 1 -- 172.21.15.123:0/2046727355 wait complete. 2024-05-08T06:10:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.825+0000 7f37af5bc740 20 client.15042 trim_cache size 0 max 16384 2024-05-08T06:10:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.841+0000 7f37af5bc740 1 Processor -- start 2024-05-08T06:10:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.841+0000 7f37af5bc740 1 -- start start 2024-05-08T06:10:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.842+0000 7f37af5bc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557866518320 0x557866516960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.842+0000 7f37af5bc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557866518d10 0x557866529800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.842+0000 7f37af5bc740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55786651de80 0x557866516ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.842+0000 7f37af5bc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f37a41b29d0 con 0x557866518320 2024-05-08T06:10:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.842+0000 7f37af5bc740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f37a41b02a0 con 0x55786651de80 2024-05-08T06:10:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.842+0000 7f37af5bc740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f37a41ae0f0 con 0x557866518d10 2024-05-08T06:10:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.842+0000 7f37a9e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557866518320 0x557866516960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.842+0000 7f37a9e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557866518320 0x557866516960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54180/0 (socket says 172.21.15.123:54180) 2024-05-08T06:10:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.842+0000 7f37a9e00640 1 -- 172.21.15.123:0/2643393247 learned_addr learned my addr 172.21.15.123:0/2643393247 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:10:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.842+0000 7f37a9400640 1 --2- 172.21.15.123:0/2643393247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557866518d10 0x557866529800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.842+0000 7f37aa800640 1 --2- 172.21.15.123:0/2643393247 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55786651de80 0x557866516ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.842+0000 7f37a9e00640 1 -- 172.21.15.123:0/2643393247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557866518d10 msgr2=0x557866529800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.842+0000 7f37a9e00640 1 --2- 172.21.15.123:0/2643393247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557866518d10 0x557866529800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.842+0000 7f37a9e00640 1 -- 172.21.15.123:0/2643393247 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55786651de80 msgr2=0x557866516ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.842+0000 7f37a9e00640 1 --2- 172.21.15.123:0/2643393247 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55786651de80 0x557866516ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.842+0000 7f37a9e00640 1 -- 172.21.15.123:0/2643393247 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557866513580 con 0x557866518320 2024-05-08T06:10:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.842+0000 7f37aa800640 1 --2- 172.21.15.123:0/2643393247 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55786651de80 0x557866516ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:10:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.843+0000 7f37a9e00640 1 --2- 172.21.15.123:0/2643393247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557866518320 0x557866516960 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3798055040 tx=0x7f3798009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24106e56d5684f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.843+0000 7f3792600640 1 -- 172.21.15.123:0/2643393247 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379804c070 con 0x557866518320 2024-05-08T06:10:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.843+0000 7f3792600640 1 -- 172.21.15.123:0/2643393247 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3798042040 con 0x557866518320 2024-05-08T06:10:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.843+0000 7f3792600640 1 -- 172.21.15.123:0/2643393247 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37980035f0 con 0x557866518320 2024-05-08T06:10:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.844+0000 7f37af5bc740 1 -- 172.21.15.123:0/2643393247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557866518320 msgr2=0x557866516960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.844+0000 7f37af5bc740 1 --2- 172.21.15.123:0/2643393247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557866518320 0x557866516960 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3798055040 tx=0x7f3798009580 comp rx=0 tx=0).stop 2024-05-08T06:10:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.844+0000 7f37af5bc740 1 -- 172.21.15.123:0/2643393247 shutdown_connections 2024-05-08T06:10:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.844+0000 7f37af5bc740 1 --2- 172.21.15.123:0/2643393247 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55786651de80 0x557866516ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.844+0000 7f37af5bc740 1 --2- 172.21.15.123:0/2643393247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557866518d10 0x557866529800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.844+0000 7f37af5bc740 1 --2- 172.21.15.123:0/2643393247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557866518320 0x557866516960 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.844+0000 7f37af5bc740 1 -- 172.21.15.123:0/2643393247 >> 172.21.15.123:0/2643393247 conn(0x5578664e0230 msgr2=0x557866512e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.844+0000 7f37af5bc740 1 -- 172.21.15.123:0/2643393247 shutdown_connections 2024-05-08T06:10:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.844+0000 7f37af5bc740 1 -- 172.21.15.123:0/2643393247 wait complete. 2024-05-08T06:10:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.844+0000 7f37af5bc740 1 -- start start 2024-05-08T06:10:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.844+0000 7f37af5bc740 1 Processor -- start 2024-05-08T06:10:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.845+0000 7f37af5bc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557866518320 0x5578665266f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.845+0000 7f37af5bc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557866518d10 0x557866526c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.845+0000 7f37af5bc740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55786651de80 0x557866596380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.845+0000 7f37af5bc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f37a41ae760 con 0x557866518320 2024-05-08T06:10:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.845+0000 7f37af5bc740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f37a41afbd0 con 0x55786651de80 2024-05-08T06:10:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.845+0000 7f37af5bc740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f37a41aee30 con 0x557866518d10 2024-05-08T06:10:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.845+0000 7f37af5bc740 10 client.0 _ll_register_callbacks cb 0x7f3794009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:10:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.845+0000 7f37a9e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557866518320 0x5578665266f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.845+0000 7f37a9e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557866518320 0x5578665266f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54196/0 (socket says 172.21.15.123:54196) 2024-05-08T06:10:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.845+0000 7f37a9e00640 1 -- 172.21.15.123:0/3765677469 learned_addr learned my addr 172.21.15.123:0/3765677469 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:10:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.845+0000 7f37a9400640 1 --2- 172.21.15.123:0/3765677469 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557866518d10 0x557866526c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.845+0000 7f37aa800640 1 --2- 172.21.15.123:0/3765677469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55786651de80 0x557866596380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.846+0000 7f37a9e00640 1 -- 172.21.15.123:0/3765677469 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557866518d10 msgr2=0x557866526c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.846+0000 7f37a9e00640 1 --2- 172.21.15.123:0/3765677469 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557866518d10 0x557866526c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.846+0000 7f37a9e00640 1 -- 172.21.15.123:0/3765677469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55786651de80 msgr2=0x557866596380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.846+0000 7f37a9e00640 1 --2- 172.21.15.123:0/3765677469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55786651de80 0x557866596380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.846+0000 7f37a9e00640 1 -- 172.21.15.123:0/3765677469 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578665968c0 con 0x557866518320 2024-05-08T06:10:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.846+0000 7f37a9400640 1 --2- 172.21.15.123:0/3765677469 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557866518d10 0x557866526c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:10:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.846+0000 7f37a9e00640 1 --2- 172.21.15.123:0/3765677469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557866518320 0x5578665266f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3798009780 tx=0x7f379803d7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb54b6585fd0c94 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.846+0000 7f3787400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-08T06:10:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.846+0000 7f3787400640 1 -- 172.21.15.123:0/3765677469 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3798061060 con 0x557866518320 2024-05-08T06:10:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.846+0000 7f3787400640 1 -- 172.21.15.123:0/3765677469 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f3798030750 con 0x557866518320 2024-05-08T06:10:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.846+0000 7f37af5bc740 1 -- 172.21.15.123:0/3765677469 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3798042040 con 0x557866518320 2024-05-08T06:10:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.846+0000 7f3787400640 1 -- 172.21.15.123:0/3765677469 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f379804c040 con 0x557866518320 2024-05-08T06:10:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.846+0000 7f37af5bc740 10 client.15048 fetch_fsmap learned FSMap version 24 2024-05-08T06:10:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.846+0000 7f37af5bc740 1 -- 172.21.15.123:0/3765677469 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557866597b80 con 0x557866518320 2024-05-08T06:10:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.847+0000 7f3787400640 1 -- 172.21.15.123:0/3765677469 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3798042040 con 0x557866518320 2024-05-08T06:10:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.847+0000 7f3787400640 1 -- 172.21.15.123:0/3765677469 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f379805a020 con 0x557866518320 2024-05-08T06:10:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.847+0000 7f37af5bc740 10 client.15048 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:10:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.847+0000 7f37af5bc740 10 client.15048 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T06:10:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.847+0000 7f37af5bc740 20 client.15048 populate_metadata read hostname 'smithi123' 2024-05-08T06:10:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.847+0000 7f37af5bc740 1 --2- 172.21.15.123:0/3765677469 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5578665982d0 0x55786659a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.847+0000 7f37af5bc740 4 client.15048 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T06:10:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.847+0000 7f37af5bc740 1 -- 172.21.15.123:0/3765677469 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557866483fd0 con 0x5578665982d0 2024-05-08T06:10:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.847+0000 7f37a9400640 1 --2- 172.21.15.123:0/3765677469 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5578665982d0 0x55786659a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.848+0000 7f37a9400640 1 --2- 172.21.15.123:0/3765677469 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5578665982d0 0x55786659a6c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a879a620f3429c9 server_cookie=52df0a1296a1bc73 in_seq=0 out_seq=0 2024-05-08T06:10:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.848+0000 7f3787400640 10 client.15048 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:10:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.860+0000 7f3787400640 1 -- 172.21.15.123:0/3765677469 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557866483fd0 con 0x5578665982d0 2024-05-08T06:10:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.860+0000 7f3787400640 10 client.15048 handle_command_reply: tid=0 2024-05-08T06:10:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.860+0000 7f37af5bc740 1 client.15048 shutdown 2024-05-08T06:10:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.860+0000 7f37af5bc740 1 -- 172.21.15.123:0/3765677469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557866518320 msgr2=0x5578665266f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.860+0000 7f37af5bc740 1 --2- 172.21.15.123:0/3765677469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557866518320 0x5578665266f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3798009780 tx=0x7f379803d7b0 comp rx=0 tx=0).stop 2024-05-08T06:10:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.861+0000 7f37af5bc740 1 -- 172.21.15.123:0/3765677469 shutdown_connections 2024-05-08T06:10:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.861+0000 7f37af5bc740 1 --2- 172.21.15.123:0/3765677469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55786651de80 0x557866596380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.861+0000 7f37af5bc740 1 --2- 172.21.15.123:0/3765677469 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5578665982d0 0x55786659a6c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.861+0000 7f37af5bc740 1 --2- 172.21.15.123:0/3765677469 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557866518d10 0x557866526c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.861+0000 7f37af5bc740 1 --2- 172.21.15.123:0/3765677469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557866518320 0x5578665266f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.861+0000 7f3787400640 0 client.15048 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:10:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.861+0000 7f37af5bc740 1 -- 172.21.15.123:0/3765677469 >> 172.21.15.123:0/3765677469 conn(0x5578664e0230 msgr2=0x557866516960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.861+0000 7f37af5bc740 1 -- 172.21.15.123:0/3765677469 shutdown_connections 2024-05-08T06:10:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.861+0000 7f37af5bc740 1 -- 172.21.15.123:0/3765677469 wait complete. 2024-05-08T06:10:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.861+0000 7f37af5bc740 20 client.15048 trim_cache size 0 max 16384 2024-05-08T06:10:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout:{ 2024-05-08T06:10:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout: "message": "", 2024-05-08T06:10:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout: "return_code": 0 2024-05-08T06:10:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout:} 2024-05-08T06:10:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.862+0000 7f37ab200640 1 -- 172.21.15.123:0/1150748361 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f378c0b28f0 msgr2=0x7f378c0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.862+0000 7f37ab200640 1 --2- 172.21.15.123:0/1150748361 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f378c0b28f0 0x7f378c0b4db0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x557866519570 tx=0x7f3794009c40 comp rx=0 tx=0).stop 2024-05-08T06:10:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.862+0000 7f37ab200640 1 -- 172.21.15.123:0/1150748361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37a410a970 msgr2=0x7f37a41a6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.862+0000 7f37ab200640 1 --2- 172.21.15.123:0/1150748361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37a410a970 0x7f37a41a6610 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f37a000ef00 tx=0x7f37a000c740 comp rx=0 tx=0).stop 2024-05-08T06:10:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.862+0000 7f37ab200640 1 -- 172.21.15.123:0/1150748361 shutdown_connections 2024-05-08T06:10:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.862+0000 7f37ab200640 1 --2- 172.21.15.123:0/1150748361 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f378c0b28f0 0x7f378c0b4db0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.862+0000 7f37ab200640 1 --2- 172.21.15.123:0/1150748361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37a410a970 0x7f37a41a6610 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.862+0000 7f37ab200640 1 --2- 172.21.15.123:0/1150748361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37a4109fb0 0x7f37a419f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.862+0000 7f37ab200640 1 --2- 172.21.15.123:0/1150748361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37a4108e60 0x7f37a419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.862+0000 7f37ab200640 1 -- 172.21.15.123:0/1150748361 >> 172.21.15.123:0/1150748361 conn(0x7f37a4077e40 msgr2=0x7f37a410d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.862+0000 7f37ab200640 1 -- 172.21.15.123:0/1150748361 shutdown_connections 2024-05-08T06:10:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:42.862+0000 7f37ab200640 1 -- 172.21.15.123:0/1150748361 wait complete. 2024-05-08T06:10:44.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:44 smithi192 ceph-mon[34664]: pgmap v468: 97 pgs: 97 active+clean; 659 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-08T06:10:44.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:44 smithi163 ceph-mon[34520]: pgmap v468: 97 pgs: 97 active+clean; 659 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-08T06:10:44.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:44 smithi123 ceph-mon[29787]: pgmap v468: 97 pgs: 97 active+clean; 659 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-08T06:10:45.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:45 smithi163 ceph-mon[34520]: scrub summary: idle 2024-05-08T06:10:45.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:45 smithi192 ceph-mon[34664]: scrub summary: idle 2024-05-08T06:10:45.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:45 smithi123 ceph-mon[29787]: scrub summary: idle 2024-05-08T06:10:46.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:46 smithi163 ceph-mon[34520]: pgmap v469: 97 pgs: 97 active+clean; 661 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-05-08T06:10:46.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:46 smithi192 ceph-mon[34664]: pgmap v469: 97 pgs: 97 active+clean; 661 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-05-08T06:10:46.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:46 smithi123 ceph-mon[29787]: pgmap v469: 97 pgs: 97 active+clean; 661 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-05-08T06:10:48.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:48 smithi163 ceph-mon[34520]: pgmap v470: 97 pgs: 97 active+clean; 661 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-05-08T06:10:48.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:48 smithi192 ceph-mon[34664]: pgmap v470: 97 pgs: 97 active+clean; 661 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-05-08T06:10:48.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:48 smithi123 ceph-mon[29787]: pgmap v470: 97 pgs: 97 active+clean; 661 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-05-08T06:10:50.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:50 smithi163 ceph-mon[34520]: pgmap v471: 97 pgs: 97 active+clean; 661 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-05-08T06:10:50.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:50 smithi192 ceph-mon[34664]: pgmap v471: 97 pgs: 97 active+clean; 661 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-05-08T06:10:50.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:50 smithi123 ceph-mon[29787]: pgmap v471: 97 pgs: 97 active+clean; 661 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-05-08T06:10:52.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:52 smithi163 ceph-mon[34520]: pgmap v472: 97 pgs: 97 active+clean; 661 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-05-08T06:10:52.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:52 smithi192 ceph-mon[34664]: pgmap v472: 97 pgs: 97 active+clean; 661 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-05-08T06:10:52.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:52 smithi123 ceph-mon[29787]: pgmap v472: 97 pgs: 97 active+clean; 661 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-05-08T06:10:54.538 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:54 smithi163 ceph-mon[34520]: pgmap v473: 97 pgs: 97 active+clean; 661 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-05-08T06:10:54.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:54 smithi192 ceph-mon[34664]: pgmap v473: 97 pgs: 97 active+clean; 661 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-05-08T06:10:54.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:54 smithi123 ceph-mon[29787]: pgmap v473: 97 pgs: 97 active+clean; 661 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-05-08T06:10:56.003 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.003+0000 7f2aa7800640 1 -- 172.21.15.123:0/2339718417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2aa010a230 msgr2=0x7f2aa010a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:56.003 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.003+0000 7f2aa7800640 1 --2- 172.21.15.123:0/2339718417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2aa010a230 0x7f2aa010a6b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2a88009900 tx=0x7f2a8802a3f0 comp rx=0 tx=0).stop 2024-05-08T06:10:56.003 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.003+0000 7f2aa7800640 1 -- 172.21.15.123:0/2339718417 shutdown_connections 2024-05-08T06:10:56.004 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.003+0000 7f2aa7800640 1 --2- 172.21.15.123:0/2339718417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2aa010abf0 0x7f2aa0111480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:56.004 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.003+0000 7f2aa7800640 1 --2- 172.21.15.123:0/2339718417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2aa010a230 0x7f2aa010a6b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:56.004 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.003+0000 7f2aa7800640 1 --2- 172.21.15.123:0/2339718417 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2aa0109040 0x7f2aa0109460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:56.004 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.003+0000 7f2aa7800640 1 -- 172.21.15.123:0/2339718417 >> 172.21.15.123:0/2339718417 conn(0x7f2aa0078000 msgr2=0x7f2aa007a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:56.004 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.003+0000 7f2aa7800640 1 -- 172.21.15.123:0/2339718417 shutdown_connections 2024-05-08T06:10:56.004 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.003+0000 7f2aa7800640 1 -- 172.21.15.123:0/2339718417 wait complete. 2024-05-08T06:10:56.006 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.004+0000 7f2aa7800640 1 Processor -- start 2024-05-08T06:10:56.006 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.004+0000 7f2aa7800640 1 -- start start 2024-05-08T06:10:56.006 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.004+0000 7f2aa7800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2aa0109040 0x7f2aa019f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.004+0000 7f2aa7800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2aa010a230 0x7f2aa019f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.005+0000 7f2aa7800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2aa010abf0 0x7f2aa01a6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.005+0000 7f2aa7800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa0114bd0 con 0x7f2aa0109040 2024-05-08T06:10:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.005+0000 7f2aa7800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa0114a50 con 0x7f2aa010a230 2024-05-08T06:10:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.005+0000 7f2aa7800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa0114d50 con 0x7f2aa010abf0 2024-05-08T06:10:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.005+0000 7f2aa4e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2aa0109040 0x7f2aa019f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.005+0000 7f2aa4e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2aa0109040 0x7f2aa019f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38006/0 (socket says 172.21.15.123:38006) 2024-05-08T06:10:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.005+0000 7f2aa4e00640 1 -- 172.21.15.123:0/120521381 learned_addr learned my addr 172.21.15.123:0/120521381 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:10:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.005+0000 7f2aa5800640 1 --2- 172.21.15.123:0/120521381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2aa010abf0 0x7f2aa01a6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.005+0000 7f2a97e00640 1 --2- 172.21.15.123:0/120521381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2aa010a230 0x7f2aa019f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.005+0000 7f2aa4e00640 1 -- 172.21.15.123:0/120521381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2aa010abf0 msgr2=0x7f2aa01a6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.005+0000 7f2aa4e00640 1 --2- 172.21.15.123:0/120521381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2aa010abf0 0x7f2aa01a6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.005+0000 7f2aa4e00640 1 -- 172.21.15.123:0/120521381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2aa010a230 msgr2=0x7f2aa019f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.005+0000 7f2aa4e00640 1 --2- 172.21.15.123:0/120521381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2aa010a230 0x7f2aa019f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.005+0000 7f2aa4e00640 1 -- 172.21.15.123:0/120521381 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aa01a6dc0 con 0x7f2aa0109040 2024-05-08T06:10:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.006+0000 7f2aa4e00640 1 --2- 172.21.15.123:0/120521381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2aa0109040 0x7f2aa019f1a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2a9000e9c0 tx=0x7f2a9000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.006+0000 7f2a9e000640 1 -- 172.21.15.123:0/120521381 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9000cd80 con 0x7f2aa0109040 2024-05-08T06:10:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.006+0000 7f2a9e000640 1 -- 172.21.15.123:0/120521381 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a900035f0 con 0x7f2aa0109040 2024-05-08T06:10:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.006+0000 7f2aa7800640 1 -- 172.21.15.123:0/120521381 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aa01a70b0 con 0x7f2aa0109040 2024-05-08T06:10:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.006+0000 7f2a9e000640 1 -- 172.21.15.123:0/120521381 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a90010690 con 0x7f2aa0109040 2024-05-08T06:10:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.006+0000 7f2aa7800640 1 -- 172.21.15.123:0/120521381 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aa01a7620 con 0x7f2aa0109040 2024-05-08T06:10:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.009+0000 7f2aaa7ec740 1 Processor -- start 2024-05-08T06:10:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.009+0000 7f2aaa7ec740 1 -- start start 2024-05-08T06:10:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.010+0000 7f2a9e000640 1 -- 172.21.15.123:0/120521381 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 19) ==== 149704+0+0 (secure 0 0 0) 0x7f2a900037a0 con 0x7f2aa0109040 2024-05-08T06:10:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.010+0000 7f2aaa7ec740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559568413120 0x559568413540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.010+0000 7f2aaa7ec740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559568413b10 0x559568418610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.010+0000 7f2aaa7ec740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559568418c80 0x55956841b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.010+0000 7f2aaa7ec740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559568377550 con 0x559568413120 2024-05-08T06:10:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.010+0000 7f2aaa7ec740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55956831cbc0 con 0x559568418c80 2024-05-08T06:10:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.010+0000 7f2aaa7ec740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x559568373510 con 0x559568413b10 2024-05-08T06:10:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.010+0000 7f2aa5800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559568413120 0x559568413540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.010+0000 7f2aa4e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559568413b10 0x559568418610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.010+0000 7f2aa5800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559568413120 0x559568413540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38014/0 (socket says 172.21.15.123:38014) 2024-05-08T06:10:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.010+0000 7f2aa5800640 1 -- 172.21.15.123:0/1431601492 learned_addr learned my addr 172.21.15.123:0/1431601492 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:10:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.010+0000 7f2a97e00640 1 --2- 172.21.15.123:0/1431601492 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559568418c80 0x55956841b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.011+0000 7f2aa4e00640 1 -- 172.21.15.123:0/1431601492 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559568418c80 msgr2=0x55956841b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.011+0000 7f2aa4e00640 1 --2- 172.21.15.123:0/1431601492 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559568418c80 0x55956841b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.011+0000 7f2aa4e00640 1 -- 172.21.15.123:0/1431601492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559568413120 msgr2=0x559568413540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.011+0000 7f2aa4e00640 1 --2- 172.21.15.123:0/1431601492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559568413120 0x559568413540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.011+0000 7f2aa4e00640 1 -- 172.21.15.123:0/1431601492 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559568089210 con 0x559568413b10 2024-05-08T06:10:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.011+0000 7f2aa5800640 1 --2- 172.21.15.123:0/1431601492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559568413120 0x559568413540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:10:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.011+0000 7f2aa4e00640 1 --2- 172.21.15.123:0/1431601492 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559568413b10 0x559568418610 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2a900e0030 tx=0x7f2a900e8b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b1c781b67b33519 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.011+0000 7f2a9e000640 1 --2- 172.21.15.123:0/120521381 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f2a840b29a0 0x7f2a840b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.011+0000 7f2a96000640 1 -- 172.21.15.123:0/1431601492 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a900f1070 con 0x559568413b10 2024-05-08T06:10:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.011+0000 7f2aa5800640 1 --2- 172.21.15.123:0/120521381 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f2a840b29a0 0x7f2a840b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.011+0000 7f2a9e000640 1 -- 172.21.15.123:0/120521381 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f2a900e2700 con 0x7f2aa0109040 2024-05-08T06:10:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.011+0000 7f2a96000640 1 -- 172.21.15.123:0/1431601492 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a900a9680 con 0x559568413b10 2024-05-08T06:10:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.011+0000 7f2a96000640 1 -- 172.21.15.123:0/1431601492 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a900a94c0 con 0x559568413b10 2024-05-08T06:10:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.012+0000 7f2aaa7ec740 1 -- 172.21.15.123:0/1431601492 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559568413b10 msgr2=0x559568418610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.012+0000 7f2aaa7ec740 1 --2- 172.21.15.123:0/1431601492 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559568413b10 0x559568418610 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2a900e0030 tx=0x7f2a900e8b20 comp rx=0 tx=0).stop 2024-05-08T06:10:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.012+0000 7f2aa5800640 1 --2- 172.21.15.123:0/120521381 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f2a840b29a0 0x7f2a840b4e60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x559568414370 tx=0x7f2a98009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.012+0000 7f2aaa7ec740 1 -- 172.21.15.123:0/1431601492 shutdown_connections 2024-05-08T06:10:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.012+0000 7f2aaa7ec740 1 --2- 172.21.15.123:0/1431601492 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559568418c80 0x55956841b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.012+0000 7f2aaa7ec740 1 --2- 172.21.15.123:0/1431601492 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559568413b10 0x559568418610 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.012+0000 7f2aaa7ec740 1 --2- 172.21.15.123:0/1431601492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559568413120 0x559568413540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.012+0000 7f2aaa7ec740 1 -- 172.21.15.123:0/1431601492 >> 172.21.15.123:0/1431601492 conn(0x5595683db230 msgr2=0x55956840eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.012+0000 7f2aaa7ec740 1 -- 172.21.15.123:0/1431601492 shutdown_connections 2024-05-08T06:10:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.012+0000 7f2aaa7ec740 1 -- 172.21.15.123:0/1431601492 wait complete. 2024-05-08T06:10:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.013+0000 7f2aaa7ec740 1 -- start start 2024-05-08T06:10:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.013+0000 7f2aaa7ec740 1 Processor -- start 2024-05-08T06:10:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.014+0000 7f2aaa7ec740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559568413120 0x55956841d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.014+0000 7f2aaa7ec740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559568413b10 0x55956841d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.014+0000 7f2aaa7ec740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559568418c80 0x559568424690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.014+0000 7f2aaa7ec740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55956841bc30 con 0x559568413120 2024-05-08T06:10:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.014+0000 7f2aaa7ec740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x559568330660 con 0x559568413b10 2024-05-08T06:10:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.014+0000 7f2aaa7ec740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55956841bdb0 con 0x559568418c80 2024-05-08T06:10:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.014+0000 7f2aaa7ec740 10 client.0 _ll_register_callbacks cb 0x5595682fa400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:10:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.014+0000 7f2aa4e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559568413120 0x55956841d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.014+0000 7f2aa5800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559568418c80 0x559568424690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.014+0000 7f2aa4e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559568413120 0x55956841d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38026/0 (socket says 172.21.15.123:38026) 2024-05-08T06:10:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.014+0000 7f2aa4e00640 1 -- 172.21.15.123:0/228362649 learned_addr learned my addr 172.21.15.123:0/228362649 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:10:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.014+0000 7f2a97e00640 1 --2- 172.21.15.123:0/228362649 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559568413b10 0x55956841d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.014+0000 7f2aa4e00640 1 -- 172.21.15.123:0/228362649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559568418c80 msgr2=0x559568424690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.014+0000 7f2aa4e00640 1 --2- 172.21.15.123:0/228362649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559568418c80 0x559568424690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.014+0000 7f2aa4e00640 1 -- 172.21.15.123:0/228362649 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559568413b10 msgr2=0x55956841d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.014+0000 7f2aa4e00640 1 --2- 172.21.15.123:0/228362649 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559568413b10 0x55956841d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.014+0000 7f2aa4e00640 1 -- 172.21.15.123:0/228362649 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559568155620 con 0x559568413120 2024-05-08T06:10:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.014+0000 7f2aa5800640 1 --2- 172.21.15.123:0/228362649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559568418c80 0x559568424690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:10:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.015+0000 7f2aa4e00640 1 --2- 172.21.15.123:0/228362649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559568413120 0x55956841d180 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2a900a5ba0 tx=0x7f2a900a9a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c338c6ecfc38203f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.015+0000 7f2a8ea00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-08T06:10:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.015+0000 7f2a8ea00640 1 -- 172.21.15.123:0/228362649 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a900f1070 con 0x559568413120 2024-05-08T06:10:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.015+0000 7f2a8ea00640 1 -- 172.21.15.123:0/228362649 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f2a900a9dd0 con 0x559568413120 2024-05-08T06:10:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.015+0000 7f2aaa7ec740 1 -- 172.21.15.123:0/228362649 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a900a9680 con 0x559568413120 2024-05-08T06:10:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.015+0000 7f2a8ea00640 1 -- 172.21.15.123:0/228362649 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2a900a92e0 con 0x559568413120 2024-05-08T06:10:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.015+0000 7f2aaa7ec740 10 client.15060 fetch_fsmap learned FSMap version 24 2024-05-08T06:10:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.015+0000 7f2aaa7ec740 1 -- 172.21.15.123:0/228362649 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559568427670 con 0x559568413120 2024-05-08T06:10:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.015+0000 7f2a8ea00640 1 -- 172.21.15.123:0/228362649 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a900a9680 con 0x559568413120 2024-05-08T06:10:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.016+0000 7f2a8ea00640 1 -- 172.21.15.123:0/228362649 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f2a900e5ce0 con 0x559568413120 2024-05-08T06:10:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.016+0000 7f2aaa7ec740 10 client.15060 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:10:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.016+0000 7f2aaa7ec740 10 client.15060 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T06:10:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.016+0000 7f2aaa7ec740 20 client.15060 populate_metadata read hostname 'smithi123' 2024-05-08T06:10:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.016+0000 7f2aaa7ec740 1 --2- 172.21.15.123:0/228362649 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5595684289c0 0x55956842adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.016+0000 7f2aaa7ec740 4 client.15060 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:10:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.016+0000 7f2aaa7ec740 1 -- 172.21.15.123:0/228362649 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559568385160 con 0x5595684289c0 2024-05-08T06:10:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.016+0000 7f2a97e00640 1 --2- 172.21.15.123:0/228362649 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5595684289c0 0x55956842adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.017+0000 7f2a97e00640 1 --2- 172.21.15.123:0/228362649 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5595684289c0 0x55956842adb0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1375926849e82152 server_cookie=3d1b03b77cb6bb85 in_seq=0 out_seq=0 2024-05-08T06:10:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.017+0000 7f2a8ea00640 10 client.15060 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:10:56.021 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.018+0000 7f2a8ea00640 1 -- 172.21.15.123:0/228362649 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559568385160 con 0x5595684289c0 2024-05-08T06:10:56.021 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.018+0000 7f2a8ea00640 10 client.15060 handle_command_reply: tid=0 2024-05-08T06:10:56.021 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.018+0000 7f2aaa7ec740 1 client.15060 shutdown 2024-05-08T06:10:56.022 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.018+0000 7f2aaa7ec740 1 -- 172.21.15.123:0/228362649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559568413120 msgr2=0x55956841d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:56.022 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.018+0000 7f2aaa7ec740 1 --2- 172.21.15.123:0/228362649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559568413120 0x55956841d180 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2a900a5ba0 tx=0x7f2a900a9a20 comp rx=0 tx=0).stop 2024-05-08T06:10:56.022 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.018+0000 7f2aaa7ec740 1 -- 172.21.15.123:0/228362649 shutdown_connections 2024-05-08T06:10:56.022 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.018+0000 7f2aaa7ec740 1 --2- 172.21.15.123:0/228362649 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5595684289c0 0x55956842adb0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:56.022 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.018+0000 7f2aaa7ec740 1 --2- 172.21.15.123:0/228362649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559568418c80 0x559568424690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:56.022 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.018+0000 7f2a8ea00640 0 client.15060 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:10:56.022 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.018+0000 7f2aaa7ec740 1 --2- 172.21.15.123:0/228362649 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559568413b10 0x55956841d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:56.022 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.018+0000 7f2aaa7ec740 1 --2- 172.21.15.123:0/228362649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559568413120 0x55956841d180 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:56.022 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.018+0000 7f2aaa7ec740 1 -- 172.21.15.123:0/228362649 >> 172.21.15.123:0/228362649 conn(0x5595683db230 msgr2=0x55956840dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:56.022 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.018+0000 7f2aaa7ec740 1 -- 172.21.15.123:0/228362649 shutdown_connections 2024-05-08T06:10:56.022 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.018+0000 7f2aaa7ec740 1 -- 172.21.15.123:0/228362649 wait complete. 2024-05-08T06:10:56.022 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.018+0000 7f2aaa7ec740 20 client.15060 trim_cache size 0 max 16384 2024-05-08T06:10:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.035+0000 7f2aaa7ec740 1 Processor -- start 2024-05-08T06:10:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.035+0000 7f2aaa7ec740 1 -- start start 2024-05-08T06:10:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.035+0000 7f2aaa7ec740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559568413120 0x55956840ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.035+0000 7f2aaa7ec740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559568413b10 0x559568424650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.035+0000 7f2aaa7ec740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559568418c80 0x559568411860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.035+0000 7f2aaa7ec740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa01b2c30 con 0x559568418c80 2024-05-08T06:10:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.035+0000 7f2aaa7ec740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa01b0500 con 0x559568413120 2024-05-08T06:10:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.035+0000 7f2aaa7ec740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa01ae350 con 0x559568413b10 2024-05-08T06:10:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.036+0000 7f2aa5800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559568418c80 0x559568411860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.036+0000 7f2aa4e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559568413120 0x55956840ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.036+0000 7f2a97e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559568413b10 0x559568424650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.036+0000 7f2aa5800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559568418c80 0x559568411860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38040/0 (socket says 172.21.15.123:38040) 2024-05-08T06:10:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.036+0000 7f2aa5800640 1 -- 172.21.15.123:0/851438122 learned_addr learned my addr 172.21.15.123:0/851438122 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:10:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.036+0000 7f2aa4e00640 1 -- 172.21.15.123:0/851438122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559568413b10 msgr2=0x559568424650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.036+0000 7f2aa4e00640 1 --2- 172.21.15.123:0/851438122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559568413b10 0x559568424650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.036+0000 7f2aa4e00640 1 -- 172.21.15.123:0/851438122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559568418c80 msgr2=0x559568411860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.036+0000 7f2aa4e00640 1 --2- 172.21.15.123:0/851438122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559568418c80 0x559568411860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.036+0000 7f2aa4e00640 1 -- 172.21.15.123:0/851438122 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55956840a280 con 0x559568413120 2024-05-08T06:10:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.036+0000 7f2aa4e00640 1 --2- 172.21.15.123:0/851438122 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559568413120 0x55956840ed30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2a900a9ce0 tx=0x7f2a900a6010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78555e9578159b64 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.037+0000 7f2a95600640 1 -- 172.21.15.123:0/851438122 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a900f1070 con 0x559568413120 2024-05-08T06:10:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.037+0000 7f2a95600640 1 -- 172.21.15.123:0/851438122 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a900b2310 con 0x559568413120 2024-05-08T06:10:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.037+0000 7f2a95600640 1 -- 172.21.15.123:0/851438122 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a900a94c0 con 0x559568413120 2024-05-08T06:10:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.037+0000 7f2aaa7ec740 1 -- 172.21.15.123:0/851438122 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559568413120 msgr2=0x55956840ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.037+0000 7f2aaa7ec740 1 --2- 172.21.15.123:0/851438122 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559568413120 0x55956840ed30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2a900a9ce0 tx=0x7f2a900a6010 comp rx=0 tx=0).stop 2024-05-08T06:10:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.037+0000 7f2aaa7ec740 1 -- 172.21.15.123:0/851438122 shutdown_connections 2024-05-08T06:10:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.037+0000 7f2aaa7ec740 1 --2- 172.21.15.123:0/851438122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559568418c80 0x559568411860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.037+0000 7f2aaa7ec740 1 --2- 172.21.15.123:0/851438122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559568413b10 0x559568424650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.037+0000 7f2aaa7ec740 1 --2- 172.21.15.123:0/851438122 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559568413120 0x55956840ed30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.037+0000 7f2aaa7ec740 1 -- 172.21.15.123:0/851438122 >> 172.21.15.123:0/851438122 conn(0x5595683db230 msgr2=0x55956840e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.037+0000 7f2aaa7ec740 1 -- 172.21.15.123:0/851438122 shutdown_connections 2024-05-08T06:10:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.038+0000 7f2aaa7ec740 1 -- 172.21.15.123:0/851438122 wait complete. 2024-05-08T06:10:56.041 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.038+0000 7f2aaa7ec740 1 -- start start 2024-05-08T06:10:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.038+0000 7f2aaa7ec740 1 Processor -- start 2024-05-08T06:10:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.039+0000 7f2aaa7ec740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559568413120 0x559568421790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.039+0000 7f2aaa7ec740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559568413b10 0x559568421cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.039+0000 7f2aaa7ec740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559568418c80 0x559568422bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.039+0000 7f2aaa7ec740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa01ae9c0 con 0x559568413120 2024-05-08T06:10:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.039+0000 7f2aaa7ec740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa01afe30 con 0x559568418c80 2024-05-08T06:10:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.039+0000 7f2aaa7ec740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa01af090 con 0x559568413b10 2024-05-08T06:10:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.039+0000 7f2aaa7ec740 10 client.0 _ll_register_callbacks cb 0x7f2a98009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:10:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.039+0000 7f2aa4e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559568413120 0x559568421790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.039+0000 7f2aa4e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559568413120 0x559568421790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:38050/0 (socket says 172.21.15.123:38050) 2024-05-08T06:10:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.039+0000 7f2aa4e00640 1 -- 172.21.15.123:0/3334136443 learned_addr learned my addr 172.21.15.123:0/3334136443 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:10:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.039+0000 7f2aa5800640 1 --2- 172.21.15.123:0/3334136443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559568418c80 0x559568422bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.039+0000 7f2a97e00640 1 --2- 172.21.15.123:0/3334136443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559568413b10 0x559568421cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.039+0000 7f2aa4e00640 1 -- 172.21.15.123:0/3334136443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559568413b10 msgr2=0x559568421cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.039+0000 7f2aa4e00640 1 --2- 172.21.15.123:0/3334136443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559568413b10 0x559568421cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.039+0000 7f2aa4e00640 1 -- 172.21.15.123:0/3334136443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559568418c80 msgr2=0x559568422bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:10:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.039+0000 7f2aa4e00640 1 --2- 172.21.15.123:0/3334136443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559568418c80 0x559568422bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.039+0000 7f2aa4e00640 1 -- 172.21.15.123:0/3334136443 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595684231a0 con 0x559568413120 2024-05-08T06:10:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.039+0000 7f2a97e00640 1 --2- 172.21.15.123:0/3334136443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559568413b10 0x559568421cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:10:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.039+0000 7f2aa4e00640 1 --2- 172.21.15.123:0/3334136443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559568413120 0x559568421790 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2a900a9b00 tx=0x7f2a900c4390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a728751a05a0e02b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:10:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.039+0000 7f2a8ea00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-08T06:10:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.040+0000 7f2a8ea00640 1 -- 172.21.15.123:0/3334136443 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a900f1070 con 0x559568413120 2024-05-08T06:10:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.040+0000 7f2a8ea00640 1 -- 172.21.15.123:0/3334136443 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f2a900b2750 con 0x559568413120 2024-05-08T06:10:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.040+0000 7f2aaa7ec740 1 -- 172.21.15.123:0/3334136443 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55956837efd0 con 0x559568413120 2024-05-08T06:10:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.040+0000 7f2a8ea00640 1 -- 172.21.15.123:0/3334136443 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2a900a94c0 con 0x559568413120 2024-05-08T06:10:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.040+0000 7f2aaa7ec740 10 client.15066 fetch_fsmap learned FSMap version 24 2024-05-08T06:10:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.040+0000 7f2aaa7ec740 1 -- 172.21.15.123:0/3334136443 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5595684231a0 con 0x559568413120 2024-05-08T06:10:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.040+0000 7f2a8ea00640 1 -- 172.21.15.123:0/3334136443 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55956837efd0 con 0x559568413120 2024-05-08T06:10:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.040+0000 7f2a8ea00640 1 -- 172.21.15.123:0/3334136443 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f2a900e5ce0 con 0x559568413120 2024-05-08T06:10:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.040+0000 7f2aaa7ec740 10 client.15066 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:10:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.040+0000 7f2aaa7ec740 10 client.15066 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T06:10:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.040+0000 7f2aaa7ec740 20 client.15066 populate_metadata read hostname 'smithi123' 2024-05-08T06:10:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.040+0000 7f2aaa7ec740 1 --2- 172.21.15.123:0/3334136443 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x559568493010 0x559568495400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:10:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.040+0000 7f2aaa7ec740 4 client.15066 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T06:10:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.040+0000 7f2aaa7ec740 1 -- 172.21.15.123:0/3334136443 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5595682fa4b0 con 0x559568493010 2024-05-08T06:10:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.041+0000 7f2a97e00640 1 --2- 172.21.15.123:0/3334136443 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x559568493010 0x559568495400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:10:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.042+0000 7f2a97e00640 1 --2- 172.21.15.123:0/3334136443 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x559568493010 0x559568495400 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81408ed065c0083f server_cookie=79758f4ec4529eab in_seq=0 out_seq=0 2024-05-08T06:10:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.042+0000 7f2a8ea00640 10 client.15066 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:10:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.048+0000 7f2a8ea00640 1 -- 172.21.15.123:0/3334136443 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5595682fa4b0 con 0x559568493010 2024-05-08T06:10:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.048+0000 7f2a8ea00640 10 client.15066 handle_command_reply: tid=0 2024-05-08T06:10:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.048+0000 7f2aaa7ec740 1 client.15066 shutdown 2024-05-08T06:10:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.048+0000 7f2aaa7ec740 1 -- 172.21.15.123:0/3334136443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559568413120 msgr2=0x559568421790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.048+0000 7f2aaa7ec740 1 --2- 172.21.15.123:0/3334136443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559568413120 0x559568421790 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2a900a9b00 tx=0x7f2a900c4390 comp rx=0 tx=0).stop 2024-05-08T06:10:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.048+0000 7f2aaa7ec740 1 -- 172.21.15.123:0/3334136443 shutdown_connections 2024-05-08T06:10:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.048+0000 7f2aaa7ec740 1 --2- 172.21.15.123:0/3334136443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559568418c80 0x559568422bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.048+0000 7f2aaa7ec740 1 --2- 172.21.15.123:0/3334136443 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x559568493010 0x559568495400 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.048+0000 7f2aaa7ec740 1 --2- 172.21.15.123:0/3334136443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559568413b10 0x559568421cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.048+0000 7f2aaa7ec740 1 --2- 172.21.15.123:0/3334136443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559568413120 0x559568421790 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.048+0000 7f2a8ea00640 0 client.15066 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:10:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.048+0000 7f2aaa7ec740 1 -- 172.21.15.123:0/3334136443 >> 172.21.15.123:0/3334136443 conn(0x5595683db230 msgr2=0x55956841cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.048+0000 7f2aaa7ec740 1 -- 172.21.15.123:0/3334136443 shutdown_connections 2024-05-08T06:10:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.048+0000 7f2aaa7ec740 1 -- 172.21.15.123:0/3334136443 wait complete. 2024-05-08T06:10:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.048+0000 7f2aaa7ec740 20 client.15066 trim_cache size 0 max 16384 2024-05-08T06:10:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout:{ 2024-05-08T06:10:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout: "message": "", 2024-05-08T06:10:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout: "return_code": 0 2024-05-08T06:10:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout:} 2024-05-08T06:10:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.049+0000 7f2aa7800640 1 -- 172.21.15.123:0/120521381 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f2a840b29a0 msgr2=0x7f2a840b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.049+0000 7f2aa7800640 1 --2- 172.21.15.123:0/120521381 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f2a840b29a0 0x7f2a840b4e60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x559568414370 tx=0x7f2a98009500 comp rx=0 tx=0).stop 2024-05-08T06:10:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.049+0000 7f2aa7800640 1 -- 172.21.15.123:0/120521381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2aa0109040 msgr2=0x7f2aa019f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:10:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.049+0000 7f2aa7800640 1 --2- 172.21.15.123:0/120521381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2aa0109040 0x7f2aa019f1a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2a9000e9c0 tx=0x7f2a9000ee80 comp rx=0 tx=0).stop 2024-05-08T06:10:56.051 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.049+0000 7f2aa7800640 1 -- 172.21.15.123:0/120521381 shutdown_connections 2024-05-08T06:10:56.051 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.049+0000 7f2aa7800640 1 --2- 172.21.15.123:0/120521381 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f2a840b29a0 0x7f2a840b4e60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:56.051 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.049+0000 7f2aa7800640 1 --2- 172.21.15.123:0/120521381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2aa010abf0 0x7f2aa01a6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:56.051 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.049+0000 7f2aa7800640 1 --2- 172.21.15.123:0/120521381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2aa010a230 0x7f2aa019f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:56.051 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.049+0000 7f2aa7800640 1 --2- 172.21.15.123:0/120521381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2aa0109040 0x7f2aa019f1a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:10:56.051 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.049+0000 7f2aa7800640 1 -- 172.21.15.123:0/120521381 >> 172.21.15.123:0/120521381 conn(0x7f2aa0078000 msgr2=0x7f2aa010d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:10:56.051 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.049+0000 7f2aa7800640 1 -- 172.21.15.123:0/120521381 shutdown_connections 2024-05-08T06:10:56.051 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:10:56.049+0000 7f2aa7800640 1 -- 172.21.15.123:0/120521381 wait complete. 2024-05-08T06:10:56.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:56 smithi192 ceph-mon[34664]: pgmap v474: 97 pgs: 97 active+clean; 661 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-05-08T06:10:56.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:56 smithi123 ceph-mon[29787]: pgmap v474: 97 pgs: 97 active+clean; 661 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-05-08T06:10:56.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:56 smithi163 ceph-mon[34520]: pgmap v474: 97 pgs: 97 active+clean; 661 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-05-08T06:10:58.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:10:58 smithi192 ceph-mon[34664]: pgmap v475: 97 pgs: 97 active+clean; 661 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-08T06:10:58.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:10:58 smithi123 ceph-mon[29787]: pgmap v475: 97 pgs: 97 active+clean; 661 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-08T06:10:58.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:10:58 smithi163 ceph-mon[34520]: pgmap v475: 97 pgs: 97 active+clean; 661 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-08T06:11:00.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:00 smithi123 ceph-mon[29787]: pgmap v476: 97 pgs: 97 active+clean; 662 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-08T06:11:00.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:00 smithi163 ceph-mon[34520]: pgmap v476: 97 pgs: 97 active+clean; 662 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-08T06:11:00.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:00 smithi192 ceph-mon[34664]: pgmap v476: 97 pgs: 97 active+clean; 662 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-08T06:11:02.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:02 smithi123 ceph-mon[29787]: pgmap v477: 97 pgs: 97 active+clean; 662 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-08T06:11:02.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:02 smithi163 ceph-mon[34520]: pgmap v477: 97 pgs: 97 active+clean; 662 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-08T06:11:02.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:02 smithi192 ceph-mon[34664]: pgmap v477: 97 pgs: 97 active+clean; 662 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-08T06:11:04.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:04 smithi123 ceph-mon[29787]: pgmap v478: 97 pgs: 97 active+clean; 662 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-08T06:11:04.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:04 smithi163 ceph-mon[34520]: pgmap v478: 97 pgs: 97 active+clean; 662 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-08T06:11:04.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:04 smithi192 ceph-mon[34664]: pgmap v478: 97 pgs: 97 active+clean; 662 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-08T06:11:06.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:06 smithi123 ceph-mon[29787]: pgmap v479: 97 pgs: 97 active+clean; 672 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-05-08T06:11:06.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:06 smithi163 ceph-mon[34520]: pgmap v479: 97 pgs: 97 active+clean; 672 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-05-08T06:11:06.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:06 smithi192 ceph-mon[34664]: pgmap v479: 97 pgs: 97 active+clean; 672 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-05-08T06:11:07.468 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-08T06:11:07.899 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:11:08.602 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:08 smithi192 ceph-mon[34664]: pgmap v480: 97 pgs: 97 active+clean; 672 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-05-08T06:11:08.608 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:08 smithi123 ceph-mon[29787]: pgmap v480: 97 pgs: 97 active+clean; 672 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-05-08T06:11:08.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:08 smithi163 ceph-mon[34520]: pgmap v480: 97 pgs: 97 active+clean; 672 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-05-08T06:11:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.194+0000 7f0e88800640 1 -- 172.21.15.123:0/2123709361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e8010efe0 msgr2=0x7f0e801113d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.194+0000 7f0e88800640 1 --2- 172.21.15.123:0/2123709361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e8010efe0 0x7f0e801113d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c009ca0 tx=0x7f0e7c02a8c0 comp rx=0 tx=0).stop 2024-05-08T06:11:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.194+0000 7f0e88800640 1 -- 172.21.15.123:0/2123709361 shutdown_connections 2024-05-08T06:11:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.194+0000 7f0e88800640 1 --2- 172.21.15.123:0/2123709361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e8010efe0 0x7f0e801113d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.194+0000 7f0e88800640 1 --2- 172.21.15.123:0/2123709361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e80075410 0x7f0e80075890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.194+0000 7f0e88800640 1 --2- 172.21.15.123:0/2123709361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e800769f0 0x7f0e80074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:09.195 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.194+0000 7f0e88800640 1 -- 172.21.15.123:0/2123709361 >> 172.21.15.123:0/2123709361 conn(0x7f0e800fdfa0 msgr2=0x7f0e801003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:09.195 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.194+0000 7f0e88800640 1 -- 172.21.15.123:0/2123709361 shutdown_connections 2024-05-08T06:11:09.195 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.194+0000 7f0e88800640 1 -- 172.21.15.123:0/2123709361 wait complete. 2024-05-08T06:11:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.195+0000 7f0e88800640 1 Processor -- start 2024-05-08T06:11:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.195+0000 7f0e88800640 1 -- start start 2024-05-08T06:11:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.195+0000 7f0e88800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e80075410 0x7f0e8019f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.195+0000 7f0e88800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e800769f0 0x7f0e8019f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.195+0000 7f0e88800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e8010efe0 0x7f0e801a6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.195+0000 7f0e88800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0e80114ba0 con 0x7f0e800769f0 2024-05-08T06:11:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.195+0000 7f0e88800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0e80114a20 con 0x7f0e8010efe0 2024-05-08T06:11:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.195+0000 7f0e88800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0e80114d20 con 0x7f0e80075410 2024-05-08T06:11:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.196+0000 7f0e85400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e800769f0 0x7f0e8019f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.196+0000 7f0e86800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e8010efe0 0x7f0e801a6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.196+0000 7f0e85400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e800769f0 0x7f0e8019f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41352/0 (socket says 172.21.15.123:41352) 2024-05-08T06:11:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.196+0000 7f0e85e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e80075410 0x7f0e8019f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.196+0000 7f0e85400640 1 -- 172.21.15.123:0/930679044 learned_addr learned my addr 172.21.15.123:0/930679044 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.196+0000 7f0e85e00640 1 -- 172.21.15.123:0/930679044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e8010efe0 msgr2=0x7f0e801a6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.196+0000 7f0e85e00640 1 --2- 172.21.15.123:0/930679044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e8010efe0 0x7f0e801a6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.196+0000 7f0e85e00640 1 -- 172.21.15.123:0/930679044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e800769f0 msgr2=0x7f0e8019f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.196+0000 7f0e85e00640 1 --2- 172.21.15.123:0/930679044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e800769f0 0x7f0e8019f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.196+0000 7f0e85e00640 1 -- 172.21.15.123:0/930679044 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e801a6d00 con 0x7f0e80075410 2024-05-08T06:11:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.196+0000 7f0e85400640 1 --2- 172.21.15.123:0/930679044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e800769f0 0x7f0e8019f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:11:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.197+0000 7f0e85e00640 1 --2- 172.21.15.123:0/930679044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e80075410 0x7f0e8019f0e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0e7400e9f0 tx=0x7f0e7400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.197+0000 7f0e7aa00640 1 -- 172.21.15.123:0/930679044 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7400ccc0 con 0x7f0e80075410 2024-05-08T06:11:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.197+0000 7f0e7aa00640 1 -- 172.21.15.123:0/930679044 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e74005020 con 0x7f0e80075410 2024-05-08T06:11:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.197+0000 7f0e88800640 1 -- 172.21.15.123:0/930679044 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e801a6ff0 con 0x7f0e80075410 2024-05-08T06:11:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.197+0000 7f0e7aa00640 1 -- 172.21.15.123:0/930679044 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e74004a00 con 0x7f0e80075410 2024-05-08T06:11:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.197+0000 7f0e88800640 1 -- 172.21.15.123:0/930679044 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e801a7450 con 0x7f0e80075410 2024-05-08T06:11:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.200+0000 7f0e8b644740 1 Processor -- start 2024-05-08T06:11:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.200+0000 7f0e8b644740 1 -- start start 2024-05-08T06:11:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.201+0000 7f0e8b644740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e6c6ab120 0x561e6c6ab540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.201+0000 7f0e8b644740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e6c6abb10 0x561e6c6b0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.201+0000 7f0e8b644740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e6c6b0c80 0x561e6c6b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.201+0000 7f0e8b644740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561e6c60f550 con 0x561e6c6b0c80 2024-05-08T06:11:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.201+0000 7f0e8b644740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x561e6c5b4bc0 con 0x561e6c6ab120 2024-05-08T06:11:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.201+0000 7f0e8b644740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x561e6c60b510 con 0x561e6c6abb10 2024-05-08T06:11:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.201+0000 7f0e7aa00640 1 -- 172.21.15.123:0/930679044 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) ==== 149704+0+0 (secure 0 0 0) 0x7f0e74004ba0 con 0x7f0e80075410 2024-05-08T06:11:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.201+0000 7f0e85400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e6c6b0c80 0x561e6c6b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.201+0000 7f0e86800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e6c6ab120 0x561e6c6ab540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.201+0000 7f0e85400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e6c6b0c80 0x561e6c6b3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41356/0 (socket says 172.21.15.123:41356) 2024-05-08T06:11:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.201+0000 7f0e85400640 1 -- 172.21.15.123:0/2888765420 learned_addr learned my addr 172.21.15.123:0/2888765420 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.201+0000 7f0e85e00640 1 --2- 172.21.15.123:0/2888765420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e6c6abb10 0x561e6c6b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.202+0000 7f0e86800640 1 -- 172.21.15.123:0/2888765420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e6c6abb10 msgr2=0x561e6c6b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.202+0000 7f0e86800640 1 --2- 172.21.15.123:0/2888765420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e6c6abb10 0x561e6c6b0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.202+0000 7f0e86800640 1 -- 172.21.15.123:0/2888765420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e6c6b0c80 msgr2=0x561e6c6b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.202+0000 7f0e86800640 1 --2- 172.21.15.123:0/2888765420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e6c6b0c80 0x561e6c6b3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.202+0000 7f0e86800640 1 -- 172.21.15.123:0/2888765420 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e6c321210 con 0x561e6c6ab120 2024-05-08T06:11:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.202+0000 7f0e85400640 1 --2- 172.21.15.123:0/2888765420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e6c6b0c80 0x561e6c6b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:11:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.202+0000 7f0e86800640 1 --2- 172.21.15.123:0/2888765420 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e6c6ab120 0x561e6c6ab540 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c02ade0 tx=0x7f0e7c0093c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f2cac0c7ef5db65 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.202+0000 7f0e6f000640 1 -- 172.21.15.123:0/2888765420 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c031de0 con 0x561e6c6ab120 2024-05-08T06:11:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.202+0000 7f0e6f000640 1 -- 172.21.15.123:0/2888765420 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e7c036030 con 0x561e6c6ab120 2024-05-08T06:11:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.202+0000 7f0e6f000640 1 -- 172.21.15.123:0/2888765420 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c0042a0 con 0x561e6c6ab120 2024-05-08T06:11:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.203+0000 7f0e7aa00640 1 --2- 172.21.15.123:0/930679044 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f0e680b29a0 0x7f0e680b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.203+0000 7f0e8b644740 1 -- 172.21.15.123:0/2888765420 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e6c6ab120 msgr2=0x561e6c6ab540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.203+0000 7f0e8b644740 1 --2- 172.21.15.123:0/2888765420 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e6c6ab120 0x561e6c6ab540 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c02ade0 tx=0x7f0e7c0093c0 comp rx=0 tx=0).stop 2024-05-08T06:11:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.203+0000 7f0e7aa00640 1 -- 172.21.15.123:0/930679044 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f0e740e2290 con 0x7f0e80075410 2024-05-08T06:11:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.203+0000 7f0e85e00640 1 --2- 172.21.15.123:0/930679044 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f0e680b29a0 0x7f0e680b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.203+0000 7f0e8b644740 1 -- 172.21.15.123:0/2888765420 shutdown_connections 2024-05-08T06:11:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.203+0000 7f0e8b644740 1 --2- 172.21.15.123:0/2888765420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e6c6b0c80 0x561e6c6b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.203+0000 7f0e8b644740 1 --2- 172.21.15.123:0/2888765420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e6c6abb10 0x561e6c6b0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.203+0000 7f0e8b644740 1 --2- 172.21.15.123:0/2888765420 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e6c6ab120 0x561e6c6ab540 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.203+0000 7f0e8b644740 1 -- 172.21.15.123:0/2888765420 >> 172.21.15.123:0/2888765420 conn(0x561e6c673230 msgr2=0x561e6c6a6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.203+0000 7f0e8b644740 1 -- 172.21.15.123:0/2888765420 shutdown_connections 2024-05-08T06:11:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.203+0000 7f0e8b644740 1 -- 172.21.15.123:0/2888765420 wait complete. 2024-05-08T06:11:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.204+0000 7f0e8b644740 1 -- start start 2024-05-08T06:11:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.204+0000 7f0e85e00640 1 --2- 172.21.15.123:0/930679044 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f0e680b29a0 0x7f0e680b4e60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x561e6c6ac580 tx=0x7f0e74009f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.204+0000 7f0e8b644740 1 Processor -- start 2024-05-08T06:11:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.204+0000 7f0e8b644740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e6c6ab120 0x561e6c6b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.204+0000 7f0e8b644740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e6c6abb10 0x561e6c6b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.204+0000 7f0e8b644740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e6c6b0c80 0x561e6c6bc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.204+0000 7f0e8b644740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561e6c6b3a50 con 0x561e6c6ab120 2024-05-08T06:11:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.204+0000 7f0e8b644740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x561e6c5c8660 con 0x561e6c6b0c80 2024-05-08T06:11:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.204+0000 7f0e8b644740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x561e6c6b3bd0 con 0x561e6c6abb10 2024-05-08T06:11:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.205+0000 7f0e8b644740 10 client.0 _ll_register_callbacks cb 0x561e6c592400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:11:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.205+0000 7f0e86800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e6c6ab120 0x561e6c6b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.205+0000 7f0e86800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e6c6ab120 0x561e6c6b4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41372/0 (socket says 172.21.15.123:41372) 2024-05-08T06:11:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.205+0000 7f0e86800640 1 -- 172.21.15.123:0/2987876789 learned_addr learned my addr 172.21.15.123:0/2987876789 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.205+0000 7f0e85400640 1 --2- 172.21.15.123:0/2987876789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e6c6abb10 0x561e6c6b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.205+0000 7f0e86800640 1 --2- 172.21.15.123:0/2987876789 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e6c6b0c80 0x561e6c6bc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.205+0000 7f0e86800640 1 -- 172.21.15.123:0/2987876789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e6c6abb10 msgr2=0x561e6c6b54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.205+0000 7f0e86800640 1 --2- 172.21.15.123:0/2987876789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e6c6abb10 0x561e6c6b54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.205+0000 7f0e86800640 1 -- 172.21.15.123:0/2987876789 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e6c6b0c80 msgr2=0x561e6c6bc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.205+0000 7f0e86800640 1 --2- 172.21.15.123:0/2987876789 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e6c6b0c80 0x561e6c6bc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.205+0000 7f0e86800640 1 -- 172.21.15.123:0/2987876789 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e6c3ed620 con 0x561e6c6ab120 2024-05-08T06:11:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.205+0000 7f0e85400640 1 --2- 172.21.15.123:0/2987876789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e6c6abb10 0x561e6c6b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:11:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.206+0000 7f0e86800640 1 --2- 172.21.15.123:0/2987876789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e6c6ab120 0x561e6c6b4f80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c005c40 tx=0x7f0e7c009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a1ceb77f8db272f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.206+0000 7f0e63400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-08T06:11:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.206+0000 7f0e63400640 1 -- 172.21.15.123:0/2987876789 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c058060 con 0x561e6c6ab120 2024-05-08T06:11:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.206+0000 7f0e63400640 1 -- 172.21.15.123:0/2987876789 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f0e7c02f610 con 0x561e6c6ab120 2024-05-08T06:11:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.206+0000 7f0e8b644740 1 -- 172.21.15.123:0/2987876789 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e7c036030 con 0x561e6c6ab120 2024-05-08T06:11:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.206+0000 7f0e63400640 1 -- 172.21.15.123:0/2987876789 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0e7c02f810 con 0x561e6c6ab120 2024-05-08T06:11:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.206+0000 7f0e8b644740 10 client.15078 fetch_fsmap learned FSMap version 24 2024-05-08T06:11:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.206+0000 7f0e8b644740 1 -- 172.21.15.123:0/2987876789 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561e6c6b3a50 con 0x561e6c6ab120 2024-05-08T06:11:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.206+0000 7f0e63400640 1 -- 172.21.15.123:0/2987876789 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e7c036030 con 0x561e6c6ab120 2024-05-08T06:11:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.207+0000 7f0e63400640 1 -- 172.21.15.123:0/2987876789 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f0e7c043070 con 0x561e6c6ab120 2024-05-08T06:11:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.207+0000 7f0e8b644740 10 client.15078 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:11:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.207+0000 7f0e8b644740 10 client.15078 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T06:11:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.207+0000 7f0e8b644740 20 client.15078 populate_metadata read hostname 'smithi123' 2024-05-08T06:11:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.207+0000 7f0e8b644740 1 --2- 172.21.15.123:0/2987876789 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x561e6c6c08e0 0x561e6c6c2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.207+0000 7f0e8b644740 4 client.15078 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:11:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.207+0000 7f0e8b644740 1 -- 172.21.15.123:0/2987876789 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e6c61d160 con 0x561e6c6c08e0 2024-05-08T06:11:09.211 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.207+0000 7f0e85400640 1 --2- 172.21.15.123:0/2987876789 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x561e6c6c08e0 0x561e6c6c2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:09.211 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.208+0000 7f0e85400640 1 --2- 172.21.15.123:0/2987876789 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x561e6c6c08e0 0x561e6c6c2cd0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e29854737342fcce server_cookie=cffa33a3a6c2fd95 in_seq=0 out_seq=0 2024-05-08T06:11:09.212 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.208+0000 7f0e63400640 10 client.15078 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:11:09.212 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.209+0000 7f0e63400640 1 -- 172.21.15.123:0/2987876789 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561e6c61d160 con 0x561e6c6c08e0 2024-05-08T06:11:09.212 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.209+0000 7f0e63400640 10 client.15078 handle_command_reply: tid=0 2024-05-08T06:11:09.212 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.209+0000 7f0e8b644740 1 client.15078 shutdown 2024-05-08T06:11:09.212 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.209+0000 7f0e8b644740 1 -- 172.21.15.123:0/2987876789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e6c6ab120 msgr2=0x561e6c6b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:09.212 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.209+0000 7f0e8b644740 1 --2- 172.21.15.123:0/2987876789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e6c6ab120 0x561e6c6b4f80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c005c40 tx=0x7f0e7c009940 comp rx=0 tx=0).stop 2024-05-08T06:11:09.212 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.209+0000 7f0e8b644740 1 -- 172.21.15.123:0/2987876789 shutdown_connections 2024-05-08T06:11:09.212 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.209+0000 7f0e8b644740 1 --2- 172.21.15.123:0/2987876789 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e6c6b0c80 0x561e6c6bc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:09.212 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.209+0000 7f0e8b644740 1 --2- 172.21.15.123:0/2987876789 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x561e6c6c08e0 0x561e6c6c2cd0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:09.212 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.209+0000 7f0e8b644740 1 --2- 172.21.15.123:0/2987876789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e6c6abb10 0x561e6c6b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:09.212 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.209+0000 7f0e8b644740 1 --2- 172.21.15.123:0/2987876789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e6c6ab120 0x561e6c6b4f80 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:09.212 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.209+0000 7f0e63400640 0 client.15078 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:11:09.212 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.209+0000 7f0e8b644740 1 -- 172.21.15.123:0/2987876789 >> 172.21.15.123:0/2987876789 conn(0x561e6c673230 msgr2=0x561e6c6a5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:09.212 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.209+0000 7f0e8b644740 1 -- 172.21.15.123:0/2987876789 shutdown_connections 2024-05-08T06:11:09.213 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.210+0000 7f0e8b644740 1 -- 172.21.15.123:0/2987876789 wait complete. 2024-05-08T06:11:09.213 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.210+0000 7f0e8b644740 20 client.15078 trim_cache size 0 max 16384 2024-05-08T06:11:09.226 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.226+0000 7f0e8b644740 1 Processor -- start 2024-05-08T06:11:09.226 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.226+0000 7f0e8b644740 1 -- start start 2024-05-08T06:11:09.226 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.226+0000 7f0e8b644740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e6c6ab120 0x561e6c6b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:09.226 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.226+0000 7f0e8b644740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e6c6abb10 0x561e6c6bc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:09.226 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.226+0000 7f0e8b644740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e6c6b0c80 0x561e6c6a5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:09.226 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.226+0000 7f0e8b644740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0e801b29f0 con 0x561e6c6abb10 2024-05-08T06:11:09.227 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.226+0000 7f0e8b644740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0e801b02c0 con 0x561e6c6ab120 2024-05-08T06:11:09.227 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.226+0000 7f0e8b644740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0e801ae110 con 0x561e6c6b0c80 2024-05-08T06:11:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.227+0000 7f0e86800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e6c6ab120 0x561e6c6b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.227+0000 7f0e86800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e6c6ab120 0x561e6c6b52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:56472/0 (socket says 172.21.15.123:56472) 2024-05-08T06:11:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.227+0000 7f0e86800640 1 -- 172.21.15.123:0/1164299187 learned_addr learned my addr 172.21.15.123:0/1164299187 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.227+0000 7f0e86800640 1 --2- 172.21.15.123:0/1164299187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e6c6b0c80 0x561e6c6a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.227+0000 7f0e85400640 1 --2- 172.21.15.123:0/1164299187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e6c6abb10 0x561e6c6bc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.227+0000 7f0e86800640 1 -- 172.21.15.123:0/1164299187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e6c6b0c80 msgr2=0x561e6c6a5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.227+0000 7f0e86800640 1 --2- 172.21.15.123:0/1164299187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e6c6b0c80 0x561e6c6a5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.227+0000 7f0e86800640 1 -- 172.21.15.123:0/1164299187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e6c6abb10 msgr2=0x561e6c6bc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.227+0000 7f0e86800640 1 --2- 172.21.15.123:0/1164299187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e6c6abb10 0x561e6c6bc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.227+0000 7f0e86800640 1 -- 172.21.15.123:0/1164299187 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e6c6abef0 con 0x561e6c6ab120 2024-05-08T06:11:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.228+0000 7f0e86800640 1 --2- 172.21.15.123:0/1164299187 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e6c6ab120 0x561e6c6b52a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c0039f0 tx=0x7f0e7c005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db43484d4f76004d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.228+0000 7f0e6e600640 1 -- 172.21.15.123:0/1164299187 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c058070 con 0x561e6c6ab120 2024-05-08T06:11:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.228+0000 7f0e6e600640 1 -- 172.21.15.123:0/1164299187 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e7c036030 con 0x561e6c6ab120 2024-05-08T06:11:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.228+0000 7f0e6e600640 1 -- 172.21.15.123:0/1164299187 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c062040 con 0x561e6c6ab120 2024-05-08T06:11:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.229+0000 7f0e8b644740 1 -- 172.21.15.123:0/1164299187 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e6c6ab120 msgr2=0x561e6c6b52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.229+0000 7f0e8b644740 1 --2- 172.21.15.123:0/1164299187 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e6c6ab120 0x561e6c6b52a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c0039f0 tx=0x7f0e7c005bd0 comp rx=0 tx=0).stop 2024-05-08T06:11:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.229+0000 7f0e8b644740 1 -- 172.21.15.123:0/1164299187 shutdown_connections 2024-05-08T06:11:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.229+0000 7f0e8b644740 1 --2- 172.21.15.123:0/1164299187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e6c6b0c80 0x561e6c6a5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.229+0000 7f0e8b644740 1 --2- 172.21.15.123:0/1164299187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e6c6abb10 0x561e6c6bc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.229+0000 7f0e8b644740 1 --2- 172.21.15.123:0/1164299187 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e6c6ab120 0x561e6c6b52a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.229+0000 7f0e8b644740 1 -- 172.21.15.123:0/1164299187 >> 172.21.15.123:0/1164299187 conn(0x561e6c673230 msgr2=0x561e6c6b4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.229+0000 7f0e8b644740 1 -- 172.21.15.123:0/1164299187 shutdown_connections 2024-05-08T06:11:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.229+0000 7f0e8b644740 1 -- 172.21.15.123:0/1164299187 wait complete. 2024-05-08T06:11:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.229+0000 7f0e8b644740 1 -- start start 2024-05-08T06:11:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.229+0000 7f0e8b644740 1 Processor -- start 2024-05-08T06:11:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.230+0000 7f0e8b644740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e6c6ab120 0x561e6c6b9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.230+0000 7f0e8b644740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e6c6abb10 0x561e6c6ba310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.230+0000 7f0e8b644740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e6c6b0c80 0x561e6c6bb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.230+0000 7f0e8b644740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0e801ae780 con 0x561e6c6b0c80 2024-05-08T06:11:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.230+0000 7f0e8b644740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0e801afbf0 con 0x561e6c6abb10 2024-05-08T06:11:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.230+0000 7f0e8b644740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0e801aee50 con 0x561e6c6ab120 2024-05-08T06:11:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.230+0000 7f0e8b644740 10 client.0 _ll_register_callbacks cb 0x7f0e70009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:11:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.230+0000 7f0e86800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e6c6ab120 0x561e6c6b9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.230+0000 7f0e86800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e6c6ab120 0x561e6c6b9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:45440/0 (socket says 172.21.15.123:45440) 2024-05-08T06:11:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.230+0000 7f0e86800640 1 -- 172.21.15.123:0/2223317722 learned_addr learned my addr 172.21.15.123:0/2223317722 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.230+0000 7f0e86800640 1 --2- 172.21.15.123:0/2223317722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e6c6b0c80 0x561e6c6bb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.230+0000 7f0e85400640 1 --2- 172.21.15.123:0/2223317722 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e6c6abb10 0x561e6c6ba310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.231+0000 7f0e86800640 1 -- 172.21.15.123:0/2223317722 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e6c6ab120 msgr2=0x561e6c6b9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.231+0000 7f0e86800640 1 --2- 172.21.15.123:0/2223317722 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e6c6ab120 0x561e6c6b9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.231+0000 7f0e86800640 1 -- 172.21.15.123:0/2223317722 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e6c6abb10 msgr2=0x561e6c6ba310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.231+0000 7f0e86800640 1 --2- 172.21.15.123:0/2223317722 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e6c6abb10 0x561e6c6ba310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.231+0000 7f0e86800640 1 -- 172.21.15.123:0/2223317722 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e6c6bb910 con 0x561e6c6b0c80 2024-05-08T06:11:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.231+0000 7f0e86800640 1 --2- 172.21.15.123:0/2223317722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e6c6b0c80 0x561e6c6bb320 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c031f50 tx=0x7f0e7c003a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef245a59bac1b1ce server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.231+0000 7f0e63400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-08T06:11:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.231+0000 7f0e63400640 1 -- 172.21.15.123:0/2223317722 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c058030 con 0x561e6c6b0c80 2024-05-08T06:11:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.231+0000 7f0e63400640 1 -- 172.21.15.123:0/2223317722 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f0e7c02fc20 con 0x561e6c6b0c80 2024-05-08T06:11:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.231+0000 7f0e8b644740 1 -- 172.21.15.123:0/2223317722 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e7c036030 con 0x561e6c6b0c80 2024-05-08T06:11:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.231+0000 7f0e63400640 1 -- 172.21.15.123:0/2223317722 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0e7c062040 con 0x561e6c6b0c80 2024-05-08T06:11:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.231+0000 7f0e8b644740 10 client.15084 fetch_fsmap learned FSMap version 24 2024-05-08T06:11:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.231+0000 7f0e8b644740 1 -- 172.21.15.123:0/2223317722 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561e6c72b2a0 con 0x561e6c6b0c80 2024-05-08T06:11:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.232+0000 7f0e63400640 1 -- 172.21.15.123:0/2223317722 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e7c036030 con 0x561e6c6b0c80 2024-05-08T06:11:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.232+0000 7f0e63400640 1 -- 172.21.15.123:0/2223317722 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f0e7c074020 con 0x561e6c6b0c80 2024-05-08T06:11:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.232+0000 7f0e8b644740 10 client.15084 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:11:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.232+0000 7f0e8b644740 10 client.15084 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T06:11:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.232+0000 7f0e8b644740 20 client.15084 populate_metadata read hostname 'smithi123' 2024-05-08T06:11:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.232+0000 7f0e8b644740 1 --2- 172.21.15.123:0/2223317722 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x561e6c72bb30 0x561e6c72df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.232+0000 7f0e8b644740 4 client.15084 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T06:11:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.232+0000 7f0e8b644740 1 -- 172.21.15.123:0/2223317722 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561e6c616fd0 con 0x561e6c72bb30 2024-05-08T06:11:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.232+0000 7f0e85400640 1 --2- 172.21.15.123:0/2223317722 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x561e6c72bb30 0x561e6c72df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.233+0000 7f0e85400640 1 --2- 172.21.15.123:0/2223317722 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x561e6c72bb30 0x561e6c72df20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48ae94a45888348e server_cookie=20649721d66a78b5 in_seq=0 out_seq=0 2024-05-08T06:11:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.233+0000 7f0e63400640 10 client.15084 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:11:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.240+0000 7f0e63400640 1 -- 172.21.15.123:0/2223317722 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561e6c616fd0 con 0x561e6c72bb30 2024-05-08T06:11:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.240+0000 7f0e63400640 10 client.15084 handle_command_reply: tid=0 2024-05-08T06:11:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.240+0000 7f0e8b644740 1 client.15084 shutdown 2024-05-08T06:11:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.240+0000 7f0e8b644740 1 -- 172.21.15.123:0/2223317722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e6c6b0c80 msgr2=0x561e6c6bb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.240+0000 7f0e8b644740 1 --2- 172.21.15.123:0/2223317722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e6c6b0c80 0x561e6c6bb320 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c031f50 tx=0x7f0e7c003a20 comp rx=0 tx=0).stop 2024-05-08T06:11:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.240+0000 7f0e8b644740 1 -- 172.21.15.123:0/2223317722 shutdown_connections 2024-05-08T06:11:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.240+0000 7f0e8b644740 1 --2- 172.21.15.123:0/2223317722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e6c6b0c80 0x561e6c6bb320 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.240+0000 7f0e8b644740 1 --2- 172.21.15.123:0/2223317722 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e6c6abb10 0x561e6c6ba310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.240+0000 7f0e8b644740 1 --2- 172.21.15.123:0/2223317722 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x561e6c72bb30 0x561e6c72df20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.240+0000 7f0e8b644740 1 --2- 172.21.15.123:0/2223317722 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e6c6ab120 0x561e6c6b9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.240+0000 7f0e63400640 0 client.15084 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:11:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.240+0000 7f0e8b644740 1 -- 172.21.15.123:0/2223317722 >> 172.21.15.123:0/2223317722 conn(0x561e6c673230 msgr2=0x561e6c6a9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.240+0000 7f0e8b644740 1 -- 172.21.15.123:0/2223317722 shutdown_connections 2024-05-08T06:11:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.240+0000 7f0e8b644740 1 -- 172.21.15.123:0/2223317722 wait complete. 2024-05-08T06:11:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.241+0000 7f0e8b644740 20 client.15084 trim_cache size 0 max 16384 2024-05-08T06:11:09.242 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout:{ 2024-05-08T06:11:09.242 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout: "message": "", 2024-05-08T06:11:09.242 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout: "return_code": 0 2024-05-08T06:11:09.242 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout:} 2024-05-08T06:11:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.241+0000 7f0e88800640 1 -- 172.21.15.123:0/930679044 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f0e680b29a0 msgr2=0x7f0e680b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.241+0000 7f0e88800640 1 --2- 172.21.15.123:0/930679044 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f0e680b29a0 0x7f0e680b4e60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x561e6c6ac580 tx=0x7f0e74009f60 comp rx=0 tx=0).stop 2024-05-08T06:11:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.241+0000 7f0e88800640 1 -- 172.21.15.123:0/930679044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e80075410 msgr2=0x7f0e8019f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.241+0000 7f0e88800640 1 --2- 172.21.15.123:0/930679044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e80075410 0x7f0e8019f0e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0e7400e9f0 tx=0x7f0e7400eeb0 comp rx=0 tx=0).stop 2024-05-08T06:11:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.242+0000 7f0e88800640 1 -- 172.21.15.123:0/930679044 shutdown_connections 2024-05-08T06:11:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.242+0000 7f0e88800640 1 --2- 172.21.15.123:0/930679044 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f0e680b29a0 0x7f0e680b4e60 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.242+0000 7f0e88800640 1 --2- 172.21.15.123:0/930679044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e8010efe0 0x7f0e801a6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.242+0000 7f0e88800640 1 --2- 172.21.15.123:0/930679044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e800769f0 0x7f0e8019f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.242+0000 7f0e88800640 1 --2- 172.21.15.123:0/930679044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e80075410 0x7f0e8019f0e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.242+0000 7f0e88800640 1 -- 172.21.15.123:0/930679044 >> 172.21.15.123:0/930679044 conn(0x7f0e800fdfa0 msgr2=0x7f0e80102270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.242+0000 7f0e88800640 1 -- 172.21.15.123:0/930679044 shutdown_connections 2024-05-08T06:11:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:09.242+0000 7f0e88800640 1 -- 172.21.15.123:0/930679044 wait complete. 2024-05-08T06:11:10.401 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:10 smithi123 ceph-mon[29787]: pgmap v481: 97 pgs: 97 active+clean; 672 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-08T06:11:10.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.441+0000 7fdf18a00700 1 -- 172.21.15.123:0/1634979407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf1410fb60 msgr2=0x7fdf1410ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:10.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.441+0000 7fdf18a00700 1 --2- 172.21.15.123:0/1634979407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf1410fb60 0x7fdf1410ff40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdf04009990 tx=0x7fdf04009ca0 comp rx=0 tx=0).stop 2024-05-08T06:11:10.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.441+0000 7fdf18a00700 1 -- 172.21.15.123:0/1634979407 shutdown_connections 2024-05-08T06:11:10.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.441+0000 7fdf18a00700 1 --2- 172.21.15.123:0/1634979407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf1410fb60 0x7fdf1410ff40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:10.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.441+0000 7fdf18a00700 1 --2- 172.21.15.123:0/1634979407 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf1410a490 0x7fdf141169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:10.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.441+0000 7fdf18a00700 1 --2- 172.21.15.123:0/1634979407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf14109ad0 0x7fdf14109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:10.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.441+0000 7fdf18a00700 1 -- 172.21.15.123:0/1634979407 >> 172.21.15.123:0/1634979407 conn(0x7fdf14076c70 msgr2=0x7fdf141082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:10.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.441+0000 7fdf18a00700 1 -- 172.21.15.123:0/1634979407 shutdown_connections 2024-05-08T06:11:10.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.441+0000 7fdf18a00700 1 -- 172.21.15.123:0/1634979407 wait complete. 2024-05-08T06:11:10.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.442+0000 7fdf18a00700 1 Processor -- start 2024-05-08T06:11:10.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.443+0000 7fdf18a00700 1 -- start start 2024-05-08T06:11:10.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.443+0000 7fdf18a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf14109ad0 0x7fdf141a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:10.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.443+0000 7fdf18a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf1410a490 0x7fdf141a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:10.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.443+0000 7fdf18a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf1410fb60 0x7fdf141a4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:10.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.443+0000 7fdf18a00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdf14119d20 con 0x7fdf1410a490 2024-05-08T06:11:10.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.443+0000 7fdf18a00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fdf14119ba0 con 0x7fdf1410fb60 2024-05-08T06:11:10.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.443+0000 7fdf18a00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fdf14119ea0 con 0x7fdf14109ad0 2024-05-08T06:11:10.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.443+0000 7fdf11400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf1410a490 0x7fdf141a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:10.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.443+0000 7fdf11400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf1410a490 0x7fdf141a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41412/0 (socket says 172.21.15.123:41412) 2024-05-08T06:11:10.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.443+0000 7fdf11400700 1 -- 172.21.15.123:0/3882718018 learned_addr learned my addr 172.21.15.123:0/3882718018 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:10.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.443+0000 7fdf11e00700 1 --2- 172.21.15.123:0/3882718018 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf14109ad0 0x7fdf141a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:10.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.443+0000 7fdf12800700 1 --2- 172.21.15.123:0/3882718018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf1410fb60 0x7fdf141a4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:10.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.444+0000 7fdf11400700 1 -- 172.21.15.123:0/3882718018 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf14109ad0 msgr2=0x7fdf141a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:10.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.444+0000 7fdf11400700 1 --2- 172.21.15.123:0/3882718018 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf14109ad0 0x7fdf141a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:10.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.444+0000 7fdf11400700 1 -- 172.21.15.123:0/3882718018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf1410fb60 msgr2=0x7fdf141a4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:10.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.444+0000 7fdf11400700 1 --2- 172.21.15.123:0/3882718018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf1410fb60 0x7fdf141a4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:10.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.444+0000 7fdf11400700 1 -- 172.21.15.123:0/3882718018 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf141a51f0 con 0x7fdf1410a490 2024-05-08T06:11:10.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.444+0000 7fdf11400700 1 --2- 172.21.15.123:0/3882718018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf1410a490 0x7fdf141a0770 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000e9c0 tx=0x7fdf0000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:10.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.444+0000 7fdf0aa00700 1 -- 172.21.15.123:0/3882718018 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0000cbd0 con 0x7fdf1410a490 2024-05-08T06:11:10.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.444+0000 7fdf0aa00700 1 -- 172.21.15.123:0/3882718018 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf0000ee60 con 0x7fdf1410a490 2024-05-08T06:11:10.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.444+0000 7fdf18a00700 1 -- 172.21.15.123:0/3882718018 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf141a54e0 con 0x7fdf1410a490 2024-05-08T06:11:10.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.445+0000 7fdf0aa00700 1 -- 172.21.15.123:0/3882718018 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf000055c0 con 0x7fdf1410a490 2024-05-08T06:11:10.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.445+0000 7fdf18a00700 1 -- 172.21.15.123:0/3882718018 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf141acd60 con 0x7fdf1410a490 2024-05-08T06:11:10.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.449+0000 7fdf0aa00700 1 -- 172.21.15.123:0/3882718018 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 19) ==== 149704+0+0 (secure 0 0 0) 0x7fdf00004750 con 0x7fdf1410a490 2024-05-08T06:11:10.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.449+0000 7fdf23396b80 1 Processor -- start 2024-05-08T06:11:10.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.449+0000 7fdf23396b80 1 -- start start 2024-05-08T06:11:10.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.450+0000 7fdf23396b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56279e772060 0x56279e772440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:10.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.450+0000 7fdf23396b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56279e772980 0x56279e776490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:10.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.450+0000 7fdf23396b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56279e7769d0 0x56279e776e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:10.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.450+0000 7fdf23396b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56279e6c4a20 con 0x56279e772060 2024-05-08T06:11:10.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.450+0000 7fdf23396b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x56279e662760 con 0x56279e7769d0 2024-05-08T06:11:10.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.450+0000 7fdf23396b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56279e6c09e0 con 0x56279e772980 2024-05-08T06:11:10.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.450+0000 7fdf12800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56279e772060 0x56279e772440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:10.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.450+0000 7fdf11e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56279e772980 0x56279e776490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:10.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.450+0000 7fdf12800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56279e772060 0x56279e772440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41424/0 (socket says 172.21.15.123:41424) 2024-05-08T06:11:10.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.450+0000 7fdf12800700 1 -- 172.21.15.123:0/3807319353 learned_addr learned my addr 172.21.15.123:0/3807319353 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:10.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.450+0000 7fdf11400700 1 --2- 172.21.15.123:0/3807319353 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56279e7769d0 0x56279e776e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:10.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.451+0000 7fdf0aa00700 1 --2- 172.21.15.123:0/3882718018 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fdef80b2990 0x7fdef80b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:10.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.451+0000 7fdf11e00700 1 -- 172.21.15.123:0/3807319353 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56279e7769d0 msgr2=0x56279e776e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:10.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.451+0000 7fdf11e00700 1 --2- 172.21.15.123:0/3807319353 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56279e7769d0 0x56279e776e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:10.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.451+0000 7fdf11e00700 1 -- 172.21.15.123:0/3807319353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56279e772060 msgr2=0x56279e772440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:10.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.451+0000 7fdf11e00700 1 --2- 172.21.15.123:0/3807319353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56279e772060 0x56279e772440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:10.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.451+0000 7fdf11e00700 1 -- 172.21.15.123:0/3807319353 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56279e6118d0 con 0x56279e772980 2024-05-08T06:11:10.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.451+0000 7fdf0aa00700 1 -- 172.21.15.123:0/3882718018 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fdf000e25f0 con 0x7fdf1410a490 2024-05-08T06:11:10.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.451+0000 7fdf12800700 1 --2- 172.21.15.123:0/3882718018 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fdef80b2990 0x7fdef80b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:10.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.451+0000 7fdf11e00700 1 --2- 172.21.15.123:0/3807319353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56279e772980 0x56279e776490 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdf04000c00 tx=0x7fdf04005ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a09eac039fec9eb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:10.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.452+0000 7fdeff400700 1 -- 172.21.15.123:0/3807319353 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf04027070 con 0x56279e772980 2024-05-08T06:11:10.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.452+0000 7fdeff400700 1 -- 172.21.15.123:0/3807319353 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf040056a0 con 0x56279e772980 2024-05-08T06:11:10.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.452+0000 7fdeff400700 1 -- 172.21.15.123:0/3807319353 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf04003b50 con 0x56279e772980 2024-05-08T06:11:10.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.453+0000 7fdf23396b80 1 -- 172.21.15.123:0/3807319353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56279e772980 msgr2=0x56279e776490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:10.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.453+0000 7fdf23396b80 1 --2- 172.21.15.123:0/3807319353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56279e772980 0x56279e776490 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdf04000c00 tx=0x7fdf04005ec0 comp rx=0 tx=0).stop 2024-05-08T06:11:10.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.453+0000 7fdf12800700 1 --2- 172.21.15.123:0/3882718018 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fdef80b2990 0x7fdef80b4e50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c009a60 tx=0x7fdf0c004ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:10.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.453+0000 7fdf23396b80 1 -- 172.21.15.123:0/3807319353 shutdown_connections 2024-05-08T06:11:10.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.453+0000 7fdf23396b80 1 --2- 172.21.15.123:0/3807319353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56279e772060 0x56279e772440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:10.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.453+0000 7fdf23396b80 1 --2- 172.21.15.123:0/3807319353 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56279e7769d0 0x56279e776e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:10.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.453+0000 7fdf23396b80 1 --2- 172.21.15.123:0/3807319353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56279e772980 0x56279e776490 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:10.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.453+0000 7fdf23396b80 1 -- 172.21.15.123:0/3807319353 >> 172.21.15.123:0/3807319353 conn(0x56279e6a0b60 msgr2=0x56279e76c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:10.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.453+0000 7fdf23396b80 1 -- 172.21.15.123:0/3807319353 shutdown_connections 2024-05-08T06:11:10.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.453+0000 7fdf23396b80 1 -- 172.21.15.123:0/3807319353 wait complete. 2024-05-08T06:11:10.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.453+0000 7fdf23396b80 1 -- start start 2024-05-08T06:11:10.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.454+0000 7fdf23396b80 1 Processor -- start 2024-05-08T06:11:10.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.454+0000 7fdf23396b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56279e772060 0x56279e77de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:10.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.454+0000 7fdf23396b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56279e772980 0x56279e77b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:10.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.454+0000 7fdf23396b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56279e7769d0 0x56279e77b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:10.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.454+0000 7fdf23396b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56279e77eb80 con 0x56279e7769d0 2024-05-08T06:11:10.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.454+0000 7fdf23396b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x56279e62ab20 con 0x56279e772060 2024-05-08T06:11:10.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.454+0000 7fdf23396b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56279e77ed00 con 0x56279e772980 2024-05-08T06:11:10.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.454+0000 7fdf23396b80 10 client.0 _ll_register_callbacks cb 0x56279e66bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:11:10.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.454+0000 7fdf11e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56279e772060 0x56279e77de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:10.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.454+0000 7fdf12800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56279e7769d0 0x56279e77b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:10.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.454+0000 7fdf11e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56279e772060 0x56279e77de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:56524/0 (socket says 172.21.15.123:56524) 2024-05-08T06:11:10.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.454+0000 7fdf11e00700 1 -- 172.21.15.123:0/860129817 learned_addr learned my addr 172.21.15.123:0/860129817 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:10.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.455+0000 7fdf11400700 1 --2- 172.21.15.123:0/860129817 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56279e772980 0x56279e77b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:10.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.455+0000 7fdf11e00700 1 -- 172.21.15.123:0/860129817 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56279e772980 msgr2=0x56279e77b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:10.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.455+0000 7fdf11e00700 1 --2- 172.21.15.123:0/860129817 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56279e772980 0x56279e77b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:10.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.455+0000 7fdf11e00700 1 -- 172.21.15.123:0/860129817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56279e7769d0 msgr2=0x56279e77b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:10.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.455+0000 7fdf11e00700 1 --2- 172.21.15.123:0/860129817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56279e7769d0 0x56279e77b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:10.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.455+0000 7fdf11e00700 1 -- 172.21.15.123:0/860129817 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56279e68a420 con 0x56279e772060 2024-05-08T06:11:10.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.455+0000 7fdf11400700 1 --2- 172.21.15.123:0/860129817 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56279e772980 0x56279e77b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:11:10.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.455+0000 7fdf11e00700 1 --2- 172.21.15.123:0/860129817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56279e772060 0x56279e77de90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400be00 tx=0x7fdf0400be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4046973bb3cc723a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:10.456 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.455+0000 7fdef3e00700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-05-08T06:11:10.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.455+0000 7fdef3e00700 1 -- 172.21.15.123:0/860129817 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf04027070 con 0x56279e772060 2024-05-08T06:11:10.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.456+0000 7fdf23396b80 1 -- 172.21.15.123:0/860129817 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf040056a0 con 0x56279e772060 2024-05-08T06:11:10.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.456+0000 7fdef3e00700 1 -- 172.21.15.123:0/860129817 <== mon.2 v2:172.21.15.163:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fdf04003890 con 0x56279e772060 2024-05-08T06:11:10.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.456+0000 7fdef3e00700 1 -- 172.21.15.123:0/860129817 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdf04003a90 con 0x56279e772060 2024-05-08T06:11:10.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.456+0000 7fdf23396b80 10 client.25211 fetch_fsmap learned FSMap version 24 2024-05-08T06:11:10.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.456+0000 7fdf23396b80 1 -- 172.21.15.123:0/860129817 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56279e789430 con 0x56279e772060 2024-05-08T06:11:10.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.456+0000 7fdef3e00700 1 -- 172.21.15.123:0/860129817 <== mon.2 v2:172.21.15.163:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf040056a0 con 0x56279e772060 2024-05-08T06:11:10.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.457+0000 7fdef3e00700 1 -- 172.21.15.123:0/860129817 <== mon.2 v2:172.21.15.163:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fdf04034030 con 0x56279e772060 2024-05-08T06:11:10.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.457+0000 7fdf23396b80 10 client.25211 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:11:10.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.457+0000 7fdf23396b80 10 client.25211 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T06:11:10.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.457+0000 7fdf23396b80 20 client.25211 populate_metadata read hostname 'smithi123' 2024-05-08T06:11:10.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.457+0000 7fdf23396b80 1 --2- 172.21.15.123:0/860129817 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x56279e78b900 0x56279e78dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:10.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.457+0000 7fdf23396b80 4 client.25211 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:11:10.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.457+0000 7fdf23396b80 1 -- 172.21.15.123:0/860129817 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56279e73f0a0 con 0x56279e78b900 2024-05-08T06:11:10.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.457+0000 7fdf11400700 1 --2- 172.21.15.123:0/860129817 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x56279e78b900 0x56279e78dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:10.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.458+0000 7fdf11400700 1 --2- 172.21.15.123:0/860129817 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x56279e78b900 0x56279e78dcf0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c13192a436718c5 server_cookie=1cf2b72591e1c255 in_seq=0 out_seq=0 2024-05-08T06:11:10.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.458+0000 7fdef3e00700 10 client.25211 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:11:10.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.458+0000 7fdef3e00700 1 -- 172.21.15.123:0/860129817 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56279e73f0a0 con 0x56279e78b900 2024-05-08T06:11:10.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.458+0000 7fdef3e00700 10 client.25211 handle_command_reply: tid=0 2024-05-08T06:11:10.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.459+0000 7fdf23396b80 1 client.25211 shutdown 2024-05-08T06:11:10.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.459+0000 7fdf23396b80 1 -- 172.21.15.123:0/860129817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56279e772060 msgr2=0x56279e77de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:10.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.459+0000 7fdf23396b80 1 --2- 172.21.15.123:0/860129817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56279e772060 0x56279e77de90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400be00 tx=0x7fdf0400be30 comp rx=0 tx=0).stop 2024-05-08T06:11:10.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.459+0000 7fdf23396b80 1 -- 172.21.15.123:0/860129817 shutdown_connections 2024-05-08T06:11:10.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.459+0000 7fdf23396b80 1 --2- 172.21.15.123:0/860129817 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x56279e78b900 0x56279e78dcf0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:10.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.459+0000 7fdf23396b80 1 --2- 172.21.15.123:0/860129817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56279e7769d0 0x56279e77b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:10.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.459+0000 7fdf23396b80 1 --2- 172.21.15.123:0/860129817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56279e772060 0x56279e77de90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:10.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.459+0000 7fdef3e00700 0 client.25211 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:11:10.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.459+0000 7fdf23396b80 1 --2- 172.21.15.123:0/860129817 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56279e772980 0x56279e77b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:10.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.459+0000 7fdf23396b80 1 -- 172.21.15.123:0/860129817 >> 172.21.15.123:0/860129817 conn(0x56279e6a0b60 msgr2=0x56279e7672b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:10.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.459+0000 7fdf23396b80 1 -- 172.21.15.123:0/860129817 shutdown_connections 2024-05-08T06:11:10.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.459+0000 7fdf23396b80 1 -- 172.21.15.123:0/860129817 wait complete. 2024-05-08T06:11:10.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.459+0000 7fdf23396b80 20 client.25211 trim_cache size 0 max 16384 2024-05-08T06:11:10.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.481+0000 7fdf23396b80 1 Processor -- start 2024-05-08T06:11:10.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.481+0000 7fdf23396b80 1 -- start start 2024-05-08T06:11:10.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.481+0000 7fdf23396b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56279e772060 0x56279e770e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:10.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.481+0000 7fdf23396b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56279e772980 0x56279e7713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:10.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.481+0000 7fdf23396b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56279e7769d0 0x56279e7718f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:10.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.481+0000 7fdf23396b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdf141b69b0 con 0x56279e772060 2024-05-08T06:11:10.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.481+0000 7fdf23396b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fdf141b4240 con 0x56279e7769d0 2024-05-08T06:11:10.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.481+0000 7fdf23396b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fdf141b2090 con 0x56279e772980 2024-05-08T06:11:10.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.481+0000 7fdf11e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56279e772060 0x56279e770e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:10.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.482+0000 7fdf11e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56279e772060 0x56279e770e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41432/0 (socket says 172.21.15.123:41432) 2024-05-08T06:11:10.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.482+0000 7fdf11e00700 1 -- 172.21.15.123:0/1903008588 learned_addr learned my addr 172.21.15.123:0/1903008588 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:10.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.482+0000 7fdf11400700 1 --2- 172.21.15.123:0/1903008588 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56279e772980 0x56279e7713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:10.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.482+0000 7fdf12800700 1 --2- 172.21.15.123:0/1903008588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56279e7769d0 0x56279e7718f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:10.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.482+0000 7fdf11e00700 1 -- 172.21.15.123:0/1903008588 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56279e772980 msgr2=0x56279e7713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:10.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.482+0000 7fdf11e00700 1 --2- 172.21.15.123:0/1903008588 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56279e772980 0x56279e7713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:10.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.482+0000 7fdf11e00700 1 -- 172.21.15.123:0/1903008588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56279e7769d0 msgr2=0x56279e7718f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:10.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.482+0000 7fdf11e00700 1 --2- 172.21.15.123:0/1903008588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56279e7769d0 0x56279e7718f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:10.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.482+0000 7fdf11e00700 1 -- 172.21.15.123:0/1903008588 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56279e77bbd0 con 0x56279e772060 2024-05-08T06:11:10.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.482+0000 7fdf11e00700 1 --2- 172.21.15.123:0/1903008588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56279e772060 0x56279e770e70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdf04009990 tx=0x7fdf04003a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2774694bf988ffbd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:10.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.482+0000 7fdefea00700 1 -- 172.21.15.123:0/1903008588 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf04027070 con 0x56279e772060 2024-05-08T06:11:10.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.483+0000 7fdefea00700 1 -- 172.21.15.123:0/1903008588 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf04028040 con 0x56279e772060 2024-05-08T06:11:10.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.483+0000 7fdefea00700 1 -- 172.21.15.123:0/1903008588 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0400f460 con 0x56279e772060 2024-05-08T06:11:10.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.483+0000 7fdf23396b80 1 -- 172.21.15.123:0/1903008588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56279e772060 msgr2=0x56279e770e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:10.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.483+0000 7fdf23396b80 1 --2- 172.21.15.123:0/1903008588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56279e772060 0x56279e770e70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdf04009990 tx=0x7fdf04003a20 comp rx=0 tx=0).stop 2024-05-08T06:11:10.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.483+0000 7fdf23396b80 1 -- 172.21.15.123:0/1903008588 shutdown_connections 2024-05-08T06:11:10.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.483+0000 7fdf23396b80 1 --2- 172.21.15.123:0/1903008588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56279e772060 0x56279e770e70 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:10.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.483+0000 7fdf23396b80 1 --2- 172.21.15.123:0/1903008588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56279e7769d0 0x56279e7718f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:10.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.483+0000 7fdf23396b80 1 --2- 172.21.15.123:0/1903008588 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56279e772980 0x56279e7713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:10.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.483+0000 7fdf23396b80 1 -- 172.21.15.123:0/1903008588 >> 172.21.15.123:0/1903008588 conn(0x56279e6a0b60 msgr2=0x56279e7672b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:10.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.483+0000 7fdf23396b80 1 -- 172.21.15.123:0/1903008588 shutdown_connections 2024-05-08T06:11:10.485 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.483+0000 7fdf23396b80 1 -- 172.21.15.123:0/1903008588 wait complete. 2024-05-08T06:11:10.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.484+0000 7fdf23396b80 1 -- start start 2024-05-08T06:11:10.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.484+0000 7fdf23396b80 1 Processor -- start 2024-05-08T06:11:10.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.484+0000 7fdf23396b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56279e772060 0x56279e7911d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:10.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.485+0000 7fdf23396b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56279e772980 0x56279e78c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:10.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.485+0000 7fdf23396b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56279e7769d0 0x56279e78cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:10.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.485+0000 7fdf23396b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdf141b2700 con 0x56279e772980 2024-05-08T06:11:10.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.485+0000 7fdf23396b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fdf141b3b70 con 0x56279e772060 2024-05-08T06:11:10.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.485+0000 7fdf23396b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fdf141b2dd0 con 0x56279e7769d0 2024-05-08T06:11:10.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.485+0000 7fdf23396b80 10 client.0 _ll_register_callbacks cb 0x56279e66bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:11:10.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.485+0000 7fdf11e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56279e772060 0x56279e7911d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:10.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.485+0000 7fdf11e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56279e772060 0x56279e7911d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:56546/0 (socket says 172.21.15.123:56546) 2024-05-08T06:11:10.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.485+0000 7fdf11e00700 1 -- 172.21.15.123:0/1292981043 learned_addr learned my addr 172.21.15.123:0/1292981043 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:10.486 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.485+0000 7fdf11400700 1 --2- 172.21.15.123:0/1292981043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56279e772980 0x56279e78c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:10.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.485+0000 7fdf12800700 1 --2- 172.21.15.123:0/1292981043 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56279e7769d0 0x56279e78cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:10.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.485+0000 7fdf11e00700 1 -- 172.21.15.123:0/1292981043 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56279e7769d0 msgr2=0x56279e78cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:10.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.485+0000 7fdf11e00700 1 --2- 172.21.15.123:0/1292981043 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56279e7769d0 0x56279e78cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:10.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.485+0000 7fdf11e00700 1 -- 172.21.15.123:0/1292981043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56279e772980 msgr2=0x56279e78c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:10.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.485+0000 7fdf11e00700 1 --2- 172.21.15.123:0/1292981043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56279e772980 0x56279e78c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:10.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.485+0000 7fdf11e00700 1 -- 172.21.15.123:0/1292981043 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56279e78d370 con 0x56279e772060 2024-05-08T06:11:10.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.485+0000 7fdf11400700 1 --2- 172.21.15.123:0/1292981043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56279e772980 0x56279e78c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:11:10.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.485+0000 7fdf12800700 1 --2- 172.21.15.123:0/1292981043 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56279e7769d0 0x56279e78cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:11:10.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.486+0000 7fdf11e00700 1 --2- 172.21.15.123:0/1292981043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56279e772060 0x56279e7911d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdf04005010 tx=0x7fdf040039b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e10f6a0ef1bdccae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:10.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.486+0000 7fdef3e00700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-05-08T06:11:10.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.486+0000 7fdef3e00700 1 -- 172.21.15.123:0/1292981043 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf04027040 con 0x56279e772060 2024-05-08T06:11:10.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.486+0000 7fdef3e00700 1 -- 172.21.15.123:0/1292981043 <== mon.2 v2:172.21.15.163:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fdf0402c440 con 0x56279e772060 2024-05-08T06:11:10.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.486+0000 7fdf23396b80 1 -- 172.21.15.123:0/1292981043 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf04028040 con 0x56279e772060 2024-05-08T06:11:10.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.486+0000 7fdef3e00700 1 -- 172.21.15.123:0/1292981043 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdf0400fb60 con 0x56279e772060 2024-05-08T06:11:10.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.486+0000 7fdf23396b80 10 client.25217 fetch_fsmap learned FSMap version 24 2024-05-08T06:11:10.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.486+0000 7fdf23396b80 1 -- 172.21.15.123:0/1292981043 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56279e7f5dd0 con 0x56279e772060 2024-05-08T06:11:10.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.487+0000 7fdef3e00700 1 -- 172.21.15.123:0/1292981043 <== mon.2 v2:172.21.15.163:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf04028040 con 0x56279e772060 2024-05-08T06:11:10.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.487+0000 7fdef3e00700 1 -- 172.21.15.123:0/1292981043 <== mon.2 v2:172.21.15.163:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fdf04034040 con 0x56279e772060 2024-05-08T06:11:10.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.487+0000 7fdf23396b80 10 client.25217 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:11:10.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.487+0000 7fdf23396b80 10 client.25217 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T06:11:10.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.487+0000 7fdf23396b80 20 client.25217 populate_metadata read hostname 'smithi123' 2024-05-08T06:11:10.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.487+0000 7fdf23396b80 1 --2- 172.21.15.123:0/1292981043 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x56279e7f6310 0x56279e7f8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:10.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.487+0000 7fdf23396b80 4 client.25217 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T06:11:10.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.487+0000 7fdf23396b80 1 -- 172.21.15.123:0/1292981043 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56279e6d2400 con 0x56279e7f6310 2024-05-08T06:11:10.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.487+0000 7fdf11400700 1 --2- 172.21.15.123:0/1292981043 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x56279e7f6310 0x56279e7f8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:10.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.488+0000 7fdf11400700 1 --2- 172.21.15.123:0/1292981043 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x56279e7f6310 0x56279e7f8730 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea0b62c6fada24bb server_cookie=f3031180d0dca2b in_seq=0 out_seq=0 2024-05-08T06:11:10.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.488+0000 7fdef3e00700 10 client.25217 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:11:10.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.488+0000 7fdef3e00700 1 -- 172.21.15.123:0/1292981043 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56279e6d2400 con 0x56279e7f6310 2024-05-08T06:11:10.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.488+0000 7fdef3e00700 10 client.25217 handle_command_reply: tid=0 2024-05-08T06:11:10.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.488+0000 7fdf23396b80 1 client.25217 shutdown 2024-05-08T06:11:10.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.488+0000 7fdf23396b80 1 -- 172.21.15.123:0/1292981043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56279e772060 msgr2=0x56279e7911d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:10.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.488+0000 7fdf23396b80 1 --2- 172.21.15.123:0/1292981043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56279e772060 0x56279e7911d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdf04005010 tx=0x7fdf040039b0 comp rx=0 tx=0).stop 2024-05-08T06:11:10.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.488+0000 7fdf23396b80 1 -- 172.21.15.123:0/1292981043 shutdown_connections 2024-05-08T06:11:10.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.488+0000 7fdf23396b80 1 --2- 172.21.15.123:0/1292981043 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x56279e7f6310 0x56279e7f8730 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:10.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.488+0000 7fdf23396b80 1 --2- 172.21.15.123:0/1292981043 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56279e7769d0 0x56279e78cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:10.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.488+0000 7fdf23396b80 1 --2- 172.21.15.123:0/1292981043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56279e772060 0x56279e7911d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:10.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.488+0000 7fdf23396b80 1 --2- 172.21.15.123:0/1292981043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56279e772980 0x56279e78c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:10.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.488+0000 7fdef3e00700 0 client.25217 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:11:10.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.488+0000 7fdf23396b80 1 -- 172.21.15.123:0/1292981043 >> 172.21.15.123:0/1292981043 conn(0x56279e6a0b60 msgr2=0x56279e776db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:10.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.488+0000 7fdf23396b80 1 -- 172.21.15.123:0/1292981043 shutdown_connections 2024-05-08T06:11:10.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.488+0000 7fdf23396b80 1 -- 172.21.15.123:0/1292981043 wait complete. 2024-05-08T06:11:10.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.489+0000 7fdf23396b80 20 client.25217 trim_cache size 0 max 16384 2024-05-08T06:11:10.492 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-05-08T06:11:10.492 INFO:teuthology.orchestra.run.smithi123.stdout: "status": "no active scrubs running", 2024-05-08T06:11:10.492 INFO:teuthology.orchestra.run.smithi123.stdout: "scrubs": {} 2024-05-08T06:11:10.492 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-05-08T06:11:10.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.489+0000 7fdf18a00700 1 -- 172.21.15.123:0/3882718018 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fdef80b2990 msgr2=0x7fdef80b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:10.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.489+0000 7fdf18a00700 1 --2- 172.21.15.123:0/3882718018 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fdef80b2990 0x7fdef80b4e50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c009a60 tx=0x7fdf0c004ad0 comp rx=0 tx=0).stop 2024-05-08T06:11:10.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.489+0000 7fdf18a00700 1 -- 172.21.15.123:0/3882718018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf1410a490 msgr2=0x7fdf141a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:10.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.489+0000 7fdf18a00700 1 --2- 172.21.15.123:0/3882718018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf1410a490 0x7fdf141a0770 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000e9c0 tx=0x7fdf0000ecd0 comp rx=0 tx=0).stop 2024-05-08T06:11:10.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.490+0000 7fdf18a00700 1 -- 172.21.15.123:0/3882718018 shutdown_connections 2024-05-08T06:11:10.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.490+0000 7fdf18a00700 1 --2- 172.21.15.123:0/3882718018 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fdef80b2990 0x7fdef80b4e50 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:10.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.490+0000 7fdf18a00700 1 --2- 172.21.15.123:0/3882718018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf1410fb60 0x7fdf141a4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:10.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.490+0000 7fdf18a00700 1 --2- 172.21.15.123:0/3882718018 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf14109ad0 0x7fdf141a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:10.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.490+0000 7fdf18a00700 1 --2- 172.21.15.123:0/3882718018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf1410a490 0x7fdf141a0770 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:10.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.490+0000 7fdf18a00700 1 -- 172.21.15.123:0/3882718018 >> 172.21.15.123:0/3882718018 conn(0x7fdf14076c70 msgr2=0x7fdf14077270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:10.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.490+0000 7fdf18a00700 1 -- 172.21.15.123:0/3882718018 shutdown_connections 2024-05-08T06:11:10.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:10.490+0000 7fdf18a00700 1 -- 172.21.15.123:0/3882718018 wait complete. 2024-05-08T06:11:10.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:10 smithi163 ceph-mon[34520]: pgmap v481: 97 pgs: 97 active+clean; 672 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-08T06:11:10.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:10 smithi192 ceph-mon[34664]: pgmap v481: 97 pgs: 97 active+clean; 672 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-08T06:11:11.584 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T06:11:11.585 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T06:11:12.026 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:11:12.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:12 smithi123 ceph-mon[29787]: pgmap v482: 97 pgs: 97 active+clean; 672 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-08T06:11:12.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:12 smithi163 ceph-mon[34520]: pgmap v482: 97 pgs: 97 active+clean; 672 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-08T06:11:12.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:12 smithi192 ceph-mon[34664]: pgmap v482: 97 pgs: 97 active+clean; 672 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-08T06:11:14.397 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:14 smithi123 ceph-mon[29787]: pgmap v483: 97 pgs: 97 active+clean; 672 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-05-08T06:11:14.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.443+0000 7fbc00000700 1 -- 172.21.15.123:0/316948514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbf806b180 msgr2=0x7fbbf806b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:14.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.443+0000 7fbc00000700 1 --2- 172.21.15.123:0/316948514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbf806b180 0x7fbbf806b600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4009990 tx=0x7fbbe4009ca0 comp rx=0 tx=0).stop 2024-05-08T06:11:14.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.443+0000 7fbc00000700 1 -- 172.21.15.123:0/316948514 shutdown_connections 2024-05-08T06:11:14.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.443+0000 7fbc00000700 1 --2- 172.21.15.123:0/316948514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbf806bb40 0x7fbbf8110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:14.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.443+0000 7fbc00000700 1 --2- 172.21.15.123:0/316948514 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbbf8109510 0x7fbbf81098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:14.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.443+0000 7fbc00000700 1 --2- 172.21.15.123:0/316948514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbf806b180 0x7fbbf806b600 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:14.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.443+0000 7fbc00000700 1 -- 172.21.15.123:0/316948514 >> 172.21.15.123:0/316948514 conn(0x7fbbf8079350 msgr2=0x7fbbf8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:14.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.443+0000 7fbc00000700 1 -- 172.21.15.123:0/316948514 shutdown_connections 2024-05-08T06:11:14.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.443+0000 7fbc00000700 1 -- 172.21.15.123:0/316948514 wait complete. 2024-05-08T06:11:14.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.445+0000 7fbc00000700 1 Processor -- start 2024-05-08T06:11:14.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.445+0000 7fbc00000700 1 -- start start 2024-05-08T06:11:14.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.445+0000 7fbc00000700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbf806b180 0x7fbbf819bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:14.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.445+0000 7fbc00000700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbbf806bb40 0x7fbbf819c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:14.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.445+0000 7fbc00000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbf8109510 0x7fbbf81a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:14.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.445+0000 7fbc00000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf8113620 con 0x7fbbf8109510 2024-05-08T06:11:14.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.445+0000 7fbc00000700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf81134a0 con 0x7fbbf806b180 2024-05-08T06:11:14.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.445+0000 7fbc00000700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf81137a0 con 0x7fbbf806bb40 2024-05-08T06:11:14.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.446+0000 7fbbfd600700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbf806b180 0x7fbbf819bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:14.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.446+0000 7fbbfe000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbf8109510 0x7fbbf81a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:14.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.446+0000 7fbbfd600700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbf806b180 0x7fbbf819bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:56570/0 (socket says 172.21.15.123:56570) 2024-05-08T06:11:14.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.446+0000 7fbbfd600700 1 -- 172.21.15.123:0/3311067726 learned_addr learned my addr 172.21.15.123:0/3311067726 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:14.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.446+0000 7fbbfcc00700 1 --2- 172.21.15.123:0/3311067726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbbf806bb40 0x7fbbf819c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:14.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.446+0000 7fbbfd600700 1 -- 172.21.15.123:0/3311067726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbbf806bb40 msgr2=0x7fbbf819c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:14.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.446+0000 7fbbfd600700 1 --2- 172.21.15.123:0/3311067726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbbf806bb40 0x7fbbf819c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:14.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.446+0000 7fbbfd600700 1 -- 172.21.15.123:0/3311067726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbf8109510 msgr2=0x7fbbf81a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:14.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.446+0000 7fbbfd600700 1 --2- 172.21.15.123:0/3311067726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbf8109510 0x7fbbf81a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:14.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.446+0000 7fbbfd600700 1 -- 172.21.15.123:0/3311067726 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf81a0d60 con 0x7fbbf806b180 2024-05-08T06:11:14.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.447+0000 7fbbfd600700 1 --2- 172.21.15.123:0/3311067726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbf806b180 0x7fbbf819bd50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbbec00d980 tx=0x7fbbec00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:14.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.447+0000 7fbbf6000700 1 -- 172.21.15.123:0/3311067726 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbec016070 con 0x7fbbf806b180 2024-05-08T06:11:14.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.447+0000 7fbbf6000700 1 -- 172.21.15.123:0/3311067726 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbec004830 con 0x7fbbf806b180 2024-05-08T06:11:14.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.447+0000 7fbbf6000700 1 -- 172.21.15.123:0/3311067726 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbec005540 con 0x7fbbf806b180 2024-05-08T06:11:14.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.447+0000 7fbc00000700 1 -- 172.21.15.123:0/3311067726 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbf81a1050 con 0x7fbbf806b180 2024-05-08T06:11:14.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.447+0000 7fbc00000700 1 -- 172.21.15.123:0/3311067726 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbf81a88d0 con 0x7fbbf806b180 2024-05-08T06:11:14.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.448+0000 7fbc00000700 1 -- 172.21.15.123:0/3311067726 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbf806a190 con 0x7fbbf806b180 2024-05-08T06:11:14.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.451+0000 7fbbf6000700 1 -- 172.21.15.123:0/3311067726 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 19) ==== 149704+0+0 (secure 0 0 0) 0x7fbbec004220 con 0x7fbbf806b180 2024-05-08T06:11:14.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.452+0000 7fbbf6000700 1 --2- 172.21.15.123:0/3311067726 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fbbe00b2910 0x7fbbe00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:14.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.452+0000 7fbbf6000700 1 -- 172.21.15.123:0/3311067726 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fbbec0e26f0 con 0x7fbbf806b180 2024-05-08T06:11:14.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.452+0000 7fbbfcc00700 1 --2- 172.21.15.123:0/3311067726 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fbbe00b2910 0x7fbbe00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:14.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.453+0000 7fbbfcc00700 1 --2- 172.21.15.123:0/3311067726 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fbbe00b2910 0x7fbbe00b4dd0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbbf81044f0 tx=0x7fbbe4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:14.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.454+0000 7fbbf6000700 1 -- 172.21.15.123:0/3311067726 <== mon.2 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbbec01f020 con 0x7fbbf806b180 2024-05-08T06:11:14.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.736+0000 7fbc00000700 1 -- 172.21.15.123:0/3311067726 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbbf81a8aa0 con 0x7fbbf806b180 2024-05-08T06:11:14.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.738+0000 7fbbf6000700 1 -- 172.21.15.123:0/3311067726 <== mon.2 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11885 (secure 0 0 0) 0x7fbbec0aa260 con 0x7fbbf806b180 2024-05-08T06:11:14.738 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:11:14.738 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3646476522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3646476522},{"type":"v1","addr":"172.21.15.123:6835","nonce":3646476522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2763706497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2763706497},{"type":"v1","addr":"172.21.15.123:6837","nonce":2763706497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/274635265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":274635265},{"type":"v1","addr":"172.21.15.123:6839","nonce":274635265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/175423115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":175423115},{"type":"v1","addr":"172.21.15.192:6833","nonce":175423115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/3263009472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3263009472},{"type":"v1","addr":"172.21.15.163:6833","nonce":3263009472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3308612826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3308612826},{"type":"v1","addr":"172.21.15.163:6835","nonce":3308612826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T06:05:54.755550+0000","modified":"2024-05-08T06:10:12.600417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24512,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/202513680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":202513680},{"type":"v1","addr":"172.21.15.192:6835","nonce":202513680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/2483545969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2483545969},{"type":"v1","addr":"172.21.15.192:6837","nonce":2483545969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/660511838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":660511838},{"type":"v1","addr":"172.21.15.163:6837","nonce":660511838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.192:6839/3598682360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":3598682360},{"type":"v1","addr":"172.21.15.192:6839","nonce":3598682360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6841/1571726231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":1571726231},{"type":"v1","addr":"172.21.15.123:6841","nonce":1571726231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.163:6839/1568161515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":1568161515},{"type":"v1","addr":"172.21.15.163:6839","nonce":1568161515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24512,24496,24524]},"id":1}]} 2024-05-08T06:11:14.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.741+0000 7fbc00000700 1 -- 172.21.15.123:0/3311067726 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fbbe00b2910 msgr2=0x7fbbe00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:14.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.741+0000 7fbc00000700 1 --2- 172.21.15.123:0/3311067726 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fbbe00b2910 0x7fbbe00b4dd0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbbf81044f0 tx=0x7fbbe4005960 comp rx=0 tx=0).stop 2024-05-08T06:11:14.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.741+0000 7fbc00000700 1 -- 172.21.15.123:0/3311067726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbf806b180 msgr2=0x7fbbf819bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:14.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.741+0000 7fbc00000700 1 --2- 172.21.15.123:0/3311067726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbf806b180 0x7fbbf819bd50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbbec00d980 tx=0x7fbbec00dc90 comp rx=0 tx=0).stop 2024-05-08T06:11:14.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.741+0000 7fbc00000700 1 -- 172.21.15.123:0/3311067726 shutdown_connections 2024-05-08T06:11:14.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.741+0000 7fbc00000700 1 --2- 172.21.15.123:0/3311067726 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fbbe00b2910 0x7fbbe00b4dd0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:14.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.741+0000 7fbc00000700 1 --2- 172.21.15.123:0/3311067726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbf8109510 0x7fbbf81a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:14.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.741+0000 7fbc00000700 1 --2- 172.21.15.123:0/3311067726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbf806b180 0x7fbbf819bd50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:14.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.741+0000 7fbc00000700 1 --2- 172.21.15.123:0/3311067726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbbf806bb40 0x7fbbf819c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:14.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.741+0000 7fbc00000700 1 -- 172.21.15.123:0/3311067726 >> 172.21.15.123:0/3311067726 conn(0x7fbbf8079350 msgr2=0x7fbbf810e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:14.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.741+0000 7fbc00000700 1 -- 172.21.15.123:0/3311067726 shutdown_connections 2024-05-08T06:11:14.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:14.742+0000 7fbc00000700 1 -- 172.21.15.123:0/3311067726 wait complete. 2024-05-08T06:11:14.743 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 24 2024-05-08T06:11:14.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:14 smithi163 ceph-mon[34520]: pgmap v483: 97 pgs: 97 active+clean; 672 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-05-08T06:11:14.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:14 smithi192 ceph-mon[34664]: pgmap v483: 97 pgs: 97 active+clean; 672 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-05-08T06:11:15.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:15 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/3311067726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:11:15.751 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-08T06:11:15.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:15 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/3311067726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:11:15.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:15 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/3311067726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:11:16.191 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:11:16.597 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:16 smithi123 ceph-mon[29787]: pgmap v484: 97 pgs: 97 active+clean; 673 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-05-08T06:11:16.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:16 smithi163 ceph-mon[34520]: pgmap v484: 97 pgs: 97 active+clean; 673 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-05-08T06:11:16.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:16 smithi192 ceph-mon[34664]: pgmap v484: 97 pgs: 97 active+clean; 673 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-05-08T06:11:18.541 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:18 smithi123 ceph-mon[29787]: pgmap v485: 97 pgs: 97 active+clean; 673 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-08T06:11:18.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.594+0000 7f52f9c00700 1 -- 172.21.15.123:0/1689602096 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52f410b6c0 msgr2=0x7f52f410baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:18.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.594+0000 7f52f9c00700 1 --2- 172.21.15.123:0/1689602096 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52f410b6c0 0x7f52f410baa0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f52e4009990 tx=0x7f52e4009ca0 comp rx=0 tx=0).stop 2024-05-08T06:11:18.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.594+0000 7f52f9c00700 1 -- 172.21.15.123:0/1689602096 shutdown_connections 2024-05-08T06:11:18.595 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.594+0000 7f52f9c00700 1 --2- 172.21.15.123:0/1689602096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52f4106080 0x7f52f4112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:18.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.594+0000 7f52f9c00700 1 --2- 172.21.15.123:0/1689602096 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52f410b6c0 0x7f52f410baa0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:18.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.594+0000 7f52f9c00700 1 --2- 172.21.15.123:0/1689602096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52f41056c0 0x7f52f4105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:18.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.594+0000 7f52f9c00700 1 -- 172.21.15.123:0/1689602096 >> 172.21.15.123:0/1689602096 conn(0x7f52f41011b0 msgr2=0x7f52f41035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:18.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.595+0000 7f52f9c00700 1 -- 172.21.15.123:0/1689602096 shutdown_connections 2024-05-08T06:11:18.596 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.595+0000 7f52f9c00700 1 -- 172.21.15.123:0/1689602096 wait complete. 2024-05-08T06:11:18.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.596+0000 7f52f9c00700 1 Processor -- start 2024-05-08T06:11:18.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.596+0000 7f52f9c00700 1 -- start start 2024-05-08T06:11:18.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.596+0000 7f52f9c00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52f41056c0 0x7f52f419bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:18.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.596+0000 7f52f9c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52f4106080 0x7f52f419c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:18.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.596+0000 7f52f9c00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52f410b6c0 0x7f52f41a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:18.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.596+0000 7f52f9c00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f52f4115950 con 0x7f52f4106080 2024-05-08T06:11:18.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.596+0000 7f52f9c00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f52f41157d0 con 0x7f52f41056c0 2024-05-08T06:11:18.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.596+0000 7f52f9c00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f52f4115ad0 con 0x7f52f410b6c0 2024-05-08T06:11:18.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.597+0000 7f52f2a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52f41056c0 0x7f52f419bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:18.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.597+0000 7f52f2a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52f41056c0 0x7f52f419bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:46300/0 (socket says 172.21.15.123:46300) 2024-05-08T06:11:18.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.597+0000 7f52f2a00700 1 -- 172.21.15.123:0/1231432069 learned_addr learned my addr 172.21.15.123:0/1231432069 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:18.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.597+0000 7f52f2000700 1 --2- 172.21.15.123:0/1231432069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52f4106080 0x7f52f419c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:18.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.597+0000 7f52f3400700 1 --2- 172.21.15.123:0/1231432069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52f410b6c0 0x7f52f41a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:18.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.597+0000 7f52f2a00700 1 -- 172.21.15.123:0/1231432069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52f410b6c0 msgr2=0x7f52f41a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:18.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.597+0000 7f52f2a00700 1 --2- 172.21.15.123:0/1231432069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52f410b6c0 0x7f52f41a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:18.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.597+0000 7f52f2a00700 1 -- 172.21.15.123:0/1231432069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52f4106080 msgr2=0x7f52f419c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:18.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.597+0000 7f52f2a00700 1 --2- 172.21.15.123:0/1231432069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52f4106080 0x7f52f419c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:18.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.597+0000 7f52f2a00700 1 -- 172.21.15.123:0/1231432069 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52f41a0db0 con 0x7f52f41056c0 2024-05-08T06:11:18.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.597+0000 7f52f2a00700 1 --2- 172.21.15.123:0/1231432069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52f41056c0 0x7f52f419bda0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f52e4000c00 tx=0x7f52e40057a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:18.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.598+0000 7f52eb400700 1 -- 172.21.15.123:0/1231432069 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e4027070 con 0x7f52f41056c0 2024-05-08T06:11:18.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.598+0000 7f52f9c00700 1 -- 172.21.15.123:0/1231432069 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52f41a1040 con 0x7f52f41056c0 2024-05-08T06:11:18.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.598+0000 7f52eb400700 1 -- 172.21.15.123:0/1231432069 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52e4003680 con 0x7f52f41056c0 2024-05-08T06:11:18.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.598+0000 7f52eb400700 1 -- 172.21.15.123:0/1231432069 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e4022aa0 con 0x7f52f41056c0 2024-05-08T06:11:18.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.598+0000 7f52f9c00700 1 -- 172.21.15.123:0/1231432069 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52f41a8920 con 0x7f52f41056c0 2024-05-08T06:11:18.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.602+0000 7f52eb400700 1 -- 172.21.15.123:0/1231432069 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 19) ==== 149704+0+0 (secure 0 0 0) 0x7f52e402a070 con 0x7f52f41056c0 2024-05-08T06:11:18.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.603+0000 7f530456eb80 1 Processor -- start 2024-05-08T06:11:18.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.603+0000 7f530456eb80 1 -- start start 2024-05-08T06:11:18.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.603+0000 7f530456eb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c22841060 0x557c22841440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:18.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.603+0000 7f530456eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c22841980 0x557c22845490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:18.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.603+0000 7f530456eb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c228459d0 0x557c22845e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:18.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.603+0000 7f530456eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557c22793a20 con 0x557c22841980 2024-05-08T06:11:18.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.603+0000 7f530456eb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x557c22731760 con 0x557c228459d0 2024-05-08T06:11:18.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.603+0000 7f530456eb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557c2278f9e0 con 0x557c22841060 2024-05-08T06:11:18.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.603+0000 7f52eb400700 1 --2- 172.21.15.123:0/1231432069 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f52d80b2910 0x7f52d80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:18.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.604+0000 7f52f2a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c22841980 0x557c22845490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:18.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.604+0000 7f52f2a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c22841980 0x557c22845490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34684/0 (socket says 172.21.15.123:34684) 2024-05-08T06:11:18.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.604+0000 7f52f2a00700 1 -- 172.21.15.123:0/3681757114 learned_addr learned my addr 172.21.15.123:0/3681757114 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:18.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.604+0000 7f52f2000700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c228459d0 0x557c22845e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:18.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.604+0000 7f52eb400700 1 -- 172.21.15.123:0/1231432069 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f52e40e33c0 con 0x7f52f41056c0 2024-05-08T06:11:18.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.604+0000 7f52f3400700 1 --2- 172.21.15.123:0/3681757114 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c22841060 0x557c22841440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:18.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.604+0000 7f52f3400700 1 --2- 172.21.15.123:0/1231432069 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f52d80b2910 0x7f52d80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:18.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.604+0000 7f52f2000700 1 -- 172.21.15.123:0/3681757114 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c22841060 msgr2=0x557c22841440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:18.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.604+0000 7f52f2000700 1 --2- 172.21.15.123:0/3681757114 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c22841060 0x557c22841440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:18.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.604+0000 7f52f2000700 1 -- 172.21.15.123:0/3681757114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c22841980 msgr2=0x557c22845490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:18.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.604+0000 7f52f2000700 1 --2- 172.21.15.123:0/3681757114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c22841980 0x557c22845490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:18.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.604+0000 7f52f2000700 1 -- 172.21.15.123:0/3681757114 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c226e08d0 con 0x557c228459d0 2024-05-08T06:11:18.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.605+0000 7f52f2000700 1 --2- 172.21.15.123:0/3681757114 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c228459d0 0x557c22845e30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f52e0007bc0 tx=0x7f52e0007ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eea01c03e8a1ec10 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:18.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.605+0000 7f52dfe00700 1 -- 172.21.15.123:0/3681757114 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e0004310 con 0x557c228459d0 2024-05-08T06:11:18.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.605+0000 7f52dfe00700 1 -- 172.21.15.123:0/3681757114 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52e0015070 con 0x557c228459d0 2024-05-08T06:11:18.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.605+0000 7f52dfe00700 1 -- 172.21.15.123:0/3681757114 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e0005020 con 0x557c228459d0 2024-05-08T06:11:18.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.605+0000 7f52f3400700 1 --2- 172.21.15.123:0/1231432069 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f52d80b2910 0x7f52d80b4dd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f52ec009e00 tx=0x7f52ec009100 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:18.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.606+0000 7f530456eb80 1 -- 172.21.15.123:0/3681757114 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c228459d0 msgr2=0x557c22845e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:18.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.606+0000 7f530456eb80 1 --2- 172.21.15.123:0/3681757114 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c228459d0 0x557c22845e30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f52e0007bc0 tx=0x7f52e0007ed0 comp rx=0 tx=0).stop 2024-05-08T06:11:18.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.606+0000 7f530456eb80 1 -- 172.21.15.123:0/3681757114 shutdown_connections 2024-05-08T06:11:18.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.606+0000 7f530456eb80 1 --2- 172.21.15.123:0/3681757114 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c228459d0 0x557c22845e30 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:18.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.606+0000 7f530456eb80 1 --2- 172.21.15.123:0/3681757114 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c22841060 0x557c22841440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:18.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.606+0000 7f530456eb80 1 --2- 172.21.15.123:0/3681757114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c22841980 0x557c22845490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:18.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.606+0000 7f530456eb80 1 -- 172.21.15.123:0/3681757114 >> 172.21.15.123:0/3681757114 conn(0x557c2276fb60 msgr2=0x557c2283b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:18.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.606+0000 7f530456eb80 1 -- 172.21.15.123:0/3681757114 shutdown_connections 2024-05-08T06:11:18.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.606+0000 7f530456eb80 1 -- 172.21.15.123:0/3681757114 wait complete. 2024-05-08T06:11:18.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.607+0000 7f530456eb80 1 -- start start 2024-05-08T06:11:18.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.607+0000 7f530456eb80 1 Processor -- start 2024-05-08T06:11:18.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.608+0000 7f530456eb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c22841060 0x557c2284cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:18.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.608+0000 7f530456eb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c22841980 0x557c2284a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:18.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.608+0000 7f530456eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c228459d0 0x557c2284a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:18.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.608+0000 7f530456eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557c2284dbe0 con 0x557c228459d0 2024-05-08T06:11:18.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.608+0000 7f530456eb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x557c226f9b20 con 0x557c22841060 2024-05-08T06:11:18.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.608+0000 7f530456eb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557c2284dd60 con 0x557c22841980 2024-05-08T06:11:18.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.608+0000 7f530456eb80 10 client.0 _ll_register_callbacks cb 0x557c2273ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:11:18.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.609+0000 7f52f2a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c22841060 0x557c2284cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:18.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.609+0000 7f52f2a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c22841060 0x557c2284cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:46318/0 (socket says 172.21.15.123:46318) 2024-05-08T06:11:18.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.609+0000 7f52f2000700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c22841980 0x557c2284a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:18.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.609+0000 7f52f2a00700 1 -- 172.21.15.123:0/693600939 learned_addr learned my addr 172.21.15.123:0/693600939 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:18.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.609+0000 7f52f3400700 1 --2- 172.21.15.123:0/693600939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c228459d0 0x557c2284a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:18.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.609+0000 7f52f2a00700 1 -- 172.21.15.123:0/693600939 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c22841980 msgr2=0x557c2284a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:18.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.609+0000 7f52f2a00700 1 --2- 172.21.15.123:0/693600939 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c22841980 0x557c2284a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:18.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.609+0000 7f52f2a00700 1 -- 172.21.15.123:0/693600939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c228459d0 msgr2=0x557c2284a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:18.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.609+0000 7f52f2a00700 1 --2- 172.21.15.123:0/693600939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c228459d0 0x557c2284a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:18.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.609+0000 7f52f2a00700 1 -- 172.21.15.123:0/693600939 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c22759420 con 0x557c22841060 2024-05-08T06:11:18.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.609+0000 7f52f2a00700 1 --2- 172.21.15.123:0/693600939 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c22841060 0x557c2284cdc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f52e4022520 tx=0x7f52e40e1250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c813f48e8becfa69 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:18.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.609+0000 7f52dcc00700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-05-08T06:11:18.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.610+0000 7f52dcc00700 1 -- 172.21.15.123:0/693600939 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e40aa690 con 0x557c22841060 2024-05-08T06:11:18.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.610+0000 7f530456eb80 1 -- 172.21.15.123:0/693600939 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52e0015070 con 0x557c22841060 2024-05-08T06:11:18.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.610+0000 7f52dcc00700 1 -- 172.21.15.123:0/693600939 <== mon.2 v2:172.21.15.163:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f52e40a6eb0 con 0x557c22841060 2024-05-08T06:11:18.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.610+0000 7f52dcc00700 1 -- 172.21.15.123:0/693600939 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f52e40aa030 con 0x557c22841060 2024-05-08T06:11:18.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.610+0000 7f530456eb80 10 client.25235 fetch_fsmap learned FSMap version 24 2024-05-08T06:11:18.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.610+0000 7f530456eb80 1 -- 172.21.15.123:0/693600939 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557c228583f0 con 0x557c22841060 2024-05-08T06:11:18.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.610+0000 7f52dcc00700 1 -- 172.21.15.123:0/693600939 <== mon.2 v2:172.21.15.163:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52e0015070 con 0x557c22841060 2024-05-08T06:11:18.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.610+0000 7f52dcc00700 1 -- 172.21.15.123:0/693600939 <== mon.2 v2:172.21.15.163:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f52e40b3200 con 0x557c22841060 2024-05-08T06:11:18.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.610+0000 7f530456eb80 10 client.25235 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:11:18.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.610+0000 7f530456eb80 10 client.25235 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T06:11:18.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.610+0000 7f530456eb80 20 client.25235 populate_metadata read hostname 'smithi123' 2024-05-08T06:11:18.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.611+0000 7f530456eb80 1 --2- 172.21.15.123:0/693600939 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x557c2285aa40 0x557c2285ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:18.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.611+0000 7f530456eb80 4 client.25235 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:11:18.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.611+0000 7f530456eb80 1 -- 172.21.15.123:0/693600939 --> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c2280e0a0 con 0x557c2285aa40 2024-05-08T06:11:18.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.611+0000 7f52f2000700 1 --2- 172.21.15.123:0/693600939 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x557c2285aa40 0x557c2285ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:18.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.612+0000 7f52f2000700 1 --2- 172.21.15.123:0/693600939 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x557c2285aa40 0x557c2285ce80 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2b6ecbec40616611 server_cookie=a60493cfd4f89ff6 in_seq=0 out_seq=0 2024-05-08T06:11:18.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.612+0000 7f52dcc00700 10 client.25235 ms_handle_connect on v2:172.21.15.192:6838/3598682360 2024-05-08T06:11:18.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.612+0000 7f52dcc00700 1 -- 172.21.15.123:0/693600939 <== mds.2 v2:172.21.15.192:6838/3598682360 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557c2280e0a0 con 0x557c2285aa40 2024-05-08T06:11:18.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.612+0000 7f52dcc00700 10 client.25235 handle_command_reply: tid=0 2024-05-08T06:11:18.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.612+0000 7f530456eb80 1 client.25235 shutdown 2024-05-08T06:11:18.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.613+0000 7f530456eb80 1 -- 172.21.15.123:0/693600939 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c22841060 msgr2=0x557c2284cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:18.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.613+0000 7f530456eb80 1 --2- 172.21.15.123:0/693600939 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c22841060 0x557c2284cdc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f52e4022520 tx=0x7f52e40e1250 comp rx=0 tx=0).stop 2024-05-08T06:11:18.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.613+0000 7f530456eb80 1 -- 172.21.15.123:0/693600939 shutdown_connections 2024-05-08T06:11:18.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.613+0000 7f530456eb80 1 --2- 172.21.15.123:0/693600939 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x557c2285aa40 0x557c2285ce80 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:18.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.613+0000 7f530456eb80 1 --2- 172.21.15.123:0/693600939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c228459d0 0x557c2284a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:18.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.613+0000 7f530456eb80 1 --2- 172.21.15.123:0/693600939 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c22841060 0x557c2284cdc0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:18.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.613+0000 7f530456eb80 1 --2- 172.21.15.123:0/693600939 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c22841980 0x557c2284a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:18.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.613+0000 7f52dcc00700 0 client.25235 ms_handle_reset on v2:172.21.15.192:6838/3598682360 2024-05-08T06:11:18.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.613+0000 7f530456eb80 1 -- 172.21.15.123:0/693600939 >> 172.21.15.123:0/693600939 conn(0x557c2276fb60 msgr2=0x557c228362b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:18.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.613+0000 7f530456eb80 1 -- 172.21.15.123:0/693600939 shutdown_connections 2024-05-08T06:11:18.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.613+0000 7f530456eb80 1 -- 172.21.15.123:0/693600939 wait complete. 2024-05-08T06:11:18.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.613+0000 7f530456eb80 20 client.25235 trim_cache size 0 max 16384 2024-05-08T06:11:18.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.630+0000 7f530456eb80 1 Processor -- start 2024-05-08T06:11:18.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.630+0000 7f530456eb80 1 -- start start 2024-05-08T06:11:18.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.630+0000 7f530456eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c22841060 0x557c22856ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:18.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.631+0000 7f530456eb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c22841980 0x557c22862750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:18.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.631+0000 7f530456eb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c228459d0 0x557c22862c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:18.631 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.631+0000 7f530456eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f52f41b2460 con 0x557c22841060 2024-05-08T06:11:18.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.631+0000 7f530456eb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f52f41afcf0 con 0x557c22841980 2024-05-08T06:11:18.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.631+0000 7f530456eb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f52f41adb40 con 0x557c228459d0 2024-05-08T06:11:18.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.631+0000 7f52f2a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c22841060 0x557c22856ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:18.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.631+0000 7f52f2a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c22841060 0x557c22856ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34704/0 (socket says 172.21.15.123:34704) 2024-05-08T06:11:18.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.631+0000 7f52f2a00700 1 -- 172.21.15.123:0/2549975101 learned_addr learned my addr 172.21.15.123:0/2549975101 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:18.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.631+0000 7f52f2000700 1 --2- 172.21.15.123:0/2549975101 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c22841980 0x557c22862750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:18.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.631+0000 7f52f3400700 1 --2- 172.21.15.123:0/2549975101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c228459d0 0x557c22862c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:18.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.631+0000 7f52f2a00700 1 -- 172.21.15.123:0/2549975101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c228459d0 msgr2=0x557c22862c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:18.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.631+0000 7f52f2a00700 1 --2- 172.21.15.123:0/2549975101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c228459d0 0x557c22862c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:18.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.631+0000 7f52f2a00700 1 -- 172.21.15.123:0/2549975101 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c22841980 msgr2=0x557c22862750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:18.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.631+0000 7f52f2a00700 1 --2- 172.21.15.123:0/2549975101 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c22841980 0x557c22862750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:18.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.632+0000 7f52f2a00700 1 -- 172.21.15.123:0/2549975101 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c228383f0 con 0x557c22841060 2024-05-08T06:11:18.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.632+0000 7f52f3400700 1 --2- 172.21.15.123:0/2549975101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c228459d0 0x557c22862c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:11:18.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.632+0000 7f52f2a00700 1 --2- 172.21.15.123:0/2549975101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c22841060 0x557c22856ca0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f52e40a7b30 tx=0x7f52e40b3120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1720c2a5f3138f5c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:18.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.632+0000 7f52df400700 1 -- 172.21.15.123:0/2549975101 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e40aa210 con 0x557c22841060 2024-05-08T06:11:18.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.632+0000 7f52df400700 1 -- 172.21.15.123:0/2549975101 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52e40aa3d0 con 0x557c22841060 2024-05-08T06:11:18.633 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.632+0000 7f52df400700 1 -- 172.21.15.123:0/2549975101 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e40701c0 con 0x557c22841060 2024-05-08T06:11:18.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.633+0000 7f530456eb80 1 -- 172.21.15.123:0/2549975101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c22841060 msgr2=0x557c22856ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:18.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.633+0000 7f530456eb80 1 --2- 172.21.15.123:0/2549975101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c22841060 0x557c22856ca0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f52e40a7b30 tx=0x7f52e40b3120 comp rx=0 tx=0).stop 2024-05-08T06:11:18.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.633+0000 7f530456eb80 1 -- 172.21.15.123:0/2549975101 shutdown_connections 2024-05-08T06:11:18.634 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.633+0000 7f530456eb80 1 --2- 172.21.15.123:0/2549975101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c22841060 0x557c22856ca0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:18.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.633+0000 7f530456eb80 1 --2- 172.21.15.123:0/2549975101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c228459d0 0x557c22862c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:18.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.633+0000 7f530456eb80 1 --2- 172.21.15.123:0/2549975101 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c22841980 0x557c22862750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:18.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.633+0000 7f530456eb80 1 -- 172.21.15.123:0/2549975101 >> 172.21.15.123:0/2549975101 conn(0x557c2276fb60 msgr2=0x557c22848a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:18.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.633+0000 7f530456eb80 1 -- 172.21.15.123:0/2549975101 shutdown_connections 2024-05-08T06:11:18.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.633+0000 7f530456eb80 1 -- 172.21.15.123:0/2549975101 wait complete. 2024-05-08T06:11:18.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.634+0000 7f530456eb80 1 -- start start 2024-05-08T06:11:18.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.634+0000 7f530456eb80 1 Processor -- start 2024-05-08T06:11:18.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.634+0000 7f530456eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c22841060 0x557c22853430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:18.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.634+0000 7f530456eb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c22841980 0x557c2284e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:18.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.634+0000 7f530456eb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c228459d0 0x557c2284ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:18.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.634+0000 7f530456eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f52f41ae1b0 con 0x557c22841060 2024-05-08T06:11:18.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.634+0000 7f530456eb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f52f41af620 con 0x557c22841980 2024-05-08T06:11:18.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.634+0000 7f530456eb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f52f41ae880 con 0x557c228459d0 2024-05-08T06:11:18.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.635+0000 7f530456eb80 10 client.0 _ll_register_callbacks cb 0x557c2273ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:11:18.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.635+0000 7f52f2a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c22841060 0x557c22853430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:18.637 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.635+0000 7f52f2a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c22841060 0x557c22853430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34708/0 (socket says 172.21.15.123:34708) 2024-05-08T06:11:18.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.635+0000 7f52f2a00700 1 -- 172.21.15.123:0/2725600745 learned_addr learned my addr 172.21.15.123:0/2725600745 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:18.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.635+0000 7f52f2000700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c22841980 0x557c2284e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:18.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.635+0000 7f52f3400700 1 --2- 172.21.15.123:0/2725600745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c228459d0 0x557c2284ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:18.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.635+0000 7f52f2000700 1 -- 172.21.15.123:0/2725600745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c228459d0 msgr2=0x557c2284ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:18.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.635+0000 7f52f2000700 1 --2- 172.21.15.123:0/2725600745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c228459d0 0x557c2284ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:18.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.635+0000 7f52f2000700 1 -- 172.21.15.123:0/2725600745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c22841060 msgr2=0x557c22853430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:18.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.635+0000 7f52f2000700 1 --2- 172.21.15.123:0/2725600745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c22841060 0x557c22853430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:18.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.635+0000 7f52f2000700 1 -- 172.21.15.123:0/2725600745 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c2284f3f0 con 0x557c22841980 2024-05-08T06:11:18.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.635+0000 7f52f2a00700 1 --2- 172.21.15.123:0/2725600745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c22841060 0x557c22853430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:11:18.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.636+0000 7f52f2000700 1 --2- 172.21.15.123:0/2725600745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c22841980 0x557c2284e7b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x557c2284b730 tx=0x7f52e00043f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82e7d7667a4008f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:18.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.636+0000 7f52dcc00700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-05-08T06:11:18.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.636+0000 7f52dcc00700 1 -- 172.21.15.123:0/2725600745 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e0010b80 con 0x557c22841980 2024-05-08T06:11:18.638 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.636+0000 7f52dcc00700 1 -- 172.21.15.123:0/2725600745 <== mon.2 v2:172.21.15.163:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f52e0010d20 con 0x557c22841980 2024-05-08T06:11:18.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.636+0000 7f530456eb80 1 -- 172.21.15.123:0/2725600745 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52e40aa3d0 con 0x557c22841980 2024-05-08T06:11:18.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.636+0000 7f52dcc00700 1 -- 172.21.15.123:0/2725600745 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f52e0003680 con 0x557c22841980 2024-05-08T06:11:18.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.636+0000 7f530456eb80 10 client.25241 fetch_fsmap learned FSMap version 24 2024-05-08T06:11:18.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.636+0000 7f530456eb80 1 -- 172.21.15.123:0/2725600745 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557c228c5160 con 0x557c22841980 2024-05-08T06:11:18.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.637+0000 7f52dcc00700 1 -- 172.21.15.123:0/2725600745 <== mon.2 v2:172.21.15.163:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52e40aa3d0 con 0x557c22841980 2024-05-08T06:11:18.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.637+0000 7f52dcc00700 1 -- 172.21.15.123:0/2725600745 <== mon.2 v2:172.21.15.163:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f52e00293d0 con 0x557c22841980 2024-05-08T06:11:18.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.637+0000 7f530456eb80 10 client.25241 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:11:18.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.637+0000 7f530456eb80 10 client.25241 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T06:11:18.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.637+0000 7f530456eb80 20 client.25241 populate_metadata read hostname 'smithi123' 2024-05-08T06:11:18.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.637+0000 7f530456eb80 1 --2- 172.21.15.123:0/2725600745 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x557c228c5540 0x557c228c7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:18.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.637+0000 7f530456eb80 4 client.25241 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T06:11:18.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.637+0000 7f530456eb80 1 -- 172.21.15.123:0/2725600745 --> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557c227a1400 con 0x557c228c5540 2024-05-08T06:11:18.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.637+0000 7f52f2a00700 1 --2- 172.21.15.123:0/2725600745 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x557c228c5540 0x557c228c7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:18.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.638+0000 7f52f2a00700 1 --2- 172.21.15.123:0/2725600745 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x557c228c5540 0x557c228c7960 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c25142b340a2388d server_cookie=e5edac2c95d24859 in_seq=0 out_seq=0 2024-05-08T06:11:18.639 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.638+0000 7f52dcc00700 10 client.25241 ms_handle_connect on v2:172.21.15.192:6838/3598682360 2024-05-08T06:11:18.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.638+0000 7f52dcc00700 1 -- 172.21.15.123:0/2725600745 <== mds.2 v2:172.21.15.192:6838/3598682360 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557c227a1400 con 0x557c228c5540 2024-05-08T06:11:18.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.638+0000 7f52dcc00700 10 client.25241 handle_command_reply: tid=0 2024-05-08T06:11:18.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.638+0000 7f530456eb80 1 client.25241 shutdown 2024-05-08T06:11:18.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.638+0000 7f530456eb80 1 -- 172.21.15.123:0/2725600745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c22841980 msgr2=0x557c2284e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:18.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.638+0000 7f530456eb80 1 --2- 172.21.15.123:0/2725600745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c22841980 0x557c2284e7b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x557c2284b730 tx=0x7f52e00043f0 comp rx=0 tx=0).stop 2024-05-08T06:11:18.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.638+0000 7f530456eb80 1 -- 172.21.15.123:0/2725600745 shutdown_connections 2024-05-08T06:11:18.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.638+0000 7f530456eb80 1 --2- 172.21.15.123:0/2725600745 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x557c228c5540 0x557c228c7960 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:18.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.638+0000 7f530456eb80 1 --2- 172.21.15.123:0/2725600745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c22841060 0x557c22853430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:18.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.638+0000 7f530456eb80 1 --2- 172.21.15.123:0/2725600745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c228459d0 0x557c2284ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:18.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.638+0000 7f530456eb80 1 --2- 172.21.15.123:0/2725600745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557c22841980 0x557c2284e7b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:18.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.638+0000 7f52dcc00700 0 client.25241 ms_handle_reset on v2:172.21.15.192:6838/3598682360 2024-05-08T06:11:18.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.638+0000 7f530456eb80 1 -- 172.21.15.123:0/2725600745 >> 172.21.15.123:0/2725600745 conn(0x557c2276fb60 msgr2=0x557c228362b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:18.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.638+0000 7f530456eb80 1 -- 172.21.15.123:0/2725600745 shutdown_connections 2024-05-08T06:11:18.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.639+0000 7f530456eb80 1 -- 172.21.15.123:0/2725600745 wait complete. 2024-05-08T06:11:18.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.639+0000 7f530456eb80 20 client.25241 trim_cache size 0 max 16384 2024-05-08T06:11:18.642 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-05-08T06:11:18.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.639+0000 7f52f9c00700 1 -- 172.21.15.123:0/1231432069 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f52d80b2910 msgr2=0x7f52d80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:18.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.639+0000 7f52f9c00700 1 --2- 172.21.15.123:0/1231432069 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f52d80b2910 0x7f52d80b4dd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f52ec009e00 tx=0x7f52ec009100 comp rx=0 tx=0).stop 2024-05-08T06:11:18.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.639+0000 7f52f9c00700 1 -- 172.21.15.123:0/1231432069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52f41056c0 msgr2=0x7f52f419bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:18.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.639+0000 7f52f9c00700 1 --2- 172.21.15.123:0/1231432069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52f41056c0 0x7f52f419bda0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f52e4000c00 tx=0x7f52e40057a0 comp rx=0 tx=0).stop 2024-05-08T06:11:18.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.640+0000 7f52f9c00700 1 -- 172.21.15.123:0/1231432069 shutdown_connections 2024-05-08T06:11:18.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.640+0000 7f52f9c00700 1 --2- 172.21.15.123:0/1231432069 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f52d80b2910 0x7f52d80b4dd0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:18.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.640+0000 7f52f9c00700 1 --2- 172.21.15.123:0/1231432069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52f410b6c0 0x7f52f41a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:18.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.640+0000 7f52f9c00700 1 --2- 172.21.15.123:0/1231432069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52f41056c0 0x7f52f419bda0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:18.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.640+0000 7f52f9c00700 1 --2- 172.21.15.123:0/1231432069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52f4106080 0x7f52f419c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:18.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.640+0000 7f52f9c00700 1 -- 172.21.15.123:0/1231432069 >> 172.21.15.123:0/1231432069 conn(0x7f52f41011b0 msgr2=0x7f52f4101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:18.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.640+0000 7f52f9c00700 1 -- 172.21.15.123:0/1231432069 shutdown_connections 2024-05-08T06:11:18.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:18.640+0000 7f52f9c00700 1 -- 172.21.15.123:0/1231432069 wait complete. 2024-05-08T06:11:18.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:18 smithi163 ceph-mon[34520]: pgmap v485: 97 pgs: 97 active+clean; 673 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-08T06:11:18.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:18 smithi192 ceph-mon[34664]: pgmap v485: 97 pgs: 97 active+clean; 673 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-08T06:11:19.664 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-08T06:11:20.093 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:11:20.506 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:20 smithi123 ceph-mon[29787]: pgmap v486: 97 pgs: 97 active+clean; 673 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-08T06:11:20.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:20 smithi163 ceph-mon[34520]: pgmap v486: 97 pgs: 97 active+clean; 673 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-08T06:11:20.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:20 smithi192 ceph-mon[34664]: pgmap v486: 97 pgs: 97 active+clean; 673 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-08T06:11:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.382+0000 7f878f000640 1 -- 172.21.15.123:0/3525415014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8788103310 msgr2=0x7f8788105700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.382+0000 7f878f000640 1 --2- 172.21.15.123:0/3525415014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8788103310 0x7f8788105700 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f877c009900 tx=0x7f877c02a400 comp rx=0 tx=0).stop 2024-05-08T06:11:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.382+0000 7f878f000640 1 -- 172.21.15.123:0/3525415014 shutdown_connections 2024-05-08T06:11:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.382+0000 7f878f000640 1 --2- 172.21.15.123:0/3525415014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8788108570 0x7f878810a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.382+0000 7f878f000640 1 --2- 172.21.15.123:0/3525415014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8788105c40 0x7f8788108030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.382+0000 7f878f000640 1 --2- 172.21.15.123:0/3525415014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8788103310 0x7f8788105700 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.382+0000 7f878f000640 1 -- 172.21.15.123:0/3525415014 >> 172.21.15.123:0/3525415014 conn(0x7f87880fcca0 msgr2=0x7f87880ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.382+0000 7f878f000640 1 -- 172.21.15.123:0/3525415014 shutdown_connections 2024-05-08T06:11:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.382+0000 7f878f000640 1 -- 172.21.15.123:0/3525415014 wait complete. 2024-05-08T06:11:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.383+0000 7f878f000640 1 Processor -- start 2024-05-08T06:11:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.383+0000 7f878f000640 1 -- start start 2024-05-08T06:11:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.383+0000 7f878f000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8788103310 0x7f8788102c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.383+0000 7f878f000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8788105c40 0x7f87881012c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.384+0000 7f878f000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8788108570 0x7f8788101800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.384+0000 7f878f000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f878810e380 con 0x7f8788105c40 2024-05-08T06:11:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.384+0000 7f878f000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f878810e200 con 0x7f8788103310 2024-05-08T06:11:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.384+0000 7f878f000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f878810e500 con 0x7f8788108570 2024-05-08T06:11:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.384+0000 7f878dc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8788103310 0x7f8788102c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.384+0000 7f878d200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8788105c40 0x7f87881012c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.384+0000 7f878dc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8788103310 0x7f8788102c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:46342/0 (socket says 172.21.15.123:46342) 2024-05-08T06:11:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.384+0000 7f878dc00640 1 -- 172.21.15.123:0/3081622766 learned_addr learned my addr 172.21.15.123:0/3081622766 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.384+0000 7f878e600640 1 --2- 172.21.15.123:0/3081622766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8788108570 0x7f8788101800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.384+0000 7f878d200640 1 -- 172.21.15.123:0/3081622766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8788108570 msgr2=0x7f8788101800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.384+0000 7f878d200640 1 --2- 172.21.15.123:0/3081622766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8788108570 0x7f8788101800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.384+0000 7f878d200640 1 -- 172.21.15.123:0/3081622766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8788103310 msgr2=0x7f8788102c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.384+0000 7f878d200640 1 --2- 172.21.15.123:0/3081622766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8788103310 0x7f8788102c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.384+0000 7f878d200640 1 -- 172.21.15.123:0/3081622766 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8788102090 con 0x7f8788105c40 2024-05-08T06:11:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.384+0000 7f878dc00640 1 --2- 172.21.15.123:0/3081622766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8788103310 0x7f8788102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:11:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.385+0000 7f878d200640 1 --2- 172.21.15.123:0/3081622766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8788105c40 0x7f87881012c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f877400e9c0 tx=0x7f877400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.385+0000 7f8786000640 1 -- 172.21.15.123:0/3081622766 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877400ccc0 con 0x7f8788105c40 2024-05-08T06:11:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.385+0000 7f8786000640 1 -- 172.21.15.123:0/3081622766 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8774005020 con 0x7f8788105c40 2024-05-08T06:11:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.385+0000 7f878f000640 1 -- 172.21.15.123:0/3081622766 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87881a2c40 con 0x7f8788105c40 2024-05-08T06:11:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.385+0000 7f8786000640 1 -- 172.21.15.123:0/3081622766 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877400ce60 con 0x7f8788105c40 2024-05-08T06:11:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.385+0000 7f878f000640 1 -- 172.21.15.123:0/3081622766 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87881a3190 con 0x7f8788105c40 2024-05-08T06:11:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.389+0000 7f87933fb740 1 Processor -- start 2024-05-08T06:11:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.389+0000 7f87933fb740 1 -- start start 2024-05-08T06:11:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.389+0000 7f8786000640 1 -- 172.21.15.123:0/3081622766 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 19) ==== 149704+0+0 (secure 0 0 0) 0x7f8774004ac0 con 0x7f8788105c40 2024-05-08T06:11:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.389+0000 7f87933fb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e29f4ab320 0x55e29f4ab740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.389+0000 7f87933fb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e29f4abd10 0x55e29f4b0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.389+0000 7f87933fb740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e29f4b0e80 0x55e29f4b3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.389+0000 7f87933fb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e29f40f550 con 0x55e29f4ab320 2024-05-08T06:11:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.389+0000 7f87933fb740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55e29f3b4bc0 con 0x55e29f4b0e80 2024-05-08T06:11:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.389+0000 7f87933fb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55e29f40b510 con 0x55e29f4abd10 2024-05-08T06:11:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.390+0000 7f878e600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e29f4ab320 0x55e29f4ab740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.390+0000 7f878e600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e29f4ab320 0x55e29f4ab740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34742/0 (socket says 172.21.15.123:34742) 2024-05-08T06:11:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.390+0000 7f878e600640 1 -- 172.21.15.123:0/1877465378 learned_addr learned my addr 172.21.15.123:0/1877465378 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:22.393 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.390+0000 7f878dc00640 1 --2- 172.21.15.123:0/1877465378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e29f4abd10 0x55e29f4b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:22.393 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.390+0000 7f878e600640 1 -- 172.21.15.123:0/1877465378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e29f4abd10 msgr2=0x55e29f4b0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:22.393 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.390+0000 7f878e600640 1 --2- 172.21.15.123:0/1877465378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e29f4abd10 0x55e29f4b0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.393 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.390+0000 7f878e600640 1 -- 172.21.15.123:0/1877465378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e29f4b0e80 msgr2=0x55e29f4b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:22.393 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.390+0000 7f878e600640 1 --2- 172.21.15.123:0/1877465378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e29f4b0e80 0x55e29f4b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.393 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.390+0000 7f878e600640 1 -- 172.21.15.123:0/1877465378 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e29f121210 con 0x55e29f4ab320 2024-05-08T06:11:22.393 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.390+0000 7f878dc00640 1 --2- 172.21.15.123:0/1877465378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e29f4abd10 0x55e29f4b0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:11:22.393 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.390+0000 7f878e600640 1 --2- 172.21.15.123:0/1877465378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e29f4ab320 0x55e29f4ab740 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8780011440 tx=0x7f8780011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b464a0192305d874 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:22.393 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.391+0000 7f877aa00640 1 -- 172.21.15.123:0/1877465378 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8780003c30 con 0x55e29f4ab320 2024-05-08T06:11:22.393 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.391+0000 7f877aa00640 1 -- 172.21.15.123:0/1877465378 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8780003dd0 con 0x55e29f4ab320 2024-05-08T06:11:22.393 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.391+0000 7f877aa00640 1 -- 172.21.15.123:0/1877465378 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878000fae0 con 0x55e29f4ab320 2024-05-08T06:11:22.393 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.391+0000 7f8786000640 1 --2- 172.21.15.123:0/3081622766 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f87700b28f0 0x7f87700b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:22.393 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.391+0000 7f8786000640 1 -- 172.21.15.123:0/3081622766 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f87740e2310 con 0x7f8788105c40 2024-05-08T06:11:22.393 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.391+0000 7f878dc00640 1 --2- 172.21.15.123:0/3081622766 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f87700b28f0 0x7f87700b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:22.394 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.392+0000 7f87933fb740 1 -- 172.21.15.123:0/1877465378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e29f4ab320 msgr2=0x55e29f4ab740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:22.394 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.392+0000 7f87933fb740 1 --2- 172.21.15.123:0/1877465378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e29f4ab320 0x55e29f4ab740 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8780011440 tx=0x7f8780011900 comp rx=0 tx=0).stop 2024-05-08T06:11:22.394 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.392+0000 7f87933fb740 1 -- 172.21.15.123:0/1877465378 shutdown_connections 2024-05-08T06:11:22.394 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.392+0000 7f87933fb740 1 --2- 172.21.15.123:0/1877465378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e29f4b0e80 0x55e29f4b3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.394 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.392+0000 7f87933fb740 1 --2- 172.21.15.123:0/1877465378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e29f4abd10 0x55e29f4b0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.394 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.392+0000 7f87933fb740 1 --2- 172.21.15.123:0/1877465378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e29f4ab320 0x55e29f4ab740 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.394 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.392+0000 7f87933fb740 1 -- 172.21.15.123:0/1877465378 >> 172.21.15.123:0/1877465378 conn(0x55e29f473230 msgr2=0x55e29f4a6da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:22.394 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.392+0000 7f87933fb740 1 -- 172.21.15.123:0/1877465378 shutdown_connections 2024-05-08T06:11:22.394 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.392+0000 7f87933fb740 1 -- 172.21.15.123:0/1877465378 wait complete. 2024-05-08T06:11:22.394 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.392+0000 7f878dc00640 1 --2- 172.21.15.123:0/3081622766 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f87700b28f0 0x7f87700b4db0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f877c009230 tx=0x7f877c0058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:22.394 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.393+0000 7f87933fb740 1 -- start start 2024-05-08T06:11:22.394 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.393+0000 7f87933fb740 1 Processor -- start 2024-05-08T06:11:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.393+0000 7f87933fb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e29f4ab320 0x55e29f4b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:22.398 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.393+0000 7f87933fb740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e29f4abd10 0x55e29f4b58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.394+0000 7f87933fb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e29f4b0e80 0x55e29f4bc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.394+0000 7f87933fb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e29f4b3e30 con 0x55e29f4b0e80 2024-05-08T06:11:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.394+0000 7f87933fb740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55e29f3c8660 con 0x55e29f4abd10 2024-05-08T06:11:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.394+0000 7f87933fb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55e29f4b3fb0 con 0x55e29f4ab320 2024-05-08T06:11:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.394+0000 7f87933fb740 10 client.0 _ll_register_callbacks cb 0x55e29f392400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:11:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.394+0000 7f878e600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e29f4b0e80 0x55e29f4bc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.394+0000 7f878dc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e29f4ab320 0x55e29f4b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.394+0000 7f878e600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e29f4b0e80 0x55e29f4bc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34754/0 (socket says 172.21.15.123:34754) 2024-05-08T06:11:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.394+0000 7f878e600640 1 -- 172.21.15.123:0/591063392 learned_addr learned my addr 172.21.15.123:0/591063392 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.394+0000 7f878d200640 1 --2- 172.21.15.123:0/591063392 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e29f4abd10 0x55e29f4b58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.394+0000 7f878d200640 1 -- 172.21.15.123:0/591063392 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e29f4ab320 msgr2=0x55e29f4b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.394+0000 7f878d200640 1 --2- 172.21.15.123:0/591063392 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e29f4ab320 0x55e29f4b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.399 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.394+0000 7f878d200640 1 -- 172.21.15.123:0/591063392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e29f4b0e80 msgr2=0x55e29f4bc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:22.400 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.394+0000 7f878d200640 1 --2- 172.21.15.123:0/591063392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e29f4b0e80 0x55e29f4bc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.400 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.394+0000 7f878d200640 1 -- 172.21.15.123:0/591063392 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e29f1ed620 con 0x55e29f4abd10 2024-05-08T06:11:22.400 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.395+0000 7f878d200640 1 --2- 172.21.15.123:0/591063392 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e29f4abd10 0x55e29f4b58d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f87740dfc70 tx=0x7f87740a9020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69c0ba0fb0ed6bb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:22.400 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.395+0000 7f876f400640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-05-08T06:11:22.400 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.395+0000 7f876f400640 1 -- 172.21.15.123:0/591063392 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877401c070 con 0x55e29f4abd10 2024-05-08T06:11:22.400 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.395+0000 7f876f400640 1 -- 172.21.15.123:0/591063392 <== mon.2 v2:172.21.15.163:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f87740a5a10 con 0x55e29f4abd10 2024-05-08T06:11:22.400 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.395+0000 7f87933fb740 1 -- 172.21.15.123:0/591063392 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8780003dd0 con 0x55e29f4abd10 2024-05-08T06:11:22.400 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.395+0000 7f876f400640 1 -- 172.21.15.123:0/591063392 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f87740a9780 con 0x55e29f4abd10 2024-05-08T06:11:22.400 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.395+0000 7f87933fb740 10 client.25250 fetch_fsmap learned FSMap version 24 2024-05-08T06:11:22.400 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.395+0000 7f87933fb740 1 -- 172.21.15.123:0/591063392 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e29f4bf8d0 con 0x55e29f4abd10 2024-05-08T06:11:22.400 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.396+0000 7f876f400640 1 -- 172.21.15.123:0/591063392 <== mon.2 v2:172.21.15.163:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8780003dd0 con 0x55e29f4abd10 2024-05-08T06:11:22.400 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.396+0000 7f876f400640 1 -- 172.21.15.123:0/591063392 <== mon.2 v2:172.21.15.163:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f877401f030 con 0x55e29f4abd10 2024-05-08T06:11:22.400 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.396+0000 7f87933fb740 10 client.25250 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:11:22.400 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.396+0000 7f87933fb740 10 client.25250 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T06:11:22.401 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.396+0000 7f87933fb740 20 client.25250 populate_metadata read hostname 'smithi123' 2024-05-08T06:11:22.401 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.396+0000 7f87933fb740 1 --2- 172.21.15.123:0/591063392 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55e29f4c0f70 0x55e29f4c3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:22.401 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.396+0000 7f87933fb740 4 client.25250 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:11:22.401 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.396+0000 7f87933fb740 1 -- 172.21.15.123:0/591063392 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e29f41d160 con 0x55e29f4c0f70 2024-05-08T06:11:22.401 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.396+0000 7f878dc00640 1 --2- 172.21.15.123:0/591063392 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55e29f4c0f70 0x55e29f4c3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:22.401 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.397+0000 7f878dc00640 1 --2- 172.21.15.123:0/591063392 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55e29f4c0f70 0x55e29f4c3380 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=639a193425729371 server_cookie=14d383fec1affbba in_seq=0 out_seq=0 2024-05-08T06:11:22.401 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.397+0000 7f876f400640 10 client.25250 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:11:22.401 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.398+0000 7f876f400640 1 -- 172.21.15.123:0/591063392 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e29f41d160 con 0x55e29f4c0f70 2024-05-08T06:11:22.401 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.398+0000 7f876f400640 10 client.25250 handle_command_reply: tid=0 2024-05-08T06:11:22.402 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.398+0000 7f87933fb740 1 client.25250 shutdown 2024-05-08T06:11:22.402 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.398+0000 7f87933fb740 1 -- 172.21.15.123:0/591063392 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e29f4abd10 msgr2=0x55e29f4b58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:22.402 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.398+0000 7f87933fb740 1 --2- 172.21.15.123:0/591063392 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e29f4abd10 0x55e29f4b58d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f87740dfc70 tx=0x7f87740a9020 comp rx=0 tx=0).stop 2024-05-08T06:11:22.402 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.398+0000 7f87933fb740 1 -- 172.21.15.123:0/591063392 shutdown_connections 2024-05-08T06:11:22.402 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.398+0000 7f87933fb740 1 --2- 172.21.15.123:0/591063392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e29f4b0e80 0x55e29f4bc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.402 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.398+0000 7f87933fb740 1 --2- 172.21.15.123:0/591063392 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e29f4abd10 0x55e29f4b58d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.402 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.398+0000 7f87933fb740 1 --2- 172.21.15.123:0/591063392 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55e29f4c0f70 0x55e29f4c3380 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.402 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.398+0000 7f87933fb740 1 --2- 172.21.15.123:0/591063392 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e29f4ab320 0x55e29f4b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.402 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.398+0000 7f87933fb740 1 -- 172.21.15.123:0/591063392 >> 172.21.15.123:0/591063392 conn(0x55e29f473230 msgr2=0x55e29f4a5e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:22.402 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.398+0000 7f876f400640 0 client.25250 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:11:22.402 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.398+0000 7f87933fb740 1 -- 172.21.15.123:0/591063392 shutdown_connections 2024-05-08T06:11:22.402 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.398+0000 7f87933fb740 1 -- 172.21.15.123:0/591063392 wait complete. 2024-05-08T06:11:22.403 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.398+0000 7f87933fb740 20 client.25250 trim_cache size 0 max 16384 2024-05-08T06:11:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.417+0000 7f87933fb740 1 Processor -- start 2024-05-08T06:11:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.417+0000 7f87933fb740 1 -- start start 2024-05-08T06:11:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.417+0000 7f87933fb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e29f4ab320 0x55e29f4b49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.417+0000 7f87933fb740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e29f4abd10 0x55e29f4bcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.417+0000 7f87933fb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e29f4b0e80 0x55e29f4bd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.417+0000 7f87933fb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f87881ae6d0 con 0x55e29f4b0e80 2024-05-08T06:11:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.417+0000 7f87933fb740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f87881abfa0 con 0x55e29f4abd10 2024-05-08T06:11:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.417+0000 7f87933fb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f87881a9df0 con 0x55e29f4ab320 2024-05-08T06:11:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.418+0000 7f878d200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e29f4abd10 0x55e29f4bcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.418+0000 7f878e600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e29f4b0e80 0x55e29f4bd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.418+0000 7f878d200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e29f4abd10 0x55e29f4bcfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:46374/0 (socket says 172.21.15.123:46374) 2024-05-08T06:11:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.418+0000 7f878d200640 1 -- 172.21.15.123:0/3136640329 learned_addr learned my addr 172.21.15.123:0/3136640329 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.418+0000 7f878dc00640 1 --2- 172.21.15.123:0/3136640329 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e29f4ab320 0x55e29f4b49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.418+0000 7f878d200640 1 -- 172.21.15.123:0/3136640329 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e29f4ab320 msgr2=0x55e29f4b49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.418+0000 7f878d200640 1 --2- 172.21.15.123:0/3136640329 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e29f4ab320 0x55e29f4b49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.418+0000 7f878d200640 1 -- 172.21.15.123:0/3136640329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e29f4b0e80 msgr2=0x55e29f4bd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.418+0000 7f878d200640 1 --2- 172.21.15.123:0/3136640329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e29f4b0e80 0x55e29f4bd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.418+0000 7f878d200640 1 -- 172.21.15.123:0/3136640329 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e29f4a7a50 con 0x55e29f4abd10 2024-05-08T06:11:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.418+0000 7f878dc00640 1 --2- 172.21.15.123:0/3136640329 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e29f4ab320 0x55e29f4b49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:11:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.418+0000 7f878d200640 1 --2- 172.21.15.123:0/3136640329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e29f4abd10 0x55e29f4bcfa0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f87740a5800 tx=0x7f8774009f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6898848b47eb24b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.419+0000 7f877a000640 1 -- 172.21.15.123:0/3136640329 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877401c070 con 0x55e29f4abd10 2024-05-08T06:11:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.419+0000 7f877a000640 1 -- 172.21.15.123:0/3136640329 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87740adb30 con 0x55e29f4abd10 2024-05-08T06:11:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.419+0000 7f877a000640 1 -- 172.21.15.123:0/3136640329 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8774091070 con 0x55e29f4abd10 2024-05-08T06:11:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.419+0000 7f87933fb740 1 -- 172.21.15.123:0/3136640329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e29f4abd10 msgr2=0x55e29f4bcfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.419+0000 7f87933fb740 1 --2- 172.21.15.123:0/3136640329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e29f4abd10 0x55e29f4bcfa0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f87740a5800 tx=0x7f8774009f00 comp rx=0 tx=0).stop 2024-05-08T06:11:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.420+0000 7f87933fb740 1 -- 172.21.15.123:0/3136640329 shutdown_connections 2024-05-08T06:11:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.420+0000 7f87933fb740 1 --2- 172.21.15.123:0/3136640329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e29f4b0e80 0x55e29f4bd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.420+0000 7f87933fb740 1 --2- 172.21.15.123:0/3136640329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e29f4abd10 0x55e29f4bcfa0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.420+0000 7f87933fb740 1 --2- 172.21.15.123:0/3136640329 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e29f4ab320 0x55e29f4b49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.420+0000 7f87933fb740 1 -- 172.21.15.123:0/3136640329 >> 172.21.15.123:0/3136640329 conn(0x55e29f473230 msgr2=0x55e29f4acdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.420+0000 7f87933fb740 1 -- 172.21.15.123:0/3136640329 shutdown_connections 2024-05-08T06:11:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.420+0000 7f87933fb740 1 -- 172.21.15.123:0/3136640329 wait complete. 2024-05-08T06:11:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.420+0000 7f87933fb740 1 -- start start 2024-05-08T06:11:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.420+0000 7f87933fb740 1 Processor -- start 2024-05-08T06:11:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.421+0000 7f87933fb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e29f4ab320 0x55e29f4b8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.421+0000 7f87933fb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e29f4abd10 0x55e29f4b93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.421+0000 7f87933fb740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e29f4b0e80 0x55e29f529830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.421+0000 7f87933fb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f87881aa460 con 0x55e29f4ab320 2024-05-08T06:11:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.421+0000 7f87933fb740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f87881ab8d0 con 0x55e29f4b0e80 2024-05-08T06:11:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.421+0000 7f87933fb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f87881aab30 con 0x55e29f4abd10 2024-05-08T06:11:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.421+0000 7f87933fb740 10 client.0 _ll_register_callbacks cb 0x7f87740046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:11:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.421+0000 7f878dc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e29f4ab320 0x55e29f4b8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.421+0000 7f878dc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e29f4ab320 0x55e29f4b8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34770/0 (socket says 172.21.15.123:34770) 2024-05-08T06:11:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.421+0000 7f878dc00640 1 -- 172.21.15.123:0/1820908942 learned_addr learned my addr 172.21.15.123:0/1820908942 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.421+0000 7f878d200640 1 --2- 172.21.15.123:0/1820908942 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e29f4abd10 0x55e29f4b93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.421+0000 7f878e600640 1 --2- 172.21.15.123:0/1820908942 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e29f4b0e80 0x55e29f529830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.422+0000 7f878d200640 1 -- 172.21.15.123:0/1820908942 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e29f4b0e80 msgr2=0x55e29f529830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.422+0000 7f878d200640 1 --2- 172.21.15.123:0/1820908942 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e29f4b0e80 0x55e29f529830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.422+0000 7f878d200640 1 -- 172.21.15.123:0/1820908942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e29f4ab320 msgr2=0x55e29f4b8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.422+0000 7f878d200640 1 --2- 172.21.15.123:0/1820908942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e29f4ab320 0x55e29f4b8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.422+0000 7f878d200640 1 -- 172.21.15.123:0/1820908942 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e29f529d70 con 0x55e29f4abd10 2024-05-08T06:11:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.422+0000 7f878e600640 1 --2- 172.21.15.123:0/1820908942 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e29f4b0e80 0x55e29f529830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:11:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.422+0000 7f878d200640 1 --2- 172.21.15.123:0/1820908942 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e29f4abd10 0x55e29f4b93d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f87740a69f0 tx=0x7f87740042b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6919ffb6af63997 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.422+0000 7f876f400640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-05-08T06:11:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.422+0000 7f876f400640 1 -- 172.21.15.123:0/1820908942 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877401c070 con 0x55e29f4abd10 2024-05-08T06:11:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.422+0000 7f87933fb740 1 -- 172.21.15.123:0/1820908942 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87740adb30 con 0x55e29f4abd10 2024-05-08T06:11:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.422+0000 7f876f400640 1 -- 172.21.15.123:0/1820908942 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f8774091280 con 0x55e29f4abd10 2024-05-08T06:11:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.422+0000 7f876f400640 1 -- 172.21.15.123:0/1820908942 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f87740a90d0 con 0x55e29f4abd10 2024-05-08T06:11:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.422+0000 7f87933fb740 10 client.25246 fetch_fsmap learned FSMap version 24 2024-05-08T06:11:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.422+0000 7f87933fb740 1 -- 172.21.15.123:0/1820908942 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e29f52af20 con 0x55e29f4abd10 2024-05-08T06:11:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.423+0000 7f876f400640 1 -- 172.21.15.123:0/1820908942 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87740adb30 con 0x55e29f4abd10 2024-05-08T06:11:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.423+0000 7f876f400640 1 -- 172.21.15.123:0/1820908942 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f877401f070 con 0x55e29f4abd10 2024-05-08T06:11:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.423+0000 7f87933fb740 10 client.25246 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:11:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.423+0000 7f87933fb740 10 client.25246 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T06:11:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.423+0000 7f87933fb740 20 client.25246 populate_metadata read hostname 'smithi123' 2024-05-08T06:11:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.423+0000 7f87933fb740 1 --2- 172.21.15.123:0/1820908942 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55e29f52b630 0x55e29f52da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.423+0000 7f87933fb740 4 client.25246 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T06:11:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.423+0000 7f87933fb740 1 -- 172.21.15.123:0/1820908942 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e29f4b5180 con 0x55e29f52b630 2024-05-08T06:11:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.423+0000 7f878dc00640 1 --2- 172.21.15.123:0/1820908942 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55e29f52b630 0x55e29f52da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.424+0000 7f878dc00640 1 --2- 172.21.15.123:0/1820908942 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55e29f52b630 0x55e29f52da20 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da9c280d99a62ec7 server_cookie=7f5b9315acbe913e in_seq=0 out_seq=0 2024-05-08T06:11:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.424+0000 7f876f400640 10 client.25246 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:11:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.428+0000 7f876f400640 1 -- 172.21.15.123:0/1820908942 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e29f4b5180 con 0x55e29f52b630 2024-05-08T06:11:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.428+0000 7f876f400640 10 client.25246 handle_command_reply: tid=0 2024-05-08T06:11:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.428+0000 7f87933fb740 1 client.25246 shutdown 2024-05-08T06:11:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.428+0000 7f87933fb740 1 -- 172.21.15.123:0/1820908942 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e29f4abd10 msgr2=0x55e29f4b93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.428+0000 7f87933fb740 1 --2- 172.21.15.123:0/1820908942 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e29f4abd10 0x55e29f4b93d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f87740a69f0 tx=0x7f87740042b0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.428+0000 7f87933fb740 1 -- 172.21.15.123:0/1820908942 shutdown_connections 2024-05-08T06:11:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.428+0000 7f87933fb740 1 --2- 172.21.15.123:0/1820908942 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e29f4b0e80 0x55e29f529830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.428+0000 7f87933fb740 1 --2- 172.21.15.123:0/1820908942 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55e29f52b630 0x55e29f52da20 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.428+0000 7f87933fb740 1 --2- 172.21.15.123:0/1820908942 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e29f4abd10 0x55e29f4b93d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.428+0000 7f87933fb740 1 --2- 172.21.15.123:0/1820908942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e29f4ab320 0x55e29f4b8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.428+0000 7f876f400640 0 client.25246 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:11:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.428+0000 7f87933fb740 1 -- 172.21.15.123:0/1820908942 >> 172.21.15.123:0/1820908942 conn(0x55e29f473230 msgr2=0x55e29f4bcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.428+0000 7f87933fb740 1 -- 172.21.15.123:0/1820908942 shutdown_connections 2024-05-08T06:11:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.428+0000 7f87933fb740 1 -- 172.21.15.123:0/1820908942 wait complete. 2024-05-08T06:11:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.428+0000 7f87933fb740 20 client.25246 trim_cache size 0 max 16384 2024-05-08T06:11:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout:{ 2024-05-08T06:11:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout: "message": "", 2024-05-08T06:11:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout: "return_code": 0 2024-05-08T06:11:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout:} 2024-05-08T06:11:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.429+0000 7f878f000640 1 -- 172.21.15.123:0/3081622766 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f87700b28f0 msgr2=0x7f87700b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.429+0000 7f878f000640 1 --2- 172.21.15.123:0/3081622766 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f87700b28f0 0x7f87700b4db0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f877c009230 tx=0x7f877c0058d0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.429+0000 7f878f000640 1 -- 172.21.15.123:0/3081622766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8788105c40 msgr2=0x7f87881012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.429+0000 7f878f000640 1 --2- 172.21.15.123:0/3081622766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8788105c40 0x7f87881012c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f877400e9c0 tx=0x7f877400edc0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.429+0000 7f878f000640 1 -- 172.21.15.123:0/3081622766 shutdown_connections 2024-05-08T06:11:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.429+0000 7f878f000640 1 --2- 172.21.15.123:0/3081622766 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f87700b28f0 0x7f87700b4db0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.429+0000 7f878f000640 1 --2- 172.21.15.123:0/3081622766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8788108570 0x7f8788101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.429+0000 7f878f000640 1 --2- 172.21.15.123:0/3081622766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8788105c40 0x7f87881012c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.429+0000 7f878f000640 1 --2- 172.21.15.123:0/3081622766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8788103310 0x7f8788102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.429+0000 7f878f000640 1 -- 172.21.15.123:0/3081622766 >> 172.21.15.123:0/3081622766 conn(0x7f87880fcca0 msgr2=0x7f8788103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.429+0000 7f878f000640 1 -- 172.21.15.123:0/3081622766 shutdown_connections 2024-05-08T06:11:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:22.429+0000 7f878f000640 1 -- 172.21.15.123:0/3081622766 wait complete. 2024-05-08T06:11:22.578 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:22 smithi123 ceph-mon[29787]: pgmap v487: 97 pgs: 97 active+clean; 673 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-08T06:11:22.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.605+0000 7f17a1a00700 1 -- 172.21.15.123:0/2793777578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f179c1033d0 msgr2=0x7f179c103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:22.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.605+0000 7f17a1a00700 1 --2- 172.21.15.123:0/2793777578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f179c1033d0 0x7f179c103850 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1788009990 tx=0x7f1788009ca0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.605+0000 7f17a1a00700 1 -- 172.21.15.123:0/2793777578 shutdown_connections 2024-05-08T06:11:22.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.605+0000 7f17a1a00700 1 --2- 172.21.15.123:0/2793777578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f179c103d90 0x7f179c1102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.605+0000 7f17a1a00700 1 --2- 172.21.15.123:0/2793777578 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f179c109460 0x7f179c109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.605+0000 7f17a1a00700 1 --2- 172.21.15.123:0/2793777578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f179c1033d0 0x7f179c103850 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.605+0000 7f17a1a00700 1 -- 172.21.15.123:0/2793777578 >> 172.21.15.123:0/2793777578 conn(0x7f179c0fef20 msgr2=0x7f179c101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:22.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.605+0000 7f17a1a00700 1 -- 172.21.15.123:0/2793777578 shutdown_connections 2024-05-08T06:11:22.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.605+0000 7f17a1a00700 1 -- 172.21.15.123:0/2793777578 wait complete. 2024-05-08T06:11:22.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.606+0000 7f17a1a00700 1 Processor -- start 2024-05-08T06:11:22.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.606+0000 7f17a1a00700 1 -- start start 2024-05-08T06:11:22.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.607+0000 7f17a1a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f179c1033d0 0x7f179c0757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:22.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.607+0000 7f17a1a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f179c103d90 0x7f179c075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:22.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.607+0000 7f17a1a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f179c109460 0x7f179c076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:22.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.607+0000 7f17a1a00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f179c113570 con 0x7f179c1033d0 2024-05-08T06:11:22.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.607+0000 7f17a1a00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f179c1133f0 con 0x7f179c103d90 2024-05-08T06:11:22.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.607+0000 7f17a1a00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f179c1136f0 con 0x7f179c109460 2024-05-08T06:11:22.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.607+0000 7f179be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f179c1033d0 0x7f179c0757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:22.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.607+0000 7f179be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f179c1033d0 0x7f179c0757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34788/0 (socket says 172.21.15.123:34788) 2024-05-08T06:11:22.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.607+0000 7f179b400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f179c103d90 0x7f179c075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:22.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.607+0000 7f179be00700 1 -- 172.21.15.123:0/1881763518 learned_addr learned my addr 172.21.15.123:0/1881763518 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:22.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.608+0000 7f17a1000700 1 --2- 172.21.15.123:0/1881763518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f179c109460 0x7f179c076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:22.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.608+0000 7f179be00700 1 -- 172.21.15.123:0/1881763518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f179c109460 msgr2=0x7f179c076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:22.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.608+0000 7f179be00700 1 --2- 172.21.15.123:0/1881763518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f179c109460 0x7f179c076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.608+0000 7f179be00700 1 -- 172.21.15.123:0/1881763518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f179c103d90 msgr2=0x7f179c075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:22.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.608+0000 7f179be00700 1 --2- 172.21.15.123:0/1881763518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f179c103d90 0x7f179c075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.608+0000 7f179be00700 1 -- 172.21.15.123:0/1881763518 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f179c1a7fa0 con 0x7f179c1033d0 2024-05-08T06:11:22.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.608+0000 7f179b400700 1 --2- 172.21.15.123:0/1881763518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f179c103d90 0x7f179c075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:11:22.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.608+0000 7f179be00700 1 --2- 172.21.15.123:0/1881763518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f179c1033d0 0x7f179c0757a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f178c00d980 tx=0x7f178c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:22.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.608+0000 7f1798c00700 1 -- 172.21.15.123:0/1881763518 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c016070 con 0x7f179c1033d0 2024-05-08T06:11:22.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.609+0000 7f1798c00700 1 -- 172.21.15.123:0/1881763518 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f178c004830 con 0x7f179c1033d0 2024-05-08T06:11:22.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.609+0000 7f17a1a00700 1 -- 172.21.15.123:0/1881763518 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f179c1a8290 con 0x7f179c1033d0 2024-05-08T06:11:22.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.609+0000 7f1798c00700 1 -- 172.21.15.123:0/1881763518 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c005540 con 0x7f179c1033d0 2024-05-08T06:11:22.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.609+0000 7f17a1a00700 1 -- 172.21.15.123:0/1881763518 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f179c1a87a0 con 0x7f179c1033d0 2024-05-08T06:11:22.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.612+0000 7f1798c00700 1 -- 172.21.15.123:0/1881763518 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 19) ==== 149704+0+0 (secure 0 0 0) 0x7f178c004220 con 0x7f179c1033d0 2024-05-08T06:11:22.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.613+0000 7f17ad8d6b80 1 Processor -- start 2024-05-08T06:11:22.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.614+0000 7f17ad8d6b80 1 -- start start 2024-05-08T06:11:22.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.614+0000 7f17ad8d6b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ab329dda0 0x563ab329e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:22.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.614+0000 7f17ad8d6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ab329e6c0 0x563ab3298430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:22.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.614+0000 7f17ad8d6b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ab3298a50 0x563ab3298ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:22.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.614+0000 7f17ad8d6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563ab31eaa20 con 0x563ab329e6c0 2024-05-08T06:11:22.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.614+0000 7f17ad8d6b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x563ab3188760 con 0x563ab3298a50 2024-05-08T06:11:22.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.614+0000 7f17ad8d6b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x563ab31e69e0 con 0x563ab329dda0 2024-05-08T06:11:22.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.614+0000 7f179be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ab329e6c0 0x563ab3298430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:22.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.614+0000 7f17a1000700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ab329dda0 0x563ab329e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:22.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.614+0000 7f179be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ab329e6c0 0x563ab3298430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34804/0 (socket says 172.21.15.123:34804) 2024-05-08T06:11:22.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.614+0000 7f179be00700 1 -- 172.21.15.123:0/9523478 learned_addr learned my addr 172.21.15.123:0/9523478 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:22.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.614+0000 7f179b400700 1 --2- 172.21.15.123:0/9523478 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ab3298a50 0x563ab3298ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:22.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.615+0000 7f17a1000700 1 -- 172.21.15.123:0/9523478 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ab3298a50 msgr2=0x563ab3298ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:22.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.615+0000 7f17a1000700 1 --2- 172.21.15.123:0/9523478 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ab3298a50 0x563ab3298ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.615+0000 7f17a1000700 1 -- 172.21.15.123:0/9523478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ab329e6c0 msgr2=0x563ab3298430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:22.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.615+0000 7f17a1000700 1 --2- 172.21.15.123:0/9523478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ab329e6c0 0x563ab3298430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.615+0000 7f17a1000700 1 -- 172.21.15.123:0/9523478 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ab31378d0 con 0x563ab329dda0 2024-05-08T06:11:22.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.615+0000 7f1798c00700 1 --2- 172.21.15.123:0/1881763518 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f17800b2910 0x7f17800b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:22.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.615+0000 7f1798c00700 1 -- 172.21.15.123:0/1881763518 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f178c0e2120 con 0x7f179c1033d0 2024-05-08T06:11:22.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.615+0000 7f17a1000700 1 --2- 172.21.15.123:0/9523478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ab329dda0 0x563ab329e180 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1794011580 tx=0x7f1794009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49bc12940de06d03 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:22.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.616+0000 7f179be00700 1 --2- 172.21.15.123:0/1881763518 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f17800b2910 0x7f17800b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:22.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.616+0000 7f1791600700 1 -- 172.21.15.123:0/9523478 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1794011e80 con 0x563ab329dda0 2024-05-08T06:11:22.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.616+0000 7f1791600700 1 -- 172.21.15.123:0/9523478 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1794004bb0 con 0x563ab329dda0 2024-05-08T06:11:22.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.616+0000 7f1791600700 1 -- 172.21.15.123:0/9523478 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1794004720 con 0x563ab329dda0 2024-05-08T06:11:22.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.617+0000 7f17ad8d6b80 1 -- 172.21.15.123:0/9523478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ab329dda0 msgr2=0x563ab329e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:22.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.617+0000 7f179be00700 1 --2- 172.21.15.123:0/1881763518 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f17800b2910 0x7f17800b4dd0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f178c000c00 tx=0x7f178c0e5bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:22.617 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.617+0000 7f17ad8d6b80 1 --2- 172.21.15.123:0/9523478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ab329dda0 0x563ab329e180 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1794011580 tx=0x7f1794009920 comp rx=0 tx=0).stop 2024-05-08T06:11:22.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.617+0000 7f17ad8d6b80 1 -- 172.21.15.123:0/9523478 shutdown_connections 2024-05-08T06:11:22.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.617+0000 7f17ad8d6b80 1 --2- 172.21.15.123:0/9523478 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ab3298a50 0x563ab3298ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.617+0000 7f17ad8d6b80 1 --2- 172.21.15.123:0/9523478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ab329dda0 0x563ab329e180 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.617+0000 7f17ad8d6b80 1 --2- 172.21.15.123:0/9523478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ab329e6c0 0x563ab3298430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.617+0000 7f17ad8d6b80 1 -- 172.21.15.123:0/9523478 >> 172.21.15.123:0/9523478 conn(0x563ab32106d0 msgr2=0x563ab3292ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:22.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.617+0000 7f17ad8d6b80 1 -- 172.21.15.123:0/9523478 shutdown_connections 2024-05-08T06:11:22.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.617+0000 7f17ad8d6b80 1 -- 172.21.15.123:0/9523478 wait complete. 2024-05-08T06:11:22.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.618+0000 7f17ad8d6b80 1 -- start start 2024-05-08T06:11:22.619 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.618+0000 7f17ad8d6b80 1 Processor -- start 2024-05-08T06:11:22.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.618+0000 7f17ad8d6b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ab3298a50 0x563ab32a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:22.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.618+0000 7f17ad8d6b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ab329dda0 0x563ab32a1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:22.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.618+0000 7f17ad8d6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ab329e6c0 0x563ab32a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:22.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.618+0000 7f17ad8d6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563ab32a5150 con 0x563ab329e6c0 2024-05-08T06:11:22.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.618+0000 7f17ad8d6b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x563ab3150b20 con 0x563ab329dda0 2024-05-08T06:11:22.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.618+0000 7f17ad8d6b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x563ab32a52d0 con 0x563ab3298a50 2024-05-08T06:11:22.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.618+0000 7f17ad8d6b80 10 client.0 _ll_register_callbacks cb 0x563ab3191c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:11:22.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.619+0000 7f17a1000700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ab3298a50 0x563ab32a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:22.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.619+0000 7f179b400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ab329dda0 0x563ab32a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:22.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.619+0000 7f17a1000700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ab3298a50 0x563ab32a44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:40840/0 (socket says 172.21.15.123:40840) 2024-05-08T06:11:22.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.619+0000 7f17a1000700 1 -- 172.21.15.123:0/276972519 learned_addr learned my addr 172.21.15.123:0/276972519 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:22.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.619+0000 7f17a1000700 1 --2- 172.21.15.123:0/276972519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ab329e6c0 0x563ab32a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:22.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.619+0000 7f17a1000700 1 -- 172.21.15.123:0/276972519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ab329dda0 msgr2=0x563ab32a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:22.620 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.619+0000 7f17a1000700 1 --2- 172.21.15.123:0/276972519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ab329dda0 0x563ab32a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.619+0000 7f17a1000700 1 -- 172.21.15.123:0/276972519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ab329e6c0 msgr2=0x563ab32a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:22.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.619+0000 7f17a1000700 1 --2- 172.21.15.123:0/276972519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ab329e6c0 0x563ab32a1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.619+0000 7f17a1000700 1 -- 172.21.15.123:0/276972519 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ab31b0420 con 0x563ab3298a50 2024-05-08T06:11:22.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.620+0000 7f17a1000700 1 --2- 172.21.15.123:0/276972519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ab3298a50 0x563ab32a44b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1794000f80 tx=0x7f17940042c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb7e03842e3da5e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:22.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.620+0000 7f1786000700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-05-08T06:11:22.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.620+0000 7f1786000700 1 -- 172.21.15.123:0/276972519 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1794022020 con 0x563ab3298a50 2024-05-08T06:11:22.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.620+0000 7f1786000700 1 -- 172.21.15.123:0/276972519 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f1794003cd0 con 0x563ab3298a50 2024-05-08T06:11:22.622 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.620+0000 7f17ad8d6b80 1 -- 172.21.15.123:0/276972519 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1794004bb0 con 0x563ab3298a50 2024-05-08T06:11:22.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.621+0000 7f1786000700 1 -- 172.21.15.123:0/276972519 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1794003ed0 con 0x563ab3298a50 2024-05-08T06:11:22.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.621+0000 7f17ad8d6b80 10 client.25258 fetch_fsmap learned FSMap version 24 2024-05-08T06:11:22.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.621+0000 7f17ad8d6b80 1 -- 172.21.15.123:0/276972519 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563ab32b8290 con 0x563ab3298a50 2024-05-08T06:11:22.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.621+0000 7f1786000700 1 -- 172.21.15.123:0/276972519 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1794004bb0 con 0x563ab3298a50 2024-05-08T06:11:22.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.621+0000 7f1786000700 1 -- 172.21.15.123:0/276972519 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f1794015430 con 0x563ab3298a50 2024-05-08T06:11:22.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.621+0000 7f17ad8d6b80 10 client.25258 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:11:22.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.621+0000 7f17ad8d6b80 10 client.25258 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-08T06:11:22.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.621+0000 7f17ad8d6b80 20 client.25258 populate_metadata read hostname 'smithi123' 2024-05-08T06:11:22.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.621+0000 7f17ad8d6b80 1 --2- 172.21.15.123:0/276972519 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x563ab32ba870 0x563ab32bcc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:22.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.621+0000 7f17ad8d6b80 4 client.25258 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:11:22.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.621+0000 7f17ad8d6b80 1 -- 172.21.15.123:0/276972519 --> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ab3265140 con 0x563ab32ba870 2024-05-08T06:11:22.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.622+0000 7f179b400700 1 --2- 172.21.15.123:0/276972519 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x563ab32ba870 0x563ab32bcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:22.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.622+0000 7f179b400700 1 --2- 172.21.15.123:0/276972519 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x563ab32ba870 0x563ab32bcc60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=23678cd438ffd8e5 server_cookie=7de8c3fe8195ac76 in_seq=0 out_seq=0 2024-05-08T06:11:22.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.622+0000 7f1786000700 10 client.25258 ms_handle_connect on v2:172.21.15.123:6840/1571726231 2024-05-08T06:11:22.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.623+0000 7f1786000700 1 -- 172.21.15.123:0/276972519 <== mds.1 v2:172.21.15.123:6840/1571726231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563ab3265140 con 0x563ab32ba870 2024-05-08T06:11:22.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.623+0000 7f1786000700 10 client.25258 handle_command_reply: tid=0 2024-05-08T06:11:22.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.623+0000 7f17ad8d6b80 1 client.25258 shutdown 2024-05-08T06:11:22.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.623+0000 7f17ad8d6b80 1 -- 172.21.15.123:0/276972519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ab3298a50 msgr2=0x563ab32a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:22.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.623+0000 7f17ad8d6b80 1 --2- 172.21.15.123:0/276972519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ab3298a50 0x563ab32a44b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1794000f80 tx=0x7f17940042c0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.623+0000 7f17ad8d6b80 1 -- 172.21.15.123:0/276972519 shutdown_connections 2024-05-08T06:11:22.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.623+0000 7f17ad8d6b80 1 --2- 172.21.15.123:0/276972519 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x563ab32ba870 0x563ab32bcc60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.624+0000 7f17ad8d6b80 1 --2- 172.21.15.123:0/276972519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ab329e6c0 0x563ab32a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.624+0000 7f17ad8d6b80 1 --2- 172.21.15.123:0/276972519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ab3298a50 0x563ab32a44b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.624+0000 7f1786000700 0 client.25258 ms_handle_reset on v2:172.21.15.123:6840/1571726231 2024-05-08T06:11:22.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.624+0000 7f17ad8d6b80 1 --2- 172.21.15.123:0/276972519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ab329dda0 0x563ab32a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.624+0000 7f17ad8d6b80 1 -- 172.21.15.123:0/276972519 >> 172.21.15.123:0/276972519 conn(0x563ab32106d0 msgr2=0x563ab328d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:22.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.624+0000 7f17ad8d6b80 1 -- 172.21.15.123:0/276972519 shutdown_connections 2024-05-08T06:11:22.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.624+0000 7f17ad8d6b80 1 -- 172.21.15.123:0/276972519 wait complete. 2024-05-08T06:11:22.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.624+0000 7f17ad8d6b80 20 client.25258 trim_cache size 0 max 16384 2024-05-08T06:11:22.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.641+0000 7f17ad8d6b80 1 Processor -- start 2024-05-08T06:11:22.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.641+0000 7f17ad8d6b80 1 -- start start 2024-05-08T06:11:22.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.641+0000 7f17ad8d6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ab3298a50 0x563ab3298430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:22.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.641+0000 7f17ad8d6b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ab329dda0 0x563ab329fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:22.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.641+0000 7f17ad8d6b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ab329e6c0 0x563ab32ae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:22.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.642+0000 7f17ad8d6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f179c1ae110 con 0x563ab3298a50 2024-05-08T06:11:22.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.642+0000 7f17ad8d6b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f179c1ab9a0 con 0x563ab329dda0 2024-05-08T06:11:22.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.642+0000 7f17ad8d6b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f179c19d760 con 0x563ab329e6c0 2024-05-08T06:11:22.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.642+0000 7f17a1000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ab3298a50 0x563ab3298430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:22.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.642+0000 7f179b400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ab329dda0 0x563ab329fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:22.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.642+0000 7f179b400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ab329dda0 0x563ab329fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:46438/0 (socket says 172.21.15.123:46438) 2024-05-08T06:11:22.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.642+0000 7f179b400700 1 -- 172.21.15.123:0/1460585990 learned_addr learned my addr 172.21.15.123:0/1460585990 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:22.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.642+0000 7f17a1000700 1 --2- 172.21.15.123:0/1460585990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ab329e6c0 0x563ab32ae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:22.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.642+0000 7f179b400700 1 -- 172.21.15.123:0/1460585990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ab329e6c0 msgr2=0x563ab32ae1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:22.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.642+0000 7f179b400700 1 --2- 172.21.15.123:0/1460585990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ab329e6c0 0x563ab32ae1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.642+0000 7f179b400700 1 -- 172.21.15.123:0/1460585990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ab3298a50 msgr2=0x563ab3298430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:22.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.642+0000 7f179b400700 1 --2- 172.21.15.123:0/1460585990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ab3298a50 0x563ab3298430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.642+0000 7f179b400700 1 -- 172.21.15.123:0/1460585990 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ab32939c0 con 0x563ab329dda0 2024-05-08T06:11:22.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.642+0000 7f17a1000700 1 --2- 172.21.15.123:0/1460585990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ab3298a50 0x563ab3298430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:11:22.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.643+0000 7f179b400700 1 --2- 172.21.15.123:0/1460585990 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ab329dda0 0x563ab329fae0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1788009990 tx=0x7f1788005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b791f48f4058e978 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:22.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.643+0000 7f1790c00700 1 -- 172.21.15.123:0/1460585990 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788005690 con 0x563ab329dda0 2024-05-08T06:11:22.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.643+0000 7f1790c00700 1 -- 172.21.15.123:0/1460585990 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1788029030 con 0x563ab329dda0 2024-05-08T06:11:22.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.643+0000 7f1790c00700 1 -- 172.21.15.123:0/1460585990 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17880295c0 con 0x563ab329dda0 2024-05-08T06:11:22.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.644+0000 7f17ad8d6b80 1 -- 172.21.15.123:0/1460585990 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ab329dda0 msgr2=0x563ab329fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:22.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.644+0000 7f17ad8d6b80 1 --2- 172.21.15.123:0/1460585990 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ab329dda0 0x563ab329fae0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1788009990 tx=0x7f1788005040 comp rx=0 tx=0).stop 2024-05-08T06:11:22.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.644+0000 7f17ad8d6b80 1 -- 172.21.15.123:0/1460585990 shutdown_connections 2024-05-08T06:11:22.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.644+0000 7f17ad8d6b80 1 --2- 172.21.15.123:0/1460585990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ab3298a50 0x563ab3298430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.644+0000 7f17ad8d6b80 1 --2- 172.21.15.123:0/1460585990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ab329e6c0 0x563ab32ae1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.644+0000 7f17ad8d6b80 1 --2- 172.21.15.123:0/1460585990 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ab329dda0 0x563ab329fae0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.644+0000 7f17ad8d6b80 1 -- 172.21.15.123:0/1460585990 >> 172.21.15.123:0/1460585990 conn(0x563ab32106d0 msgr2=0x563ab32a3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:22.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.644+0000 7f17ad8d6b80 1 -- 172.21.15.123:0/1460585990 shutdown_connections 2024-05-08T06:11:22.645 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.644+0000 7f17ad8d6b80 1 -- 172.21.15.123:0/1460585990 wait complete. 2024-05-08T06:11:22.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.645+0000 7f17ad8d6b80 1 -- start start 2024-05-08T06:11:22.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.645+0000 7f17ad8d6b80 1 Processor -- start 2024-05-08T06:11:22.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.645+0000 7f17ad8d6b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ab3298a50 0x563ab32aaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:22.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.645+0000 7f17ad8d6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ab329dda0 0x563ab32c1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:22.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.645+0000 7f17ad8d6b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ab329e6c0 0x563ab32c1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:22.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.645+0000 7f17ad8d6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f179c1a9e60 con 0x563ab329dda0 2024-05-08T06:11:22.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.645+0000 7f17ad8d6b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f179c1ab2d0 con 0x563ab3298a50 2024-05-08T06:11:22.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.645+0000 7f17ad8d6b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f179c1aa530 con 0x563ab329e6c0 2024-05-08T06:11:22.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.645+0000 7f17ad8d6b80 10 client.0 _ll_register_callbacks cb 0x563ab3191c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:11:22.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.646+0000 7f17a1000700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ab3298a50 0x563ab32aaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:22.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.646+0000 7f17a1000700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ab3298a50 0x563ab32aaba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:46448/0 (socket says 172.21.15.123:46448) 2024-05-08T06:11:22.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.646+0000 7f17a1000700 1 -- 172.21.15.123:0/2785106996 learned_addr learned my addr 172.21.15.123:0/2785106996 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:22.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.646+0000 7f179b400700 1 --2- 172.21.15.123:0/2785106996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ab329dda0 0x563ab32c1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:22.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.646+0000 7f17a1000700 1 --2- 172.21.15.123:0/2785106996 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ab329e6c0 0x563ab32c1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:22.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.646+0000 7f17a1000700 1 -- 172.21.15.123:0/2785106996 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ab329e6c0 msgr2=0x563ab32c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:22.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.646+0000 7f17a1000700 1 --2- 172.21.15.123:0/2785106996 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ab329e6c0 0x563ab32c1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.646+0000 7f17a1000700 1 -- 172.21.15.123:0/2785106996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ab329dda0 msgr2=0x563ab32c1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:22.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.646+0000 7f17a1000700 1 --2- 172.21.15.123:0/2785106996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ab329dda0 0x563ab32c1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.646+0000 7f17a1000700 1 -- 172.21.15.123:0/2785106996 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ab3323ea0 con 0x563ab3298a50 2024-05-08T06:11:22.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.646+0000 7f17a1000700 1 --2- 172.21.15.123:0/2785106996 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ab3298a50 0x563ab32aaba0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1794004490 tx=0x7f179400b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adc350552c9376ff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:22.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.646+0000 7f1786000700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-05-08T06:11:22.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.647+0000 7f1786000700 1 -- 172.21.15.123:0/2785106996 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1794022040 con 0x563ab3298a50 2024-05-08T06:11:22.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.647+0000 7f17ad8d6b80 1 -- 172.21.15.123:0/2785106996 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1788029030 con 0x563ab3298a50 2024-05-08T06:11:22.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.647+0000 7f1786000700 1 -- 172.21.15.123:0/2785106996 <== mon.2 v2:172.21.15.163:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f1794004840 con 0x563ab3298a50 2024-05-08T06:11:22.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.647+0000 7f1786000700 1 -- 172.21.15.123:0/2785106996 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1794004a40 con 0x563ab3298a50 2024-05-08T06:11:22.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.647+0000 7f17ad8d6b80 10 client.25274 fetch_fsmap learned FSMap version 24 2024-05-08T06:11:22.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.647+0000 7f17ad8d6b80 1 -- 172.21.15.123:0/2785106996 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563ab3324ff0 con 0x563ab3298a50 2024-05-08T06:11:22.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.647+0000 7f1786000700 1 -- 172.21.15.123:0/2785106996 <== mon.2 v2:172.21.15.163:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1788029030 con 0x563ab3298a50 2024-05-08T06:11:22.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.648+0000 7f1786000700 1 -- 172.21.15.123:0/2785106996 <== mon.2 v2:172.21.15.163:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f1794034040 con 0x563ab3298a50 2024-05-08T06:11:22.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.648+0000 7f17ad8d6b80 10 client.25274 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:11:22.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.648+0000 7f17ad8d6b80 10 client.25274 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-08T06:11:22.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.648+0000 7f17ad8d6b80 20 client.25274 populate_metadata read hostname 'smithi123' 2024-05-08T06:11:22.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.648+0000 7f17ad8d6b80 1 --2- 172.21.15.123:0/2785106996 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x563ab3325640 0x563ab3327a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:22.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.648+0000 7f17ad8d6b80 4 client.25274 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T06:11:22.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.648+0000 7f17ad8d6b80 1 -- 172.21.15.123:0/2785106996 --> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563ab31f8400 con 0x563ab3325640 2024-05-08T06:11:22.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.648+0000 7f179b400700 1 --2- 172.21.15.123:0/2785106996 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x563ab3325640 0x563ab3327a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:22.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.649+0000 7f179b400700 1 --2- 172.21.15.123:0/2785106996 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x563ab3325640 0x563ab3327a60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f9f3704fd2db39a6 server_cookie=9f95cf3bca5c0a95 in_seq=0 out_seq=0 2024-05-08T06:11:22.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.649+0000 7f1786000700 10 client.25274 ms_handle_connect on v2:172.21.15.123:6840/1571726231 2024-05-08T06:11:22.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.649+0000 7f1786000700 1 -- 172.21.15.123:0/2785106996 <== mds.1 v2:172.21.15.123:6840/1571726231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563ab31f8400 con 0x563ab3325640 2024-05-08T06:11:22.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.649+0000 7f1786000700 10 client.25274 handle_command_reply: tid=0 2024-05-08T06:11:22.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.649+0000 7f17ad8d6b80 1 client.25274 shutdown 2024-05-08T06:11:22.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.649+0000 7f17ad8d6b80 1 -- 172.21.15.123:0/2785106996 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ab3298a50 msgr2=0x563ab32aaba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:22.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.649+0000 7f17ad8d6b80 1 --2- 172.21.15.123:0/2785106996 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ab3298a50 0x563ab32aaba0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1794004490 tx=0x7f179400b040 comp rx=0 tx=0).stop 2024-05-08T06:11:22.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.649+0000 7f17ad8d6b80 1 -- 172.21.15.123:0/2785106996 shutdown_connections 2024-05-08T06:11:22.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.649+0000 7f17ad8d6b80 1 --2- 172.21.15.123:0/2785106996 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x563ab3325640 0x563ab3327a60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.649+0000 7f17ad8d6b80 1 --2- 172.21.15.123:0/2785106996 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563ab329e6c0 0x563ab32c1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.649+0000 7f1786000700 0 client.25274 ms_handle_reset on v2:172.21.15.123:6840/1571726231 2024-05-08T06:11:22.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.649+0000 7f17ad8d6b80 1 --2- 172.21.15.123:0/2785106996 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ab3298a50 0x563ab32aaba0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.649+0000 7f17ad8d6b80 1 --2- 172.21.15.123:0/2785106996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ab329dda0 0x563ab32c1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.651 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.649+0000 7f17ad8d6b80 1 -- 172.21.15.123:0/2785106996 >> 172.21.15.123:0/2785106996 conn(0x563ab32106d0 msgr2=0x563ab32a3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:22.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.650+0000 7f17ad8d6b80 1 -- 172.21.15.123:0/2785106996 shutdown_connections 2024-05-08T06:11:22.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.650+0000 7f17ad8d6b80 1 -- 172.21.15.123:0/2785106996 wait complete. 2024-05-08T06:11:22.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.650+0000 7f17ad8d6b80 20 client.25274 trim_cache size 0 max 16384 2024-05-08T06:11:22.652 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-05-08T06:11:22.652 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.650+0000 7f17a1a00700 1 -- 172.21.15.123:0/1881763518 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f17800b2910 msgr2=0x7f17800b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:22.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.650+0000 7f17a1a00700 1 --2- 172.21.15.123:0/1881763518 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f17800b2910 0x7f17800b4dd0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f178c000c00 tx=0x7f178c0e5bd0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.650+0000 7f17a1a00700 1 -- 172.21.15.123:0/1881763518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f179c1033d0 msgr2=0x7f179c0757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:22.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.650+0000 7f17a1a00700 1 --2- 172.21.15.123:0/1881763518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f179c1033d0 0x7f179c0757a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f178c00d980 tx=0x7f178c00dc90 comp rx=0 tx=0).stop 2024-05-08T06:11:22.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.651+0000 7f17a1a00700 1 -- 172.21.15.123:0/1881763518 shutdown_connections 2024-05-08T06:11:22.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.651+0000 7f17a1a00700 1 --2- 172.21.15.123:0/1881763518 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f17800b2910 0x7f17800b4dd0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.651+0000 7f17a1a00700 1 --2- 172.21.15.123:0/1881763518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f179c1033d0 0x7f179c0757a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.651+0000 7f17a1a00700 1 --2- 172.21.15.123:0/1881763518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f179c109460 0x7f179c076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.651+0000 7f17a1a00700 1 --2- 172.21.15.123:0/1881763518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f179c103d90 0x7f179c075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:22.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.651+0000 7f17a1a00700 1 -- 172.21.15.123:0/1881763518 >> 172.21.15.123:0/1881763518 conn(0x7f179c0fef20 msgr2=0x7f179c0ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:22.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.651+0000 7f17a1a00700 1 -- 172.21.15.123:0/1881763518 shutdown_connections 2024-05-08T06:11:22.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:22.651+0000 7f17a1a00700 1 -- 172.21.15.123:0/1881763518 wait complete. 2024-05-08T06:11:22.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:22 smithi163 ceph-mon[34520]: pgmap v487: 97 pgs: 97 active+clean; 673 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-08T06:11:22.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:22 smithi192 ceph-mon[34664]: pgmap v487: 97 pgs: 97 active+clean; 673 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-08T06:11:23.799 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-08T06:11:24.241 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:11:24.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:24 smithi123 ceph-mon[29787]: pgmap v488: 97 pgs: 97 active+clean; 673 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-08T06:11:24.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:24 smithi163 ceph-mon[34520]: pgmap v488: 97 pgs: 97 active+clean; 673 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-08T06:11:24.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:24 smithi192 ceph-mon[34664]: pgmap v488: 97 pgs: 97 active+clean; 673 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-08T06:11:26.425 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:26 smithi123 ceph-mon[29787]: pgmap v489: 97 pgs: 97 active+clean; 674 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-08T06:11:26.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.712+0000 7fa2dfe00700 1 -- 172.21.15.123:0/219604420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2e0105f90 msgr2=0x7fa2e0112450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:26.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.712+0000 7fa2dfe00700 1 --2- 172.21.15.123:0/219604420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2e0105f90 0x7fa2e0112450 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa2d800b0d0 tx=0x7fa2d800b3e0 comp rx=0 tx=0).stop 2024-05-08T06:11:26.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.712+0000 7fa2dfe00700 1 -- 172.21.15.123:0/219604420 shutdown_connections 2024-05-08T06:11:26.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.712+0000 7fa2dfe00700 1 --2- 172.21.15.123:0/219604420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2e0105f90 0x7fa2e0112450 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:26.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.712+0000 7fa2dfe00700 1 --2- 172.21.15.123:0/219604420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2e010b680 0x7fa2e010ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:26.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.712+0000 7fa2dfe00700 1 --2- 172.21.15.123:0/219604420 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2e01055d0 0x7fa2e0105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:26.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.712+0000 7fa2dfe00700 1 -- 172.21.15.123:0/219604420 >> 172.21.15.123:0/219604420 conn(0x7fa2e0101080 msgr2=0x7fa2e01034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:26.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.712+0000 7fa2dfe00700 1 -- 172.21.15.123:0/219604420 shutdown_connections 2024-05-08T06:11:26.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.712+0000 7fa2dfe00700 1 -- 172.21.15.123:0/219604420 wait complete. 2024-05-08T06:11:26.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.713+0000 7fa2dfe00700 1 Processor -- start 2024-05-08T06:11:26.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.713+0000 7fa2dfe00700 1 -- start start 2024-05-08T06:11:26.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.714+0000 7fa2dfe00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2e01055d0 0x7fa2e01a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:26.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.714+0000 7fa2dfe00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2e0105f90 0x7fa2e01a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:26.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.714+0000 7fa2dfe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2e010b680 0x7fa2e019b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:26.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.714+0000 7fa2dfe00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e0115830 con 0x7fa2e010b680 2024-05-08T06:11:26.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.714+0000 7fa2dfe00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e01156b0 con 0x7fa2e01055d0 2024-05-08T06:11:26.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.714+0000 7fa2dfe00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e01159b0 con 0x7fa2e0105f90 2024-05-08T06:11:26.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.714+0000 7fa2df400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2e010b680 0x7fa2e019b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:26.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.714+0000 7fa2de000700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2e0105f90 0x7fa2e01a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:26.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.714+0000 7fa2df400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2e010b680 0x7fa2e019b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:46970/0 (socket says 172.21.15.123:46970) 2024-05-08T06:11:26.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.714+0000 7fa2de000700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2e0105f90 0x7fa2e01a1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:46602/0 (socket says 172.21.15.123:46602) 2024-05-08T06:11:26.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.714+0000 7fa2df400700 1 -- 172.21.15.123:0/3809524489 learned_addr learned my addr 172.21.15.123:0/3809524489 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:26.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.714+0000 7fa2dea00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2e01055d0 0x7fa2e01a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:26.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.715+0000 7fa2de000700 1 -- 172.21.15.123:0/3809524489 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2e01055d0 msgr2=0x7fa2e01a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:26.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.715+0000 7fa2de000700 1 --2- 172.21.15.123:0/3809524489 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2e01055d0 0x7fa2e01a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:26.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.715+0000 7fa2de000700 1 -- 172.21.15.123:0/3809524489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2e010b680 msgr2=0x7fa2e019b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:26.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.715+0000 7fa2de000700 1 --2- 172.21.15.123:0/3809524489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2e010b680 0x7fa2e019b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:26.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.715+0000 7fa2de000700 1 -- 172.21.15.123:0/3809524489 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e019c150 con 0x7fa2e0105f90 2024-05-08T06:11:26.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.715+0000 7fa2de000700 1 --2- 172.21.15.123:0/3809524489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2e0105f90 0x7fa2e01a1d70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00e9f0 tx=0x7fa2cc00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:26.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.715+0000 7fa2d7400700 1 -- 172.21.15.123:0/3809524489 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc00c9c0 con 0x7fa2e0105f90 2024-05-08T06:11:26.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.715+0000 7fa2dfe00700 1 -- 172.21.15.123:0/3809524489 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2e019c440 con 0x7fa2e0105f90 2024-05-08T06:11:26.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.715+0000 7fa2d7400700 1 -- 172.21.15.123:0/3809524489 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2cc00cb60 con 0x7fa2e0105f90 2024-05-08T06:11:26.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.715+0000 7fa2d7400700 1 -- 172.21.15.123:0/3809524489 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc018680 con 0x7fa2e0105f90 2024-05-08T06:11:26.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.715+0000 7fa2dfe00700 1 -- 172.21.15.123:0/3809524489 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2e01a8700 con 0x7fa2e0105f90 2024-05-08T06:11:26.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.720+0000 7fa2d7400700 1 -- 172.21.15.123:0/3809524489 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) ==== 149704+0+0 (secure 0 0 0) 0x7fa2cc010430 con 0x7fa2e0105f90 2024-05-08T06:11:26.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.720+0000 7fa2f0050b80 1 Processor -- start 2024-05-08T06:11:26.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.721+0000 7fa2f0050b80 1 -- start start 2024-05-08T06:11:26.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.721+0000 7fa2f0050b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571b917eda0 0x5571b917f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:26.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.721+0000 7fa2f0050b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571b917f6c0 0x5571b9179430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:26.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.721+0000 7fa2f0050b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571b9179a50 0x5571b9179ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:26.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.721+0000 7fa2f0050b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5571b90cba20 con 0x5571b9179a50 2024-05-08T06:11:26.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.721+0000 7fa2f0050b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5571b9069760 con 0x5571b917eda0 2024-05-08T06:11:26.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.721+0000 7fa2f0050b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5571b90c79e0 con 0x5571b917f6c0 2024-05-08T06:11:26.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.721+0000 7fa2df400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571b917eda0 0x5571b917f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:26.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.722+0000 7fa2df400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571b917eda0 0x5571b917f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:39248/0 (socket says 172.21.15.123:39248) 2024-05-08T06:11:26.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.722+0000 7fa2df400700 1 -- 172.21.15.123:0/1673579471 learned_addr learned my addr 172.21.15.123:0/1673579471 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:26.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.722+0000 7fa2de000700 1 --2- 172.21.15.123:0/1673579471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571b9179a50 0x5571b9179ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:26.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.722+0000 7fa2dea00700 1 --2- 172.21.15.123:0/1673579471 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571b917f6c0 0x5571b9179430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:26.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.722+0000 7fa2df400700 1 -- 172.21.15.123:0/1673579471 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571b917f6c0 msgr2=0x5571b9179430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:26.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.722+0000 7fa2df400700 1 --2- 172.21.15.123:0/1673579471 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571b917f6c0 0x5571b9179430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:26.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.722+0000 7fa2df400700 1 -- 172.21.15.123:0/1673579471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571b9179a50 msgr2=0x5571b9179ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:26.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.722+0000 7fa2df400700 1 --2- 172.21.15.123:0/1673579471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571b9179a50 0x5571b9179ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:26.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.722+0000 7fa2df400700 1 -- 172.21.15.123:0/1673579471 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571b90188d0 con 0x5571b917eda0 2024-05-08T06:11:26.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.722+0000 7fa2de000700 1 --2- 172.21.15.123:0/1673579471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571b9179a50 0x5571b9179ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:11:26.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.722+0000 7fa2d7400700 1 --2- 172.21.15.123:0/3809524489 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa2c40b2990 0x7fa2c40b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:26.723 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.723+0000 7fa2df400700 1 --2- 172.21.15.123:0/1673579471 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571b917eda0 0x5571b917f180 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009e20 tx=0x7fa2d80092e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a5dbeba37b2fe85 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:26.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.723+0000 7fa2d7400700 1 -- 172.21.15.123:0/3809524489 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fa2cc0e24d0 con 0x7fa2e0105f90 2024-05-08T06:11:26.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.723+0000 7fa2dea00700 1 --2- 172.21.15.123:0/3809524489 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa2c40b2990 0x7fa2c40b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:26.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.723+0000 7fa2cbe00700 1 -- 172.21.15.123:0/1673579471 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d800bca0 con 0x5571b917eda0 2024-05-08T06:11:26.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.723+0000 7fa2cbe00700 1 -- 172.21.15.123:0/1673579471 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2d800be40 con 0x5571b917eda0 2024-05-08T06:11:26.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.723+0000 7fa2cbe00700 1 -- 172.21.15.123:0/1673579471 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8005040 con 0x5571b917eda0 2024-05-08T06:11:26.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.724+0000 7fa2dea00700 1 --2- 172.21.15.123:0/3809524489 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa2c40b2990 0x7fa2c40b4e50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x5571b9175970 tx=0x7fa2d0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:26.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.724+0000 7fa2f0050b80 1 -- 172.21.15.123:0/1673579471 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571b917eda0 msgr2=0x5571b917f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:26.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.724+0000 7fa2f0050b80 1 --2- 172.21.15.123:0/1673579471 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571b917eda0 0x5571b917f180 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009e20 tx=0x7fa2d80092e0 comp rx=0 tx=0).stop 2024-05-08T06:11:26.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.724+0000 7fa2f0050b80 1 -- 172.21.15.123:0/1673579471 shutdown_connections 2024-05-08T06:11:26.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.724+0000 7fa2f0050b80 1 --2- 172.21.15.123:0/1673579471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571b9179a50 0x5571b9179ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:26.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.724+0000 7fa2f0050b80 1 --2- 172.21.15.123:0/1673579471 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571b917eda0 0x5571b917f180 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:26.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.724+0000 7fa2f0050b80 1 --2- 172.21.15.123:0/1673579471 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571b917f6c0 0x5571b9179430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:26.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.724+0000 7fa2f0050b80 1 -- 172.21.15.123:0/1673579471 >> 172.21.15.123:0/1673579471 conn(0x5571b90f16d0 msgr2=0x5571b9173ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:26.725 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.724+0000 7fa2f0050b80 1 -- 172.21.15.123:0/1673579471 shutdown_connections 2024-05-08T06:11:26.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.724+0000 7fa2f0050b80 1 -- 172.21.15.123:0/1673579471 wait complete. 2024-05-08T06:11:26.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.725+0000 7fa2f0050b80 1 -- start start 2024-05-08T06:11:26.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.725+0000 7fa2f0050b80 1 Processor -- start 2024-05-08T06:11:26.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.726+0000 7fa2f0050b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571b9179a50 0x5571b91853f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:26.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.726+0000 7fa2f0050b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571b917eda0 0x5571b9182790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:26.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.726+0000 7fa2f0050b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571b917f6c0 0x5571b9182cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:26.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.726+0000 7fa2f0050b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5571b9186090 con 0x5571b917f6c0 2024-05-08T06:11:26.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.726+0000 7fa2f0050b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5571b9031b20 con 0x5571b9179a50 2024-05-08T06:11:26.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.726+0000 7fa2f0050b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5571b9186210 con 0x5571b917eda0 2024-05-08T06:11:26.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.726+0000 7fa2f0050b80 10 client.0 _ll_register_callbacks cb 0x5571b9072c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:11:26.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.726+0000 7fa2de000700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571b917eda0 0x5571b9182790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:26.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.726+0000 7fa2de000700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571b917eda0 0x5571b9182790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:46624/0 (socket says 172.21.15.123:46624) 2024-05-08T06:11:26.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.726+0000 7fa2de000700 1 -- 172.21.15.123:0/920015266 learned_addr learned my addr 172.21.15.123:0/920015266 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:26.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.726+0000 7fa2dea00700 1 --2- 172.21.15.123:0/920015266 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571b9179a50 0x5571b91853f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:26.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.726+0000 7fa2df400700 1 --2- 172.21.15.123:0/920015266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571b917f6c0 0x5571b9182cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:26.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.726+0000 7fa2de000700 1 -- 172.21.15.123:0/920015266 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571b9179a50 msgr2=0x5571b91853f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:26.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.726+0000 7fa2de000700 1 --2- 172.21.15.123:0/920015266 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571b9179a50 0x5571b91853f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:26.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.726+0000 7fa2de000700 1 -- 172.21.15.123:0/920015266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571b917f6c0 msgr2=0x5571b9182cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:26.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.726+0000 7fa2de000700 1 --2- 172.21.15.123:0/920015266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571b917f6c0 0x5571b9182cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:26.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.726+0000 7fa2de000700 1 -- 172.21.15.123:0/920015266 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571b9091420 con 0x5571b917eda0 2024-05-08T06:11:26.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.727+0000 7fa2de000700 1 --2- 172.21.15.123:0/920015266 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571b917eda0 0x5571b9182790 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc0a59c0 tx=0x7fa2cc0a9e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e27efdde74a1b6a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:26.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.727+0000 7fa2c8c00700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-05-08T06:11:26.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.727+0000 7fa2c8c00700 1 -- 172.21.15.123:0/920015266 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc0e5e50 con 0x5571b917eda0 2024-05-08T06:11:26.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.727+0000 7fa2f0050b80 1 -- 172.21.15.123:0/920015266 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5571b9146140 con 0x5571b917eda0 2024-05-08T06:11:26.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.727+0000 7fa2c8c00700 1 -- 172.21.15.123:0/920015266 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fa2cc0e20d0 con 0x5571b917eda0 2024-05-08T06:11:26.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.727+0000 7fa2c8c00700 1 -- 172.21.15.123:0/920015266 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa2cc0e22d0 con 0x5571b917eda0 2024-05-08T06:11:26.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.727+0000 7fa2f0050b80 10 client.25276 fetch_fsmap learned FSMap version 24 2024-05-08T06:11:26.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.727+0000 7fa2f0050b80 1 -- 172.21.15.123:0/920015266 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5571b9190940 con 0x5571b917eda0 2024-05-08T06:11:26.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.728+0000 7fa2c8c00700 1 -- 172.21.15.123:0/920015266 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5571b9146140 con 0x5571b917eda0 2024-05-08T06:11:26.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.728+0000 7fa2c8c00700 1 -- 172.21.15.123:0/920015266 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fa2cc010720 con 0x5571b917eda0 2024-05-08T06:11:26.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.728+0000 7fa2f0050b80 10 client.25276 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:11:26.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.728+0000 7fa2f0050b80 10 client.25276 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T06:11:26.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.728+0000 7fa2f0050b80 20 client.25276 populate_metadata read hostname 'smithi123' 2024-05-08T06:11:26.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.728+0000 7fa2f0050b80 1 --2- 172.21.15.123:0/920015266 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5571b9192f80 0x5571b9195370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:26.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.728+0000 7fa2f0050b80 4 client.25276 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:11:26.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.728+0000 7fa2f0050b80 1 -- 172.21.15.123:0/920015266 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571b90d9400 con 0x5571b9192f80 2024-05-08T06:11:26.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.728+0000 7fa2dea00700 1 --2- 172.21.15.123:0/920015266 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5571b9192f80 0x5571b9195370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:26.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.729+0000 7fa2dea00700 1 --2- 172.21.15.123:0/920015266 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5571b9192f80 0x5571b9195370 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3b1747c9feb1d2e server_cookie=43286c6a992b21c5 in_seq=0 out_seq=0 2024-05-08T06:11:26.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.729+0000 7fa2c8c00700 10 client.25276 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:11:26.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.730+0000 7fa2c8c00700 1 -- 172.21.15.123:0/920015266 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5571b90d9400 con 0x5571b9192f80 2024-05-08T06:11:26.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.730+0000 7fa2c8c00700 10 client.25276 handle_command_reply: tid=0 2024-05-08T06:11:26.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.730+0000 7fa2f0050b80 1 client.25276 shutdown 2024-05-08T06:11:26.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.730+0000 7fa2f0050b80 1 -- 172.21.15.123:0/920015266 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571b917eda0 msgr2=0x5571b9182790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:26.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.730+0000 7fa2f0050b80 1 --2- 172.21.15.123:0/920015266 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571b917eda0 0x5571b9182790 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc0a59c0 tx=0x7fa2cc0a9e80 comp rx=0 tx=0).stop 2024-05-08T06:11:26.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.730+0000 7fa2f0050b80 1 -- 172.21.15.123:0/920015266 shutdown_connections 2024-05-08T06:11:26.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.730+0000 7fa2f0050b80 1 --2- 172.21.15.123:0/920015266 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5571b9192f80 0x5571b9195370 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:26.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.730+0000 7fa2f0050b80 1 --2- 172.21.15.123:0/920015266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571b917f6c0 0x5571b9182cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:26.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.730+0000 7fa2f0050b80 1 --2- 172.21.15.123:0/920015266 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571b9179a50 0x5571b91853f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:26.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.730+0000 7fa2f0050b80 1 --2- 172.21.15.123:0/920015266 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571b917eda0 0x5571b9182790 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:26.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.730+0000 7fa2f0050b80 1 -- 172.21.15.123:0/920015266 >> 172.21.15.123:0/920015266 conn(0x5571b90f16d0 msgr2=0x5571b916e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:26.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.730+0000 7fa2c8c00700 0 client.25276 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:11:26.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.730+0000 7fa2f0050b80 1 -- 172.21.15.123:0/920015266 shutdown_connections 2024-05-08T06:11:26.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.731+0000 7fa2f0050b80 1 -- 172.21.15.123:0/920015266 wait complete. 2024-05-08T06:11:26.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.731+0000 7fa2f0050b80 20 client.25276 trim_cache size 0 max 16384 2024-05-08T06:11:26.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.747+0000 7fa2f0050b80 1 Processor -- start 2024-05-08T06:11:26.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.747+0000 7fa2f0050b80 1 -- start start 2024-05-08T06:11:26.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.747+0000 7fa2f0050b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571b9179a50 0x5571b918f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:26.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.747+0000 7fa2f0050b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571b917eda0 0x5571b916e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:26.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.747+0000 7fa2f0050b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571b917f6c0 0x5571b91817b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:26.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.747+0000 7fa2f0050b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e01b22a0 con 0x5571b917f6c0 2024-05-08T06:11:26.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.747+0000 7fa2f0050b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e01afb30 con 0x5571b9179a50 2024-05-08T06:11:26.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.747+0000 7fa2f0050b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e01ad980 con 0x5571b917eda0 2024-05-08T06:11:26.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.748+0000 7fa2dea00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571b9179a50 0x5571b918f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:26.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.748+0000 7fa2dea00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571b9179a50 0x5571b918f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:39260/0 (socket says 172.21.15.123:39260) 2024-05-08T06:11:26.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.748+0000 7fa2dea00700 1 -- 172.21.15.123:0/1958085659 learned_addr learned my addr 172.21.15.123:0/1958085659 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:26.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.748+0000 7fa2de000700 1 --2- 172.21.15.123:0/1958085659 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571b917eda0 0x5571b916e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:26.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.748+0000 7fa2df400700 1 --2- 172.21.15.123:0/1958085659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571b917f6c0 0x5571b91817b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:26.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.748+0000 7fa2dea00700 1 -- 172.21.15.123:0/1958085659 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571b917eda0 msgr2=0x5571b916e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:26.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.748+0000 7fa2dea00700 1 --2- 172.21.15.123:0/1958085659 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571b917eda0 0x5571b916e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:26.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.748+0000 7fa2dea00700 1 -- 172.21.15.123:0/1958085659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571b917f6c0 msgr2=0x5571b91817b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:26.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.748+0000 7fa2dea00700 1 --2- 172.21.15.123:0/1958085659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571b917f6c0 0x5571b91817b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:26.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.748+0000 7fa2dea00700 1 -- 172.21.15.123:0/1958085659 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571b9174ac0 con 0x5571b9179a50 2024-05-08T06:11:26.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.748+0000 7fa2df400700 1 --2- 172.21.15.123:0/1958085659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571b917f6c0 0x5571b91817b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:11:26.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.748+0000 7fa2dea00700 1 --2- 172.21.15.123:0/1958085659 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571b9179a50 0x5571b918f120 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa2e019cb30 tx=0x7fa2d001b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24c0bc8b40324a2d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:26.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.749+0000 7fa2cb400700 1 -- 172.21.15.123:0/1958085659 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0003800 con 0x5571b9179a50 2024-05-08T06:11:26.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.749+0000 7fa2cb400700 1 -- 172.21.15.123:0/1958085659 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2d0030070 con 0x5571b9179a50 2024-05-08T06:11:26.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.749+0000 7fa2cb400700 1 -- 172.21.15.123:0/1958085659 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d001f400 con 0x5571b9179a50 2024-05-08T06:11:26.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.750+0000 7fa2f0050b80 1 -- 172.21.15.123:0/1958085659 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571b9179a50 msgr2=0x5571b918f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:26.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.750+0000 7fa2f0050b80 1 --2- 172.21.15.123:0/1958085659 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571b9179a50 0x5571b918f120 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa2e019cb30 tx=0x7fa2d001b000 comp rx=0 tx=0).stop 2024-05-08T06:11:26.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.750+0000 7fa2f0050b80 1 -- 172.21.15.123:0/1958085659 shutdown_connections 2024-05-08T06:11:26.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.750+0000 7fa2f0050b80 1 --2- 172.21.15.123:0/1958085659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571b917f6c0 0x5571b91817b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:26.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.750+0000 7fa2f0050b80 1 --2- 172.21.15.123:0/1958085659 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571b9179a50 0x5571b918f120 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:26.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.750+0000 7fa2f0050b80 1 --2- 172.21.15.123:0/1958085659 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571b917eda0 0x5571b916e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:26.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.750+0000 7fa2f0050b80 1 -- 172.21.15.123:0/1958085659 >> 172.21.15.123:0/1958085659 conn(0x5571b90f16d0 msgr2=0x5571b91810a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:26.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.750+0000 7fa2f0050b80 1 -- 172.21.15.123:0/1958085659 shutdown_connections 2024-05-08T06:11:26.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.750+0000 7fa2f0050b80 1 -- 172.21.15.123:0/1958085659 wait complete. 2024-05-08T06:11:26.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.750+0000 7fa2f0050b80 1 -- start start 2024-05-08T06:11:26.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.750+0000 7fa2f0050b80 1 Processor -- start 2024-05-08T06:11:26.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.751+0000 7fa2f0050b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571b9179a50 0x5571b918b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:26.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.751+0000 7fa2f0050b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571b917eda0 0x5571b918bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:26.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.751+0000 7fa2f0050b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571b917f6c0 0x5571b9186c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:26.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.751+0000 7fa2f0050b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e01adff0 con 0x5571b917f6c0 2024-05-08T06:11:26.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.751+0000 7fa2f0050b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e01af460 con 0x5571b9179a50 2024-05-08T06:11:26.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.751+0000 7fa2f0050b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e01ae6c0 con 0x5571b917eda0 2024-05-08T06:11:26.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.751+0000 7fa2f0050b80 10 client.0 _ll_register_callbacks cb 0x5571b9072c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:11:26.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.751+0000 7fa2dea00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571b9179a50 0x5571b918b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:26.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.751+0000 7fa2dea00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571b9179a50 0x5571b918b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:39272/0 (socket says 172.21.15.123:39272) 2024-05-08T06:11:26.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.751+0000 7fa2dea00700 1 -- 172.21.15.123:0/1210139487 learned_addr learned my addr 172.21.15.123:0/1210139487 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:26.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.751+0000 7fa2df400700 1 --2- 172.21.15.123:0/1210139487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571b917f6c0 0x5571b9186c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:26.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.751+0000 7fa2de000700 1 --2- 172.21.15.123:0/1210139487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571b917eda0 0x5571b918bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:26.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.752+0000 7fa2de000700 1 -- 172.21.15.123:0/1210139487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571b9179a50 msgr2=0x5571b918b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:26.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.752+0000 7fa2de000700 1 --2- 172.21.15.123:0/1210139487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571b9179a50 0x5571b918b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:26.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.752+0000 7fa2de000700 1 -- 172.21.15.123:0/1210139487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571b917f6c0 msgr2=0x5571b9186c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:26.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.752+0000 7fa2de000700 1 --2- 172.21.15.123:0/1210139487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571b917f6c0 0x5571b9186c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:26.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.752+0000 7fa2de000700 1 -- 172.21.15.123:0/1210139487 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571b9187480 con 0x5571b917eda0 2024-05-08T06:11:26.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.752+0000 7fa2de000700 1 --2- 172.21.15.123:0/1210139487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571b917eda0 0x5571b918bb40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc0a9640 tx=0x7fa2cc0a8dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e56fd5b48cb6fb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:26.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.752+0000 7fa2c8c00700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-05-08T06:11:26.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.752+0000 7fa2c8c00700 1 -- 172.21.15.123:0/1210139487 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc0a8ec0 con 0x5571b917eda0 2024-05-08T06:11:26.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.753+0000 7fa2c8c00700 1 -- 172.21.15.123:0/1210139487 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fa2cc0a5f30 con 0x5571b917eda0 2024-05-08T06:11:26.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.753+0000 7fa2f0050b80 1 -- 172.21.15.123:0/1210139487 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2d0030070 con 0x5571b917eda0 2024-05-08T06:11:26.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.753+0000 7fa2c8c00700 1 -- 172.21.15.123:0/1210139487 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa2cc0e5ab0 con 0x5571b917eda0 2024-05-08T06:11:26.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.753+0000 7fa2f0050b80 10 client.25282 fetch_fsmap learned FSMap version 24 2024-05-08T06:11:26.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.753+0000 7fa2f0050b80 1 -- 172.21.15.123:0/1210139487 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5571b91fd380 con 0x5571b917eda0 2024-05-08T06:11:26.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.753+0000 7fa2c8c00700 1 -- 172.21.15.123:0/1210139487 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2d0030070 con 0x5571b917eda0 2024-05-08T06:11:26.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.753+0000 7fa2c8c00700 1 -- 172.21.15.123:0/1210139487 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fa2cc0c8070 con 0x5571b917eda0 2024-05-08T06:11:26.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.753+0000 7fa2f0050b80 10 client.25282 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:11:26.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.753+0000 7fa2f0050b80 10 client.25282 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T06:11:26.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.753+0000 7fa2f0050b80 20 client.25282 populate_metadata read hostname 'smithi123' 2024-05-08T06:11:26.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.753+0000 7fa2f0050b80 1 --2- 172.21.15.123:0/1210139487 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5571b91fda90 0x5571b91ffeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:26.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.753+0000 7fa2f0050b80 4 client.25282 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T06:11:26.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.753+0000 7fa2f0050b80 1 -- 172.21.15.123:0/1210139487 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5571b90d34a0 con 0x5571b91fda90 2024-05-08T06:11:26.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.754+0000 7fa2dea00700 1 --2- 172.21.15.123:0/1210139487 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5571b91fda90 0x5571b91ffeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:26.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.754+0000 7fa2c8c00700 10 client.25282 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:11:26.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.754+0000 7fa2dea00700 1 --2- 172.21.15.123:0/1210139487 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5571b91fda90 0x5571b91ffeb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a76a6785d7c2521 server_cookie=7a6afdc1a2c965e5 in_seq=0 out_seq=0 2024-05-08T06:11:26.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.755+0000 7fa2c8c00700 1 -- 172.21.15.123:0/1210139487 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5571b90d34a0 con 0x5571b91fda90 2024-05-08T06:11:26.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.755+0000 7fa2c8c00700 10 client.25282 handle_command_reply: tid=0 2024-05-08T06:11:26.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.755+0000 7fa2f0050b80 1 client.25282 shutdown 2024-05-08T06:11:26.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.755+0000 7fa2f0050b80 1 -- 172.21.15.123:0/1210139487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571b917eda0 msgr2=0x5571b918bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:26.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.755+0000 7fa2f0050b80 1 --2- 172.21.15.123:0/1210139487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571b917eda0 0x5571b918bb40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc0a9640 tx=0x7fa2cc0a8dd0 comp rx=0 tx=0).stop 2024-05-08T06:11:26.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.755+0000 7fa2f0050b80 1 -- 172.21.15.123:0/1210139487 shutdown_connections 2024-05-08T06:11:26.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.755+0000 7fa2f0050b80 1 --2- 172.21.15.123:0/1210139487 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5571b91fda90 0x5571b91ffeb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:26.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.755+0000 7fa2f0050b80 1 --2- 172.21.15.123:0/1210139487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5571b917f6c0 0x5571b9186c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:26.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.755+0000 7fa2f0050b80 1 --2- 172.21.15.123:0/1210139487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5571b9179a50 0x5571b918b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:26.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.755+0000 7fa2f0050b80 1 --2- 172.21.15.123:0/1210139487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571b917eda0 0x5571b918bb40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:26.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.755+0000 7fa2f0050b80 1 -- 172.21.15.123:0/1210139487 >> 172.21.15.123:0/1210139487 conn(0x5571b90f16d0 msgr2=0x5571b9184620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:26.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.755+0000 7fa2c8c00700 0 client.25282 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:11:26.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.755+0000 7fa2f0050b80 1 -- 172.21.15.123:0/1210139487 shutdown_connections 2024-05-08T06:11:26.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.755+0000 7fa2f0050b80 1 -- 172.21.15.123:0/1210139487 wait complete. 2024-05-08T06:11:26.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.755+0000 7fa2f0050b80 20 client.25282 trim_cache size 0 max 16384 2024-05-08T06:11:26.760 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-05-08T06:11:26.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.756+0000 7fa2dfe00700 1 -- 172.21.15.123:0/3809524489 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa2c40b2990 msgr2=0x7fa2c40b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:26.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.756+0000 7fa2dfe00700 1 --2- 172.21.15.123:0/3809524489 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa2c40b2990 0x7fa2c40b4e50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x5571b9175970 tx=0x7fa2d0009260 comp rx=0 tx=0).stop 2024-05-08T06:11:26.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.756+0000 7fa2dfe00700 1 -- 172.21.15.123:0/3809524489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2e0105f90 msgr2=0x7fa2e01a1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:26.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.756+0000 7fa2dfe00700 1 --2- 172.21.15.123:0/3809524489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2e0105f90 0x7fa2e01a1d70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00e9f0 tx=0x7fa2cc00ed00 comp rx=0 tx=0).stop 2024-05-08T06:11:26.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.756+0000 7fa2dfe00700 1 -- 172.21.15.123:0/3809524489 shutdown_connections 2024-05-08T06:11:26.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.756+0000 7fa2dfe00700 1 --2- 172.21.15.123:0/3809524489 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa2c40b2990 0x7fa2c40b4e50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:26.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.756+0000 7fa2dfe00700 1 --2- 172.21.15.123:0/3809524489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2e010b680 0x7fa2e019b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:26.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.756+0000 7fa2dfe00700 1 --2- 172.21.15.123:0/3809524489 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2e01055d0 0x7fa2e01a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:26.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.756+0000 7fa2dfe00700 1 --2- 172.21.15.123:0/3809524489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2e0105f90 0x7fa2e01a1d70 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:26.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.756+0000 7fa2dfe00700 1 -- 172.21.15.123:0/3809524489 >> 172.21.15.123:0/3809524489 conn(0x7fa2e0101080 msgr2=0x7fa2e01026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:26.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.757+0000 7fa2dfe00700 1 -- 172.21.15.123:0/3809524489 shutdown_connections 2024-05-08T06:11:26.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:26.757+0000 7fa2dfe00700 1 -- 172.21.15.123:0/3809524489 wait complete. 2024-05-08T06:11:26.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:26 smithi163 ceph-mon[34520]: pgmap v489: 97 pgs: 97 active+clean; 674 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-08T06:11:26.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:26 smithi192 ceph-mon[34664]: pgmap v489: 97 pgs: 97 active+clean; 674 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-08T06:11:28.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:28 smithi123 ceph-mon[29787]: pgmap v490: 97 pgs: 97 active+clean; 674 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:11:28.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:28 smithi163 ceph-mon[34520]: pgmap v490: 97 pgs: 97 active+clean; 674 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:11:28.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:28 smithi192 ceph-mon[34664]: pgmap v490: 97 pgs: 97 active+clean; 674 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:11:28.819 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T06:11:28.820 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T06:11:29.286 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:11:29.657 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:29 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:11:29.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:29 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:11:29.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:29 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:11:30.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:30 smithi123 ceph-mon[29787]: pgmap v491: 97 pgs: 97 active+clean; 674 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:11:30.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:30 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:11:30.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:30 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:11:30.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:30 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:11:30.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:30 smithi163 ceph-mon[34520]: pgmap v491: 97 pgs: 97 active+clean; 674 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:11:30.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:30 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:11:30.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:30 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:11:30.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:30 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:11:30.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:30 smithi192 ceph-mon[34664]: pgmap v491: 97 pgs: 97 active+clean; 674 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:11:30.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:30 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:11:30.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:30 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:11:30.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:30 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:11:31.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.788+0000 7f8003e00700 1 -- 172.21.15.123:0/2090022145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ffc104100 msgr2=0x7f7ffc108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:31.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.788+0000 7f8003e00700 1 --2- 172.21.15.123:0/2090022145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ffc104100 0x7f7ffc108120 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7fe80099c0 tx=0x7f7fe8009cd0 comp rx=0 tx=0).stop 2024-05-08T06:11:31.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.788+0000 7f8003e00700 1 -- 172.21.15.123:0/2090022145 shutdown_connections 2024-05-08T06:11:31.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.788+0000 7f8003e00700 1 --2- 172.21.15.123:0/2090022145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ffc108850 0x7f7ffc1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:31.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.788+0000 7f8003e00700 1 --2- 172.21.15.123:0/2090022145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ffc103750 0x7f7ffc103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:31.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.788+0000 7f8003e00700 1 --2- 172.21.15.123:0/2090022145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ffc104100 0x7f7ffc108120 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:31.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.788+0000 7f8003e00700 1 -- 172.21.15.123:0/2090022145 >> 172.21.15.123:0/2090022145 conn(0x7f7ffc07a600 msgr2=0x7f7ffc07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:31.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.789+0000 7f8003e00700 1 -- 172.21.15.123:0/2090022145 shutdown_connections 2024-05-08T06:11:31.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.789+0000 7f8003e00700 1 -- 172.21.15.123:0/2090022145 wait complete. 2024-05-08T06:11:31.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.790+0000 7f8003e00700 1 Processor -- start 2024-05-08T06:11:31.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.791+0000 7f8003e00700 1 -- start start 2024-05-08T06:11:31.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.791+0000 7f8003e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ffc103750 0x7f7ffc1a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:31.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.791+0000 7f8003e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ffc104100 0x7f7ffc1a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:31.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.791+0000 7f8003e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ffc108850 0x7f7ffc19baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:31.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.791+0000 7f8003e00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc113630 con 0x7f7ffc104100 2024-05-08T06:11:31.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.791+0000 7f8003e00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc1134b0 con 0x7f7ffc103750 2024-05-08T06:11:31.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.791+0000 7f8003e00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc1137b0 con 0x7f7ffc108850 2024-05-08T06:11:31.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.792+0000 7f8001400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ffc103750 0x7f7ffc1a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:31.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.792+0000 7f8001e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ffc108850 0x7f7ffc19baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:31.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.792+0000 7f8001400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ffc103750 0x7f7ffc1a1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:39290/0 (socket says 172.21.15.123:39290) 2024-05-08T06:11:31.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.792+0000 7f8001400700 1 -- 172.21.15.123:0/1527776202 learned_addr learned my addr 172.21.15.123:0/1527776202 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:31.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.792+0000 7f8000a00700 1 --2- 172.21.15.123:0/1527776202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ffc104100 0x7f7ffc1a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:31.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.792+0000 7f8001e00700 1 -- 172.21.15.123:0/1527776202 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ffc103750 msgr2=0x7f7ffc1a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:31.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.792+0000 7f8001e00700 1 --2- 172.21.15.123:0/1527776202 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ffc103750 0x7f7ffc1a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:31.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.792+0000 7f8001e00700 1 -- 172.21.15.123:0/1527776202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ffc104100 msgr2=0x7f7ffc1a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:31.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.792+0000 7f8001e00700 1 --2- 172.21.15.123:0/1527776202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ffc104100 0x7f7ffc1a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:31.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.792+0000 7f8001e00700 1 -- 172.21.15.123:0/1527776202 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ffc19c250 con 0x7f7ffc108850 2024-05-08T06:11:31.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.792+0000 7f8001400700 1 --2- 172.21.15.123:0/1527776202 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ffc103750 0x7f7ffc1a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:11:31.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.792+0000 7f8001e00700 1 --2- 172.21.15.123:0/1527776202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ffc108850 0x7f7ffc19baf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7ff400efc0 tx=0x7f7ff400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:31.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.793+0000 7f7ffa000700 1 -- 172.21.15.123:0/1527776202 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff40092e0 con 0x7f7ffc108850 2024-05-08T06:11:31.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.793+0000 7f7ffa000700 1 -- 172.21.15.123:0/1527776202 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ff4004bb0 con 0x7f7ffc108850 2024-05-08T06:11:31.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.793+0000 7f7ffa000700 1 -- 172.21.15.123:0/1527776202 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4004020 con 0x7f7ffc108850 2024-05-08T06:11:31.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.793+0000 7f8003e00700 1 -- 172.21.15.123:0/1527776202 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ffc19c4e0 con 0x7f7ffc108850 2024-05-08T06:11:31.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.793+0000 7f8003e00700 1 -- 172.21.15.123:0/1527776202 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ffc1a8800 con 0x7f7ffc108850 2024-05-08T06:11:31.798 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.798+0000 7f7ffa000700 1 -- 172.21.15.123:0/1527776202 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) ==== 149704+0+0 (secure 0 0 0) 0x7f7ff401b030 con 0x7f7ffc108850 2024-05-08T06:11:31.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.799+0000 7f800e71cb80 1 Processor -- start 2024-05-08T06:11:31.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.799+0000 7f800e71cb80 1 -- start start 2024-05-08T06:11:31.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.800+0000 7f800e71cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563abe0b50e0 0x563abe0b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:31.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.800+0000 7f800e71cb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563abe0b5a00 0x563abe0b9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:31.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.800+0000 7f800e71cb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563abe0b9a50 0x563abe0b9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:31.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.800+0000 7f800e71cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563abe007aa0 con 0x563abe0b50e0 2024-05-08T06:11:31.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.800+0000 7f800e71cb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x563abdfa57e0 con 0x563abe0b5a00 2024-05-08T06:11:31.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.800+0000 7f800e71cb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x563abe003a60 con 0x563abe0b9a50 2024-05-08T06:11:31.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.800+0000 7f8000a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563abe0b50e0 0x563abe0b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:31.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.800+0000 7f7ffa000700 1 --2- 172.21.15.123:0/1527776202 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f7fe40b2910 0x7f7fe40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:31.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.800+0000 7f8000a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563abe0b50e0 0x563abe0b54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47048/0 (socket says 172.21.15.123:47048) 2024-05-08T06:11:31.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.800+0000 7f8000a00700 1 -- 172.21.15.123:0/718266286 learned_addr learned my addr 172.21.15.123:0/718266286 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:31.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.800+0000 7f8001400700 1 --2- 172.21.15.123:0/718266286 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563abe0b5a00 0x563abe0b9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:31.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.801+0000 7f7ffa000700 1 -- 172.21.15.123:0/1527776202 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f7ff40e1f00 con 0x7f7ffc108850 2024-05-08T06:11:31.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.801+0000 7f8001e00700 1 --2- 172.21.15.123:0/1527776202 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f7fe40b2910 0x7f7fe40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:31.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.801+0000 7f8000a00700 1 -- 172.21.15.123:0/718266286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563abe0b9a50 msgr2=0x563abe0b9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:31.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.801+0000 7f8000a00700 1 --2- 172.21.15.123:0/718266286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563abe0b9a50 0x563abe0b9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:31.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.801+0000 7f8000a00700 1 -- 172.21.15.123:0/718266286 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563abe0b5a00 msgr2=0x563abe0b9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:31.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.801+0000 7f8000a00700 1 --2- 172.21.15.123:0/718266286 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563abe0b5a00 0x563abe0b9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:31.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.801+0000 7f8000a00700 1 -- 172.21.15.123:0/718266286 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563abdf54950 con 0x563abe0b50e0 2024-05-08T06:11:31.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.802+0000 7f8001e00700 1 --2- 172.21.15.123:0/1527776202 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f7fe40b2910 0x7f7fe40b4dd0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7ff40a6670 tx=0x7f7ff4009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:31.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.802+0000 7f8000a00700 1 --2- 172.21.15.123:0/718266286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563abe0b50e0 0x563abe0b54c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7fe80098e0 tx=0x7f7fe8005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6977ca96c9821b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:31.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.802+0000 7f7feea00700 1 -- 172.21.15.123:0/718266286 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe80095d0 con 0x563abe0b50e0 2024-05-08T06:11:31.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.802+0000 7f7feea00700 1 -- 172.21.15.123:0/718266286 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fe8005570 con 0x563abe0b50e0 2024-05-08T06:11:31.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.802+0000 7f7feea00700 1 -- 172.21.15.123:0/718266286 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe800b790 con 0x563abe0b50e0 2024-05-08T06:11:31.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.803+0000 7f800e71cb80 1 -- 172.21.15.123:0/718266286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563abe0b50e0 msgr2=0x563abe0b54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:31.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.803+0000 7f800e71cb80 1 --2- 172.21.15.123:0/718266286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563abe0b50e0 0x563abe0b54c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7fe80098e0 tx=0x7f7fe8005c50 comp rx=0 tx=0).stop 2024-05-08T06:11:31.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.803+0000 7f800e71cb80 1 -- 172.21.15.123:0/718266286 shutdown_connections 2024-05-08T06:11:31.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.803+0000 7f800e71cb80 1 --2- 172.21.15.123:0/718266286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563abe0b50e0 0x563abe0b54c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:31.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.803+0000 7f800e71cb80 1 --2- 172.21.15.123:0/718266286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563abe0b9a50 0x563abe0b9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:31.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.803+0000 7f800e71cb80 1 --2- 172.21.15.123:0/718266286 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563abe0b5a00 0x563abe0b9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:31.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.803+0000 7f800e71cb80 1 -- 172.21.15.123:0/718266286 >> 172.21.15.123:0/718266286 conn(0x563abdfe3be0 msgr2=0x563abe0af820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:31.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.803+0000 7f800e71cb80 1 -- 172.21.15.123:0/718266286 shutdown_connections 2024-05-08T06:11:31.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.804+0000 7f800e71cb80 1 -- 172.21.15.123:0/718266286 wait complete. 2024-05-08T06:11:31.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.804+0000 7f800e71cb80 1 -- start start 2024-05-08T06:11:31.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.805+0000 7f800e71cb80 1 Processor -- start 2024-05-08T06:11:31.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.805+0000 7f800e71cb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563abe0b50e0 0x563abe0c0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:31.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.805+0000 7f800e71cb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563abe0b5a00 0x563abe0be2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:31.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.805+0000 7f800e71cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563abe0b9a50 0x563abe0be820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:31.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.805+0000 7f800e71cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563abe0c1c00 con 0x563abe0b9a50 2024-05-08T06:11:31.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.805+0000 7f800e71cb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x563abdf6dba0 con 0x563abe0b50e0 2024-05-08T06:11:31.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.805+0000 7f800e71cb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x563abe0c1d80 con 0x563abe0b5a00 2024-05-08T06:11:31.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.805+0000 7f800e71cb80 10 client.0 _ll_register_callbacks cb 0x563abdfaecf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:11:31.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.806+0000 7f8001400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563abe0b5a00 0x563abe0be2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:31.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.806+0000 7f8001400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563abe0b5a00 0x563abe0be2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:46684/0 (socket says 172.21.15.123:46684) 2024-05-08T06:11:31.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.806+0000 7f8001400700 1 -- 172.21.15.123:0/2455006194 learned_addr learned my addr 172.21.15.123:0/2455006194 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:31.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.806+0000 7f8000a00700 1 --2- 172.21.15.123:0/2455006194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563abe0b9a50 0x563abe0be820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:31.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.806+0000 7f8000a00700 1 --2- 172.21.15.123:0/2455006194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563abe0b50e0 0x563abe0c0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:31.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.806+0000 7f8001400700 1 -- 172.21.15.123:0/2455006194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563abe0b50e0 msgr2=0x563abe0c0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:31.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.806+0000 7f8001400700 1 --2- 172.21.15.123:0/2455006194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563abe0b50e0 0x563abe0c0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:31.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.806+0000 7f8001400700 1 -- 172.21.15.123:0/2455006194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563abe0b9a50 msgr2=0x563abe0be820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:31.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.806+0000 7f8001400700 1 --2- 172.21.15.123:0/2455006194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563abe0b9a50 0x563abe0be820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:31.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.806+0000 7f8001400700 1 -- 172.21.15.123:0/2455006194 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563abdfcd4a0 con 0x563abe0b5a00 2024-05-08T06:11:31.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.806+0000 7f8000a00700 1 --2- 172.21.15.123:0/2455006194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563abe0b50e0 0x563abe0c0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:11:31.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.807+0000 7f8001400700 1 --2- 172.21.15.123:0/2455006194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563abe0b5a00 0x563abe0be2e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0012d20 tx=0x7f7ff000f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70339d955890c90d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:31.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.807+0000 7f7fe3400700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-05-08T06:11:31.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.807+0000 7f7fe3400700 1 -- 172.21.15.123:0/2455006194 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff000fc30 con 0x563abe0b5a00 2024-05-08T06:11:31.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.807+0000 7f7fe3400700 1 -- 172.21.15.123:0/2455006194 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f7ff0012e00 con 0x563abe0b5a00 2024-05-08T06:11:31.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.807+0000 7f800e71cb80 1 -- 172.21.15.123:0/2455006194 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fe8005570 con 0x563abe0b5a00 2024-05-08T06:11:31.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.807+0000 7f7fe3400700 1 -- 172.21.15.123:0/2455006194 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7ff000fdd0 con 0x563abe0b5a00 2024-05-08T06:11:31.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.807+0000 7f800e71cb80 10 client.25291 fetch_fsmap learned FSMap version 24 2024-05-08T06:11:31.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.807+0000 7f800e71cb80 1 -- 172.21.15.123:0/2455006194 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563abe0cc530 con 0x563abe0b5a00 2024-05-08T06:11:31.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.808+0000 7f7fe3400700 1 -- 172.21.15.123:0/2455006194 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fe8005570 con 0x563abe0b5a00 2024-05-08T06:11:31.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.808+0000 7f7fe3400700 1 -- 172.21.15.123:0/2455006194 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f7ff001c420 con 0x563abe0b5a00 2024-05-08T06:11:31.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.808+0000 7f800e71cb80 10 client.25291 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:11:31.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.808+0000 7f800e71cb80 10 client.25291 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T06:11:31.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.808+0000 7f800e71cb80 20 client.25291 populate_metadata read hostname 'smithi123' 2024-05-08T06:11:31.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.808+0000 7f800e71cb80 1 --2- 172.21.15.123:0/2455006194 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x563abe0ceb10 0x563abe0d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:31.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.808+0000 7f800e71cb80 4 client.25291 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:11:31.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.808+0000 7f800e71cb80 1 -- 172.21.15.123:0/2455006194 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563abe082120 con 0x563abe0ceb10 2024-05-08T06:11:31.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.808+0000 7f8000a00700 1 --2- 172.21.15.123:0/2455006194 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x563abe0ceb10 0x563abe0d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:31.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.809+0000 7f8000a00700 1 --2- 172.21.15.123:0/2455006194 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x563abe0ceb10 0x563abe0d0f50 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fa042ee06947b2 server_cookie=906b50fe2979c31a in_seq=0 out_seq=0 2024-05-08T06:11:31.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.809+0000 7f7fe3400700 10 client.25291 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:11:31.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.810+0000 7f7fe3400700 1 -- 172.21.15.123:0/2455006194 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563abe082120 con 0x563abe0ceb10 2024-05-08T06:11:31.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.810+0000 7f7fe3400700 10 client.25291 handle_command_reply: tid=0 2024-05-08T06:11:31.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.810+0000 7f800e71cb80 1 client.25291 shutdown 2024-05-08T06:11:31.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.810+0000 7f800e71cb80 1 -- 172.21.15.123:0/2455006194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563abe0b5a00 msgr2=0x563abe0be2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:31.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.810+0000 7f800e71cb80 1 --2- 172.21.15.123:0/2455006194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563abe0b5a00 0x563abe0be2e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0012d20 tx=0x7f7ff000f6a0 comp rx=0 tx=0).stop 2024-05-08T06:11:31.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.810+0000 7f800e71cb80 1 -- 172.21.15.123:0/2455006194 shutdown_connections 2024-05-08T06:11:31.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.810+0000 7f800e71cb80 1 --2- 172.21.15.123:0/2455006194 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x563abe0ceb10 0x563abe0d0f50 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:31.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.810+0000 7f800e71cb80 1 --2- 172.21.15.123:0/2455006194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563abe0b9a50 0x563abe0be820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:31.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.810+0000 7f800e71cb80 1 --2- 172.21.15.123:0/2455006194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563abe0b50e0 0x563abe0c0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:31.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.810+0000 7f800e71cb80 1 --2- 172.21.15.123:0/2455006194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563abe0b5a00 0x563abe0be2e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:31.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.810+0000 7f800e71cb80 1 -- 172.21.15.123:0/2455006194 >> 172.21.15.123:0/2455006194 conn(0x563abdfe3be0 msgr2=0x563abe0af7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:31.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.810+0000 7f7fe3400700 0 client.25291 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:11:31.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.810+0000 7f800e71cb80 1 -- 172.21.15.123:0/2455006194 shutdown_connections 2024-05-08T06:11:31.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.811+0000 7f800e71cb80 1 -- 172.21.15.123:0/2455006194 wait complete. 2024-05-08T06:11:31.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.811+0000 7f800e71cb80 20 client.25291 trim_cache size 0 max 16384 2024-05-08T06:11:31.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.835+0000 7f800e71cb80 1 Processor -- start 2024-05-08T06:11:31.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.835+0000 7f800e71cb80 1 -- start start 2024-05-08T06:11:31.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.836+0000 7f800e71cb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563abe0b50e0 0x563abe0b9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:31.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.836+0000 7f800e71cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563abe0b5a00 0x563abe0d6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:31.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.836+0000 7f800e71cb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563abe0b9a50 0x563abe0d6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:31.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.836+0000 7f800e71cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc1b2480 con 0x563abe0b5a00 2024-05-08T06:11:31.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.836+0000 7f800e71cb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc1afd10 con 0x563abe0b9a50 2024-05-08T06:11:31.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.836+0000 7f800e71cb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc1adb60 con 0x563abe0b50e0 2024-05-08T06:11:31.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.836+0000 7f8000a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563abe0b50e0 0x563abe0b9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:31.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.836+0000 7f8001400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563abe0b5a00 0x563abe0d6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:31.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.836+0000 7f8000a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563abe0b50e0 0x563abe0b9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:46686/0 (socket says 172.21.15.123:46686) 2024-05-08T06:11:31.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.836+0000 7f8000a00700 1 -- 172.21.15.123:0/3514101182 learned_addr learned my addr 172.21.15.123:0/3514101182 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:31.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.836+0000 7f8000a00700 1 --2- 172.21.15.123:0/3514101182 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563abe0b9a50 0x563abe0d6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:31.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.837+0000 7f8000a00700 1 -- 172.21.15.123:0/3514101182 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563abe0b9a50 msgr2=0x563abe0d6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:31.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.837+0000 7f8000a00700 1 --2- 172.21.15.123:0/3514101182 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563abe0b9a50 0x563abe0d6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:31.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.837+0000 7f8000a00700 1 -- 172.21.15.123:0/3514101182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563abe0b5a00 msgr2=0x563abe0d6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:31.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.837+0000 7f8000a00700 1 --2- 172.21.15.123:0/3514101182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563abe0b5a00 0x563abe0d6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:31.837 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.837+0000 7f8000a00700 1 -- 172.21.15.123:0/3514101182 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563abe0b0690 con 0x563abe0b50e0 2024-05-08T06:11:31.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.837+0000 7f8000a00700 1 --2- 172.21.15.123:0/3514101182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563abe0b50e0 0x563abe0b9040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8005b40 tx=0x7f7fe8019040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a3228957d44fab5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:31.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.837+0000 7f7fee000700 1 -- 172.21.15.123:0/3514101182 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe8031e60 con 0x563abe0b50e0 2024-05-08T06:11:31.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.838+0000 7f7fee000700 1 -- 172.21.15.123:0/3514101182 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fe8005560 con 0x563abe0b50e0 2024-05-08T06:11:31.838 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.838+0000 7f7fee000700 1 -- 172.21.15.123:0/3514101182 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe800bcb0 con 0x563abe0b50e0 2024-05-08T06:11:31.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.839+0000 7f800e71cb80 1 -- 172.21.15.123:0/3514101182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563abe0b50e0 msgr2=0x563abe0b9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:31.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.839+0000 7f800e71cb80 1 --2- 172.21.15.123:0/3514101182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563abe0b50e0 0x563abe0b9040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8005b40 tx=0x7f7fe8019040 comp rx=0 tx=0).stop 2024-05-08T06:11:31.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.839+0000 7f800e71cb80 1 -- 172.21.15.123:0/3514101182 shutdown_connections 2024-05-08T06:11:31.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.839+0000 7f800e71cb80 1 --2- 172.21.15.123:0/3514101182 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563abe0b9a50 0x563abe0d6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:31.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.839+0000 7f800e71cb80 1 --2- 172.21.15.123:0/3514101182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563abe0b50e0 0x563abe0b9040 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:31.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.839+0000 7f800e71cb80 1 --2- 172.21.15.123:0/3514101182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563abe0b5a00 0x563abe0d6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:31.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.839+0000 7f800e71cb80 1 -- 172.21.15.123:0/3514101182 >> 172.21.15.123:0/3514101182 conn(0x563abdfe3be0 msgr2=0x563abe0b8930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:31.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.839+0000 7f800e71cb80 1 -- 172.21.15.123:0/3514101182 shutdown_connections 2024-05-08T06:11:31.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.839+0000 7f800e71cb80 1 -- 172.21.15.123:0/3514101182 wait complete. 2024-05-08T06:11:31.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.840+0000 7f800e71cb80 1 -- start start 2024-05-08T06:11:31.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.840+0000 7f800e71cb80 1 Processor -- start 2024-05-08T06:11:31.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.840+0000 7f800e71cb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563abe0b50e0 0x563abe0c6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:31.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.840+0000 7f800e71cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563abe0b5a00 0x563abe0c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:31.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.840+0000 7f800e71cb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563abe0b9a50 0x563abe1323b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:31.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.840+0000 7f800e71cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc1ae1d0 con 0x563abe0b5a00 2024-05-08T06:11:31.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.840+0000 7f800e71cb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc1af640 con 0x563abe0b50e0 2024-05-08T06:11:31.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.840+0000 7f800e71cb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc1ae8a0 con 0x563abe0b9a50 2024-05-08T06:11:31.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.840+0000 7f800e71cb80 10 client.0 _ll_register_callbacks cb 0x563abdfaecf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:11:31.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.841+0000 7f8001400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563abe0b5a00 0x563abe0c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:31.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.841+0000 7f8001400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563abe0b5a00 0x563abe0c6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47064/0 (socket says 172.21.15.123:47064) 2024-05-08T06:11:31.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.841+0000 7f8001400700 1 -- 172.21.15.123:0/1731332502 learned_addr learned my addr 172.21.15.123:0/1731332502 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:31.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.841+0000 7f8001400700 1 -- 172.21.15.123:0/1731332502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563abe0b9a50 msgr2=0x563abe1323b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T06:11:31.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.841+0000 7f8001400700 1 --2- 172.21.15.123:0/1731332502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563abe0b9a50 0x563abe1323b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:31.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.841+0000 7f8001400700 1 -- 172.21.15.123:0/1731332502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563abe0b50e0 msgr2=0x563abe0c6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:31.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.841+0000 7f8001400700 1 --2- 172.21.15.123:0/1731332502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563abe0b50e0 0x563abe0c6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:31.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.841+0000 7f8001400700 1 -- 172.21.15.123:0/1731332502 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563abe0ac470 con 0x563abe0b5a00 2024-05-08T06:11:31.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.842+0000 7f8001400700 1 --2- 172.21.15.123:0/1731332502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563abe0b5a00 0x563abe0c6b90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0000c00 tx=0x7f7ff00122e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=524f63336ab7626a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:31.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.842+0000 7f7fe3400700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-08T06:11:31.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.842+0000 7f7fe3400700 1 -- 172.21.15.123:0/1731332502 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff000fc30 con 0x563abe0b5a00 2024-05-08T06:11:31.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.842+0000 7f7fe3400700 1 -- 172.21.15.123:0/1731332502 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f7ff0018040 con 0x563abe0b5a00 2024-05-08T06:11:31.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.842+0000 7f800e71cb80 1 -- 172.21.15.123:0/1731332502 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563abe015480 con 0x563abe0b5a00 2024-05-08T06:11:31.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.843+0000 7f7fe3400700 1 -- 172.21.15.123:0/1731332502 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7ff001ccc0 con 0x563abe0b5a00 2024-05-08T06:11:31.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.843+0000 7f800e71cb80 10 client.15195 fetch_fsmap learned FSMap version 24 2024-05-08T06:11:31.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.843+0000 7f800e71cb80 1 -- 172.21.15.123:0/1731332502 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563abe13adb0 con 0x563abe0b5a00 2024-05-08T06:11:31.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.843+0000 7f7fe3400700 1 -- 172.21.15.123:0/1731332502 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563abe015480 con 0x563abe0b5a00 2024-05-08T06:11:31.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.843+0000 7f7fe3400700 1 -- 172.21.15.123:0/1731332502 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f7ff0004900 con 0x563abe0b5a00 2024-05-08T06:11:31.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.844+0000 7f800e71cb80 10 client.15195 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:11:31.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.844+0000 7f800e71cb80 10 client.15195 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T06:11:31.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.844+0000 7f800e71cb80 20 client.15195 populate_metadata read hostname 'smithi123' 2024-05-08T06:11:31.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.844+0000 7f800e71cb80 1 --2- 172.21.15.123:0/1731332502 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x563abe13b460 0x563abe13d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:31.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.844+0000 7f800e71cb80 4 client.15195 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T06:11:31.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.844+0000 7f800e71cb80 1 -- 172.21.15.123:0/1731332502 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563abe00f520 con 0x563abe13b460 2024-05-08T06:11:31.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.844+0000 7f8000a00700 1 --2- 172.21.15.123:0/1731332502 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x563abe13b460 0x563abe13d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:31.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.845+0000 7f7fe3400700 10 client.15195 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:11:31.845 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.845+0000 7f8000a00700 1 --2- 172.21.15.123:0/1731332502 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x563abe13b460 0x563abe13d8a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2758264e7b15e9d9 server_cookie=ea96e573070bef02 in_seq=0 out_seq=0 2024-05-08T06:11:31.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.846+0000 7f7fe3400700 1 -- 172.21.15.123:0/1731332502 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563abe00f520 con 0x563abe13b460 2024-05-08T06:11:31.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.846+0000 7f7fe3400700 10 client.15195 handle_command_reply: tid=0 2024-05-08T06:11:31.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.846+0000 7f800e71cb80 1 client.15195 shutdown 2024-05-08T06:11:31.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.846+0000 7f800e71cb80 1 -- 172.21.15.123:0/1731332502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563abe0b5a00 msgr2=0x563abe0c6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:31.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.846+0000 7f800e71cb80 1 --2- 172.21.15.123:0/1731332502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563abe0b5a00 0x563abe0c6b90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0000c00 tx=0x7f7ff00122e0 comp rx=0 tx=0).stop 2024-05-08T06:11:31.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.847+0000 7f800e71cb80 1 -- 172.21.15.123:0/1731332502 shutdown_connections 2024-05-08T06:11:31.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.847+0000 7f800e71cb80 1 --2- 172.21.15.123:0/1731332502 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x563abe13b460 0x563abe13d8a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:31.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.847+0000 7f800e71cb80 1 --2- 172.21.15.123:0/1731332502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563abe0b9a50 0x563abe1323b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:31.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.847+0000 7f800e71cb80 1 --2- 172.21.15.123:0/1731332502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563abe0b50e0 0x563abe0c6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:31.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.847+0000 7f800e71cb80 1 --2- 172.21.15.123:0/1731332502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563abe0b5a00 0x563abe0c6b90 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:31.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.847+0000 7f7fe3400700 0 client.15195 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:11:31.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.847+0000 7f800e71cb80 1 -- 172.21.15.123:0/1731332502 >> 172.21.15.123:0/1731332502 conn(0x563abdfe3be0 msgr2=0x563abe0b4780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:31.847 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.847+0000 7f800e71cb80 1 -- 172.21.15.123:0/1731332502 shutdown_connections 2024-05-08T06:11:31.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.847+0000 7f800e71cb80 1 -- 172.21.15.123:0/1731332502 wait complete. 2024-05-08T06:11:31.848 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.848+0000 7f800e71cb80 20 client.15195 trim_cache size 0 max 16384 2024-05-08T06:11:31.848 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-05-08T06:11:31.848 INFO:teuthology.orchestra.run.smithi123.stdout: "return_code": 0, 2024-05-08T06:11:31.849 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_tag": "4e8ccb46-1ab4-475c-9986-b1337d750dec", 2024-05-08T06:11:31.849 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": "asynchronous" 2024-05-08T06:11:31.849 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-05-08T06:11:31.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.848+0000 7f8003e00700 1 -- 172.21.15.123:0/1527776202 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f7fe40b2910 msgr2=0x7f7fe40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:31.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.848+0000 7f8003e00700 1 --2- 172.21.15.123:0/1527776202 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f7fe40b2910 0x7f7fe40b4dd0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7ff40a6670 tx=0x7f7ff4009040 comp rx=0 tx=0).stop 2024-05-08T06:11:31.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.848+0000 7f8003e00700 1 -- 172.21.15.123:0/1527776202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ffc108850 msgr2=0x7f7ffc19baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:31.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.848+0000 7f8003e00700 1 --2- 172.21.15.123:0/1527776202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ffc108850 0x7f7ffc19baf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7ff400efc0 tx=0x7f7ff400c3a0 comp rx=0 tx=0).stop 2024-05-08T06:11:31.849 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.849+0000 7f8003e00700 1 -- 172.21.15.123:0/1527776202 shutdown_connections 2024-05-08T06:11:31.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.849+0000 7f8003e00700 1 --2- 172.21.15.123:0/1527776202 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f7fe40b2910 0x7f7fe40b4dd0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:31.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.849+0000 7f8003e00700 1 --2- 172.21.15.123:0/1527776202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ffc108850 0x7f7ffc19baf0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:31.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.849+0000 7f8003e00700 1 --2- 172.21.15.123:0/1527776202 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ffc103750 0x7f7ffc1a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:31.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.849+0000 7f8003e00700 1 --2- 172.21.15.123:0/1527776202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ffc104100 0x7f7ffc1a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:31.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.849+0000 7f8003e00700 1 -- 172.21.15.123:0/1527776202 >> 172.21.15.123:0/1527776202 conn(0x7f7ffc07a600 msgr2=0x7f7ffc1018e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:31.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.849+0000 7f8003e00700 1 -- 172.21.15.123:0/1527776202 shutdown_connections 2024-05-08T06:11:31.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:31.849+0000 7f8003e00700 1 -- 172.21.15.123:0/1527776202 wait complete. 2024-05-08T06:11:32.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:32 smithi123 ceph-mon[29787]: pgmap v492: 97 pgs: 97 active+clean; 674 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:11:32.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:32 smithi123 ceph-mon[29787]: scrub queued for path: / 2024-05-08T06:11:32.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:32 smithi123 ceph-mon[29787]: scrub summary: idle+waiting paths [/] 2024-05-08T06:11:32.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:32 smithi123 ceph-mon[29787]: scrub summary: active paths [/] 2024-05-08T06:11:32.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:32 smithi123 ceph-mon[29787]: scrub summary: idle+waiting paths [/] 2024-05-08T06:11:32.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:32 smithi163 ceph-mon[34520]: pgmap v492: 97 pgs: 97 active+clean; 674 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:11:32.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:32 smithi163 ceph-mon[34520]: scrub queued for path: / 2024-05-08T06:11:32.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:32 smithi163 ceph-mon[34520]: scrub summary: idle+waiting paths [/] 2024-05-08T06:11:32.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:32 smithi163 ceph-mon[34520]: scrub summary: active paths [/] 2024-05-08T06:11:32.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:32 smithi163 ceph-mon[34520]: scrub summary: idle+waiting paths [/] 2024-05-08T06:11:32.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:32 smithi192 ceph-mon[34664]: pgmap v492: 97 pgs: 97 active+clean; 674 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:11:32.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:32 smithi192 ceph-mon[34664]: scrub queued for path: / 2024-05-08T06:11:32.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:32 smithi192 ceph-mon[34664]: scrub summary: idle+waiting paths [/] 2024-05-08T06:11:32.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:32 smithi192 ceph-mon[34664]: scrub summary: active paths [/] 2024-05-08T06:11:32.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:32 smithi192 ceph-mon[34664]: scrub summary: idle+waiting paths [/] 2024-05-08T06:11:33.085 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-08T06:11:33.529 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:11:33.694 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:33 smithi123 ceph-mon[29787]: mgrmap e20: x(active, since 14m), standbys: z, y 2024-05-08T06:11:33.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:33 smithi163 ceph-mon[34520]: mgrmap e20: x(active, since 14m), standbys: z, y 2024-05-08T06:11:33.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:33 smithi192 ceph-mon[34664]: mgrmap e20: x(active, since 14m), standbys: z, y 2024-05-08T06:11:34.627 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:34 smithi123 ceph-mon[29787]: pgmap v493: 97 pgs: 97 active+clean; 674 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-08T06:11:34.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:34 smithi163 ceph-mon[34520]: pgmap v493: 97 pgs: 97 active+clean; 674 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-08T06:11:34.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:34 smithi192 ceph-mon[34664]: pgmap v493: 97 pgs: 97 active+clean; 674 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-08T06:11:35.574 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.574+0000 7f0e0d000640 1 -- 172.21.15.123:0/670254236 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e0810a790 msgr2=0x7f0e0810cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:35.574 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.574+0000 7f0e0d000640 1 --2- 172.21.15.123:0/670254236 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e0810a790 0x7f0e0810cc20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009be0 tx=0x7f0e0002a6c0 comp rx=0 tx=0).stop 2024-05-08T06:11:35.574 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.574+0000 7f0e0d000640 1 -- 172.21.15.123:0/670254236 shutdown_connections 2024-05-08T06:11:35.574 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.574+0000 7f0e0d000640 1 --2- 172.21.15.123:0/670254236 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e0810a790 0x7f0e0810cc20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:35.575 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.574+0000 7f0e0d000640 1 --2- 172.21.15.123:0/670254236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e08075410 0x7f0e08075890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:35.575 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.574+0000 7f0e0d000640 1 --2- 172.21.15.123:0/670254236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e080769f0 0x7f0e08074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:35.575 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.574+0000 7f0e0d000640 1 -- 172.21.15.123:0/670254236 >> 172.21.15.123:0/670254236 conn(0x7f0e080fde40 msgr2=0x7f0e08100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:35.575 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.574+0000 7f0e0d000640 1 -- 172.21.15.123:0/670254236 shutdown_connections 2024-05-08T06:11:35.575 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.574+0000 7f0e0d000640 1 -- 172.21.15.123:0/670254236 wait complete. 2024-05-08T06:11:35.576 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.575+0000 7f0e0d000640 1 Processor -- start 2024-05-08T06:11:35.576 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.575+0000 7f0e0d000640 1 -- start start 2024-05-08T06:11:35.576 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.576+0000 7f0e0d000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e08075410 0x7f0e0819ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:35.576 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.576+0000 7f0e0d000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e080769f0 0x7f0e0819b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.576+0000 7f0e0d000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e0810a790 0x7f0e081a21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.576+0000 7f0e0d000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0e08110570 con 0x7f0e08075410 2024-05-08T06:11:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.576+0000 7f0e0d000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0e081103f0 con 0x7f0e0810a790 2024-05-08T06:11:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.576+0000 7f0e0d000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0e081106f0 con 0x7f0e080769f0 2024-05-08T06:11:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.576+0000 7f0e07400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e08075410 0x7f0e0819ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.576+0000 7f0e07400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e08075410 0x7f0e0819ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53344/0 (socket says 172.21.15.123:53344) 2024-05-08T06:11:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.576+0000 7f0e07400640 1 -- 172.21.15.123:0/787893140 learned_addr learned my addr 172.21.15.123:0/787893140 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.576+0000 7f0e06a00640 1 --2- 172.21.15.123:0/787893140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e080769f0 0x7f0e0819b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.576+0000 7f0e07e00640 1 --2- 172.21.15.123:0/787893140 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e0810a790 0x7f0e081a21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.576+0000 7f0e07400640 1 -- 172.21.15.123:0/787893140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e080769f0 msgr2=0x7f0e0819b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.576+0000 7f0e07400640 1 --2- 172.21.15.123:0/787893140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e080769f0 0x7f0e0819b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.576+0000 7f0e07400640 1 -- 172.21.15.123:0/787893140 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e0810a790 msgr2=0x7f0e081a21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.577+0000 7f0e07400640 1 --2- 172.21.15.123:0/787893140 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e0810a790 0x7f0e081a21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.577+0000 7f0e07400640 1 -- 172.21.15.123:0/787893140 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e081a2840 con 0x7f0e08075410 2024-05-08T06:11:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.577+0000 7f0e07e00640 1 --2- 172.21.15.123:0/787893140 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e0810a790 0x7f0e081a21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:11:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.577+0000 7f0e07400640 1 --2- 172.21.15.123:0/787893140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e08075410 0x7f0e0819ac30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0df800e9c0 tx=0x7f0df800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.577+0000 7f0dffe00640 1 -- 172.21.15.123:0/787893140 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df800cde0 con 0x7f0e08075410 2024-05-08T06:11:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.577+0000 7f0dffe00640 1 -- 172.21.15.123:0/787893140 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0df80035f0 con 0x7f0e08075410 2024-05-08T06:11:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.577+0000 7f0e0d000640 1 -- 172.21.15.123:0/787893140 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e081a2b30 con 0x7f0e08075410 2024-05-08T06:11:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.577+0000 7f0dffe00640 1 -- 172.21.15.123:0/787893140 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df8010690 con 0x7f0e08075410 2024-05-08T06:11:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.577+0000 7f0e0d000640 1 -- 172.21.15.123:0/787893140 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e081a3080 con 0x7f0e08075410 2024-05-08T06:11:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.581+0000 7f0e11423740 1 Processor -- start 2024-05-08T06:11:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.581+0000 7f0e11423740 1 -- start start 2024-05-08T06:11:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.582+0000 7f0e11423740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c8abef320 0x556c8abef740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.582+0000 7f0e11423740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c8abefd10 0x556c8abf4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.582+0000 7f0e11423740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c8abf4e80 0x556c8abf7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.582+0000 7f0e11423740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556c8ab53550 con 0x556c8abf4e80 2024-05-08T06:11:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.582+0000 7f0e11423740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x556c8aaf8bc0 con 0x556c8abef320 2024-05-08T06:11:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.582+0000 7f0e11423740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x556c8ab4f510 con 0x556c8abefd10 2024-05-08T06:11:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.582+0000 7f0e07e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c8abef320 0x556c8abef740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.582+0000 7f0e07e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c8abef320 0x556c8abef740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:44580/0 (socket says 172.21.15.123:44580) 2024-05-08T06:11:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.582+0000 7f0e07e00640 1 -- 172.21.15.123:0/908551335 learned_addr learned my addr 172.21.15.123:0/908551335 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.583+0000 7f0e06a00640 1 --2- 172.21.15.123:0/908551335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c8abf4e80 0x556c8abf7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.583+0000 7f0e07e00640 1 -- 172.21.15.123:0/908551335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c8abefd10 msgr2=0x556c8abf4810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T06:11:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.583+0000 7f0e07e00640 1 --2- 172.21.15.123:0/908551335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c8abefd10 0x556c8abf4810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.583+0000 7f0e07e00640 1 -- 172.21.15.123:0/908551335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c8abf4e80 msgr2=0x556c8abf7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.583+0000 7f0e07e00640 1 --2- 172.21.15.123:0/908551335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c8abf4e80 0x556c8abf7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.583+0000 7f0e07e00640 1 -- 172.21.15.123:0/908551335 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c8a865210 con 0x556c8abef320 2024-05-08T06:11:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.583+0000 7f0e07e00640 1 --2- 172.21.15.123:0/908551335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c8abef320 0x556c8abef740 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009be0 tx=0x7f0e00030410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2564cf6c6548e2df server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.584+0000 7f0dffe00640 1 -- 172.21.15.123:0/787893140 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7f0df8004000 con 0x7f0e08075410 2024-05-08T06:11:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.584+0000 7f0dfcc00640 1 -- 172.21.15.123:0/908551335 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e00040030 con 0x556c8abef320 2024-05-08T06:11:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.584+0000 7f0dfcc00640 1 -- 172.21.15.123:0/908551335 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e00003970 con 0x556c8abef320 2024-05-08T06:11:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.584+0000 7f0dfcc00640 1 -- 172.21.15.123:0/908551335 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e00036040 con 0x556c8abef320 2024-05-08T06:11:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.585+0000 7f0e11423740 1 -- 172.21.15.123:0/908551335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c8abef320 msgr2=0x556c8abef740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.585+0000 7f0e11423740 1 --2- 172.21.15.123:0/908551335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c8abef320 0x556c8abef740 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009be0 tx=0x7f0e00030410 comp rx=0 tx=0).stop 2024-05-08T06:11:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.585+0000 7f0e11423740 1 -- 172.21.15.123:0/908551335 shutdown_connections 2024-05-08T06:11:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.585+0000 7f0e11423740 1 --2- 172.21.15.123:0/908551335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c8abf4e80 0x556c8abf7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.585+0000 7f0e11423740 1 --2- 172.21.15.123:0/908551335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c8abefd10 0x556c8abf4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.585+0000 7f0e11423740 1 --2- 172.21.15.123:0/908551335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c8abef320 0x556c8abef740 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.585+0000 7f0e11423740 1 -- 172.21.15.123:0/908551335 >> 172.21.15.123:0/908551335 conn(0x556c8abb7230 msgr2=0x556c8abeada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.585+0000 7f0e11423740 1 -- 172.21.15.123:0/908551335 shutdown_connections 2024-05-08T06:11:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.585+0000 7f0e11423740 1 -- 172.21.15.123:0/908551335 wait complete. 2024-05-08T06:11:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.586+0000 7f0dffe00640 1 --2- 172.21.15.123:0/787893140 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f0dec0b26f0 0x7f0dec0b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.586+0000 7f0e11423740 1 -- start start 2024-05-08T06:11:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.586+0000 7f0dffe00640 1 -- 172.21.15.123:0/787893140 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f0df80e2620 con 0x7f0e08075410 2024-05-08T06:11:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.586+0000 7f0e11423740 1 Processor -- start 2024-05-08T06:11:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.586+0000 7f0e07e00640 1 --2- 172.21.15.123:0/787893140 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f0dec0b26f0 0x7f0dec0b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.587+0000 7f0e11423740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c8abef320 0x556c8abf9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.587+0000 7f0e11423740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c8abefd10 0x556c8abf98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.587+0000 7f0e11423740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c8abf4e80 0x556c8ac008e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.587+0000 7f0e11423740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556c8abf7dd0 con 0x556c8abefd10 2024-05-08T06:11:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.587+0000 7f0e11423740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x556c8ab0c660 con 0x556c8abef320 2024-05-08T06:11:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.587+0000 7f0e11423740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x556c8abf7f50 con 0x556c8abf4e80 2024-05-08T06:11:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.587+0000 7f0e11423740 10 client.0 _ll_register_callbacks cb 0x556c8aad6400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:11:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.588+0000 7f0e07e00640 1 --2- 172.21.15.123:0/787893140 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f0dec0b26f0 0x7f0dec0b4bb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009b30 tx=0x7f0e00005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.588+0000 7f0e07e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c8abf4e80 0x556c8ac008e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.588+0000 7f0e06a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c8abefd10 0x556c8abf98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.588+0000 7f0e07e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c8abf4e80 0x556c8ac008e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:38722/0 (socket says 172.21.15.123:38722) 2024-05-08T06:11:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.588+0000 7f0e07e00640 1 -- 172.21.15.123:0/3559130624 learned_addr learned my addr 172.21.15.123:0/3559130624 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.588+0000 7f0e07400640 1 --2- 172.21.15.123:0/3559130624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c8abef320 0x556c8abf9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.589+0000 7f0e07400640 1 -- 172.21.15.123:0/3559130624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c8abf4e80 msgr2=0x556c8ac008e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.589+0000 7f0e07400640 1 --2- 172.21.15.123:0/3559130624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c8abf4e80 0x556c8ac008e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.589+0000 7f0e07400640 1 -- 172.21.15.123:0/3559130624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c8abefd10 msgr2=0x556c8abf98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.589+0000 7f0e07400640 1 --2- 172.21.15.123:0/3559130624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c8abefd10 0x556c8abf98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.589+0000 7f0e07400640 1 -- 172.21.15.123:0/3559130624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c8a931620 con 0x556c8abef320 2024-05-08T06:11:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.589+0000 7f0e07e00640 1 --2- 172.21.15.123:0/3559130624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c8abf4e80 0x556c8ac008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:11:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.589+0000 7f0e06a00640 1 --2- 172.21.15.123:0/3559130624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c8abefd10 0x556c8abf98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:11:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.589+0000 7f0e07400640 1 --2- 172.21.15.123:0/3559130624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c8abef320 0x556c8abf9360 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x556c8abf0780 tx=0x7f0df80a5350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59d9886949b0cdc2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.589+0000 7f0df1600640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-05-08T06:11:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.589+0000 7f0df1600640 1 -- 172.21.15.123:0/3559130624 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df80aa350 con 0x556c8abef320 2024-05-08T06:11:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.589+0000 7f0df1600640 1 -- 172.21.15.123:0/3559130624 <== mon.2 v2:172.21.15.163:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f0df80037a0 con 0x556c8abef320 2024-05-08T06:11:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.589+0000 7f0e11423740 1 -- 172.21.15.123:0/3559130624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e00003970 con 0x556c8abef320 2024-05-08T06:11:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.590+0000 7f0df1600640 1 -- 172.21.15.123:0/3559130624 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0df80a9c50 con 0x556c8abef320 2024-05-08T06:11:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.590+0000 7f0e11423740 10 client.25316 fetch_fsmap learned FSMap version 24 2024-05-08T06:11:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.590+0000 7f0e11423740 1 -- 172.21.15.123:0/3559130624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556c8ac03960 con 0x556c8abef320 2024-05-08T06:11:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.590+0000 7f0df1600640 1 -- 172.21.15.123:0/3559130624 <== mon.2 v2:172.21.15.163:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e00003970 con 0x556c8abef320 2024-05-08T06:11:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.590+0000 7f0df1600640 1 -- 172.21.15.123:0/3559130624 <== mon.2 v2:172.21.15.163:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f0df80a7ba0 con 0x556c8abef320 2024-05-08T06:11:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.591+0000 7f0e11423740 10 client.25316 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:11:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.591+0000 7f0e11423740 10 client.25316 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T06:11:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.591+0000 7f0e11423740 20 client.25316 populate_metadata read hostname 'smithi123' 2024-05-08T06:11:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.591+0000 7f0e11423740 1 --2- 172.21.15.123:0/3559130624 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x556c8ac04fe0 0x556c8ac073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.591+0000 7f0e11423740 4 client.25316 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:11:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.591+0000 7f0e11423740 1 -- 172.21.15.123:0/3559130624 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c8ab61160 con 0x556c8ac04fe0 2024-05-08T06:11:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.591+0000 7f0e06a00640 1 --2- 172.21.15.123:0/3559130624 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x556c8ac04fe0 0x556c8ac073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.592+0000 7f0e06a00640 1 --2- 172.21.15.123:0/3559130624 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x556c8ac04fe0 0x556c8ac073d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89d14c7370a18ccc server_cookie=267ac5c372b0dfc8 in_seq=0 out_seq=0 2024-05-08T06:11:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.592+0000 7f0df1600640 10 client.25316 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:11:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.592+0000 7f0df1600640 1 -- 172.21.15.123:0/3559130624 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556c8ab61160 con 0x556c8ac04fe0 2024-05-08T06:11:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.592+0000 7f0df1600640 10 client.25316 handle_command_reply: tid=0 2024-05-08T06:11:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.592+0000 7f0e11423740 1 client.25316 shutdown 2024-05-08T06:11:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.593+0000 7f0e11423740 1 -- 172.21.15.123:0/3559130624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c8abef320 msgr2=0x556c8abf9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.593+0000 7f0e11423740 1 --2- 172.21.15.123:0/3559130624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c8abef320 0x556c8abf9360 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x556c8abf0780 tx=0x7f0df80a5350 comp rx=0 tx=0).stop 2024-05-08T06:11:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.593+0000 7f0e11423740 1 -- 172.21.15.123:0/3559130624 shutdown_connections 2024-05-08T06:11:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.593+0000 7f0e11423740 1 --2- 172.21.15.123:0/3559130624 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x556c8ac04fe0 0x556c8ac073d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.593+0000 7f0e11423740 1 --2- 172.21.15.123:0/3559130624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c8abf4e80 0x556c8ac008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.593+0000 7f0e11423740 1 --2- 172.21.15.123:0/3559130624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c8abefd10 0x556c8abf98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.593+0000 7f0e11423740 1 --2- 172.21.15.123:0/3559130624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c8abef320 0x556c8abf9360 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.593+0000 7f0df1600640 0 client.25316 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:11:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.593+0000 7f0e11423740 1 -- 172.21.15.123:0/3559130624 >> 172.21.15.123:0/3559130624 conn(0x556c8abb7230 msgr2=0x556c8abe9e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.593+0000 7f0e11423740 1 -- 172.21.15.123:0/3559130624 shutdown_connections 2024-05-08T06:11:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.593+0000 7f0e11423740 1 -- 172.21.15.123:0/3559130624 wait complete. 2024-05-08T06:11:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.593+0000 7f0e11423740 20 client.25316 trim_cache size 0 max 16384 2024-05-08T06:11:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.609+0000 7f0e11423740 1 Processor -- start 2024-05-08T06:11:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.610+0000 7f0e11423740 1 -- start start 2024-05-08T06:11:35.610 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.610+0000 7f0e11423740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c8abefd10 0x556c8abe5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.610+0000 7f0e11423740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c8abf4e80 0x556c8abea880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.610+0000 7f0e11423740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c8abee3a0 0x556c8abeadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.610+0000 7f0e11423740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0e081ae720 con 0x556c8abee3a0 2024-05-08T06:11:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.610+0000 7f0e11423740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0e081abff0 con 0x556c8abefd10 2024-05-08T06:11:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.610+0000 7f0e11423740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0e081a9e40 con 0x556c8abf4e80 2024-05-08T06:11:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.610+0000 7f0e07400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c8abefd10 0x556c8abe5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.610+0000 7f0e06a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c8abf4e80 0x556c8abea880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.610+0000 7f0e07400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c8abefd10 0x556c8abe5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:44592/0 (socket says 172.21.15.123:44592) 2024-05-08T06:11:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.610+0000 7f0e07400640 1 -- 172.21.15.123:0/13874973 learned_addr learned my addr 172.21.15.123:0/13874973 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.610+0000 7f0e07e00640 1 --2- 172.21.15.123:0/13874973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c8abee3a0 0x556c8abeadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.611+0000 7f0e07400640 1 -- 172.21.15.123:0/13874973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c8abf4e80 msgr2=0x556c8abea880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.611+0000 7f0e07400640 1 --2- 172.21.15.123:0/13874973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c8abf4e80 0x556c8abea880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.611+0000 7f0e07400640 1 -- 172.21.15.123:0/13874973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c8abee3a0 msgr2=0x556c8abeadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.611+0000 7f0e07400640 1 --2- 172.21.15.123:0/13874973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c8abee3a0 0x556c8abeadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.611+0000 7f0e07400640 1 -- 172.21.15.123:0/13874973 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c8abe5280 con 0x556c8abefd10 2024-05-08T06:11:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.611+0000 7f0e07e00640 1 --2- 172.21.15.123:0/13874973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c8abee3a0 0x556c8abeadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:11:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.611+0000 7f0e07400640 1 --2- 172.21.15.123:0/13874973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c8abefd10 0x556c8abe5d00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0df800c6a0 tx=0x7f0df8005c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edc98ffbe0f558e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.611+0000 7f0df3e00640 1 -- 172.21.15.123:0/13874973 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df80a5e50 con 0x556c8abefd10 2024-05-08T06:11:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.611+0000 7f0df3e00640 1 -- 172.21.15.123:0/13874973 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0df80a9760 con 0x556c8abefd10 2024-05-08T06:11:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.611+0000 7f0df3e00640 1 -- 172.21.15.123:0/13874973 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df80037a0 con 0x556c8abefd10 2024-05-08T06:11:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.612+0000 7f0e11423740 1 -- 172.21.15.123:0/13874973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c8abefd10 msgr2=0x556c8abe5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.612+0000 7f0e11423740 1 --2- 172.21.15.123:0/13874973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c8abefd10 0x556c8abe5d00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0df800c6a0 tx=0x7f0df8005c80 comp rx=0 tx=0).stop 2024-05-08T06:11:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.612+0000 7f0e11423740 1 -- 172.21.15.123:0/13874973 shutdown_connections 2024-05-08T06:11:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.612+0000 7f0e11423740 1 --2- 172.21.15.123:0/13874973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c8abee3a0 0x556c8abeadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.612+0000 7f0e11423740 1 --2- 172.21.15.123:0/13874973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c8abf4e80 0x556c8abea880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.612+0000 7f0e11423740 1 --2- 172.21.15.123:0/13874973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c8abefd10 0x556c8abe5d00 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.612+0000 7f0e11423740 1 -- 172.21.15.123:0/13874973 >> 172.21.15.123:0/13874973 conn(0x556c8abb7230 msgr2=0x556c8abedc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.612+0000 7f0e11423740 1 -- 172.21.15.123:0/13874973 shutdown_connections 2024-05-08T06:11:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.612+0000 7f0e11423740 1 -- 172.21.15.123:0/13874973 wait complete. 2024-05-08T06:11:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.612+0000 7f0e11423740 1 -- start start 2024-05-08T06:11:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.613+0000 7f0e11423740 1 Processor -- start 2024-05-08T06:11:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.613+0000 7f0e11423740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c8abee3a0 0x556c8abfd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.613+0000 7f0e11423740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c8abefd10 0x556c8abfda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.613+0000 7f0e11423740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c8abf4e80 0x556c8abfea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.613+0000 7f0e11423740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0e081aa4b0 con 0x556c8abf4e80 2024-05-08T06:11:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.613+0000 7f0e11423740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0e081ab920 con 0x556c8abee3a0 2024-05-08T06:11:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.613+0000 7f0e11423740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0e081aab80 con 0x556c8abefd10 2024-05-08T06:11:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.613+0000 7f0e11423740 10 client.0 _ll_register_callbacks cb 0x7f0df80046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:11:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.613+0000 7f0e07400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c8abee3a0 0x556c8abfd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.613+0000 7f0e07400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c8abee3a0 0x556c8abfd530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:44602/0 (socket says 172.21.15.123:44602) 2024-05-08T06:11:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.613+0000 7f0e07400640 1 -- 172.21.15.123:0/3529742622 learned_addr learned my addr 172.21.15.123:0/3529742622 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.613+0000 7f0e07e00640 1 --2- 172.21.15.123:0/3529742622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c8abf4e80 0x556c8abfea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.614+0000 7f0e06a00640 1 --2- 172.21.15.123:0/3529742622 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c8abefd10 0x556c8abfda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.614+0000 7f0e07400640 1 -- 172.21.15.123:0/3529742622 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c8abefd10 msgr2=0x556c8abfda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.614+0000 7f0e07400640 1 --2- 172.21.15.123:0/3529742622 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c8abefd10 0x556c8abfda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:35.616 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.614+0000 7f0e07400640 1 -- 172.21.15.123:0/3529742622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c8abf4e80 msgr2=0x556c8abfea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:35.616 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.614+0000 7f0e07400640 1 --2- 172.21.15.123:0/3529742622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c8abf4e80 0x556c8abfea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:35.616 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.614+0000 7f0e07400640 1 -- 172.21.15.123:0/3529742622 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c8abff070 con 0x556c8abee3a0 2024-05-08T06:11:35.617 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.614+0000 7f0e07400640 1 --2- 172.21.15.123:0/3529742622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c8abee3a0 0x556c8abfd530 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0df80a7fb0 tx=0x7f0df80a5bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28339b8d0784ca4b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:35.617 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.614+0000 7f0df1600640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-05-08T06:11:35.617 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.614+0000 7f0df1600640 1 -- 172.21.15.123:0/3529742622 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df80ae000 con 0x556c8abee3a0 2024-05-08T06:11:35.617 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.614+0000 7f0df1600640 1 -- 172.21.15.123:0/3529742622 <== mon.2 v2:172.21.15.163:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f0df80a7ba0 con 0x556c8abee3a0 2024-05-08T06:11:35.617 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.614+0000 7f0e11423740 1 -- 172.21.15.123:0/3529742622 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0df80a9760 con 0x556c8abee3a0 2024-05-08T06:11:35.618 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.615+0000 7f0df1600640 1 -- 172.21.15.123:0/3529742622 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0df80a5530 con 0x556c8abee3a0 2024-05-08T06:11:35.618 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.615+0000 7f0e11423740 10 client.25322 fetch_fsmap learned FSMap version 24 2024-05-08T06:11:35.618 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.615+0000 7f0e11423740 1 -- 172.21.15.123:0/3529742622 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556c8abff070 con 0x556c8abee3a0 2024-05-08T06:11:35.618 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.615+0000 7f0df1600640 1 -- 172.21.15.123:0/3529742622 <== mon.2 v2:172.21.15.163:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0df80a9760 con 0x556c8abee3a0 2024-05-08T06:11:35.618 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.616+0000 7f0df1600640 1 -- 172.21.15.123:0/3529742622 <== mon.2 v2:172.21.15.163:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f0df80a8920 con 0x556c8abee3a0 2024-05-08T06:11:35.618 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.616+0000 7f0e11423740 10 client.25322 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:11:35.618 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.616+0000 7f0e11423740 10 client.25322 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T06:11:35.618 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.616+0000 7f0e11423740 20 client.25322 populate_metadata read hostname 'smithi123' 2024-05-08T06:11:35.618 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.616+0000 7f0e11423740 1 --2- 172.21.15.123:0/3529742622 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x556c8ac6fb50 0x556c8ac71f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:35.618 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.616+0000 7f0e11423740 4 client.25322 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T06:11:35.618 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.616+0000 7f0e11423740 1 -- 172.21.15.123:0/3529742622 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556c8ab5afd0 con 0x556c8ac6fb50 2024-05-08T06:11:35.618 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.616+0000 7f0e06a00640 1 --2- 172.21.15.123:0/3529742622 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x556c8ac6fb50 0x556c8ac71f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:35.618 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.617+0000 7f0e06a00640 1 --2- 172.21.15.123:0/3529742622 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x556c8ac6fb50 0x556c8ac71f70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93983d0d1141afb2 server_cookie=c77c755ada0a98c5 in_seq=0 out_seq=0 2024-05-08T06:11:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.617+0000 7f0df1600640 10 client.25322 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:11:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.623+0000 7f0df1600640 1 -- 172.21.15.123:0/3529742622 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556c8ab5afd0 con 0x556c8ac6fb50 2024-05-08T06:11:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.623+0000 7f0df1600640 10 client.25322 handle_command_reply: tid=0 2024-05-08T06:11:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.623+0000 7f0e11423740 1 client.25322 shutdown 2024-05-08T06:11:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.623+0000 7f0e11423740 1 -- 172.21.15.123:0/3529742622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c8abee3a0 msgr2=0x556c8abfd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.623+0000 7f0e11423740 1 --2- 172.21.15.123:0/3529742622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c8abee3a0 0x556c8abfd530 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0df80a7fb0 tx=0x7f0df80a5bb0 comp rx=0 tx=0).stop 2024-05-08T06:11:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.623+0000 7f0e11423740 1 -- 172.21.15.123:0/3529742622 shutdown_connections 2024-05-08T06:11:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.623+0000 7f0e11423740 1 --2- 172.21.15.123:0/3529742622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c8abf4e80 0x556c8abfea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.623+0000 7f0e11423740 1 --2- 172.21.15.123:0/3529742622 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x556c8ac6fb50 0x556c8ac71f70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.623+0000 7f0e11423740 1 --2- 172.21.15.123:0/3529742622 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c8abefd10 0x556c8abfda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.623+0000 7f0e11423740 1 --2- 172.21.15.123:0/3529742622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c8abee3a0 0x556c8abfd530 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.623+0000 7f0df1600640 0 client.25322 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:11:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.623+0000 7f0e11423740 1 -- 172.21.15.123:0/3529742622 >> 172.21.15.123:0/3529742622 conn(0x556c8abb7230 msgr2=0x556c8abf8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.623+0000 7f0e11423740 1 -- 172.21.15.123:0/3529742622 shutdown_connections 2024-05-08T06:11:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.623+0000 7f0e11423740 1 -- 172.21.15.123:0/3529742622 wait complete. 2024-05-08T06:11:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.624+0000 7f0e11423740 20 client.25322 trim_cache size 0 max 16384 2024-05-08T06:11:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout:{ 2024-05-08T06:11:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout: "message": "", 2024-05-08T06:11:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout: "return_code": 0 2024-05-08T06:11:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout:} 2024-05-08T06:11:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.624+0000 7f0e0d000640 1 -- 172.21.15.123:0/787893140 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f0dec0b26f0 msgr2=0x7f0dec0b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.624+0000 7f0e0d000640 1 --2- 172.21.15.123:0/787893140 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f0dec0b26f0 0x7f0dec0b4bb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009b30 tx=0x7f0e00005fd0 comp rx=0 tx=0).stop 2024-05-08T06:11:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.624+0000 7f0e0d000640 1 -- 172.21.15.123:0/787893140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e08075410 msgr2=0x7f0e0819ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.624+0000 7f0e0d000640 1 --2- 172.21.15.123:0/787893140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e08075410 0x7f0e0819ac30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0df800e9c0 tx=0x7f0df800ee80 comp rx=0 tx=0).stop 2024-05-08T06:11:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.625+0000 7f0e0d000640 1 -- 172.21.15.123:0/787893140 shutdown_connections 2024-05-08T06:11:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.625+0000 7f0e0d000640 1 --2- 172.21.15.123:0/787893140 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f0dec0b26f0 0x7f0dec0b4bb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.625+0000 7f0e0d000640 1 --2- 172.21.15.123:0/787893140 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e0810a790 0x7f0e081a21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.625+0000 7f0e0d000640 1 --2- 172.21.15.123:0/787893140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e080769f0 0x7f0e0819b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.625+0000 7f0e0d000640 1 --2- 172.21.15.123:0/787893140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e08075410 0x7f0e0819ac30 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.625+0000 7f0e0d000640 1 -- 172.21.15.123:0/787893140 >> 172.21.15.123:0/787893140 conn(0x7f0e080fde40 msgr2=0x7f0e08106690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.625+0000 7f0e0d000640 1 -- 172.21.15.123:0/787893140 shutdown_connections 2024-05-08T06:11:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:35.625+0000 7f0e0d000640 1 -- 172.21.15.123:0/787893140 wait complete. 2024-05-08T06:11:35.678 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:35 smithi123 ceph-mon[29787]: scrub summary: idle 2024-05-08T06:11:35.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:35 smithi163 ceph-mon[34520]: scrub summary: idle 2024-05-08T06:11:35.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:35 smithi192 ceph-mon[34664]: scrub summary: idle 2024-05-08T06:11:36.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.027+0000 7fa38d800700 1 -- 172.21.15.123:0/4095070568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa388104ea0 msgr2=0x7fa38810d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:36.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.027+0000 7fa38d800700 1 --2- 172.21.15.123:0/4095070568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa388104ea0 0x7fa38810d390 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa374009990 tx=0x7fa374009ca0 comp rx=0 tx=0).stop 2024-05-08T06:11:36.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.027+0000 7fa38d800700 1 -- 172.21.15.123:0/4095070568 shutdown_connections 2024-05-08T06:11:36.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.027+0000 7fa38d800700 1 --2- 172.21.15.123:0/4095070568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa38810d9b0 0x7fa38810fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:36.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.027+0000 7fa38d800700 1 --2- 172.21.15.123:0/4095070568 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa388104580 0x7fa388104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:36.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.027+0000 7fa38d800700 1 --2- 172.21.15.123:0/4095070568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa388104ea0 0x7fa38810d390 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:36.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.027+0000 7fa38d800700 1 -- 172.21.15.123:0/4095070568 >> 172.21.15.123:0/4095070568 conn(0x7fa3880fddb0 msgr2=0x7fa3881001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:36.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.028+0000 7fa38d800700 1 -- 172.21.15.123:0/4095070568 shutdown_connections 2024-05-08T06:11:36.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.028+0000 7fa38d800700 1 -- 172.21.15.123:0/4095070568 wait complete. 2024-05-08T06:11:36.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.029+0000 7fa38d800700 1 Processor -- start 2024-05-08T06:11:36.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.029+0000 7fa38d800700 1 -- start start 2024-05-08T06:11:36.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.030+0000 7fa38d800700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa388104580 0x7fa3881021e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:36.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.030+0000 7fa38d800700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa388104ea0 0x7fa388102720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:36.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.030+0000 7fa38d800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa38810d9b0 0x7fa388102c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:36.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.030+0000 7fa38d800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa3881135d0 con 0x7fa38810d9b0 2024-05-08T06:11:36.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.030+0000 7fa38d800700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa388113450 con 0x7fa388104ea0 2024-05-08T06:11:36.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.030+0000 7fa38d800700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa388113750 con 0x7fa388104580 2024-05-08T06:11:36.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.030+0000 7fa387e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa388104580 0x7fa3881021e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:36.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.030+0000 7fa387e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa388104580 0x7fa3881021e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:38754/0 (socket says 172.21.15.123:38754) 2024-05-08T06:11:36.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.030+0000 7fa387e00700 1 -- 172.21.15.123:0/3555804440 learned_addr learned my addr 172.21.15.123:0/3555804440 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:36.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.031+0000 7fa387e00700 1 -- 172.21.15.123:0/3555804440 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa388104ea0 msgr2=0x7fa388102720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:36.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.031+0000 7fa387e00700 1 --2- 172.21.15.123:0/3555804440 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa388104ea0 0x7fa388102720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:36.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.031+0000 7fa387e00700 1 -- 172.21.15.123:0/3555804440 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa38810d9b0 msgr2=0x7fa388102c60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T06:11:36.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.031+0000 7fa387e00700 1 --2- 172.21.15.123:0/3555804440 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa38810d9b0 0x7fa388102c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:36.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.031+0000 7fa387e00700 1 -- 172.21.15.123:0/3555804440 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3881a8060 con 0x7fa388104580 2024-05-08T06:11:36.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.032+0000 7fa387e00700 1 --2- 172.21.15.123:0/3555804440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa388104580 0x7fa3881021e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa37800e9a0 tx=0x7fa37800ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:36.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.032+0000 7fa384c00700 1 -- 172.21.15.123:0/3555804440 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37800cbd0 con 0x7fa388104580 2024-05-08T06:11:36.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.032+0000 7fa384c00700 1 -- 172.21.15.123:0/3555804440 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa37800cd70 con 0x7fa388104580 2024-05-08T06:11:36.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.032+0000 7fa38d800700 1 -- 172.21.15.123:0/3555804440 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3881a8350 con 0x7fa388104580 2024-05-08T06:11:36.032 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.032+0000 7fa384c00700 1 -- 172.21.15.123:0/3555804440 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa378018680 con 0x7fa388104580 2024-05-08T06:11:36.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.032+0000 7fa38d800700 1 -- 172.21.15.123:0/3555804440 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3881a88e0 con 0x7fa388104580 2024-05-08T06:11:36.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.037+0000 7fa384c00700 1 -- 172.21.15.123:0/3555804440 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7fa378010430 con 0x7fa388104580 2024-05-08T06:11:36.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.038+0000 7fa399834b80 1 Processor -- start 2024-05-08T06:11:36.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.038+0000 7fa399834b80 1 -- start start 2024-05-08T06:11:36.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.038+0000 7fa399834b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b71d059e0 0x563b71d05dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:36.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.038+0000 7fa399834b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b71d06300 0x563b71cfffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:36.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.038+0000 7fa399834b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b71d00600 0x563b71d00a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:36.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.038+0000 7fa399834b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563b71c52a20 con 0x563b71d059e0 2024-05-08T06:11:36.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.038+0000 7fa399834b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x563b71bf0760 con 0x563b71d00600 2024-05-08T06:11:36.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.038+0000 7fa399834b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x563b71c4e9e0 con 0x563b71d06300 2024-05-08T06:11:36.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.039+0000 7fa387e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b71d06300 0x563b71cfffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:36.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.039+0000 7fa387e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b71d06300 0x563b71cfffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:38756/0 (socket says 172.21.15.123:38756) 2024-05-08T06:11:36.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.039+0000 7fa387e00700 1 -- 172.21.15.123:0/404878040 learned_addr learned my addr 172.21.15.123:0/404878040 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:36.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.039+0000 7fa387400700 1 --2- 172.21.15.123:0/404878040 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b71d00600 0x563b71d00a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:36.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.039+0000 7fa384c00700 1 --2- 172.21.15.123:0/3555804440 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa36c0b2900 0x7fa36c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:36.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.040+0000 7fa387e00700 1 -- 172.21.15.123:0/404878040 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b71d00600 msgr2=0x563b71d00a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:36.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.040+0000 7fa387e00700 1 --2- 172.21.15.123:0/404878040 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b71d00600 0x563b71d00a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:36.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.040+0000 7fa387e00700 1 -- 172.21.15.123:0/404878040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b71d059e0 msgr2=0x563b71d05dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T06:11:36.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.040+0000 7fa384c00700 1 -- 172.21.15.123:0/3555804440 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fa3780e2500 con 0x7fa388104580 2024-05-08T06:11:36.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.040+0000 7fa38ce00700 1 --2- 172.21.15.123:0/404878040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b71d059e0 0x563b71d05dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:36.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.040+0000 7fa387e00700 1 --2- 172.21.15.123:0/404878040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b71d059e0 0x563b71d05dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:36.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.041+0000 7fa387e00700 1 -- 172.21.15.123:0/404878040 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b71b9f8d0 con 0x563b71d06300 2024-05-08T06:11:36.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.041+0000 7fa38ce00700 1 --2- 172.21.15.123:0/404878040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b71d059e0 0x563b71d05dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:11:36.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.041+0000 7fa387e00700 1 --2- 172.21.15.123:0/404878040 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b71d06300 0x563b71cfffe0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa3780dffb0 tx=0x7fa378010cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=520c24b0ee4a88ec server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:36.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.041+0000 7fa38ce00700 1 --2- 172.21.15.123:0/3555804440 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa36c0b2900 0x7fa36c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:36.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.041+0000 7fa37d600700 1 -- 172.21.15.123:0/404878040 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3780a98a0 con 0x563b71d06300 2024-05-08T06:11:36.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.041+0000 7fa37d600700 1 -- 172.21.15.123:0/404878040 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3780a95e0 con 0x563b71d06300 2024-05-08T06:11:36.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.041+0000 7fa37d600700 1 -- 172.21.15.123:0/404878040 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3780a9240 con 0x563b71d06300 2024-05-08T06:11:36.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.042+0000 7fa399834b80 1 -- 172.21.15.123:0/404878040 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b71d06300 msgr2=0x563b71cfffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:36.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.042+0000 7fa38ce00700 1 --2- 172.21.15.123:0/3555804440 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa36c0b2900 0x7fa36c0b4dc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa380009fd0 tx=0x7fa38000d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:36.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.042+0000 7fa399834b80 1 --2- 172.21.15.123:0/404878040 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b71d06300 0x563b71cfffe0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa3780dffb0 tx=0x7fa378010cb0 comp rx=0 tx=0).stop 2024-05-08T06:11:36.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.043+0000 7fa399834b80 1 -- 172.21.15.123:0/404878040 shutdown_connections 2024-05-08T06:11:36.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.043+0000 7fa399834b80 1 --2- 172.21.15.123:0/404878040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b71d059e0 0x563b71d05dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:36.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.043+0000 7fa399834b80 1 --2- 172.21.15.123:0/404878040 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b71d00600 0x563b71d00a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:36.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.043+0000 7fa399834b80 1 --2- 172.21.15.123:0/404878040 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b71d06300 0x563b71cfffe0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:36.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.043+0000 7fa399834b80 1 -- 172.21.15.123:0/404878040 >> 172.21.15.123:0/404878040 conn(0x563b71c2eb60 msgr2=0x563b71cfa9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:36.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.043+0000 7fa399834b80 1 -- 172.21.15.123:0/404878040 shutdown_connections 2024-05-08T06:11:36.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.043+0000 7fa399834b80 1 -- 172.21.15.123:0/404878040 wait complete. 2024-05-08T06:11:36.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.044+0000 7fa399834b80 1 -- start start 2024-05-08T06:11:36.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.044+0000 7fa399834b80 1 Processor -- start 2024-05-08T06:11:36.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.044+0000 7fa399834b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b71d00600 0x563b71d0bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:36.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.044+0000 7fa399834b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b71d059e0 0x563b71d09290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:36.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.044+0000 7fa399834b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b71d06300 0x563b71d097d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:36.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.044+0000 7fa399834b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563b71d0cd50 con 0x563b71d00600 2024-05-08T06:11:36.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.044+0000 7fa399834b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x563b71bb8b20 con 0x563b71d06300 2024-05-08T06:11:36.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.044+0000 7fa399834b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x563b71d0ced0 con 0x563b71d059e0 2024-05-08T06:11:36.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.045+0000 7fa399834b80 10 client.0 _ll_register_callbacks cb 0x563b71bf9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:11:36.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.045+0000 7fa387e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b71d00600 0x563b71d0bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:36.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.045+0000 7fa387400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b71d059e0 0x563b71d09290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:36.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.045+0000 7fa387400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b71d059e0 0x563b71d09290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:38762/0 (socket says 172.21.15.123:38762) 2024-05-08T06:11:36.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.045+0000 7fa387400700 1 -- 172.21.15.123:0/2760416370 learned_addr learned my addr 172.21.15.123:0/2760416370 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:36.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.045+0000 7fa38ce00700 1 --2- 172.21.15.123:0/2760416370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b71d06300 0x563b71d097d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:36.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.046+0000 7fa387400700 1 -- 172.21.15.123:0/2760416370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b71d06300 msgr2=0x563b71d097d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:36.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.046+0000 7fa387400700 1 --2- 172.21.15.123:0/2760416370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b71d06300 0x563b71d097d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:36.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.046+0000 7fa387400700 1 -- 172.21.15.123:0/2760416370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b71d00600 msgr2=0x563b71d0bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:36.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.046+0000 7fa387400700 1 --2- 172.21.15.123:0/2760416370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b71d00600 0x563b71d0bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:36.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.046+0000 7fa387400700 1 -- 172.21.15.123:0/2760416370 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b71c18420 con 0x563b71d059e0 2024-05-08T06:11:36.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.046+0000 7fa387e00700 1 --2- 172.21.15.123:0/2760416370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b71d00600 0x563b71d0bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:11:36.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.046+0000 7fa387400700 1 --2- 172.21.15.123:0/2760416370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b71d059e0 0x563b71d09290 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa374009990 tx=0x7fa374005640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1411a092d243f9bb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:36.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.046+0000 7fa372000700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-05-08T06:11:36.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.047+0000 7fa372000700 1 -- 172.21.15.123:0/2760416370 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa374029060 con 0x563b71d059e0 2024-05-08T06:11:36.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.047+0000 7fa372000700 1 -- 172.21.15.123:0/2760416370 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fa37400bd00 con 0x563b71d059e0 2024-05-08T06:11:36.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.047+0000 7fa399834b80 1 -- 172.21.15.123:0/2760416370 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3780a95e0 con 0x563b71d059e0 2024-05-08T06:11:36.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.047+0000 7fa372000700 1 -- 172.21.15.123:0/2760416370 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa374003680 con 0x563b71d059e0 2024-05-08T06:11:36.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.047+0000 7fa399834b80 10 client.25321 fetch_fsmap learned FSMap version 24 2024-05-08T06:11:36.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.047+0000 7fa399834b80 1 -- 172.21.15.123:0/2760416370 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563b71d17410 con 0x563b71d059e0 2024-05-08T06:11:36.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.047+0000 7fa372000700 1 -- 172.21.15.123:0/2760416370 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3780a95e0 con 0x563b71d059e0 2024-05-08T06:11:36.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.048+0000 7fa372000700 1 -- 172.21.15.123:0/2760416370 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fa37400f600 con 0x563b71d059e0 2024-05-08T06:11:36.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.048+0000 7fa399834b80 10 client.25321 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:11:36.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.048+0000 7fa399834b80 10 client.25321 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T06:11:36.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.048+0000 7fa399834b80 20 client.25321 populate_metadata read hostname 'smithi123' 2024-05-08T06:11:36.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.048+0000 7fa399834b80 1 --2- 172.21.15.123:0/2760416370 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x563b71d19a50 0x563b71d1be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:36.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.048+0000 7fa399834b80 4 client.25321 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:11:36.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.048+0000 7fa399834b80 1 -- 172.21.15.123:0/2760416370 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b71ccd0a0 con 0x563b71d19a50 2024-05-08T06:11:36.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.048+0000 7fa387e00700 1 --2- 172.21.15.123:0/2760416370 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x563b71d19a50 0x563b71d1be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:36.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.049+0000 7fa372000700 10 client.25321 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:11:36.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.049+0000 7fa387e00700 1 --2- 172.21.15.123:0/2760416370 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x563b71d19a50 0x563b71d1be40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72e626aa7c522ae3 server_cookie=c55bc14700708e74 in_seq=0 out_seq=0 2024-05-08T06:11:36.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.050+0000 7fa372000700 1 -- 172.21.15.123:0/2760416370 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563b71ccd0a0 con 0x563b71d19a50 2024-05-08T06:11:36.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.050+0000 7fa372000700 10 client.25321 handle_command_reply: tid=0 2024-05-08T06:11:36.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.050+0000 7fa399834b80 1 client.25321 shutdown 2024-05-08T06:11:36.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.050+0000 7fa399834b80 1 -- 172.21.15.123:0/2760416370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b71d059e0 msgr2=0x563b71d09290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:36.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.050+0000 7fa399834b80 1 --2- 172.21.15.123:0/2760416370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b71d059e0 0x563b71d09290 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa374009990 tx=0x7fa374005640 comp rx=0 tx=0).stop 2024-05-08T06:11:36.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.050+0000 7fa399834b80 1 -- 172.21.15.123:0/2760416370 shutdown_connections 2024-05-08T06:11:36.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.050+0000 7fa399834b80 1 --2- 172.21.15.123:0/2760416370 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x563b71d19a50 0x563b71d1be40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:36.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.050+0000 7fa399834b80 1 --2- 172.21.15.123:0/2760416370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b71d00600 0x563b71d0bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:36.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.050+0000 7fa399834b80 1 --2- 172.21.15.123:0/2760416370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b71d06300 0x563b71d097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:36.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.050+0000 7fa399834b80 1 --2- 172.21.15.123:0/2760416370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b71d059e0 0x563b71d09290 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:36.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.050+0000 7fa372000700 0 client.25321 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:11:36.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.050+0000 7fa399834b80 1 -- 172.21.15.123:0/2760416370 >> 172.21.15.123:0/2760416370 conn(0x563b71c2eb60 msgr2=0x563b71cf5460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:36.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.051+0000 7fa399834b80 1 -- 172.21.15.123:0/2760416370 shutdown_connections 2024-05-08T06:11:36.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.051+0000 7fa399834b80 1 -- 172.21.15.123:0/2760416370 wait complete. 2024-05-08T06:11:36.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.051+0000 7fa399834b80 20 client.25321 trim_cache size 0 max 16384 2024-05-08T06:11:36.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.079+0000 7fa399834b80 1 Processor -- start 2024-05-08T06:11:36.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.079+0000 7fa399834b80 1 -- start start 2024-05-08T06:11:36.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.079+0000 7fa399834b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b71d00600 0x563b71cfffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:36.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.079+0000 7fa399834b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b71d059e0 0x563b71d06fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:36.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.079+0000 7fa399834b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b71d06300 0x563b71d081f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:36.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.080+0000 7fa399834b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa3881b2630 con 0x563b71d00600 2024-05-08T06:11:36.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.080+0000 7fa399834b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa3881afec0 con 0x563b71d06300 2024-05-08T06:11:36.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.080+0000 7fa399834b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa3881add30 con 0x563b71d059e0 2024-05-08T06:11:36.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.080+0000 7fa387e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b71d00600 0x563b71cfffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:36.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.080+0000 7fa387e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b71d00600 0x563b71cfffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53396/0 (socket says 172.21.15.123:53396) 2024-05-08T06:11:36.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.080+0000 7fa387400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b71d059e0 0x563b71d06fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:36.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.080+0000 7fa387e00700 1 -- 172.21.15.123:0/2437573064 learned_addr learned my addr 172.21.15.123:0/2437573064 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:36.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.080+0000 7fa38ce00700 1 --2- 172.21.15.123:0/2437573064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b71d06300 0x563b71d081f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:36.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.080+0000 7fa387e00700 1 -- 172.21.15.123:0/2437573064 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b71d059e0 msgr2=0x563b71d06fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:36.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.080+0000 7fa387e00700 1 --2- 172.21.15.123:0/2437573064 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b71d059e0 0x563b71d06fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:36.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.080+0000 7fa387e00700 1 -- 172.21.15.123:0/2437573064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b71d06300 msgr2=0x563b71d081f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:36.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.080+0000 7fa387e00700 1 --2- 172.21.15.123:0/2437573064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b71d06300 0x563b71d081f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:36.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.080+0000 7fa387e00700 1 -- 172.21.15.123:0/2437573064 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b71d1c4a0 con 0x563b71d00600 2024-05-08T06:11:36.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.081+0000 7fa387e00700 1 --2- 172.21.15.123:0/2437573064 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b71d00600 0x563b71cfffe0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa3780a97d0 tx=0x7fa3780e5f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc77af557954e1d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:36.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.081+0000 7fa37cc00700 1 -- 172.21.15.123:0/2437573064 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3780a9420 con 0x563b71d00600 2024-05-08T06:11:36.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.081+0000 7fa37cc00700 1 -- 172.21.15.123:0/2437573064 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3780a9ad0 con 0x563b71d00600 2024-05-08T06:11:36.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.081+0000 7fa37cc00700 1 -- 172.21.15.123:0/2437573064 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3780a9d30 con 0x563b71d00600 2024-05-08T06:11:36.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.082+0000 7fa399834b80 1 -- 172.21.15.123:0/2437573064 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b71d00600 msgr2=0x563b71cfffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:36.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.082+0000 7fa399834b80 1 --2- 172.21.15.123:0/2437573064 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b71d00600 0x563b71cfffe0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa3780a97d0 tx=0x7fa3780e5f40 comp rx=0 tx=0).stop 2024-05-08T06:11:36.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.082+0000 7fa399834b80 1 -- 172.21.15.123:0/2437573064 shutdown_connections 2024-05-08T06:11:36.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.082+0000 7fa399834b80 1 --2- 172.21.15.123:0/2437573064 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b71d00600 0x563b71cfffe0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:36.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.082+0000 7fa399834b80 1 --2- 172.21.15.123:0/2437573064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b71d06300 0x563b71d081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:36.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.082+0000 7fa399834b80 1 --2- 172.21.15.123:0/2437573064 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b71d059e0 0x563b71d06fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:36.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.082+0000 7fa399834b80 1 -- 172.21.15.123:0/2437573064 >> 172.21.15.123:0/2437573064 conn(0x563b71c2eb60 msgr2=0x563b71cf4bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:36.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.083+0000 7fa399834b80 1 -- 172.21.15.123:0/2437573064 shutdown_connections 2024-05-08T06:11:36.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.083+0000 7fa399834b80 1 -- 172.21.15.123:0/2437573064 wait complete. 2024-05-08T06:11:36.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.083+0000 7fa399834b80 1 -- start start 2024-05-08T06:11:36.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.084+0000 7fa399834b80 1 Processor -- start 2024-05-08T06:11:36.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.084+0000 7fa399834b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b71d00600 0x563b71d20da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:36.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.084+0000 7fa399834b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b71d059e0 0x563b71d212e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:36.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.084+0000 7fa399834b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b71d06300 0x563b71d21820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:36.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.084+0000 7fa399834b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa3881ae380 con 0x563b71d00600 2024-05-08T06:11:36.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.084+0000 7fa399834b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa3881af7f0 con 0x563b71d06300 2024-05-08T06:11:36.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.084+0000 7fa399834b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa3881aea50 con 0x563b71d059e0 2024-05-08T06:11:36.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.084+0000 7fa399834b80 10 client.0 _ll_register_callbacks cb 0x563b71bf9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:11:36.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.085+0000 7fa387e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b71d00600 0x563b71d20da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:36.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.085+0000 7fa387400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b71d059e0 0x563b71d212e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:36.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.085+0000 7fa387400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b71d059e0 0x563b71d212e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:38776/0 (socket says 172.21.15.123:38776) 2024-05-08T06:11:36.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.085+0000 7fa387400700 1 -- 172.21.15.123:0/1855474024 learned_addr learned my addr 172.21.15.123:0/1855474024 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:36.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.085+0000 7fa387e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b71d00600 0x563b71d20da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53406/0 (socket says 172.21.15.123:53406) 2024-05-08T06:11:36.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.085+0000 7fa38ce00700 1 --2- 172.21.15.123:0/1855474024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b71d06300 0x563b71d21820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:36.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.085+0000 7fa387400700 1 -- 172.21.15.123:0/1855474024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b71d06300 msgr2=0x563b71d21820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:36.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.085+0000 7fa387400700 1 --2- 172.21.15.123:0/1855474024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b71d06300 0x563b71d21820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:36.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.085+0000 7fa387400700 1 -- 172.21.15.123:0/1855474024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b71d00600 msgr2=0x563b71d20da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:36.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.085+0000 7fa387400700 1 --2- 172.21.15.123:0/1855474024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b71d00600 0x563b71d20da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:36.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.085+0000 7fa387400700 1 -- 172.21.15.123:0/1855474024 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b71d8ba90 con 0x563b71d059e0 2024-05-08T06:11:36.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.085+0000 7fa38ce00700 1 --2- 172.21.15.123:0/1855474024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b71d06300 0x563b71d21820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:11:36.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.085+0000 7fa387e00700 1 --2- 172.21.15.123:0/1855474024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b71d00600 0x563b71d20da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:11:36.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.086+0000 7fa387400700 1 --2- 172.21.15.123:0/1855474024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b71d059e0 0x563b71d212e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa374005950 tx=0x7fa37400f710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46b3c2d3ecaa1c67 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:36.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.086+0000 7fa372000700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-05-08T06:11:36.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.086+0000 7fa372000700 1 -- 172.21.15.123:0/1855474024 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa374034070 con 0x563b71d059e0 2024-05-08T06:11:36.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.086+0000 7fa372000700 1 -- 172.21.15.123:0/1855474024 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fa37400be10 con 0x563b71d059e0 2024-05-08T06:11:36.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.086+0000 7fa399834b80 1 -- 172.21.15.123:0/1855474024 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563b71c60400 con 0x563b71d059e0 2024-05-08T06:11:36.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.087+0000 7fa372000700 1 -- 172.21.15.123:0/1855474024 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa374029040 con 0x563b71d059e0 2024-05-08T06:11:36.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.087+0000 7fa399834b80 10 client.25327 fetch_fsmap learned FSMap version 24 2024-05-08T06:11:36.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.087+0000 7fa399834b80 1 -- 172.21.15.123:0/1855474024 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563b71d8cc60 con 0x563b71d059e0 2024-05-08T06:11:36.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.087+0000 7fa372000700 1 -- 172.21.15.123:0/1855474024 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563b71c60400 con 0x563b71d059e0 2024-05-08T06:11:36.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.088+0000 7fa372000700 1 -- 172.21.15.123:0/1855474024 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fa3740247b0 con 0x563b71d059e0 2024-05-08T06:11:36.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.088+0000 7fa399834b80 10 client.25327 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:11:36.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.088+0000 7fa399834b80 10 client.25327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T06:11:36.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.088+0000 7fa399834b80 20 client.25327 populate_metadata read hostname 'smithi123' 2024-05-08T06:11:36.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.088+0000 7fa399834b80 1 --2- 172.21.15.123:0/1855474024 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x563b71d8d3a0 0x563b71d8f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:36.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.088+0000 7fa399834b80 4 client.25327 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T06:11:36.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.088+0000 7fa399834b80 1 -- 172.21.15.123:0/1855474024 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563b71c5a4a0 con 0x563b71d8d3a0 2024-05-08T06:11:36.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.088+0000 7fa387e00700 1 --2- 172.21.15.123:0/1855474024 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x563b71d8d3a0 0x563b71d8f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:36.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.089+0000 7fa387e00700 1 --2- 172.21.15.123:0/1855474024 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x563b71d8d3a0 0x563b71d8f7c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b390c7e0fe37a97 server_cookie=3319ee150a5448c9 in_seq=0 out_seq=0 2024-05-08T06:11:36.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.089+0000 7fa372000700 10 client.25327 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:11:36.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.089+0000 7fa372000700 1 -- 172.21.15.123:0/1855474024 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563b71c5a4a0 con 0x563b71d8d3a0 2024-05-08T06:11:36.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.089+0000 7fa372000700 10 client.25327 handle_command_reply: tid=0 2024-05-08T06:11:36.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.089+0000 7fa399834b80 1 client.25327 shutdown 2024-05-08T06:11:36.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.089+0000 7fa399834b80 1 -- 172.21.15.123:0/1855474024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b71d059e0 msgr2=0x563b71d212e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:36.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.089+0000 7fa399834b80 1 --2- 172.21.15.123:0/1855474024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b71d059e0 0x563b71d212e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa374005950 tx=0x7fa37400f710 comp rx=0 tx=0).stop 2024-05-08T06:11:36.090 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-05-08T06:11:36.090 INFO:teuthology.orchestra.run.smithi123.stdout: "status": "no active scrubs running", 2024-05-08T06:11:36.090 INFO:teuthology.orchestra.run.smithi123.stdout: "scrubs": {} 2024-05-08T06:11:36.090 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-05-08T06:11:36.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.089+0000 7fa399834b80 1 -- 172.21.15.123:0/1855474024 shutdown_connections 2024-05-08T06:11:36.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.089+0000 7fa399834b80 1 --2- 172.21.15.123:0/1855474024 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x563b71d8d3a0 0x563b71d8f7c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:36.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.089+0000 7fa399834b80 1 --2- 172.21.15.123:0/1855474024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b71d00600 0x563b71d20da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:36.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.089+0000 7fa399834b80 1 --2- 172.21.15.123:0/1855474024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563b71d06300 0x563b71d21820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:36.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.089+0000 7fa399834b80 1 --2- 172.21.15.123:0/1855474024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b71d059e0 0x563b71d212e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:36.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.089+0000 7fa399834b80 1 -- 172.21.15.123:0/1855474024 >> 172.21.15.123:0/1855474024 conn(0x563b71c2eb60 msgr2=0x563b71cf5460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:36.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.089+0000 7fa372000700 0 client.25327 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:11:36.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.090+0000 7fa399834b80 1 -- 172.21.15.123:0/1855474024 shutdown_connections 2024-05-08T06:11:36.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.090+0000 7fa399834b80 1 -- 172.21.15.123:0/1855474024 wait complete. 2024-05-08T06:11:36.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.090+0000 7fa399834b80 20 client.25327 trim_cache size 0 max 16384 2024-05-08T06:11:36.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.091+0000 7fa38d800700 1 -- 172.21.15.123:0/3555804440 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa36c0b2900 msgr2=0x7fa36c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:36.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.091+0000 7fa38d800700 1 --2- 172.21.15.123:0/3555804440 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa36c0b2900 0x7fa36c0b4dc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa380009fd0 tx=0x7fa38000d040 comp rx=0 tx=0).stop 2024-05-08T06:11:36.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.091+0000 7fa38d800700 1 -- 172.21.15.123:0/3555804440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa388104580 msgr2=0x7fa3881021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:36.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.091+0000 7fa38d800700 1 --2- 172.21.15.123:0/3555804440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa388104580 0x7fa3881021e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa37800e9a0 tx=0x7fa37800ed60 comp rx=0 tx=0).stop 2024-05-08T06:11:36.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.091+0000 7fa38d800700 1 -- 172.21.15.123:0/3555804440 shutdown_connections 2024-05-08T06:11:36.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.091+0000 7fa38d800700 1 --2- 172.21.15.123:0/3555804440 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa36c0b2900 0x7fa36c0b4dc0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:36.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.091+0000 7fa38d800700 1 --2- 172.21.15.123:0/3555804440 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa38810d9b0 0x7fa388102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:36.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.091+0000 7fa38d800700 1 --2- 172.21.15.123:0/3555804440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa388104580 0x7fa3881021e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:36.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.091+0000 7fa38d800700 1 --2- 172.21.15.123:0/3555804440 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa388104ea0 0x7fa388102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:36.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.091+0000 7fa38d800700 1 -- 172.21.15.123:0/3555804440 >> 172.21.15.123:0/3555804440 conn(0x7fa3880fddb0 msgr2=0x7fa3880feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:36.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.091+0000 7fa38d800700 1 -- 172.21.15.123:0/3555804440 shutdown_connections 2024-05-08T06:11:36.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:36.091+0000 7fa38d800700 1 -- 172.21.15.123:0/3555804440 wait complete. 2024-05-08T06:11:36.425 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:36 smithi123 ceph-mon[29787]: pgmap v494: 97 pgs: 97 active+clean; 674 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-08T06:11:36.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:36 smithi163 ceph-mon[34520]: pgmap v494: 97 pgs: 97 active+clean; 674 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-08T06:11:36.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:36 smithi192 ceph-mon[34664]: pgmap v494: 97 pgs: 97 active+clean; 674 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-08T06:11:37.127 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T06:11:37.127 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T06:11:37.667 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:11:38.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:38 smithi123 ceph-mon[29787]: pgmap v495: 97 pgs: 97 active+clean; 674 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-08T06:11:38.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:38 smithi163 ceph-mon[34520]: pgmap v495: 97 pgs: 97 active+clean; 674 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-08T06:11:38.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:38 smithi192 ceph-mon[34664]: pgmap v495: 97 pgs: 97 active+clean; 674 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-08T06:11:40.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:40 smithi123 ceph-mon[29787]: pgmap v496: 97 pgs: 97 active+clean; 675 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-08T06:11:40.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:40 smithi163 ceph-mon[34520]: pgmap v496: 97 pgs: 97 active+clean; 675 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-08T06:11:40.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:40 smithi192 ceph-mon[34664]: pgmap v496: 97 pgs: 97 active+clean; 675 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-08T06:11:42.603 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:42 smithi123 ceph-mon[29787]: pgmap v497: 97 pgs: 97 active+clean; 675 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-08T06:11:42.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:42 smithi163 ceph-mon[34520]: pgmap v497: 97 pgs: 97 active+clean; 675 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-08T06:11:42.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:42 smithi192 ceph-mon[34664]: pgmap v497: 97 pgs: 97 active+clean; 675 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-08T06:11:43.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.190+0000 7f9eaf200700 1 -- 172.21.15.123:0/3393501522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ea8109460 msgr2=0x7f9ea8109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:43.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.190+0000 7f9eaf200700 1 --2- 172.21.15.123:0/3393501522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ea8109460 0x7f9ea8109840 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c009990 tx=0x7f9e9c009ca0 comp rx=0 tx=0).stop 2024-05-08T06:11:43.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.190+0000 7f9eaf200700 1 -- 172.21.15.123:0/3393501522 shutdown_connections 2024-05-08T06:11:43.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.190+0000 7f9eaf200700 1 --2- 172.21.15.123:0/3393501522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ea8109460 0x7f9ea8109840 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:43.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.190+0000 7f9eaf200700 1 --2- 172.21.15.123:0/3393501522 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ea8103d90 0x7f9ea81102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:43.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.190+0000 7f9eaf200700 1 --2- 172.21.15.123:0/3393501522 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ea81033d0 0x7f9ea8103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:43.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.190+0000 7f9eaf200700 1 -- 172.21.15.123:0/3393501522 >> 172.21.15.123:0/3393501522 conn(0x7f9ea80fef20 msgr2=0x7f9ea8101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:43.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.191+0000 7f9eaf200700 1 -- 172.21.15.123:0/3393501522 shutdown_connections 2024-05-08T06:11:43.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.191+0000 7f9eaf200700 1 -- 172.21.15.123:0/3393501522 wait complete. 2024-05-08T06:11:43.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.192+0000 7f9eaf200700 1 Processor -- start 2024-05-08T06:11:43.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.192+0000 7f9eaf200700 1 -- start start 2024-05-08T06:11:43.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.193+0000 7f9eaf200700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ea81033d0 0x7f9ea80757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:43.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.193+0000 7f9eaf200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ea8103d90 0x7f9ea8075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:43.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.193+0000 7f9eaf200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ea8109460 0x7f9ea8076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:43.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.193+0000 7f9eaf200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea81135e0 con 0x7f9ea8103d90 2024-05-08T06:11:43.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.193+0000 7f9eaf200700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea8113460 con 0x7f9ea8109460 2024-05-08T06:11:43.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.193+0000 7f9eaf200700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea8113760 con 0x7f9ea81033d0 2024-05-08T06:11:43.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.194+0000 7f9eade00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ea81033d0 0x7f9ea80757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:43.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.194+0000 7f9ead400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ea8103d90 0x7f9ea8075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:43.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.194+0000 7f9eade00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ea81033d0 0x7f9ea80757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:38798/0 (socket says 172.21.15.123:38798) 2024-05-08T06:11:43.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.194+0000 7f9eade00700 1 -- 172.21.15.123:0/738932846 learned_addr learned my addr 172.21.15.123:0/738932846 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:43.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.194+0000 7f9eae800700 1 --2- 172.21.15.123:0/738932846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ea8109460 0x7f9ea8076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:43.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.194+0000 7f9eade00700 1 -- 172.21.15.123:0/738932846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ea8109460 msgr2=0x7f9ea8076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:43.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.194+0000 7f9eade00700 1 --2- 172.21.15.123:0/738932846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ea8109460 0x7f9ea8076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:43.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.194+0000 7f9eade00700 1 -- 172.21.15.123:0/738932846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ea8103d90 msgr2=0x7f9ea8075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:43.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.194+0000 7f9eade00700 1 --2- 172.21.15.123:0/738932846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ea8103d90 0x7f9ea8075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:43.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.194+0000 7f9eade00700 1 -- 172.21.15.123:0/738932846 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea81a8000 con 0x7f9ea81033d0 2024-05-08T06:11:43.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.194+0000 7f9ead400700 1 --2- 172.21.15.123:0/738932846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ea8103d90 0x7f9ea8075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:11:43.196 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.194+0000 7f9eae800700 1 --2- 172.21.15.123:0/738932846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ea8109460 0x7f9ea8076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:11:43.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.195+0000 7f9eade00700 1 --2- 172.21.15.123:0/738932846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ea81033d0 0x7f9ea80757a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c0047b0 tx=0x7f9e9c009550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:43.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.195+0000 7f9ea2a00700 1 -- 172.21.15.123:0/738932846 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c027070 con 0x7f9ea81033d0 2024-05-08T06:11:43.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.195+0000 7f9ea2a00700 1 -- 172.21.15.123:0/738932846 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e9c022470 con 0x7f9ea81033d0 2024-05-08T06:11:43.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.195+0000 7f9ea2a00700 1 -- 172.21.15.123:0/738932846 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c017650 con 0x7f9ea81033d0 2024-05-08T06:11:43.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.196+0000 7f9eaf200700 1 -- 172.21.15.123:0/738932846 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ea81a8290 con 0x7f9ea81033d0 2024-05-08T06:11:43.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.196+0000 7f9eaf200700 1 -- 172.21.15.123:0/738932846 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ea81a8690 con 0x7f9ea81033d0 2024-05-08T06:11:43.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.197+0000 7f9eaf200700 1 -- 172.21.15.123:0/738932846 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ea806a190 con 0x7f9ea81033d0 2024-05-08T06:11:43.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.199+0000 7f9ea2a00700 1 -- 172.21.15.123:0/738932846 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7f9e9c02a070 con 0x7f9ea81033d0 2024-05-08T06:11:43.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.201+0000 7f9ea2a00700 1 --2- 172.21.15.123:0/738932846 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9e900b2900 0x7f9e900b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:43.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.201+0000 7f9ea2a00700 1 -- 172.21.15.123:0/738932846 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f9e9c0e2e50 con 0x7f9ea81033d0 2024-05-08T06:11:43.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.201+0000 7f9ead400700 1 --2- 172.21.15.123:0/738932846 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9e900b2900 0x7f9e900b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:43.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.202+0000 7f9ead400700 1 --2- 172.21.15.123:0/738932846 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9e900b2900 0x7f9e900b4dc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9e98009680 tx=0x7f9e98009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:43.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.205+0000 7f9ea2a00700 1 -- 172.21.15.123:0/738932846 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9e9c013070 con 0x7f9ea81033d0 2024-05-08T06:11:43.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.719+0000 7f9eaf200700 1 -- 172.21.15.123:0/738932846 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ea802e510 con 0x7f9ea81033d0 2024-05-08T06:11:43.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.720+0000 7f9ea2a00700 1 -- 172.21.15.123:0/738932846 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11885 (secure 0 0 0) 0x7f9e9c0aabb0 con 0x7f9ea81033d0 2024-05-08T06:11:43.722 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:11:43.722 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3646476522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3646476522},{"type":"v1","addr":"172.21.15.123:6835","nonce":3646476522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2763706497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2763706497},{"type":"v1","addr":"172.21.15.123:6837","nonce":2763706497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/274635265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":274635265},{"type":"v1","addr":"172.21.15.123:6839","nonce":274635265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/175423115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":175423115},{"type":"v1","addr":"172.21.15.192:6833","nonce":175423115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/3263009472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3263009472},{"type":"v1","addr":"172.21.15.163:6833","nonce":3263009472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3308612826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3308612826},{"type":"v1","addr":"172.21.15.163:6835","nonce":3308612826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T06:05:54.755550+0000","modified":"2024-05-08T06:10:12.600417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24512,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/202513680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":202513680},{"type":"v1","addr":"172.21.15.192:6835","nonce":202513680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/2483545969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2483545969},{"type":"v1","addr":"172.21.15.192:6837","nonce":2483545969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/660511838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":660511838},{"type":"v1","addr":"172.21.15.163:6837","nonce":660511838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.192:6839/3598682360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":3598682360},{"type":"v1","addr":"172.21.15.192:6839","nonce":3598682360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6841/1571726231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":1571726231},{"type":"v1","addr":"172.21.15.123:6841","nonce":1571726231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.163:6839/1568161515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":1568161515},{"type":"v1","addr":"172.21.15.163:6839","nonce":1568161515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24512,24496,24524]},"id":1}]} 2024-05-08T06:11:43.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.724+0000 7f9eaf200700 1 -- 172.21.15.123:0/738932846 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9e900b2900 msgr2=0x7f9e900b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:43.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.724+0000 7f9eaf200700 1 --2- 172.21.15.123:0/738932846 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9e900b2900 0x7f9e900b4dc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9e98009680 tx=0x7f9e98009260 comp rx=0 tx=0).stop 2024-05-08T06:11:43.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.724+0000 7f9eaf200700 1 -- 172.21.15.123:0/738932846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ea81033d0 msgr2=0x7f9ea80757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:43.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.724+0000 7f9eaf200700 1 --2- 172.21.15.123:0/738932846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ea81033d0 0x7f9ea80757a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c0047b0 tx=0x7f9e9c009550 comp rx=0 tx=0).stop 2024-05-08T06:11:43.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.724+0000 7f9eaf200700 1 -- 172.21.15.123:0/738932846 shutdown_connections 2024-05-08T06:11:43.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.724+0000 7f9eaf200700 1 --2- 172.21.15.123:0/738932846 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9e900b2900 0x7f9e900b4dc0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:43.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.724+0000 7f9eaf200700 1 --2- 172.21.15.123:0/738932846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ea8109460 0x7f9ea8076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:43.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.724+0000 7f9eaf200700 1 --2- 172.21.15.123:0/738932846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ea81033d0 0x7f9ea80757a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:43.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.724+0000 7f9eaf200700 1 --2- 172.21.15.123:0/738932846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ea8103d90 0x7f9ea8075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:43.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.725+0000 7f9eaf200700 1 -- 172.21.15.123:0/738932846 >> 172.21.15.123:0/738932846 conn(0x7f9ea80fef20 msgr2=0x7f9ea80ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:43.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.725+0000 7f9eaf200700 1 -- 172.21.15.123:0/738932846 shutdown_connections 2024-05-08T06:11:43.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:43.725+0000 7f9eaf200700 1 -- 172.21.15.123:0/738932846 wait complete. 2024-05-08T06:11:43.727 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 24 2024-05-08T06:11:44.439 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:44 smithi123 ceph-mon[29787]: pgmap v498: 97 pgs: 97 active+clean; 675 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-08T06:11:44.440 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:44 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/738932846' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:11:44.730 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-08T06:11:44.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:44 smithi163 ceph-mon[34520]: pgmap v498: 97 pgs: 97 active+clean; 675 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-08T06:11:44.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:44 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/738932846' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:11:44.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:44 smithi192 ceph-mon[34664]: pgmap v498: 97 pgs: 97 active+clean; 675 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-08T06:11:44.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:44 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/738932846' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:11:45.308 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:11:46.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:46 smithi123 ceph-mon[29787]: pgmap v499: 97 pgs: 97 active+clean; 675 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 852 B/s wr, 8 op/s 2024-05-08T06:11:46.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:46 smithi163 ceph-mon[34520]: pgmap v499: 97 pgs: 97 active+clean; 675 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 852 B/s wr, 8 op/s 2024-05-08T06:11:46.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:46 smithi192 ceph-mon[34664]: pgmap v499: 97 pgs: 97 active+clean; 675 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 852 B/s wr, 8 op/s 2024-05-08T06:11:47.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.798+0000 7f5a25c00700 1 -- 172.21.15.123:0/1603739515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a20076160 msgr2=0x7f5a20113ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:47.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.798+0000 7f5a25c00700 1 --2- 172.21.15.123:0/1603739515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a20076160 0x7f5a20113ea0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c009990 tx=0x7f5a0c009ca0 comp rx=0 tx=0).stop 2024-05-08T06:11:47.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.798+0000 7f5a25c00700 1 -- 172.21.15.123:0/1603739515 shutdown_connections 2024-05-08T06:11:47.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.798+0000 7f5a25c00700 1 --2- 172.21.15.123:0/1603739515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a201143e0 0x7f5a201167d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:47.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.798+0000 7f5a25c00700 1 --2- 172.21.15.123:0/1603739515 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a20075840 0x7f5a20075c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:47.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.798+0000 7f5a25c00700 1 --2- 172.21.15.123:0/1603739515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a20076160 0x7f5a20113ea0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:47.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.798+0000 7f5a25c00700 1 -- 172.21.15.123:0/1603739515 >> 172.21.15.123:0/1603739515 conn(0x7f5a200fef20 msgr2=0x7f5a20101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:47.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.798+0000 7f5a25c00700 1 -- 172.21.15.123:0/1603739515 shutdown_connections 2024-05-08T06:11:47.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.798+0000 7f5a25c00700 1 -- 172.21.15.123:0/1603739515 wait complete. 2024-05-08T06:11:47.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.800+0000 7f5a25c00700 1 Processor -- start 2024-05-08T06:11:47.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.800+0000 7f5a25c00700 1 -- start start 2024-05-08T06:11:47.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.801+0000 7f5a25c00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a20075840 0x7f5a201a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:47.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.801+0000 7f5a25c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a20076160 0x7f5a201a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:47.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.801+0000 7f5a25c00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a201143e0 0x7f5a201a4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:47.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.801+0000 7f5a25c00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5a20119c00 con 0x7f5a20076160 2024-05-08T06:11:47.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.801+0000 7f5a25c00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5a20119a80 con 0x7f5a201143e0 2024-05-08T06:11:47.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.801+0000 7f5a25c00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5a20119d80 con 0x7f5a20075840 2024-05-08T06:11:47.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.801+0000 7f5a1fe00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a20075840 0x7f5a201a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:47.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.801+0000 7f5a1fe00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a20075840 0x7f5a201a0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:50372/0 (socket says 172.21.15.123:50372) 2024-05-08T06:11:47.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.801+0000 7f5a1fe00700 1 -- 172.21.15.123:0/4015252497 learned_addr learned my addr 172.21.15.123:0/4015252497 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:47.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.802+0000 7f5a25200700 1 --2- 172.21.15.123:0/4015252497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a201143e0 0x7f5a201a4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:47.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.802+0000 7f5a1fe00700 1 -- 172.21.15.123:0/4015252497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a201143e0 msgr2=0x7f5a201a4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:47.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.802+0000 7f5a1fe00700 1 --2- 172.21.15.123:0/4015252497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a201143e0 0x7f5a201a4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:47.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.802+0000 7f5a1fe00700 1 -- 172.21.15.123:0/4015252497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a20076160 msgr2=0x7f5a201a0670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T06:11:47.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.802+0000 7f5a1fe00700 1 --2- 172.21.15.123:0/4015252497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a20076160 0x7f5a201a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:47.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.802+0000 7f5a1fe00700 1 -- 172.21.15.123:0/4015252497 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a201a5140 con 0x7f5a20075840 2024-05-08T06:11:47.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.803+0000 7f5a1fe00700 1 --2- 172.21.15.123:0/4015252497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a20075840 0x7f5a201a0130 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5a1000e9a0 tx=0x7f5a1000ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:47.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.803+0000 7f5a1cc00700 1 -- 172.21.15.123:0/4015252497 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1000cbd0 con 0x7f5a20075840 2024-05-08T06:11:47.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.803+0000 7f5a1cc00700 1 -- 172.21.15.123:0/4015252497 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a1000cd70 con 0x7f5a20075840 2024-05-08T06:11:47.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.803+0000 7f5a1cc00700 1 -- 172.21.15.123:0/4015252497 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a10018680 con 0x7f5a20075840 2024-05-08T06:11:47.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.804+0000 7f5a25c00700 1 -- 172.21.15.123:0/4015252497 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a201a5430 con 0x7f5a20075840 2024-05-08T06:11:47.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.804+0000 7f5a25c00700 1 -- 172.21.15.123:0/4015252497 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a201accb0 con 0x7f5a20075840 2024-05-08T06:11:47.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.808+0000 7f5a31aafb80 1 Processor -- start 2024-05-08T06:11:47.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.808+0000 7f5a31aafb80 1 -- start start 2024-05-08T06:11:47.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.809+0000 7f5a31aafb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564097f369e0 0x564097f36dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:47.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.809+0000 7f5a31aafb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564097f37300 0x564097f30fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:47.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.809+0000 7f5a31aafb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564097f31600 0x564097f31a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:47.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.809+0000 7f5a31aafb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564097e83a20 con 0x564097f37300 2024-05-08T06:11:47.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.809+0000 7f5a31aafb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x564097e21760 con 0x564097f31600 2024-05-08T06:11:47.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.809+0000 7f5a31aafb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564097e7f9e0 con 0x564097f369e0 2024-05-08T06:11:47.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.809+0000 7f5a25200700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564097f369e0 0x564097f36dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:47.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.809+0000 7f5a25200700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564097f369e0 0x564097f36dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:50374/0 (socket says 172.21.15.123:50374) 2024-05-08T06:11:47.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.809+0000 7f5a25200700 1 -- 172.21.15.123:0/1833375704 learned_addr learned my addr 172.21.15.123:0/1833375704 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:47.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.809+0000 7f5a1f400700 1 --2- 172.21.15.123:0/1833375704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564097f31600 0x564097f31a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:47.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.810+0000 7f5a1cc00700 1 -- 172.21.15.123:0/4015252497 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7f5a10010970 con 0x7f5a20075840 2024-05-08T06:11:47.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.810+0000 7f5a25200700 1 -- 172.21.15.123:0/1833375704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564097f31600 msgr2=0x564097f31a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:47.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.810+0000 7f5a25200700 1 --2- 172.21.15.123:0/1833375704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564097f31600 0x564097f31a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:47.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.810+0000 7f5a25200700 1 -- 172.21.15.123:0/1833375704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564097f37300 msgr2=0x564097f30fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:47.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.810+0000 7f5a1fe00700 1 --2- 172.21.15.123:0/1833375704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564097f37300 0x564097f30fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:47.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.811+0000 7f5a25200700 1 --2- 172.21.15.123:0/1833375704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564097f37300 0x564097f30fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:47.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.811+0000 7f5a25200700 1 -- 172.21.15.123:0/1833375704 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564097dd08d0 con 0x564097f369e0 2024-05-08T06:11:47.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.811+0000 7f5a1fe00700 1 --2- 172.21.15.123:0/1833375704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564097f37300 0x564097f30fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:11:47.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.811+0000 7f5a25200700 1 --2- 172.21.15.123:0/1833375704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564097f369e0 0x564097f36dc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5a180093a0 tx=0x7f5a18009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56f4f1d9fa91d6cf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:47.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.811+0000 7f5a15600700 1 -- 172.21.15.123:0/1833375704 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a180153f0 con 0x564097f369e0 2024-05-08T06:11:47.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.812+0000 7f5a15600700 1 -- 172.21.15.123:0/1833375704 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a1801a070 con 0x564097f369e0 2024-05-08T06:11:47.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.812+0000 7f5a15600700 1 -- 172.21.15.123:0/1833375704 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a180147b0 con 0x564097f369e0 2024-05-08T06:11:47.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.812+0000 7f5a1cc00700 1 --2- 172.21.15.123:0/4015252497 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5a040b2980 0x7f5a040b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:47.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.812+0000 7f5a1cc00700 1 -- 172.21.15.123:0/4015252497 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f5a100e25a0 con 0x7f5a20075840 2024-05-08T06:11:47.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.812+0000 7f5a31aafb80 1 -- 172.21.15.123:0/1833375704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564097f369e0 msgr2=0x564097f36dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:47.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.813+0000 7f5a31aafb80 1 --2- 172.21.15.123:0/1833375704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564097f369e0 0x564097f36dc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5a180093a0 tx=0x7f5a18009920 comp rx=0 tx=0).stop 2024-05-08T06:11:47.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.813+0000 7f5a31aafb80 1 -- 172.21.15.123:0/1833375704 shutdown_connections 2024-05-08T06:11:47.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.813+0000 7f5a31aafb80 1 --2- 172.21.15.123:0/1833375704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564097f31600 0x564097f31a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:47.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.813+0000 7f5a31aafb80 1 --2- 172.21.15.123:0/1833375704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564097f369e0 0x564097f36dc0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:47.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.813+0000 7f5a31aafb80 1 --2- 172.21.15.123:0/1833375704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564097f37300 0x564097f30fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:47.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.813+0000 7f5a31aafb80 1 -- 172.21.15.123:0/1833375704 >> 172.21.15.123:0/1833375704 conn(0x564097e5fb60 msgr2=0x564097f2b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:47.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.813+0000 7f5a1fe00700 1 --2- 172.21.15.123:0/4015252497 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5a040b2980 0x7f5a040b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:47.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.813+0000 7f5a31aafb80 1 -- 172.21.15.123:0/1833375704 shutdown_connections 2024-05-08T06:11:47.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.814+0000 7f5a31aafb80 1 -- 172.21.15.123:0/1833375704 wait complete. 2024-05-08T06:11:47.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.814+0000 7f5a1fe00700 1 --2- 172.21.15.123:0/4015252497 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5a040b2980 0x7f5a040b4e40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5a100a9870 tx=0x7f5a100092d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:47.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.814+0000 7f5a31aafb80 1 -- start start 2024-05-08T06:11:47.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.814+0000 7f5a31aafb80 1 Processor -- start 2024-05-08T06:11:47.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.815+0000 7f5a31aafb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564097f31600 0x564097f3ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:47.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.815+0000 7f5a31aafb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564097f369e0 0x564097f3d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:47.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.815+0000 7f5a31aafb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564097f37300 0x564097f3a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:47.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.815+0000 7f5a31aafb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564097f3ddb0 con 0x564097f369e0 2024-05-08T06:11:47.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.815+0000 7f5a31aafb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x564097de9b20 con 0x564097f37300 2024-05-08T06:11:47.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.815+0000 7f5a31aafb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564097f3df30 con 0x564097f31600 2024-05-08T06:11:47.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.815+0000 7f5a31aafb80 10 client.0 _ll_register_callbacks cb 0x564097e2ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:11:47.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.816+0000 7f5a1f400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564097f369e0 0x564097f3d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:47.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.816+0000 7f5a1f400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564097f369e0 0x564097f3d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53388/0 (socket says 172.21.15.123:53388) 2024-05-08T06:11:47.817 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.816+0000 7f5a1f400700 1 -- 172.21.15.123:0/410998394 learned_addr learned my addr 172.21.15.123:0/410998394 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:47.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.816+0000 7f5a25200700 1 --2- 172.21.15.123:0/410998394 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564097f37300 0x564097f3a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:47.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.816+0000 7f5a25200700 1 --2- 172.21.15.123:0/410998394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564097f31600 0x564097f3ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:47.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.816+0000 7f5a1f400700 1 -- 172.21.15.123:0/410998394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564097f31600 msgr2=0x564097f3ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:47.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.816+0000 7f5a1f400700 1 --2- 172.21.15.123:0/410998394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564097f31600 0x564097f3ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:47.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.817+0000 7f5a1f400700 1 -- 172.21.15.123:0/410998394 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564097f37300 msgr2=0x564097f3a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:47.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.817+0000 7f5a1f400700 1 --2- 172.21.15.123:0/410998394 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564097f37300 0x564097f3a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:47.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.817+0000 7f5a1f400700 1 -- 172.21.15.123:0/410998394 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564097e49420 con 0x564097f369e0 2024-05-08T06:11:47.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.817+0000 7f5a25200700 1 --2- 172.21.15.123:0/410998394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564097f31600 0x564097f3ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:11:47.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.817+0000 7f5a1f400700 1 --2- 172.21.15.123:0/410998394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564097f369e0 0x564097f3d3d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c015000 tx=0x7f5a0c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b1320675d1e1ae8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:47.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.817+0000 7f5a0a000700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-08T06:11:47.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.818+0000 7f5a0a000700 1 -- 172.21.15.123:0/410998394 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c00bd30 con 0x564097f369e0 2024-05-08T06:11:47.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.818+0000 7f5a0a000700 1 -- 172.21.15.123:0/410998394 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f5a0c003680 con 0x564097f369e0 2024-05-08T06:11:47.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.818+0000 7f5a31aafb80 1 -- 172.21.15.123:0/410998394 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a1801a070 con 0x564097f369e0 2024-05-08T06:11:47.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.818+0000 7f5a0a000700 1 -- 172.21.15.123:0/410998394 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5a0c003880 con 0x564097f369e0 2024-05-08T06:11:47.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.818+0000 7f5a31aafb80 10 client.15237 fetch_fsmap learned FSMap version 24 2024-05-08T06:11:47.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.818+0000 7f5a31aafb80 1 -- 172.21.15.123:0/410998394 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564097f483e0 con 0x564097f369e0 2024-05-08T06:11:47.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.819+0000 7f5a0a000700 1 -- 172.21.15.123:0/410998394 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a1801a070 con 0x564097f369e0 2024-05-08T06:11:47.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.819+0000 7f5a0a000700 1 -- 172.21.15.123:0/410998394 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f5a0c024880 con 0x564097f369e0 2024-05-08T06:11:47.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.819+0000 7f5a31aafb80 10 client.15237 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:11:47.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.819+0000 7f5a31aafb80 10 client.15237 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T06:11:47.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.819+0000 7f5a31aafb80 20 client.15237 populate_metadata read hostname 'smithi123' 2024-05-08T06:11:47.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.819+0000 7f5a31aafb80 1 --2- 172.21.15.123:0/410998394 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x564097f4aac0 0x564097f4ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:47.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.819+0000 7f5a31aafb80 4 client.15237 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:11:47.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.819+0000 7f5a31aafb80 1 -- 172.21.15.123:0/410998394 --> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564097efe0a0 con 0x564097f4aac0 2024-05-08T06:11:47.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.820+0000 7f5a25200700 1 --2- 172.21.15.123:0/410998394 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x564097f4aac0 0x564097f4ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:47.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.821+0000 7f5a25200700 1 --2- 172.21.15.123:0/410998394 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x564097f4aac0 0x564097f4ced0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6e19507cc782b2aa server_cookie=ef639f2b2d42cb1e in_seq=0 out_seq=0 2024-05-08T06:11:47.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.821+0000 7f5a0a000700 10 client.15237 ms_handle_connect on v2:172.21.15.192:6838/3598682360 2024-05-08T06:11:47.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.822+0000 7f5a0a000700 1 -- 172.21.15.123:0/410998394 <== mds.2 v2:172.21.15.192:6838/3598682360 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564097efe0a0 con 0x564097f4aac0 2024-05-08T06:11:47.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.822+0000 7f5a0a000700 10 client.15237 handle_command_reply: tid=0 2024-05-08T06:11:47.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.822+0000 7f5a31aafb80 1 client.15237 shutdown 2024-05-08T06:11:47.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.822+0000 7f5a31aafb80 1 -- 172.21.15.123:0/410998394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564097f369e0 msgr2=0x564097f3d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:47.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.822+0000 7f5a31aafb80 1 --2- 172.21.15.123:0/410998394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564097f369e0 0x564097f3d3d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c015000 tx=0x7f5a0c005780 comp rx=0 tx=0).stop 2024-05-08T06:11:47.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.822+0000 7f5a31aafb80 1 -- 172.21.15.123:0/410998394 shutdown_connections 2024-05-08T06:11:47.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.822+0000 7f5a31aafb80 1 --2- 172.21.15.123:0/410998394 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x564097f4aac0 0x564097f4ced0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:47.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.822+0000 7f5a31aafb80 1 --2- 172.21.15.123:0/410998394 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564097f37300 0x564097f3a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:47.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.822+0000 7f5a31aafb80 1 --2- 172.21.15.123:0/410998394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564097f31600 0x564097f3ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:47.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.822+0000 7f5a31aafb80 1 --2- 172.21.15.123:0/410998394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564097f369e0 0x564097f3d3d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:47.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.822+0000 7f5a31aafb80 1 -- 172.21.15.123:0/410998394 >> 172.21.15.123:0/410998394 conn(0x564097e5fb60 msgr2=0x564097f26460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:47.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.823+0000 7f5a0a000700 0 client.15237 ms_handle_reset on v2:172.21.15.192:6838/3598682360 2024-05-08T06:11:47.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.823+0000 7f5a31aafb80 1 -- 172.21.15.123:0/410998394 shutdown_connections 2024-05-08T06:11:47.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.823+0000 7f5a31aafb80 1 -- 172.21.15.123:0/410998394 wait complete. 2024-05-08T06:11:47.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.823+0000 7f5a31aafb80 20 client.15237 trim_cache size 0 max 16384 2024-05-08T06:11:47.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.850+0000 7f5a31aafb80 1 Processor -- start 2024-05-08T06:11:47.851 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.850+0000 7f5a31aafb80 1 -- start start 2024-05-08T06:11:47.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.851+0000 7f5a31aafb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564097f31600 0x564097f30fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:47.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.851+0000 7f5a31aafb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564097f369e0 0x564097f46e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:47.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.851+0000 7f5a31aafb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564097f37300 0x564097f527e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:47.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.851+0000 7f5a31aafb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5a201b69e0 con 0x564097f31600 2024-05-08T06:11:47.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.851+0000 7f5a31aafb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5a201b4270 con 0x564097f37300 2024-05-08T06:11:47.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.851+0000 7f5a31aafb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5a201b20e0 con 0x564097f369e0 2024-05-08T06:11:47.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.851+0000 7f5a1f400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564097f369e0 0x564097f46e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:47.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.851+0000 7f5a1f400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564097f369e0 0x564097f46e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:50378/0 (socket says 172.21.15.123:50378) 2024-05-08T06:11:47.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.851+0000 7f5a1f400700 1 -- 172.21.15.123:0/2356094559 learned_addr learned my addr 172.21.15.123:0/2356094559 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:47.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.851+0000 7f5a25200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564097f31600 0x564097f30fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:47.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.852+0000 7f5a1f400700 1 -- 172.21.15.123:0/2356094559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564097f37300 msgr2=0x564097f527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:47.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.852+0000 7f5a25200700 1 --2- 172.21.15.123:0/2356094559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564097f37300 0x564097f527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:47.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.852+0000 7f5a1f400700 1 --2- 172.21.15.123:0/2356094559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564097f37300 0x564097f527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:47.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.852+0000 7f5a1f400700 1 -- 172.21.15.123:0/2356094559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564097f31600 msgr2=0x564097f30fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:47.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.852+0000 7f5a1f400700 1 --2- 172.21.15.123:0/2356094559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564097f31600 0x564097f30fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:47.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.852+0000 7f5a1f400700 1 -- 172.21.15.123:0/2356094559 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564097f387e0 con 0x564097f369e0 2024-05-08T06:11:47.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.852+0000 7f5a25200700 1 --2- 172.21.15.123:0/2356094559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564097f37300 0x564097f527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:11:47.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.852+0000 7f5a1f400700 1 --2- 172.21.15.123:0/2356094559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564097f369e0 0x564097f46e40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c000c00 tx=0x7f5a0c005a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce33e4a32902e41e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:47.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.852+0000 7f5a14c00700 1 -- 172.21.15.123:0/2356094559 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c03c070 con 0x564097f369e0 2024-05-08T06:11:47.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.853+0000 7f5a14c00700 1 -- 172.21.15.123:0/2356094559 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a0c005bc0 con 0x564097f369e0 2024-05-08T06:11:47.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.853+0000 7f5a14c00700 1 -- 172.21.15.123:0/2356094559 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c00f850 con 0x564097f369e0 2024-05-08T06:11:47.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.853+0000 7f5a31aafb80 1 -- 172.21.15.123:0/2356094559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564097f369e0 msgr2=0x564097f46e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:47.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.853+0000 7f5a31aafb80 1 --2- 172.21.15.123:0/2356094559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564097f369e0 0x564097f46e40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c000c00 tx=0x7f5a0c005a80 comp rx=0 tx=0).stop 2024-05-08T06:11:47.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.853+0000 7f5a31aafb80 1 -- 172.21.15.123:0/2356094559 shutdown_connections 2024-05-08T06:11:47.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.853+0000 7f5a31aafb80 1 --2- 172.21.15.123:0/2356094559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564097f31600 0x564097f30fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:47.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.853+0000 7f5a31aafb80 1 --2- 172.21.15.123:0/2356094559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564097f37300 0x564097f527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:47.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.853+0000 7f5a31aafb80 1 --2- 172.21.15.123:0/2356094559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564097f369e0 0x564097f46e40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:47.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.853+0000 7f5a31aafb80 1 -- 172.21.15.123:0/2356094559 >> 172.21.15.123:0/2356094559 conn(0x564097e5fb60 msgr2=0x564097f38b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:47.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.854+0000 7f5a31aafb80 1 -- 172.21.15.123:0/2356094559 shutdown_connections 2024-05-08T06:11:47.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.854+0000 7f5a31aafb80 1 -- 172.21.15.123:0/2356094559 wait complete. 2024-05-08T06:11:47.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.854+0000 7f5a31aafb80 1 -- start start 2024-05-08T06:11:47.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.854+0000 7f5a31aafb80 1 Processor -- start 2024-05-08T06:11:47.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.855+0000 7f5a31aafb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564097f31600 0x564097f433a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:47.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.855+0000 7f5a31aafb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564097f369e0 0x564097f438e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:47.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.855+0000 7f5a31aafb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564097f37300 0x564097f401d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:47.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.855+0000 7f5a31aafb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5a201b2730 con 0x564097f31600 2024-05-08T06:11:47.857 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.855+0000 7f5a31aafb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5a201b3ba0 con 0x564097f369e0 2024-05-08T06:11:47.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.855+0000 7f5a31aafb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5a201b2e00 con 0x564097f37300 2024-05-08T06:11:47.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.855+0000 7f5a31aafb80 10 client.0 _ll_register_callbacks cb 0x564097e2ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:11:47.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.855+0000 7f5a1f400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564097f369e0 0x564097f438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:47.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.856+0000 7f5a1f400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564097f369e0 0x564097f438e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:44338/0 (socket says 172.21.15.123:44338) 2024-05-08T06:11:47.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.856+0000 7f5a1f400700 1 -- 172.21.15.123:0/2332000757 learned_addr learned my addr 172.21.15.123:0/2332000757 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:47.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.856+0000 7f5a1f400700 1 -- 172.21.15.123:0/2332000757 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564097f37300 msgr2=0x564097f401d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:47.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.856+0000 7f5a1f400700 1 --2- 172.21.15.123:0/2332000757 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564097f37300 0x564097f401d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:47.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.856+0000 7f5a1f400700 1 -- 172.21.15.123:0/2332000757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564097f31600 msgr2=0x564097f433a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:47.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.856+0000 7f5a1f400700 1 --2- 172.21.15.123:0/2332000757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564097f31600 0x564097f433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:47.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.856+0000 7f5a1f400700 1 -- 172.21.15.123:0/2332000757 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564097f40a10 con 0x564097f369e0 2024-05-08T06:11:47.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.856+0000 7f5a1f400700 1 --2- 172.21.15.123:0/2332000757 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564097f369e0 0x564097f438e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c006010 tx=0x7f5a0c005620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b258182121d2a0aa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:47.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.856+0000 7f5a0a000700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-05-08T06:11:47.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.857+0000 7f5a0a000700 1 -- 172.21.15.123:0/2332000757 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c03c070 con 0x564097f369e0 2024-05-08T06:11:47.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.857+0000 7f5a31aafb80 1 -- 172.21.15.123:0/2332000757 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a0c005bc0 con 0x564097f369e0 2024-05-08T06:11:47.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.857+0000 7f5a0a000700 1 -- 172.21.15.123:0/2332000757 <== mon.2 v2:172.21.15.163:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f5a0c003c30 con 0x564097f369e0 2024-05-08T06:11:47.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.857+0000 7f5a0a000700 1 -- 172.21.15.123:0/2332000757 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5a0c003e30 con 0x564097f369e0 2024-05-08T06:11:47.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.857+0000 7f5a31aafb80 10 client.25355 fetch_fsmap learned FSMap version 24 2024-05-08T06:11:47.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.857+0000 7f5a31aafb80 1 -- 172.21.15.123:0/2332000757 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564097fb5e90 con 0x564097f369e0 2024-05-08T06:11:47.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.858+0000 7f5a0a000700 1 -- 172.21.15.123:0/2332000757 <== mon.2 v2:172.21.15.163:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a0c005bc0 con 0x564097f369e0 2024-05-08T06:11:47.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.858+0000 7f5a0a000700 1 -- 172.21.15.123:0/2332000757 <== mon.2 v2:172.21.15.163:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f5a0c00f870 con 0x564097f369e0 2024-05-08T06:11:47.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.858+0000 7f5a31aafb80 10 client.25355 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:11:47.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.858+0000 7f5a31aafb80 10 client.25355 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T06:11:47.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.858+0000 7f5a31aafb80 20 client.25355 populate_metadata read hostname 'smithi123' 2024-05-08T06:11:47.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.858+0000 7f5a31aafb80 1 --2- 172.21.15.123:0/2332000757 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x564097fb6590 0x564097fb89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:47.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.858+0000 7f5a31aafb80 4 client.25355 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T06:11:47.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.858+0000 7f5a31aafb80 1 -- 172.21.15.123:0/2332000757 --> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564097e91400 con 0x564097fb6590 2024-05-08T06:11:47.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.858+0000 7f5a25200700 1 --2- 172.21.15.123:0/2332000757 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x564097fb6590 0x564097fb89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:47.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.859+0000 7f5a25200700 1 --2- 172.21.15.123:0/2332000757 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x564097fb6590 0x564097fb89b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d38488ddf65c93a0 server_cookie=caf4412d27f83ead in_seq=0 out_seq=0 2024-05-08T06:11:47.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.859+0000 7f5a0a000700 10 client.25355 ms_handle_connect on v2:172.21.15.192:6838/3598682360 2024-05-08T06:11:47.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.859+0000 7f5a0a000700 1 -- 172.21.15.123:0/2332000757 <== mds.2 v2:172.21.15.192:6838/3598682360 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564097e91400 con 0x564097fb6590 2024-05-08T06:11:47.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.859+0000 7f5a0a000700 10 client.25355 handle_command_reply: tid=0 2024-05-08T06:11:47.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.859+0000 7f5a31aafb80 1 client.25355 shutdown 2024-05-08T06:11:47.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.859+0000 7f5a31aafb80 1 -- 172.21.15.123:0/2332000757 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564097f369e0 msgr2=0x564097f438e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:47.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.859+0000 7f5a31aafb80 1 --2- 172.21.15.123:0/2332000757 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564097f369e0 0x564097f438e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c006010 tx=0x7f5a0c005620 comp rx=0 tx=0).stop 2024-05-08T06:11:47.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.860+0000 7f5a31aafb80 1 -- 172.21.15.123:0/2332000757 shutdown_connections 2024-05-08T06:11:47.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.860+0000 7f5a31aafb80 1 --2- 172.21.15.123:0/2332000757 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x564097fb6590 0x564097fb89b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:47.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.860+0000 7f5a31aafb80 1 --2- 172.21.15.123:0/2332000757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564097f31600 0x564097f433a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:47.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.860+0000 7f5a31aafb80 1 --2- 172.21.15.123:0/2332000757 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564097f37300 0x564097f401d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:47.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.860+0000 7f5a31aafb80 1 --2- 172.21.15.123:0/2332000757 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564097f369e0 0x564097f438e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:47.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.860+0000 7f5a31aafb80 1 -- 172.21.15.123:0/2332000757 >> 172.21.15.123:0/2332000757 conn(0x564097e5fb60 msgr2=0x564097f376e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:47.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.860+0000 7f5a0a000700 0 client.25355 ms_handle_reset on v2:172.21.15.192:6838/3598682360 2024-05-08T06:11:47.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.860+0000 7f5a31aafb80 1 -- 172.21.15.123:0/2332000757 shutdown_connections 2024-05-08T06:11:47.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.860+0000 7f5a31aafb80 1 -- 172.21.15.123:0/2332000757 wait complete. 2024-05-08T06:11:47.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.860+0000 7f5a31aafb80 20 client.25355 trim_cache size 0 max 16384 2024-05-08T06:11:47.863 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-05-08T06:11:47.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.861+0000 7f5a25c00700 1 -- 172.21.15.123:0/4015252497 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5a040b2980 msgr2=0x7f5a040b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:47.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.861+0000 7f5a25c00700 1 --2- 172.21.15.123:0/4015252497 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5a040b2980 0x7f5a040b4e40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5a100a9870 tx=0x7f5a100092d0 comp rx=0 tx=0).stop 2024-05-08T06:11:47.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.861+0000 7f5a25c00700 1 -- 172.21.15.123:0/4015252497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a20075840 msgr2=0x7f5a201a0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:47.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.861+0000 7f5a25c00700 1 --2- 172.21.15.123:0/4015252497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a20075840 0x7f5a201a0130 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5a1000e9a0 tx=0x7f5a1000ed60 comp rx=0 tx=0).stop 2024-05-08T06:11:47.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.861+0000 7f5a25c00700 1 -- 172.21.15.123:0/4015252497 shutdown_connections 2024-05-08T06:11:47.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.861+0000 7f5a25c00700 1 --2- 172.21.15.123:0/4015252497 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5a040b2980 0x7f5a040b4e40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:47.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.861+0000 7f5a25c00700 1 --2- 172.21.15.123:0/4015252497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a201143e0 0x7f5a201a4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:47.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.861+0000 7f5a25c00700 1 --2- 172.21.15.123:0/4015252497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a20075840 0x7f5a201a0130 secure :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5a1000e9a0 tx=0x7f5a1000ed60 comp rx=0 tx=0).stop 2024-05-08T06:11:47.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.861+0000 7f5a25c00700 1 --2- 172.21.15.123:0/4015252497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a20076160 0x7f5a201a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:47.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.861+0000 7f5a25c00700 1 -- 172.21.15.123:0/4015252497 >> 172.21.15.123:0/4015252497 conn(0x7f5a200fef20 msgr2=0x7f5a200ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:47.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.862+0000 7f5a25c00700 1 -- 172.21.15.123:0/4015252497 shutdown_connections 2024-05-08T06:11:47.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:47.862+0000 7f5a25c00700 1 -- 172.21.15.123:0/4015252497 wait complete. 2024-05-08T06:11:48.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:48 smithi123 ceph-mon[29787]: pgmap v500: 97 pgs: 97 active+clean; 675 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-08T06:11:48.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:48 smithi163 ceph-mon[34520]: pgmap v500: 97 pgs: 97 active+clean; 675 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-08T06:11:48.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:48 smithi192 ceph-mon[34664]: pgmap v500: 97 pgs: 97 active+clean; 675 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-08T06:11:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.810+0000 7f7c69c00640 1 -- 172.21.15.123:0/123820142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c641043f0 msgr2=0x7f7c6410ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.810+0000 7f7c69c00640 1 --2- 172.21.15.123:0/123820142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c641043f0 0x7f7c6410ac80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c009ca0 tx=0x7f7c5c02a8c0 comp rx=0 tx=0).stop 2024-05-08T06:11:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.811+0000 7f7c69c00640 1 -- 172.21.15.123:0/123820142 shutdown_connections 2024-05-08T06:11:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.811+0000 7f7c69c00640 1 --2- 172.21.15.123:0/123820142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c641043f0 0x7f7c6410ac80 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.811+0000 7f7c69c00640 1 --2- 172.21.15.123:0/123820142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c64103a30 0x7f7c64103eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.811+0000 7f7c69c00640 1 --2- 172.21.15.123:0/123820142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c64102840 0x7f7c64102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.811+0000 7f7c69c00640 1 -- 172.21.15.123:0/123820142 >> 172.21.15.123:0/123820142 conn(0x7f7c640fddb0 msgr2=0x7f7c641001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.811+0000 7f7c69c00640 1 -- 172.21.15.123:0/123820142 shutdown_connections 2024-05-08T06:11:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.811+0000 7f7c69c00640 1 -- 172.21.15.123:0/123820142 wait complete. 2024-05-08T06:11:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.812+0000 7f7c69c00640 1 Processor -- start 2024-05-08T06:11:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.812+0000 7f7c69c00640 1 -- start start 2024-05-08T06:11:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.813+0000 7f7c69c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c64102840 0x7f7c64076760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.813+0000 7f7c69c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c64103a30 0x7f7c64074da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.813+0000 7f7c69c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c641043f0 0x7f7c640752e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.813+0000 7f7c69c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6410e3c0 con 0x7f7c64102840 2024-05-08T06:11:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.813+0000 7f7c69c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6410e240 con 0x7f7c64103a30 2024-05-08T06:11:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.813+0000 7f7c69c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6410e540 con 0x7f7c641043f0 2024-05-08T06:11:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.813+0000 7f7c63400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c64103a30 0x7f7c64074da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.814+0000 7f7c63400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c64103a30 0x7f7c64074da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:44350/0 (socket says 172.21.15.123:44350) 2024-05-08T06:11:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.814+0000 7f7c63400640 1 -- 172.21.15.123:0/2846460195 learned_addr learned my addr 172.21.15.123:0/2846460195 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.814+0000 7f7c69200640 1 --2- 172.21.15.123:0/2846460195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c641043f0 0x7f7c640752e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:48.816 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.814+0000 7f7c63e00640 1 --2- 172.21.15.123:0/2846460195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c64102840 0x7f7c64076760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:48.816 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.814+0000 7f7c69200640 1 -- 172.21.15.123:0/2846460195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c64103a30 msgr2=0x7f7c64074da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:48.816 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.814+0000 7f7c69200640 1 --2- 172.21.15.123:0/2846460195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c64103a30 0x7f7c64074da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:48.816 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.814+0000 7f7c69200640 1 -- 172.21.15.123:0/2846460195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c64102840 msgr2=0x7f7c64076760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:48.816 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.814+0000 7f7c69200640 1 --2- 172.21.15.123:0/2846460195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c64102840 0x7f7c64076760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:48.816 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.814+0000 7f7c69200640 1 -- 172.21.15.123:0/2846460195 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c64075b70 con 0x7f7c641043f0 2024-05-08T06:11:48.817 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.815+0000 7f7c69200640 1 --2- 172.21.15.123:0/2846460195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c641043f0 0x7f7c640752e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c031a20 tx=0x7f7c5c0317c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:48.817 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.815+0000 7f7c60c00640 1 -- 172.21.15.123:0/2846460195 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c003b70 con 0x7f7c641043f0 2024-05-08T06:11:48.817 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.815+0000 7f7c69c00640 1 -- 172.21.15.123:0/2846460195 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c641a2b30 con 0x7f7c641043f0 2024-05-08T06:11:48.817 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.815+0000 7f7c60c00640 1 -- 172.21.15.123:0/2846460195 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c5c003d10 con 0x7f7c641043f0 2024-05-08T06:11:48.817 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.815+0000 7f7c60c00640 1 -- 172.21.15.123:0/2846460195 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c02fb70 con 0x7f7c641043f0 2024-05-08T06:11:48.817 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.815+0000 7f7c69c00640 1 -- 172.21.15.123:0/2846460195 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c641a30d0 con 0x7f7c641043f0 2024-05-08T06:11:48.821 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.820+0000 7f7c6dfb3740 1 Processor -- start 2024-05-08T06:11:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.821+0000 7f7c60c00640 1 -- 172.21.15.123:0/2846460195 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7f7c5c0046a0 con 0x7f7c641043f0 2024-05-08T06:11:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.821+0000 7f7c6dfb3740 1 -- start start 2024-05-08T06:11:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.822+0000 7f7c6dfb3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564fcacf0b30 0x564fcacf0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.822+0000 7f7c6dfb3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fcacf1520 0x564fcacf6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.822+0000 7f7c6dfb3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fcacf6690 0x564fcacf8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.822+0000 7f7c6dfb3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564fcac4c550 con 0x564fcacf1520 2024-05-08T06:11:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.822+0000 7f7c6dfb3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x564fcabf1bc0 con 0x564fcacf0b30 2024-05-08T06:11:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.822+0000 7f7c6dfb3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564fcac48510 con 0x564fcacf6690 2024-05-08T06:11:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.822+0000 7f7c63400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564fcacf0b30 0x564fcacf0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.822+0000 7f7c63400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564fcacf0b30 0x564fcacf0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:44358/0 (socket says 172.21.15.123:44358) 2024-05-08T06:11:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.822+0000 7f7c63400640 1 -- 172.21.15.123:0/3825766096 learned_addr learned my addr 172.21.15.123:0/3825766096 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.822+0000 7f7c63e00640 1 --2- 172.21.15.123:0/3825766096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fcacf1520 0x564fcacf6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.822+0000 7f7c63400640 1 --2- 172.21.15.123:0/3825766096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fcacf6690 0x564fcacf8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.823+0000 7f7c63400640 1 -- 172.21.15.123:0/3825766096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fcacf6690 msgr2=0x564fcacf8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.823+0000 7f7c63400640 1 --2- 172.21.15.123:0/3825766096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fcacf6690 0x564fcacf8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.823+0000 7f7c63400640 1 -- 172.21.15.123:0/3825766096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fcacf1520 msgr2=0x564fcacf6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.823+0000 7f7c63400640 1 --2- 172.21.15.123:0/3825766096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fcacf1520 0x564fcacf6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.823+0000 7f7c63400640 1 -- 172.21.15.123:0/3825766096 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fca95e210 con 0x564fcacf0b30 2024-05-08T06:11:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.823+0000 7f7c63e00640 1 --2- 172.21.15.123:0/3825766096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fcacf1520 0x564fcacf6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:11:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.823+0000 7f7c63400640 1 --2- 172.21.15.123:0/3825766096 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564fcacf0b30 0x564fcacf0f50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7c500118f0 tx=0x7f7c500098d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7297c21ac833aa98 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.824+0000 7f7c60c00640 1 --2- 172.21.15.123:0/2846460195 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f7c480b2860 0x7f7c480b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.824+0000 7f7c59600640 1 -- 172.21.15.123:0/3825766096 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5000f7c0 con 0x564fcacf0b30 2024-05-08T06:11:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.824+0000 7f7c59600640 1 -- 172.21.15.123:0/3825766096 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c5000fe10 con 0x564fcacf0b30 2024-05-08T06:11:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.824+0000 7f7c59600640 1 -- 172.21.15.123:0/3825766096 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5000ec70 con 0x564fcacf0b30 2024-05-08T06:11:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.824+0000 7f7c60c00640 1 -- 172.21.15.123:0/2846460195 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f7c5c103600 con 0x7f7c641043f0 2024-05-08T06:11:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.824+0000 7f7c63e00640 1 --2- 172.21.15.123:0/2846460195 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f7c480b2860 0x7f7c480b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.825+0000 7f7c6dfb3740 1 -- 172.21.15.123:0/3825766096 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564fcacf0b30 msgr2=0x564fcacf0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.825+0000 7f7c6dfb3740 1 --2- 172.21.15.123:0/3825766096 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564fcacf0b30 0x564fcacf0f50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7c500118f0 tx=0x7f7c500098d0 comp rx=0 tx=0).stop 2024-05-08T06:11:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.825+0000 7f7c6dfb3740 1 -- 172.21.15.123:0/3825766096 shutdown_connections 2024-05-08T06:11:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.825+0000 7f7c6dfb3740 1 --2- 172.21.15.123:0/3825766096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fcacf6690 0x564fcacf8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.825+0000 7f7c6dfb3740 1 --2- 172.21.15.123:0/3825766096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fcacf1520 0x564fcacf6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.825+0000 7f7c6dfb3740 1 --2- 172.21.15.123:0/3825766096 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564fcacf0b30 0x564fcacf0f50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.825+0000 7f7c6dfb3740 1 -- 172.21.15.123:0/3825766096 >> 172.21.15.123:0/3825766096 conn(0x564fcacb0230 msgr2=0x564fcace3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.825+0000 7f7c6dfb3740 1 -- 172.21.15.123:0/3825766096 shutdown_connections 2024-05-08T06:11:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.825+0000 7f7c63e00640 1 --2- 172.21.15.123:0/2846460195 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f7c480b2860 0x7f7c480b4d20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x564fcacf1920 tx=0x7f7c5400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.825+0000 7f7c6dfb3740 1 -- 172.21.15.123:0/3825766096 wait complete. 2024-05-08T06:11:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.826+0000 7f7c6dfb3740 1 -- start start 2024-05-08T06:11:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.826+0000 7f7c6dfb3740 1 Processor -- start 2024-05-08T06:11:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.827+0000 7f7c6dfb3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fcacf0b30 0x564fcacfaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.827+0000 7f7c6dfb3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fcacf1520 0x564fcacfb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.827+0000 7f7c6dfb3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564fcacf6690 0x564fcad02120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.827+0000 7f7c6dfb3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564fcacf9640 con 0x564fcacf1520 2024-05-08T06:11:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.827+0000 7f7c6dfb3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x564fcac05660 con 0x564fcacf6690 2024-05-08T06:11:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.827+0000 7f7c6dfb3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564fcacf97c0 con 0x564fcacf0b30 2024-05-08T06:11:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.827+0000 7f7c6dfb3740 10 client.0 _ll_register_callbacks cb 0x564fcabcf400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:11:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.827+0000 7f7c63400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fcacf1520 0x564fcacfb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.827+0000 7f7c63e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fcacf0b30 0x564fcacfaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.827+0000 7f7c63e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fcacf0b30 0x564fcacfaba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:50410/0 (socket says 172.21.15.123:50410) 2024-05-08T06:11:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.827+0000 7f7c63e00640 1 -- 172.21.15.123:0/594497763 learned_addr learned my addr 172.21.15.123:0/594497763 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.827+0000 7f7c69200640 1 --2- 172.21.15.123:0/594497763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564fcacf6690 0x564fcad02120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.828+0000 7f7c69200640 1 -- 172.21.15.123:0/594497763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fcacf0b30 msgr2=0x564fcacfaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.828+0000 7f7c69200640 1 --2- 172.21.15.123:0/594497763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fcacf0b30 0x564fcacfaba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.828+0000 7f7c69200640 1 -- 172.21.15.123:0/594497763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fcacf1520 msgr2=0x564fcacfb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.828+0000 7f7c69200640 1 --2- 172.21.15.123:0/594497763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fcacf1520 0x564fcacfb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.828+0000 7f7c69200640 1 -- 172.21.15.123:0/594497763 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fcaa2a620 con 0x564fcacf6690 2024-05-08T06:11:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.828+0000 7f7c63400640 1 --2- 172.21.15.123:0/594497763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fcacf1520 0x564fcacfb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:11:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.829+0000 7f7c69200640 1 --2- 172.21.15.123:0/594497763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564fcacf6690 0x564fcad02120 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c0049d0 tx=0x7f7c5c0ca340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f7bf34d27556f43 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.829+0000 7f7c4e000640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-05-08T06:11:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.829+0000 7f7c4e000640 1 -- 172.21.15.123:0/594497763 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c0cb2d0 con 0x564fcacf6690 2024-05-08T06:11:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.829+0000 7f7c4e000640 1 -- 172.21.15.123:0/594497763 <== mon.2 v2:172.21.15.163:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f7c5c02fd10 con 0x564fcacf6690 2024-05-08T06:11:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.829+0000 7f7c6dfb3740 1 -- 172.21.15.123:0/594497763 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c5000fe10 con 0x564fcacf6690 2024-05-08T06:11:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.830+0000 7f7c4e000640 1 -- 172.21.15.123:0/594497763 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7c5c0cabd0 con 0x564fcacf6690 2024-05-08T06:11:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.830+0000 7f7c6dfb3740 10 client.25367 fetch_fsmap learned FSMap version 24 2024-05-08T06:11:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.830+0000 7f7c6dfb3740 1 -- 172.21.15.123:0/594497763 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564fcacf9640 con 0x564fcacf6690 2024-05-08T06:11:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.830+0000 7f7c4e000640 1 -- 172.21.15.123:0/594497763 <== mon.2 v2:172.21.15.163:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c5000fe10 con 0x564fcacf6690 2024-05-08T06:11:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.831+0000 7f7c4e000640 1 -- 172.21.15.123:0/594497763 <== mon.2 v2:172.21.15.163:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f7c5c0c6040 con 0x564fcacf6690 2024-05-08T06:11:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.831+0000 7f7c6dfb3740 10 client.25367 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:11:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.831+0000 7f7c6dfb3740 10 client.25367 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T06:11:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.831+0000 7f7c6dfb3740 20 client.25367 populate_metadata read hostname 'smithi123' 2024-05-08T06:11:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.831+0000 7f7c6dfb3740 1 --2- 172.21.15.123:0/594497763 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x564fcad06800 0x564fcad08c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.831+0000 7f7c6dfb3740 4 client.25367 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:11:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.831+0000 7f7c6dfb3740 1 -- 172.21.15.123:0/594497763 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fcac5a160 con 0x564fcad06800 2024-05-08T06:11:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.831+0000 7f7c63e00640 1 --2- 172.21.15.123:0/594497763 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x564fcad06800 0x564fcad08c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:48.834 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.832+0000 7f7c63e00640 1 --2- 172.21.15.123:0/594497763 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x564fcad06800 0x564fcad08c20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3717a4bc8622cfcc server_cookie=d5485f43afaec0ec in_seq=0 out_seq=0 2024-05-08T06:11:48.834 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.833+0000 7f7c4e000640 10 client.25367 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:11:48.834 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.833+0000 7f7c4e000640 1 -- 172.21.15.123:0/594497763 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564fcac5a160 con 0x564fcad06800 2024-05-08T06:11:48.834 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.833+0000 7f7c4e000640 10 client.25367 handle_command_reply: tid=0 2024-05-08T06:11:48.834 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.833+0000 7f7c6dfb3740 1 client.25367 shutdown 2024-05-08T06:11:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.834+0000 7f7c6dfb3740 1 -- 172.21.15.123:0/594497763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564fcacf6690 msgr2=0x564fcad02120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.834+0000 7f7c6dfb3740 1 --2- 172.21.15.123:0/594497763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564fcacf6690 0x564fcad02120 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c0049d0 tx=0x7f7c5c0ca340 comp rx=0 tx=0).stop 2024-05-08T06:11:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.834+0000 7f7c6dfb3740 1 -- 172.21.15.123:0/594497763 shutdown_connections 2024-05-08T06:11:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.834+0000 7f7c6dfb3740 1 --2- 172.21.15.123:0/594497763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564fcacf6690 0x564fcad02120 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.834+0000 7f7c6dfb3740 1 --2- 172.21.15.123:0/594497763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fcacf1520 0x564fcacfb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.834+0000 7f7c6dfb3740 1 --2- 172.21.15.123:0/594497763 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x564fcad06800 0x564fcad08c20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.834+0000 7f7c6dfb3740 1 --2- 172.21.15.123:0/594497763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fcacf0b30 0x564fcacfaba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.834+0000 7f7c6dfb3740 1 -- 172.21.15.123:0/594497763 >> 172.21.15.123:0/594497763 conn(0x564fcacb0230 msgr2=0x564fcace2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.834+0000 7f7c4e000640 0 client.25367 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:11:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.837+0000 7f7c6dfb3740 1 -- 172.21.15.123:0/594497763 shutdown_connections 2024-05-08T06:11:48.838 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.837+0000 7f7c6dfb3740 1 -- 172.21.15.123:0/594497763 wait complete. 2024-05-08T06:11:48.838 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.837+0000 7f7c6dfb3740 20 client.25367 trim_cache size 0 max 16384 2024-05-08T06:11:48.869 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.868+0000 7f7c6dfb3740 1 Processor -- start 2024-05-08T06:11:48.869 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.868+0000 7f7c6dfb3740 1 -- start start 2024-05-08T06:11:48.869 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.868+0000 7f7c6dfb3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fcacf0b30 0x564fcacfa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:48.869 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.868+0000 7f7c6dfb3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564fcacf1520 0x564fcacf2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:48.869 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.868+0000 7f7c6dfb3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fcacf6690 0x564fcacf2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:48.869 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.868+0000 7f7c6dfb3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7c641aa230 con 0x564fcacf6690 2024-05-08T06:11:48.869 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.868+0000 7f7c6dfb3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7c641a7b00 con 0x564fcacf1520 2024-05-08T06:11:48.869 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.868+0000 7f7c6dfb3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7c641a5950 con 0x564fcacf0b30 2024-05-08T06:11:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.869+0000 7f7c63e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fcacf0b30 0x564fcacfa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.869+0000 7f7c63e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fcacf0b30 0x564fcacfa1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:50420/0 (socket says 172.21.15.123:50420) 2024-05-08T06:11:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.869+0000 7f7c63e00640 1 -- 172.21.15.123:0/380892725 learned_addr learned my addr 172.21.15.123:0/380892725 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.869+0000 7f7c63400640 1 --2- 172.21.15.123:0/380892725 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564fcacf1520 0x564fcacf2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.869+0000 7f7c69200640 1 --2- 172.21.15.123:0/380892725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fcacf6690 0x564fcacf2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.869+0000 7f7c63e00640 1 -- 172.21.15.123:0/380892725 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564fcacf1520 msgr2=0x564fcacf2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.869+0000 7f7c63e00640 1 --2- 172.21.15.123:0/380892725 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564fcacf1520 0x564fcacf2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.869+0000 7f7c63e00640 1 -- 172.21.15.123:0/380892725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fcacf6690 msgr2=0x564fcacf2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.869+0000 7f7c63e00640 1 --2- 172.21.15.123:0/380892725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fcacf6690 0x564fcacf2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.869+0000 7f7c63e00640 1 -- 172.21.15.123:0/380892725 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fcad08f90 con 0x564fcacf0b30 2024-05-08T06:11:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.869+0000 7f7c63e00640 1 --2- 172.21.15.123:0/380892725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fcacf0b30 0x564fcacfa1c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x564fcad09460 tx=0x7f7c54021040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f433256cc3170a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.870+0000 7f7c58c00640 1 -- 172.21.15.123:0/380892725 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5401ee60 con 0x564fcacf0b30 2024-05-08T06:11:48.872 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.870+0000 7f7c58c00640 1 -- 172.21.15.123:0/380892725 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c5402f070 con 0x564fcacf0b30 2024-05-08T06:11:48.872 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.870+0000 7f7c58c00640 1 -- 172.21.15.123:0/380892725 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5401db50 con 0x564fcacf0b30 2024-05-08T06:11:48.872 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.871+0000 7f7c6dfb3740 1 -- 172.21.15.123:0/380892725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fcacf0b30 msgr2=0x564fcacfa1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:48.872 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.871+0000 7f7c6dfb3740 1 --2- 172.21.15.123:0/380892725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fcacf0b30 0x564fcacfa1c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x564fcad09460 tx=0x7f7c54021040 comp rx=0 tx=0).stop 2024-05-08T06:11:48.872 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.871+0000 7f7c6dfb3740 1 -- 172.21.15.123:0/380892725 shutdown_connections 2024-05-08T06:11:48.872 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.871+0000 7f7c6dfb3740 1 --2- 172.21.15.123:0/380892725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fcacf6690 0x564fcacf2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.871+0000 7f7c6dfb3740 1 --2- 172.21.15.123:0/380892725 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564fcacf1520 0x564fcacf2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.871+0000 7f7c6dfb3740 1 --2- 172.21.15.123:0/380892725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fcacf0b30 0x564fcacfa1c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.871+0000 7f7c6dfb3740 1 -- 172.21.15.123:0/380892725 >> 172.21.15.123:0/380892725 conn(0x564fcacb0230 msgr2=0x564fcace6c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.871+0000 7f7c6dfb3740 1 -- 172.21.15.123:0/380892725 shutdown_connections 2024-05-08T06:11:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.871+0000 7f7c6dfb3740 1 -- 172.21.15.123:0/380892725 wait complete. 2024-05-08T06:11:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.872+0000 7f7c6dfb3740 1 -- start start 2024-05-08T06:11:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.872+0000 7f7c6dfb3740 1 Processor -- start 2024-05-08T06:11:48.874 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.873+0000 7f7c6dfb3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fcacf0b30 0x564fcacff440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.873+0000 7f7c6dfb3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fcacf1520 0x564fcacff980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.873+0000 7f7c6dfb3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564fcacf6690 0x564fcad008f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.873+0000 7f7c6dfb3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7c641a5fc0 con 0x564fcacf1520 2024-05-08T06:11:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.873+0000 7f7c6dfb3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7c641a7430 con 0x564fcacf6690 2024-05-08T06:11:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.873+0000 7f7c6dfb3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7c641a6690 con 0x564fcacf0b30 2024-05-08T06:11:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.873+0000 7f7c6dfb3740 10 client.0 _ll_register_callbacks cb 0x7f7c5c003ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:11:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.873+0000 7f7c63e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fcacf0b30 0x564fcacff440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.873+0000 7f7c63e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fcacf0b30 0x564fcacff440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:50432/0 (socket says 172.21.15.123:50432) 2024-05-08T06:11:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.873+0000 7f7c63e00640 1 -- 172.21.15.123:0/848087021 learned_addr learned my addr 172.21.15.123:0/848087021 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.874+0000 7f7c69200640 1 --2- 172.21.15.123:0/848087021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564fcacf6690 0x564fcad008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.874+0000 7f7c63400640 1 --2- 172.21.15.123:0/848087021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fcacf1520 0x564fcacff980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.874+0000 7f7c63e00640 1 -- 172.21.15.123:0/848087021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564fcacf6690 msgr2=0x564fcad008f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.874+0000 7f7c63e00640 1 --2- 172.21.15.123:0/848087021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564fcacf6690 0x564fcad008f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.874+0000 7f7c63e00640 1 -- 172.21.15.123:0/848087021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fcacf1520 msgr2=0x564fcacff980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:48.876 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.874+0000 7f7c63e00640 1 --2- 172.21.15.123:0/848087021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fcacf1520 0x564fcacff980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:48.876 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.874+0000 7f7c63e00640 1 -- 172.21.15.123:0/848087021 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fcad00ee0 con 0x564fcacf0b30 2024-05-08T06:11:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.874+0000 7f7c63e00640 1 --2- 172.21.15.123:0/848087021 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fcacf0b30 0x564fcacff440 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7c5400ba60 tx=0x7f7c5401e750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18ddc6bbacce0181 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.874+0000 7f7c4e000640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-05-08T06:11:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.875+0000 7f7c4e000640 1 -- 172.21.15.123:0/848087021 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5401d410 con 0x564fcacf0b30 2024-05-08T06:11:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.875+0000 7f7c4e000640 1 -- 172.21.15.123:0/848087021 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f7c5401d8d0 con 0x564fcacf0b30 2024-05-08T06:11:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.875+0000 7f7c6dfb3740 1 -- 172.21.15.123:0/848087021 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c5402f070 con 0x564fcacf0b30 2024-05-08T06:11:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.875+0000 7f7c4e000640 1 -- 172.21.15.123:0/848087021 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7c5401dad0 con 0x564fcacf0b30 2024-05-08T06:11:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.875+0000 7f7c6dfb3740 10 client.25363 fetch_fsmap learned FSMap version 24 2024-05-08T06:11:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.876+0000 7f7c6dfb3740 1 -- 172.21.15.123:0/848087021 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564fcace8530 con 0x564fcacf0b30 2024-05-08T06:11:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.876+0000 7f7c4e000640 1 -- 172.21.15.123:0/848087021 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c5402f070 con 0x564fcacf0b30 2024-05-08T06:11:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.876+0000 7f7c4e000640 1 -- 172.21.15.123:0/848087021 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f7c54047420 con 0x564fcacf0b30 2024-05-08T06:11:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.876+0000 7f7c6dfb3740 10 client.25363 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:11:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.876+0000 7f7c6dfb3740 10 client.25363 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T06:11:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.876+0000 7f7c6dfb3740 20 client.25363 populate_metadata read hostname 'smithi123' 2024-05-08T06:11:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.876+0000 7f7c6dfb3740 1 --2- 172.21.15.123:0/848087021 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x564fcace8e60 0x564fcaceb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.876+0000 7f7c6dfb3740 4 client.25363 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T06:11:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.876+0000 7f7c6dfb3740 1 -- 172.21.15.123:0/848087021 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564fcac53fd0 con 0x564fcace8e60 2024-05-08T06:11:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.877+0000 7f7c63400640 1 --2- 172.21.15.123:0/848087021 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x564fcace8e60 0x564fcaceb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.878+0000 7f7c63400640 1 --2- 172.21.15.123:0/848087021 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x564fcace8e60 0x564fcaceb280 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9333650a5991f712 server_cookie=8b525d3b6b87438 in_seq=0 out_seq=0 2024-05-08T06:11:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.878+0000 7f7c4e000640 10 client.25363 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:11:48.883 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.882+0000 7f7c4e000640 1 -- 172.21.15.123:0/848087021 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564fcac53fd0 con 0x564fcace8e60 2024-05-08T06:11:48.883 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.882+0000 7f7c4e000640 10 client.25363 handle_command_reply: tid=0 2024-05-08T06:11:48.883 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.883+0000 7f7c6dfb3740 1 client.25363 shutdown 2024-05-08T06:11:48.884 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.883+0000 7f7c6dfb3740 1 -- 172.21.15.123:0/848087021 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fcacf0b30 msgr2=0x564fcacff440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:48.884 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.883+0000 7f7c6dfb3740 1 --2- 172.21.15.123:0/848087021 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fcacf0b30 0x564fcacff440 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7c5400ba60 tx=0x7f7c5401e750 comp rx=0 tx=0).stop 2024-05-08T06:11:48.884 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.884+0000 7f7c6dfb3740 1 -- 172.21.15.123:0/848087021 shutdown_connections 2024-05-08T06:11:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.884+0000 7f7c6dfb3740 1 --2- 172.21.15.123:0/848087021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564fcacf6690 0x564fcad008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.884+0000 7f7c6dfb3740 1 --2- 172.21.15.123:0/848087021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fcacf1520 0x564fcacff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.884+0000 7f7c6dfb3740 1 --2- 172.21.15.123:0/848087021 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x564fcace8e60 0x564fcaceb280 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.884+0000 7f7c6dfb3740 1 --2- 172.21.15.123:0/848087021 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564fcacf0b30 0x564fcacff440 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.884+0000 7f7c6dfb3740 1 -- 172.21.15.123:0/848087021 >> 172.21.15.123:0/848087021 conn(0x564fcacb0230 msgr2=0x564fcacfaba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.884+0000 7f7c4e000640 0 client.25363 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:11:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.884+0000 7f7c6dfb3740 1 -- 172.21.15.123:0/848087021 shutdown_connections 2024-05-08T06:11:48.886 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.884+0000 7f7c6dfb3740 1 -- 172.21.15.123:0/848087021 wait complete. 2024-05-08T06:11:48.886 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.884+0000 7f7c6dfb3740 20 client.25363 trim_cache size 0 max 16384 2024-05-08T06:11:48.886 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout:{ 2024-05-08T06:11:48.886 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout: "message": "", 2024-05-08T06:11:48.886 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout: "return_code": 0 2024-05-08T06:11:48.886 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout:} 2024-05-08T06:11:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.885+0000 7f7c69c00640 1 -- 172.21.15.123:0/2846460195 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f7c480b2860 msgr2=0x7f7c480b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.885+0000 7f7c69c00640 1 --2- 172.21.15.123:0/2846460195 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f7c480b2860 0x7f7c480b4d20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x564fcacf1920 tx=0x7f7c5400b390 comp rx=0 tx=0).stop 2024-05-08T06:11:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.885+0000 7f7c69c00640 1 -- 172.21.15.123:0/2846460195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c641043f0 msgr2=0x7f7c640752e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.885+0000 7f7c69c00640 1 --2- 172.21.15.123:0/2846460195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c641043f0 0x7f7c640752e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c031a20 tx=0x7f7c5c0317c0 comp rx=0 tx=0).stop 2024-05-08T06:11:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.886+0000 7f7c69c00640 1 -- 172.21.15.123:0/2846460195 shutdown_connections 2024-05-08T06:11:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.886+0000 7f7c69c00640 1 --2- 172.21.15.123:0/2846460195 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f7c480b2860 0x7f7c480b4d20 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.886+0000 7f7c69c00640 1 --2- 172.21.15.123:0/2846460195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c641043f0 0x7f7c640752e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:48.888 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.886+0000 7f7c69c00640 1 --2- 172.21.15.123:0/2846460195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c64103a30 0x7f7c64074da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:48.888 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.886+0000 7f7c69c00640 1 --2- 172.21.15.123:0/2846460195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c64102840 0x7f7c64076760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:48.888 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.886+0000 7f7c69c00640 1 -- 172.21.15.123:0/2846460195 >> 172.21.15.123:0/2846460195 conn(0x7f7c640fddb0 msgr2=0x7f7c64106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:48.888 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.886+0000 7f7c69c00640 1 -- 172.21.15.123:0/2846460195 shutdown_connections 2024-05-08T06:11:48.888 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:11:48.886+0000 7f7c69c00640 1 -- 172.21.15.123:0/2846460195 wait complete. 2024-05-08T06:11:48.894 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-08T06:11:49.349 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:11:50.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:50 smithi123 ceph-mon[29787]: pgmap v501: 97 pgs: 97 active+clean; 675 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:11:50.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:50 smithi163 ceph-mon[34520]: pgmap v501: 97 pgs: 97 active+clean; 675 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:11:50.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:50 smithi192 ceph-mon[34664]: pgmap v501: 97 pgs: 97 active+clean; 675 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:11:51.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.808+0000 7f7fee200700 1 -- 172.21.15.123:0/2091019089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe810e7c0 msgr2=0x7f7fe811ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:51.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.808+0000 7f7fee200700 1 --2- 172.21.15.123:0/2091019089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe810e7c0 0x7f7fe811ac60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000b0a0 tx=0x7f7fe000b3b0 comp rx=0 tx=0).stop 2024-05-08T06:11:51.808 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.808+0000 7f7fee200700 1 -- 172.21.15.123:0/2091019089 shutdown_connections 2024-05-08T06:11:51.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.808+0000 7f7fee200700 1 --2- 172.21.15.123:0/2091019089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe810e7c0 0x7f7fe811ac60 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:51.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.808+0000 7f7fee200700 1 --2- 172.21.15.123:0/2091019089 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7fe8113eb0 0x7f7fe8114290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:51.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.808+0000 7f7fee200700 1 --2- 172.21.15.123:0/2091019089 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fe810de00 0x7f7fe810e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:51.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.808+0000 7f7fee200700 1 -- 172.21.15.123:0/2091019089 >> 172.21.15.123:0/2091019089 conn(0x7f7fe806e430 msgr2=0x7f7fe810c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:51.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.808+0000 7f7fee200700 1 -- 172.21.15.123:0/2091019089 shutdown_connections 2024-05-08T06:11:51.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.808+0000 7f7fee200700 1 -- 172.21.15.123:0/2091019089 wait complete. 2024-05-08T06:11:51.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.810+0000 7f7fee200700 1 Processor -- start 2024-05-08T06:11:51.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.810+0000 7f7fee200700 1 -- start start 2024-05-08T06:11:51.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.810+0000 7f7fee200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe810de00 0x7f7fe81a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:51.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.810+0000 7f7fee200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7fe810e7c0 0x7f7fe81a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:51.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.810+0000 7f7fee200700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fe8113eb0 0x7f7fe81a8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:51.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.810+0000 7f7fee200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe811dfc0 con 0x7f7fe810de00 2024-05-08T06:11:51.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.810+0000 7f7fee200700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe811de40 con 0x7f7fe810e7c0 2024-05-08T06:11:51.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.810+0000 7f7fee200700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe811e140 con 0x7f7fe8113eb0 2024-05-08T06:11:51.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.811+0000 7f7fe7e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7fe810e7c0 0x7f7fe81a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:51.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.811+0000 7f7fe7e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7fe810e7c0 0x7f7fe81a4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:44400/0 (socket says 172.21.15.123:44400) 2024-05-08T06:11:51.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.811+0000 7f7fe7e00700 1 -- 172.21.15.123:0/318152531 learned_addr learned my addr 172.21.15.123:0/318152531 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:51.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.811+0000 7f7fece00700 1 --2- 172.21.15.123:0/318152531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe810de00 0x7f7fe81a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:51.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.811+0000 7f7fed800700 1 --2- 172.21.15.123:0/318152531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fe8113eb0 0x7f7fe81a8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:51.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.811+0000 7f7fe7e00700 1 -- 172.21.15.123:0/318152531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fe8113eb0 msgr2=0x7f7fe81a8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:51.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.811+0000 7f7fe7e00700 1 --2- 172.21.15.123:0/318152531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fe8113eb0 0x7f7fe81a8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:51.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.811+0000 7f7fe7e00700 1 -- 172.21.15.123:0/318152531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe810de00 msgr2=0x7f7fe81a43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:51.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.812+0000 7f7fe7e00700 1 --2- 172.21.15.123:0/318152531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe810de00 0x7f7fe81a43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:51.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.812+0000 7f7fe7e00700 1 -- 172.21.15.123:0/318152531 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe81a9400 con 0x7f7fe810e7c0 2024-05-08T06:11:51.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.812+0000 7f7fe7e00700 1 --2- 172.21.15.123:0/318152531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7fe810e7c0 0x7f7fe81a4930 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7fd400e9a0 tx=0x7f7fd400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:51.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.812+0000 7f7fe5600700 1 -- 172.21.15.123:0/318152531 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd400cbd0 con 0x7f7fe810e7c0 2024-05-08T06:11:51.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.812+0000 7f7fe5600700 1 -- 172.21.15.123:0/318152531 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fd400cd70 con 0x7f7fe810e7c0 2024-05-08T06:11:51.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.812+0000 7f7fee200700 1 -- 172.21.15.123:0/318152531 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fe81a96f0 con 0x7f7fe810e7c0 2024-05-08T06:11:51.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.812+0000 7f7fe5600700 1 -- 172.21.15.123:0/318152531 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4018680 con 0x7f7fe810e7c0 2024-05-08T06:11:51.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.812+0000 7f7fee200700 1 -- 172.21.15.123:0/318152531 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fe81b0f70 con 0x7f7fe810e7c0 2024-05-08T06:11:51.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.817+0000 7f7ffa1a0b80 1 Processor -- start 2024-05-08T06:11:51.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.817+0000 7f7ffa1a0b80 1 -- start start 2024-05-08T06:11:51.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.817+0000 7f7ffa1a0b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6eeff41d0 0x55d6eeff45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:51.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.817+0000 7f7ffa1a0b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6eeff4af0 0x55d6eefee7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:51.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.817+0000 7f7ffa1a0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6eefeee10 0x55d6eefef270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:51.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.817+0000 7f7ffa1a0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d6eef38a20 con 0x55d6eefeee10 2024-05-08T06:11:51.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.817+0000 7f7ffa1a0b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55d6eeed6760 con 0x55d6eeff41d0 2024-05-08T06:11:51.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.817+0000 7f7ffa1a0b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55d6eef349e0 con 0x55d6eeff4af0 2024-05-08T06:11:51.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.818+0000 7f7fed800700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6eeff41d0 0x55d6eeff45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:51.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.818+0000 7f7fed800700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6eeff41d0 0x55d6eeff45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:44410/0 (socket says 172.21.15.123:44410) 2024-05-08T06:11:51.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.818+0000 7f7fed800700 1 -- 172.21.15.123:0/624056106 learned_addr learned my addr 172.21.15.123:0/624056106 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:51.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.818+0000 7f7fe5600700 1 -- 172.21.15.123:0/318152531 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7f7fd4010430 con 0x7f7fe810e7c0 2024-05-08T06:11:51.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.818+0000 7f7fece00700 1 --2- 172.21.15.123:0/624056106 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6eeff4af0 0x55d6eefee7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:51.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.819+0000 7f7fece00700 1 -- 172.21.15.123:0/624056106 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6eeff41d0 msgr2=0x55d6eeff45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:51.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.819+0000 7f7fece00700 1 --2- 172.21.15.123:0/624056106 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6eeff41d0 0x55d6eeff45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:51.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.819+0000 7f7fece00700 1 -- 172.21.15.123:0/624056106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6eefeee10 msgr2=0x55d6eefef270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:51.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.819+0000 7f7fece00700 1 --2- 172.21.15.123:0/624056106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6eefeee10 0x55d6eefef270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:51.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.819+0000 7f7fece00700 1 -- 172.21.15.123:0/624056106 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6eee858d0 con 0x55d6eeff4af0 2024-05-08T06:11:51.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.819+0000 7f7fed800700 1 --2- 172.21.15.123:0/624056106 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6eeff41d0 0x55d6eeff45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:11:51.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.820+0000 7f7fece00700 1 --2- 172.21.15.123:0/624056106 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6eeff4af0 0x55d6eefee7f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc009cb0 tx=0x7f7fdc011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55eda422f1b0b9db server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:51.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.820+0000 7f7fe5600700 1 --2- 172.21.15.123:0/318152531 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f7fd00b2880 0x7f7fd00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:51.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.820+0000 7f7fe5600700 1 -- 172.21.15.123:0/318152531 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f7fd40e24c0 con 0x7f7fe810e7c0 2024-05-08T06:11:51.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.820+0000 7f7fda000700 1 -- 172.21.15.123:0/624056106 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc011d20 con 0x55d6eeff4af0 2024-05-08T06:11:51.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.820+0000 7f7fed800700 1 --2- 172.21.15.123:0/318152531 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f7fd00b2880 0x7f7fd00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:51.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.821+0000 7f7fda000700 1 -- 172.21.15.123:0/624056106 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fdc0044e0 con 0x55d6eeff4af0 2024-05-08T06:11:51.821 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.821+0000 7f7fda000700 1 -- 172.21.15.123:0/624056106 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc005280 con 0x55d6eeff4af0 2024-05-08T06:11:51.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.822+0000 7f7ffa1a0b80 1 -- 172.21.15.123:0/624056106 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6eeff4af0 msgr2=0x55d6eefee7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:51.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.822+0000 7f7ffa1a0b80 1 --2- 172.21.15.123:0/624056106 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6eeff4af0 0x55d6eefee7f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc009cb0 tx=0x7f7fdc011440 comp rx=0 tx=0).stop 2024-05-08T06:11:51.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.822+0000 7f7fed800700 1 --2- 172.21.15.123:0/318152531 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f7fd00b2880 0x7f7fd00b4d40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0009850 tx=0x7f7fe001d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:51.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.822+0000 7f7ffa1a0b80 1 -- 172.21.15.123:0/624056106 shutdown_connections 2024-05-08T06:11:51.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.822+0000 7f7ffa1a0b80 1 --2- 172.21.15.123:0/624056106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6eefeee10 0x55d6eefef270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:51.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.822+0000 7f7ffa1a0b80 1 --2- 172.21.15.123:0/624056106 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6eeff41d0 0x55d6eeff45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:51.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.822+0000 7f7ffa1a0b80 1 --2- 172.21.15.123:0/624056106 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6eeff4af0 0x55d6eefee7f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:51.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.822+0000 7f7ffa1a0b80 1 -- 172.21.15.123:0/624056106 >> 172.21.15.123:0/624056106 conn(0x55d6eef14b60 msgr2=0x55d6eefe09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:51.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.822+0000 7f7ffa1a0b80 1 -- 172.21.15.123:0/624056106 shutdown_connections 2024-05-08T06:11:51.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.822+0000 7f7ffa1a0b80 1 -- 172.21.15.123:0/624056106 wait complete. 2024-05-08T06:11:51.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.823+0000 7f7ffa1a0b80 1 -- start start 2024-05-08T06:11:51.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.823+0000 7f7ffa1a0b80 1 Processor -- start 2024-05-08T06:11:51.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.823+0000 7f7ffa1a0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6eefeee10 0x55d6eeffa740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:51.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.823+0000 7f7ffa1a0b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6eeff41d0 0x55d6eeff7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:51.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.823+0000 7f7ffa1a0b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6eeff4af0 0x55d6eeff8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:51.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.823+0000 7f7ffa1a0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d6eeffb5f0 con 0x55d6eefeee10 2024-05-08T06:11:51.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.823+0000 7f7ffa1a0b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55d6eee9eb20 con 0x55d6eeff4af0 2024-05-08T06:11:51.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.823+0000 7f7ffa1a0b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55d6eeffb770 con 0x55d6eeff41d0 2024-05-08T06:11:51.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.824+0000 7f7ffa1a0b80 10 client.0 _ll_register_callbacks cb 0x55d6eeedfc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:11:51.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.824+0000 7f7fe7e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6eeff41d0 0x55d6eeff7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:51.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.824+0000 7f7fe7e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6eeff41d0 0x55d6eeff7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:50478/0 (socket says 172.21.15.123:50478) 2024-05-08T06:11:51.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.824+0000 7f7fe7e00700 1 -- 172.21.15.123:0/3142699077 learned_addr learned my addr 172.21.15.123:0/3142699077 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:51.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.824+0000 7f7fece00700 1 --2- 172.21.15.123:0/3142699077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6eefeee10 0x55d6eeffa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:51.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.824+0000 7f7fe7e00700 1 -- 172.21.15.123:0/3142699077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6eeff4af0 msgr2=0x55d6eeff8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:51.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.824+0000 7f7fe7e00700 1 --2- 172.21.15.123:0/3142699077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6eeff4af0 0x55d6eeff8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:51.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.824+0000 7f7fe7e00700 1 -- 172.21.15.123:0/3142699077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6eefeee10 msgr2=0x55d6eeffa740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:51.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.824+0000 7f7fe7e00700 1 --2- 172.21.15.123:0/3142699077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6eefeee10 0x55d6eeffa740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:51.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.824+0000 7f7fe7e00700 1 -- 172.21.15.123:0/3142699077 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6eeefe420 con 0x55d6eeff41d0 2024-05-08T06:11:51.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.824+0000 7f7fece00700 1 --2- 172.21.15.123:0/3142699077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6eefeee10 0x55d6eeffa740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:11:51.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.825+0000 7f7fe7e00700 1 --2- 172.21.15.123:0/3142699077 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6eeff41d0 0x55d6eeff7ae0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7fd40a9790 tx=0x7f7fd40a9330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a67c8fa55d85aca8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:51.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.825+0000 7f7fcea00700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-05-08T06:11:51.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.825+0000 7f7fcea00700 1 -- 172.21.15.123:0/3142699077 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd40a95a0 con 0x55d6eeff41d0 2024-05-08T06:11:51.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.825+0000 7f7fcea00700 1 -- 172.21.15.123:0/3142699077 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f7fd4010720 con 0x55d6eeff41d0 2024-05-08T06:11:51.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.825+0000 7f7ffa1a0b80 1 -- 172.21.15.123:0/3142699077 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fdc0044e0 con 0x55d6eeff41d0 2024-05-08T06:11:51.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.825+0000 7f7fcea00700 1 -- 172.21.15.123:0/3142699077 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7fd40aa190 con 0x55d6eeff41d0 2024-05-08T06:11:51.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.825+0000 7f7ffa1a0b80 10 client.25375 fetch_fsmap learned FSMap version 24 2024-05-08T06:11:51.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.825+0000 7f7ffa1a0b80 1 -- 172.21.15.123:0/3142699077 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d6ef005d70 con 0x55d6eeff41d0 2024-05-08T06:11:51.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.826+0000 7f7fcea00700 1 -- 172.21.15.123:0/3142699077 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fdc0044e0 con 0x55d6eeff41d0 2024-05-08T06:11:51.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.826+0000 7f7fcea00700 1 -- 172.21.15.123:0/3142699077 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f7fd40cc0b0 con 0x55d6eeff41d0 2024-05-08T06:11:51.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.826+0000 7f7ffa1a0b80 10 client.25375 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:11:51.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.826+0000 7f7ffa1a0b80 10 client.25375 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-08T06:11:51.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.826+0000 7f7ffa1a0b80 20 client.25375 populate_metadata read hostname 'smithi123' 2024-05-08T06:11:51.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.826+0000 7f7ffa1a0b80 1 --2- 172.21.15.123:0/3142699077 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x55d6ef008240 0x55d6ef00a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:51.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.826+0000 7f7ffa1a0b80 4 client.25375 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:11:51.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.826+0000 7f7ffa1a0b80 1 -- 172.21.15.123:0/3142699077 --> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6eefb30a0 con 0x55d6ef008240 2024-05-08T06:11:51.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.827+0000 7f7fece00700 1 --2- 172.21.15.123:0/3142699077 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x55d6ef008240 0x55d6ef00a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:51.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.827+0000 7f7fcea00700 10 client.25375 ms_handle_connect on v2:172.21.15.123:6840/1571726231 2024-05-08T06:11:51.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.827+0000 7f7fece00700 1 --2- 172.21.15.123:0/3142699077 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x55d6ef008240 0x55d6ef00a630 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7819883f481ba44b server_cookie=655427a85aa01ebe in_seq=0 out_seq=0 2024-05-08T06:11:51.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.828+0000 7f7fcea00700 1 -- 172.21.15.123:0/3142699077 <== mds.1 v2:172.21.15.123:6840/1571726231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d6eefb30a0 con 0x55d6ef008240 2024-05-08T06:11:51.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.828+0000 7f7fcea00700 10 client.25375 handle_command_reply: tid=0 2024-05-08T06:11:51.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.829+0000 7f7ffa1a0b80 1 client.25375 shutdown 2024-05-08T06:11:51.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.829+0000 7f7ffa1a0b80 1 -- 172.21.15.123:0/3142699077 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6eeff41d0 msgr2=0x55d6eeff7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:51.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.829+0000 7f7ffa1a0b80 1 --2- 172.21.15.123:0/3142699077 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6eeff41d0 0x55d6eeff7ae0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7fd40a9790 tx=0x7f7fd40a9330 comp rx=0 tx=0).stop 2024-05-08T06:11:51.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.829+0000 7f7ffa1a0b80 1 -- 172.21.15.123:0/3142699077 shutdown_connections 2024-05-08T06:11:51.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.829+0000 7f7ffa1a0b80 1 --2- 172.21.15.123:0/3142699077 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x55d6ef008240 0x55d6ef00a630 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:51.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.829+0000 7f7fcea00700 0 client.25375 ms_handle_reset on v2:172.21.15.123:6840/1571726231 2024-05-08T06:11:51.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.829+0000 7f7ffa1a0b80 1 --2- 172.21.15.123:0/3142699077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6eefeee10 0x55d6eeffa740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:51.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.829+0000 7f7ffa1a0b80 1 --2- 172.21.15.123:0/3142699077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6eeff4af0 0x55d6eeff8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:51.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.829+0000 7f7ffa1a0b80 1 --2- 172.21.15.123:0/3142699077 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6eeff41d0 0x55d6eeff7ae0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:51.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.829+0000 7f7ffa1a0b80 1 -- 172.21.15.123:0/3142699077 >> 172.21.15.123:0/3142699077 conn(0x55d6eef14b60 msgr2=0x55d6eefdb460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:51.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.829+0000 7f7ffa1a0b80 1 -- 172.21.15.123:0/3142699077 shutdown_connections 2024-05-08T06:11:51.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.830+0000 7f7ffa1a0b80 1 -- 172.21.15.123:0/3142699077 wait complete. 2024-05-08T06:11:51.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.830+0000 7f7ffa1a0b80 20 client.25375 trim_cache size 0 max 16384 2024-05-08T06:11:51.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.860+0000 7f7ffa1a0b80 1 Processor -- start 2024-05-08T06:11:51.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.860+0000 7f7ffa1a0b80 1 -- start start 2024-05-08T06:11:51.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.860+0000 7f7ffa1a0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6eefeee10 0x55d6eeffa740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:51.860 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.860+0000 7f7ffa1a0b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6eeff41d0 0x55d6ef00feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:51.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.860+0000 7f7ffa1a0b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6eeff4af0 0x55d6ef0103f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:51.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.860+0000 7f7ffa1a0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe81b69a0 con 0x55d6eefeee10 2024-05-08T06:11:51.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.860+0000 7f7ffa1a0b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe81b4230 con 0x55d6eeff41d0 2024-05-08T06:11:51.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.860+0000 7f7ffa1a0b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe8072f60 con 0x55d6eeff4af0 2024-05-08T06:11:51.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.861+0000 7f7fece00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6eefeee10 0x55d6eeffa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:51.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.861+0000 7f7fece00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6eefeee10 0x55d6eeffa740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53516/0 (socket says 172.21.15.123:53516) 2024-05-08T06:11:51.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.861+0000 7f7fece00700 1 -- 172.21.15.123:0/273606375 learned_addr learned my addr 172.21.15.123:0/273606375 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:51.861 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.861+0000 7f7fe7e00700 1 --2- 172.21.15.123:0/273606375 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6eeff41d0 0x55d6ef00feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:51.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.861+0000 7f7fece00700 1 -- 172.21.15.123:0/273606375 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6eeff4af0 msgr2=0x55d6ef0103f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T06:11:51.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.861+0000 7f7fece00700 1 --2- 172.21.15.123:0/273606375 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6eeff4af0 0x55d6ef0103f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:51.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.861+0000 7f7fece00700 1 -- 172.21.15.123:0/273606375 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6eeff41d0 msgr2=0x55d6ef00feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:51.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.861+0000 7f7fece00700 1 --2- 172.21.15.123:0/273606375 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6eeff41d0 0x55d6ef00feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:51.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.861+0000 7f7fece00700 1 -- 172.21.15.123:0/273606375 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6eeffa380 con 0x55d6eefeee10 2024-05-08T06:11:51.862 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.862+0000 7f7fe7e00700 1 --2- 172.21.15.123:0/273606375 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6eeff41d0 0x55d6ef00feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:11:51.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.862+0000 7f7fece00700 1 --2- 172.21.15.123:0/273606375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6eefeee10 0x55d6eeffa740 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc000c00 tx=0x7f7fdc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d2abd39cd54adc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:51.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.862+0000 7f7fd9600700 1 -- 172.21.15.123:0/273606375 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc029030 con 0x55d6eefeee10 2024-05-08T06:11:51.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.862+0000 7f7fd9600700 1 -- 172.21.15.123:0/273606375 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fdc005ce0 con 0x55d6eefeee10 2024-05-08T06:11:51.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.862+0000 7f7fd9600700 1 -- 172.21.15.123:0/273606375 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc005490 con 0x55d6eefeee10 2024-05-08T06:11:51.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.863+0000 7f7ffa1a0b80 1 -- 172.21.15.123:0/273606375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6eefeee10 msgr2=0x55d6eeffa740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:51.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.863+0000 7f7ffa1a0b80 1 --2- 172.21.15.123:0/273606375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6eefeee10 0x55d6eeffa740 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc000c00 tx=0x7f7fdc009bb0 comp rx=0 tx=0).stop 2024-05-08T06:11:51.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.863+0000 7f7ffa1a0b80 1 -- 172.21.15.123:0/273606375 shutdown_connections 2024-05-08T06:11:51.863 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.863+0000 7f7ffa1a0b80 1 --2- 172.21.15.123:0/273606375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6eefeee10 0x55d6eeffa740 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:51.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.863+0000 7f7ffa1a0b80 1 --2- 172.21.15.123:0/273606375 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6eeff4af0 0x55d6ef0103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:51.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.863+0000 7f7ffa1a0b80 1 --2- 172.21.15.123:0/273606375 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6eeff41d0 0x55d6ef00feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:51.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.863+0000 7f7ffa1a0b80 1 -- 172.21.15.123:0/273606375 >> 172.21.15.123:0/273606375 conn(0x55d6eef14b60 msgr2=0x55d6eefda400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:51.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.864+0000 7f7ffa1a0b80 1 -- 172.21.15.123:0/273606375 shutdown_connections 2024-05-08T06:11:51.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.864+0000 7f7ffa1a0b80 1 -- 172.21.15.123:0/273606375 wait complete. 2024-05-08T06:11:51.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.864+0000 7f7ffa1a0b80 1 -- start start 2024-05-08T06:11:51.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.864+0000 7f7ffa1a0b80 1 Processor -- start 2024-05-08T06:11:51.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.865+0000 7f7ffa1a0b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6eefeee10 0x55d6ef000fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:51.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.865+0000 7f7ffa1a0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6eeff41d0 0x55d6ef00f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:51.865 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.865+0000 7f7ffa1a0b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6eeff4af0 0x55d6ef00f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:51.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.865+0000 7f7ffa1a0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe81b26f0 con 0x55d6eeff41d0 2024-05-08T06:11:51.866 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.865+0000 7f7ffa1a0b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe81b3b60 con 0x55d6eefeee10 2024-05-08T06:11:51.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.865+0000 7f7ffa1a0b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe81b2dc0 con 0x55d6eeff4af0 2024-05-08T06:11:51.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.865+0000 7f7ffa1a0b80 10 client.0 _ll_register_callbacks cb 0x55d6eeedfc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:11:51.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.866+0000 7f7fece00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6eefeee10 0x55d6ef000fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:51.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.866+0000 7f7fe7e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6eeff41d0 0x55d6ef00f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:51.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.866+0000 7f7fece00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6eefeee10 0x55d6ef000fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:44438/0 (socket says 172.21.15.123:44438) 2024-05-08T06:11:51.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.866+0000 7f7fece00700 1 -- 172.21.15.123:0/721645102 learned_addr learned my addr 172.21.15.123:0/721645102 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:51.867 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.866+0000 7f7fed800700 1 --2- 172.21.15.123:0/721645102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6eeff4af0 0x55d6ef00f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:51.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.866+0000 7f7fece00700 1 -- 172.21.15.123:0/721645102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6eeff4af0 msgr2=0x55d6ef00f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:51.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.866+0000 7f7fece00700 1 --2- 172.21.15.123:0/721645102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6eeff4af0 0x55d6ef00f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:51.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.866+0000 7f7fece00700 1 -- 172.21.15.123:0/721645102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6eeff41d0 msgr2=0x55d6ef00f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:51.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.866+0000 7f7fece00700 1 --2- 172.21.15.123:0/721645102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6eeff41d0 0x55d6ef00f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:51.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.866+0000 7f7fece00700 1 -- 172.21.15.123:0/721645102 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6ef010210 con 0x55d6eefeee10 2024-05-08T06:11:51.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.866+0000 7f7fece00700 1 --2- 172.21.15.123:0/721645102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6eefeee10 0x55d6ef000fe0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc0097c0 tx=0x7f7fdc004fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e619b059a8ae8488 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:51.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.866+0000 7f7fcea00700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-05-08T06:11:51.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.867+0000 7f7fcea00700 1 -- 172.21.15.123:0/721645102 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc011020 con 0x55d6eefeee10 2024-05-08T06:11:51.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.867+0000 7f7ffa1a0b80 1 -- 172.21.15.123:0/721645102 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fdc005ce0 con 0x55d6eefeee10 2024-05-08T06:11:51.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.867+0000 7f7fcea00700 1 -- 172.21.15.123:0/721645102 <== mon.2 v2:172.21.15.163:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f7fdc0111c0 con 0x55d6eefeee10 2024-05-08T06:11:51.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.867+0000 7f7fcea00700 1 -- 172.21.15.123:0/721645102 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7fdc005110 con 0x55d6eefeee10 2024-05-08T06:11:51.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.867+0000 7f7ffa1a0b80 10 client.25388 fetch_fsmap learned FSMap version 24 2024-05-08T06:11:51.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.867+0000 7f7ffa1a0b80 1 -- 172.21.15.123:0/721645102 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d6ef072ae0 con 0x55d6eefeee10 2024-05-08T06:11:51.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.868+0000 7f7fcea00700 1 -- 172.21.15.123:0/721645102 <== mon.2 v2:172.21.15.163:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fdc005ce0 con 0x55d6eefeee10 2024-05-08T06:11:51.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.869+0000 7f7fcea00700 1 -- 172.21.15.123:0/721645102 <== mon.2 v2:172.21.15.163:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f7fdc018070 con 0x55d6eefeee10 2024-05-08T06:11:51.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.869+0000 7f7ffa1a0b80 10 client.25388 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:11:51.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.869+0000 7f7ffa1a0b80 10 client.25388 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-08T06:11:51.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.869+0000 7f7ffa1a0b80 20 client.25388 populate_metadata read hostname 'smithi123' 2024-05-08T06:11:51.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.869+0000 7f7ffa1a0b80 1 --2- 172.21.15.123:0/721645102 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x55d6ef008240 0x55d6ef074f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:51.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.869+0000 7f7ffa1a0b80 4 client.25388 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T06:11:51.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.869+0000 7f7ffa1a0b80 1 -- 172.21.15.123:0/721645102 --> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d6eef46400 con 0x55d6ef008240 2024-05-08T06:11:51.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.870+0000 7f7fe7e00700 1 --2- 172.21.15.123:0/721645102 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x55d6ef008240 0x55d6ef074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:51.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.870+0000 7f7fe7e00700 1 --2- 172.21.15.123:0/721645102 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x55d6ef008240 0x55d6ef074f20 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=62104fd93194cc8f server_cookie=466e326a3fa81f7b in_seq=0 out_seq=0 2024-05-08T06:11:51.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.870+0000 7f7fcea00700 10 client.25388 ms_handle_connect on v2:172.21.15.123:6840/1571726231 2024-05-08T06:11:51.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.871+0000 7f7fcea00700 1 -- 172.21.15.123:0/721645102 <== mds.1 v2:172.21.15.123:6840/1571726231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d6eef46400 con 0x55d6ef008240 2024-05-08T06:11:51.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.871+0000 7f7fcea00700 10 client.25388 handle_command_reply: tid=0 2024-05-08T06:11:51.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.871+0000 7f7ffa1a0b80 1 client.25388 shutdown 2024-05-08T06:11:51.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.871+0000 7f7ffa1a0b80 1 -- 172.21.15.123:0/721645102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6eefeee10 msgr2=0x55d6ef000fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:51.872 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.871+0000 7f7ffa1a0b80 1 --2- 172.21.15.123:0/721645102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6eefeee10 0x55d6ef000fe0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc0097c0 tx=0x7f7fdc004fc0 comp rx=0 tx=0).stop 2024-05-08T06:11:51.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.871+0000 7f7ffa1a0b80 1 -- 172.21.15.123:0/721645102 shutdown_connections 2024-05-08T06:11:51.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.871+0000 7f7ffa1a0b80 1 --2- 172.21.15.123:0/721645102 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x55d6ef008240 0x55d6ef074f20 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:51.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.871+0000 7f7ffa1a0b80 1 --2- 172.21.15.123:0/721645102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d6eeff4af0 0x55d6ef00f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:51.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.871+0000 7f7ffa1a0b80 1 --2- 172.21.15.123:0/721645102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d6eefeee10 0x55d6ef000fe0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:51.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.871+0000 7f7ffa1a0b80 1 --2- 172.21.15.123:0/721645102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6eeff41d0 0x55d6ef00f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:51.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.871+0000 7f7fcea00700 0 client.25388 ms_handle_reset on v2:172.21.15.123:6840/1571726231 2024-05-08T06:11:51.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.871+0000 7f7ffa1a0b80 1 -- 172.21.15.123:0/721645102 >> 172.21.15.123:0/721645102 conn(0x55d6eef14b60 msgr2=0x55d6eeff4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:51.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.871+0000 7f7ffa1a0b80 1 -- 172.21.15.123:0/721645102 shutdown_connections 2024-05-08T06:11:51.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.872+0000 7f7ffa1a0b80 1 -- 172.21.15.123:0/721645102 wait complete. 2024-05-08T06:11:51.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.872+0000 7f7ffa1a0b80 20 client.25388 trim_cache size 0 max 16384 2024-05-08T06:11:51.873 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-05-08T06:11:51.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.873+0000 7f7fee200700 1 -- 172.21.15.123:0/318152531 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f7fd00b2880 msgr2=0x7f7fd00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:51.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.873+0000 7f7fee200700 1 --2- 172.21.15.123:0/318152531 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f7fd00b2880 0x7f7fd00b4d40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0009850 tx=0x7f7fe001d040 comp rx=0 tx=0).stop 2024-05-08T06:11:51.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.873+0000 7f7fee200700 1 -- 172.21.15.123:0/318152531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7fe810e7c0 msgr2=0x7f7fe81a4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:51.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.873+0000 7f7fee200700 1 --2- 172.21.15.123:0/318152531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7fe810e7c0 0x7f7fe81a4930 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7fd400e9a0 tx=0x7f7fd400ecb0 comp rx=0 tx=0).stop 2024-05-08T06:11:51.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.873+0000 7f7fee200700 1 -- 172.21.15.123:0/318152531 shutdown_connections 2024-05-08T06:11:51.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.873+0000 7f7fee200700 1 --2- 172.21.15.123:0/318152531 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f7fd00b2880 0x7f7fd00b4d40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:51.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.873+0000 7f7fee200700 1 --2- 172.21.15.123:0/318152531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe810de00 0x7f7fe81a43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:51.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.873+0000 7f7fee200700 1 --2- 172.21.15.123:0/318152531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fe8113eb0 0x7f7fe81a8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:51.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.873+0000 7f7fee200700 1 --2- 172.21.15.123:0/318152531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7fe810e7c0 0x7f7fe81a4930 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:51.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.873+0000 7f7fee200700 1 -- 172.21.15.123:0/318152531 >> 172.21.15.123:0/318152531 conn(0x7f7fe806e430 msgr2=0x7f7fe810ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:51.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.874+0000 7f7fee200700 1 -- 172.21.15.123:0/318152531 shutdown_connections 2024-05-08T06:11:51.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:51.874+0000 7f7fee200700 1 -- 172.21.15.123:0/318152531 wait complete. 2024-05-08T06:11:52.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:52 smithi123 ceph-mon[29787]: pgmap v502: 97 pgs: 97 active+clean; 675 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:11:52.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:52 smithi163 ceph-mon[34520]: pgmap v502: 97 pgs: 97 active+clean; 675 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:11:52.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:52 smithi192 ceph-mon[34664]: pgmap v502: 97 pgs: 97 active+clean; 675 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:11:52.985 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-08T06:11:53.410 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:11:54.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:54 smithi163 ceph-mon[34520]: pgmap v503: 97 pgs: 97 active+clean; 675 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-08T06:11:54.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:54 smithi192 ceph-mon[34664]: pgmap v503: 97 pgs: 97 active+clean; 675 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-08T06:11:54.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:54 smithi123 ceph-mon[29787]: pgmap v503: 97 pgs: 97 active+clean; 675 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-08T06:11:55.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.973+0000 7f0038e00700 1 -- 172.21.15.123:0/2043793125 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0034109460 msgr2=0x7f0034109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:55.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.973+0000 7f0038e00700 1 --2- 172.21.15.123:0/2043793125 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0034109460 0x7f0034109840 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0024009990 tx=0x7f0024009ca0 comp rx=0 tx=0).stop 2024-05-08T06:11:55.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.973+0000 7f0038e00700 1 -- 172.21.15.123:0/2043793125 shutdown_connections 2024-05-08T06:11:55.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.973+0000 7f0038e00700 1 --2- 172.21.15.123:0/2043793125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0034103d90 0x7f00341102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:55.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.973+0000 7f0038e00700 1 --2- 172.21.15.123:0/2043793125 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0034109460 0x7f0034109840 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:55.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.973+0000 7f0038e00700 1 --2- 172.21.15.123:0/2043793125 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00341033d0 0x7f0034103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:55.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.973+0000 7f0038e00700 1 -- 172.21.15.123:0/2043793125 >> 172.21.15.123:0/2043793125 conn(0x7f00340fef20 msgr2=0x7f0034101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:55.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.974+0000 7f0038e00700 1 -- 172.21.15.123:0/2043793125 shutdown_connections 2024-05-08T06:11:55.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.974+0000 7f0038e00700 1 -- 172.21.15.123:0/2043793125 wait complete. 2024-05-08T06:11:55.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.975+0000 7f0038e00700 1 Processor -- start 2024-05-08T06:11:55.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.975+0000 7f0038e00700 1 -- start start 2024-05-08T06:11:55.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.976+0000 7f0038e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00341033d0 0x7f003410f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:55.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.976+0000 7f0038e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0034103d90 0x7f003410fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:55.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.976+0000 7f0038e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0034109460 0x7f00341a4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:55.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.976+0000 7f0038e00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0034113570 con 0x7f0034103d90 2024-05-08T06:11:55.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.976+0000 7f0038e00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f00341133f0 con 0x7f00341033d0 2024-05-08T06:11:55.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.976+0000 7f0038e00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f00341136f0 con 0x7f0034109460 2024-05-08T06:11:55.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.976+0000 7f0033400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00341033d0 0x7f003410f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:55.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.976+0000 7f0032a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0034103d90 0x7f003410fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:55.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.977+0000 7f0033400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00341033d0 0x7f003410f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:57254/0 (socket says 172.21.15.123:57254) 2024-05-08T06:11:55.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.977+0000 7f0033400700 1 -- 172.21.15.123:0/294515168 learned_addr learned my addr 172.21.15.123:0/294515168 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:55.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.977+0000 7f0033e00700 1 --2- 172.21.15.123:0/294515168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0034109460 0x7f00341a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:55.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.977+0000 7f0033e00700 1 -- 172.21.15.123:0/294515168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00341033d0 msgr2=0x7f003410f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:55.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.977+0000 7f0033e00700 1 --2- 172.21.15.123:0/294515168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00341033d0 0x7f003410f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:55.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.977+0000 7f0033e00700 1 -- 172.21.15.123:0/294515168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0034103d90 msgr2=0x7f003410fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:55.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.977+0000 7f0033e00700 1 --2- 172.21.15.123:0/294515168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0034103d90 0x7f003410fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:55.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.977+0000 7f0033e00700 1 -- 172.21.15.123:0/294515168 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00341a5030 con 0x7f0034109460 2024-05-08T06:11:55.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.978+0000 7f0032a00700 1 --2- 172.21.15.123:0/294515168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0034103d90 0x7f003410fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:11:55.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.978+0000 7f0033400700 1 --2- 172.21.15.123:0/294515168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00341033d0 0x7f003410f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:11:55.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.978+0000 7f0033e00700 1 --2- 172.21.15.123:0/294515168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0034109460 0x7f00341a4920 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f002c00efc0 tx=0x7f002c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:55.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.978+0000 7f002be00700 1 -- 172.21.15.123:0/294515168 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c0092e0 con 0x7f0034109460 2024-05-08T06:11:55.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.978+0000 7f0038e00700 1 -- 172.21.15.123:0/294515168 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00341a52c0 con 0x7f0034109460 2024-05-08T06:11:55.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.979+0000 7f0038e00700 1 -- 172.21.15.123:0/294515168 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00341acbd0 con 0x7f0034109460 2024-05-08T06:11:55.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.979+0000 7f002be00700 1 -- 172.21.15.123:0/294515168 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f002c004bb0 con 0x7f0034109460 2024-05-08T06:11:55.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.979+0000 7f002be00700 1 -- 172.21.15.123:0/294515168 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c004020 con 0x7f0034109460 2024-05-08T06:11:55.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.984+0000 7f002be00700 1 -- 172.21.15.123:0/294515168 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7f002c01b030 con 0x7f0034109460 2024-05-08T06:11:55.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.985+0000 7f0044cb2b80 1 Processor -- start 2024-05-08T06:11:55.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.986+0000 7f0044cb2b80 1 -- start start 2024-05-08T06:11:55.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.986+0000 7f0044cb2b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5626cb172da0 0x5626cb173180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:55.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.986+0000 7f0044cb2b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626cb1736c0 0x5626cb16d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:55.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.986+0000 7f0044cb2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626cb16da50 0x5626cb16ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:55.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.986+0000 7f0044cb2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5626cb0bfa20 con 0x5626cb16da50 2024-05-08T06:11:55.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.986+0000 7f0044cb2b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5626cb05d760 con 0x5626cb1736c0 2024-05-08T06:11:55.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.986+0000 7f0044cb2b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5626cb0bb9e0 con 0x5626cb172da0 2024-05-08T06:11:55.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.987+0000 7f0032a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5626cb172da0 0x5626cb173180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:55.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.987+0000 7f0032a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5626cb172da0 0x5626cb173180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:53146/0 (socket says 172.21.15.123:53146) 2024-05-08T06:11:55.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.987+0000 7f0032a00700 1 -- 172.21.15.123:0/2733257430 learned_addr learned my addr 172.21.15.123:0/2733257430 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:55.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.987+0000 7f0033400700 1 --2- 172.21.15.123:0/2733257430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626cb1736c0 0x5626cb16d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:55.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.988+0000 7f002be00700 1 --2- 172.21.15.123:0/294515168 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f00180b2900 0x7f00180b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:55.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.988+0000 7f002be00700 1 -- 172.21.15.123:0/294515168 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f002c0e1ac0 con 0x7f0034109460 2024-05-08T06:11:55.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.989+0000 7f0033e00700 1 --2- 172.21.15.123:0/294515168 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f00180b2900 0x7f00180b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:55.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.989+0000 7f0033400700 1 -- 172.21.15.123:0/2733257430 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5626cb172da0 msgr2=0x5626cb173180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:55.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.989+0000 7f0033400700 1 --2- 172.21.15.123:0/2733257430 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5626cb172da0 0x5626cb173180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:55.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.989+0000 7f0033400700 1 -- 172.21.15.123:0/2733257430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626cb16da50 msgr2=0x5626cb16ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:55.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.989+0000 7f0033400700 1 --2- 172.21.15.123:0/2733257430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626cb16da50 0x5626cb16ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:55.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.989+0000 7f0033400700 1 -- 172.21.15.123:0/2733257430 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626cb00c8d0 con 0x5626cb1736c0 2024-05-08T06:11:55.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.990+0000 7f0033400700 1 --2- 172.21.15.123:0/2733257430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626cb1736c0 0x5626cb16d430 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f00240098e0 tx=0x7f0024009710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=132576df34a91066 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:55.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.990+0000 7f0028c00700 1 -- 172.21.15.123:0/2733257430 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0024027070 con 0x5626cb1736c0 2024-05-08T06:11:55.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.991+0000 7f0033e00700 1 --2- 172.21.15.123:0/294515168 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f00180b2900 0x7f00180b4dc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f002c00ef90 tx=0x7f002c009480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:55.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.991+0000 7f0028c00700 1 -- 172.21.15.123:0/2733257430 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0024003680 con 0x5626cb1736c0 2024-05-08T06:11:55.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.991+0000 7f0028c00700 1 -- 172.21.15.123:0/2733257430 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002400f6c0 con 0x5626cb1736c0 2024-05-08T06:11:55.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.992+0000 7f0044cb2b80 1 -- 172.21.15.123:0/2733257430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626cb1736c0 msgr2=0x5626cb16d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:55.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.992+0000 7f0044cb2b80 1 --2- 172.21.15.123:0/2733257430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626cb1736c0 0x5626cb16d430 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f00240098e0 tx=0x7f0024009710 comp rx=0 tx=0).stop 2024-05-08T06:11:55.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.992+0000 7f0044cb2b80 1 -- 172.21.15.123:0/2733257430 shutdown_connections 2024-05-08T06:11:55.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.992+0000 7f0044cb2b80 1 --2- 172.21.15.123:0/2733257430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626cb16da50 0x5626cb16ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:55.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.992+0000 7f0044cb2b80 1 --2- 172.21.15.123:0/2733257430 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5626cb172da0 0x5626cb173180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:55.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.992+0000 7f0044cb2b80 1 --2- 172.21.15.123:0/2733257430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626cb1736c0 0x5626cb16d430 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:55.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.992+0000 7f0044cb2b80 1 -- 172.21.15.123:0/2733257430 >> 172.21.15.123:0/2733257430 conn(0x5626cb0e56d0 msgr2=0x5626cb167ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:55.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.992+0000 7f0044cb2b80 1 -- 172.21.15.123:0/2733257430 shutdown_connections 2024-05-08T06:11:55.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.993+0000 7f0044cb2b80 1 -- 172.21.15.123:0/2733257430 wait complete. 2024-05-08T06:11:55.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.993+0000 7f0044cb2b80 1 -- start start 2024-05-08T06:11:55.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.994+0000 7f0044cb2b80 1 Processor -- start 2024-05-08T06:11:55.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.994+0000 7f0044cb2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626cb16da50 0x5626cb179410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:55.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.994+0000 7f0044cb2b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626cb172da0 0x5626cb1767b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:55.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.994+0000 7f0044cb2b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5626cb1736c0 0x5626cb176cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:55.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.994+0000 7f0044cb2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5626cb17a0f0 con 0x5626cb16da50 2024-05-08T06:11:55.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.994+0000 7f0044cb2b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5626cb025b20 con 0x5626cb172da0 2024-05-08T06:11:55.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.994+0000 7f0044cb2b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5626cb17a270 con 0x5626cb1736c0 2024-05-08T06:11:55.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.994+0000 7f0044cb2b80 10 client.0 _ll_register_callbacks cb 0x5626cb066c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:11:55.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.995+0000 7f0032a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626cb16da50 0x5626cb179410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:55.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.995+0000 7f0032a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626cb16da50 0x5626cb179410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50560/0 (socket says 172.21.15.123:50560) 2024-05-08T06:11:55.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.995+0000 7f0032a00700 1 -- 172.21.15.123:0/3705396994 learned_addr learned my addr 172.21.15.123:0/3705396994 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:55.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.995+0000 7f0032a00700 1 --2- 172.21.15.123:0/3705396994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5626cb1736c0 0x5626cb176cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:55.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.996+0000 7f0032a00700 1 -- 172.21.15.123:0/3705396994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626cb172da0 msgr2=0x5626cb1767b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T06:11:55.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.996+0000 7f0032a00700 1 --2- 172.21.15.123:0/3705396994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626cb172da0 0x5626cb1767b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:55.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.996+0000 7f0032a00700 1 -- 172.21.15.123:0/3705396994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626cb16da50 msgr2=0x5626cb179410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:55.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.996+0000 7f0032a00700 1 --2- 172.21.15.123:0/3705396994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626cb16da50 0x5626cb179410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:55.998 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.996+0000 7f0032a00700 1 -- 172.21.15.123:0/3705396994 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626cb085420 con 0x5626cb1736c0 2024-05-08T06:11:55.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.996+0000 7f0032a00700 1 --2- 172.21.15.123:0/3705396994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5626cb1736c0 0x5626cb176cf0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0020004790 tx=0x7f00200047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75b7c6ff11f5c3b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:55.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.996+0000 7f001d600700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-05-08T06:11:55.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.997+0000 7f001d600700 1 -- 172.21.15.123:0/3705396994 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002000fe70 con 0x5626cb1736c0 2024-05-08T06:11:55.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.997+0000 7f001d600700 1 -- 172.21.15.123:0/3705396994 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f0020004030 con 0x5626cb1736c0 2024-05-08T06:11:55.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.997+0000 7f0044cb2b80 1 -- 172.21.15.123:0/3705396994 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0024003680 con 0x5626cb1736c0 2024-05-08T06:11:55.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.998+0000 7f001d600700 1 -- 172.21.15.123:0/3705396994 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0020010900 con 0x5626cb1736c0 2024-05-08T06:11:55.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.998+0000 7f0044cb2b80 10 client.25390 fetch_fsmap learned FSMap version 24 2024-05-08T06:11:55.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.998+0000 7f0044cb2b80 1 -- 172.21.15.123:0/3705396994 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5626cb184a90 con 0x5626cb1736c0 2024-05-08T06:11:55.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.998+0000 7f001d600700 1 -- 172.21.15.123:0/3705396994 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0024003680 con 0x5626cb1736c0 2024-05-08T06:11:56.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.999+0000 7f001d600700 1 -- 172.21.15.123:0/3705396994 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f002002a420 con 0x5626cb1736c0 2024-05-08T06:11:56.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.999+0000 7f0044cb2b80 10 client.25390 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:11:56.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.999+0000 7f0044cb2b80 10 client.25390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T06:11:56.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.999+0000 7f0044cb2b80 20 client.25390 populate_metadata read hostname 'smithi123' 2024-05-08T06:11:56.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.999+0000 7f0044cb2b80 1 --2- 172.21.15.123:0/3705396994 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5626cb187020 0x5626cb189480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:56.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.999+0000 7f0044cb2b80 4 client.25390 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:11:56.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:55.999+0000 7f0044cb2b80 1 -- 172.21.15.123:0/3705396994 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626cb13a140 con 0x5626cb187020 2024-05-08T06:11:56.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.000+0000 7f0033400700 1 --2- 172.21.15.123:0/3705396994 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5626cb187020 0x5626cb189480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:56.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.001+0000 7f0033400700 1 --2- 172.21.15.123:0/3705396994 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5626cb187020 0x5626cb189480 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af37d66db8e0c512 server_cookie=a817dbcc4c61a694 in_seq=0 out_seq=0 2024-05-08T06:11:56.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.001+0000 7f001d600700 10 client.25390 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:11:56.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.002+0000 7f001d600700 1 -- 172.21.15.123:0/3705396994 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5626cb13a140 con 0x5626cb187020 2024-05-08T06:11:56.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.002+0000 7f001d600700 10 client.25390 handle_command_reply: tid=0 2024-05-08T06:11:56.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.003+0000 7f0044cb2b80 1 client.25390 shutdown 2024-05-08T06:11:56.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.003+0000 7f0044cb2b80 1 -- 172.21.15.123:0/3705396994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5626cb1736c0 msgr2=0x5626cb176cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:56.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.003+0000 7f0044cb2b80 1 --2- 172.21.15.123:0/3705396994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5626cb1736c0 0x5626cb176cf0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0020004790 tx=0x7f00200047c0 comp rx=0 tx=0).stop 2024-05-08T06:11:56.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.003+0000 7f0044cb2b80 1 -- 172.21.15.123:0/3705396994 shutdown_connections 2024-05-08T06:11:56.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.003+0000 7f0044cb2b80 1 --2- 172.21.15.123:0/3705396994 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5626cb187020 0x5626cb189480 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:56.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.003+0000 7f0044cb2b80 1 --2- 172.21.15.123:0/3705396994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626cb16da50 0x5626cb179410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:56.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.003+0000 7f0044cb2b80 1 --2- 172.21.15.123:0/3705396994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5626cb1736c0 0x5626cb176cf0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:56.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.003+0000 7f0044cb2b80 1 --2- 172.21.15.123:0/3705396994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626cb172da0 0x5626cb1767b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:56.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.003+0000 7f001d600700 0 client.25390 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:11:56.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.003+0000 7f0044cb2b80 1 -- 172.21.15.123:0/3705396994 >> 172.21.15.123:0/3705396994 conn(0x5626cb0e56d0 msgr2=0x5626cb162750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:56.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.003+0000 7f0044cb2b80 1 -- 172.21.15.123:0/3705396994 shutdown_connections 2024-05-08T06:11:56.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.003+0000 7f0044cb2b80 1 -- 172.21.15.123:0/3705396994 wait complete. 2024-05-08T06:11:56.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.004+0000 7f0044cb2b80 20 client.25390 trim_cache size 0 max 16384 2024-05-08T06:11:56.033 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.034+0000 7f0044cb2b80 1 Processor -- start 2024-05-08T06:11:56.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.034+0000 7f0044cb2b80 1 -- start start 2024-05-08T06:11:56.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.034+0000 7f0044cb2b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5626cb16da50 0x5626cb16d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:56.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.034+0000 7f0044cb2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626cb172da0 0x5626cb170f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:56.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.034+0000 7f0044cb2b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626cb1736c0 0x5626cb189410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:56.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.034+0000 7f0044cb2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f001400fb90 con 0x5626cb172da0 2024-05-08T06:11:56.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.034+0000 7f0044cb2b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f001400d420 con 0x5626cb1736c0 2024-05-08T06:11:56.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.034+0000 7f0044cb2b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f001400b270 con 0x5626cb16da50 2024-05-08T06:11:56.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.035+0000 7f0033400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626cb172da0 0x5626cb170f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:56.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.035+0000 7f0033400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626cb172da0 0x5626cb170f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50572/0 (socket says 172.21.15.123:50572) 2024-05-08T06:11:56.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.035+0000 7f0033400700 1 -- 172.21.15.123:0/1051099430 learned_addr learned my addr 172.21.15.123:0/1051099430 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:56.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.035+0000 7f0032a00700 1 --2- 172.21.15.123:0/1051099430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626cb1736c0 0x5626cb189410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:56.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.035+0000 7f0032a00700 1 --2- 172.21.15.123:0/1051099430 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5626cb16da50 0x5626cb16d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:56.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.035+0000 7f0033400700 1 -- 172.21.15.123:0/1051099430 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5626cb16da50 msgr2=0x5626cb16d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:56.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.035+0000 7f0033400700 1 --2- 172.21.15.123:0/1051099430 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5626cb16da50 0x5626cb16d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:56.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.035+0000 7f0033400700 1 -- 172.21.15.123:0/1051099430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626cb1736c0 msgr2=0x5626cb189410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:56.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.035+0000 7f0033400700 1 --2- 172.21.15.123:0/1051099430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626cb1736c0 0x5626cb189410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:56.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.035+0000 7f0033400700 1 -- 172.21.15.123:0/1051099430 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626cb1697a0 con 0x5626cb172da0 2024-05-08T06:11:56.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.036+0000 7f0032a00700 1 --2- 172.21.15.123:0/1051099430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626cb1736c0 0x5626cb189410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:11:56.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.036+0000 7f0033400700 1 --2- 172.21.15.123:0/1051099430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626cb172da0 0x5626cb170f80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0024025040 tx=0x7f0024009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6b4dbd95d086f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:56.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.037+0000 7f001fe00700 1 -- 172.21.15.123:0/1051099430 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0024027040 con 0x5626cb172da0 2024-05-08T06:11:56.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.037+0000 7f001fe00700 1 -- 172.21.15.123:0/1051099430 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0024005570 con 0x5626cb172da0 2024-05-08T06:11:56.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.037+0000 7f001fe00700 1 -- 172.21.15.123:0/1051099430 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002402d830 con 0x5626cb172da0 2024-05-08T06:11:56.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.038+0000 7f0044cb2b80 1 -- 172.21.15.123:0/1051099430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626cb172da0 msgr2=0x5626cb170f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:56.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.038+0000 7f0044cb2b80 1 --2- 172.21.15.123:0/1051099430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626cb172da0 0x5626cb170f80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0024025040 tx=0x7f0024009310 comp rx=0 tx=0).stop 2024-05-08T06:11:56.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.038+0000 7f0044cb2b80 1 -- 172.21.15.123:0/1051099430 shutdown_connections 2024-05-08T06:11:56.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.038+0000 7f0044cb2b80 1 --2- 172.21.15.123:0/1051099430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626cb1736c0 0x5626cb189410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:56.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.038+0000 7f0044cb2b80 1 --2- 172.21.15.123:0/1051099430 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5626cb16da50 0x5626cb16d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:56.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.038+0000 7f0044cb2b80 1 --2- 172.21.15.123:0/1051099430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626cb172da0 0x5626cb170f80 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:56.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.038+0000 7f0044cb2b80 1 -- 172.21.15.123:0/1051099430 >> 172.21.15.123:0/1051099430 conn(0x5626cb0e56d0 msgr2=0x5626cb170870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:56.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.038+0000 7f0044cb2b80 1 -- 172.21.15.123:0/1051099430 shutdown_connections 2024-05-08T06:11:56.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.039+0000 7f0044cb2b80 1 -- 172.21.15.123:0/1051099430 wait complete. 2024-05-08T06:11:56.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.039+0000 7f0044cb2b80 1 -- start start 2024-05-08T06:11:56.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.039+0000 7f0044cb2b80 1 Processor -- start 2024-05-08T06:11:56.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.040+0000 7f0044cb2b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626cb16da50 0x5626cb17f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:56.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.040+0000 7f0044cb2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626cb172da0 0x5626cb17fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:56.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.040+0000 7f0044cb2b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5626cb1736c0 0x5626cb18dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:56.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.040+0000 7f0044cb2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f001400b8e0 con 0x5626cb172da0 2024-05-08T06:11:56.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.040+0000 7f0044cb2b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f001400cd50 con 0x5626cb16da50 2024-05-08T06:11:56.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.040+0000 7f0044cb2b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f001400bfb0 con 0x5626cb1736c0 2024-05-08T06:11:56.040 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.040+0000 7f0044cb2b80 10 client.0 _ll_register_callbacks cb 0x5626cb066c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:11:56.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.041+0000 7f0032a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626cb16da50 0x5626cb17f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:56.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.041+0000 7f0032a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626cb16da50 0x5626cb17f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:57294/0 (socket says 172.21.15.123:57294) 2024-05-08T06:11:56.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.041+0000 7f0033400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626cb172da0 0x5626cb17fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:56.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.041+0000 7f0032a00700 1 -- 172.21.15.123:0/4193939212 learned_addr learned my addr 172.21.15.123:0/4193939212 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:11:56.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.041+0000 7f0032a00700 1 --2- 172.21.15.123:0/4193939212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5626cb1736c0 0x5626cb18dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:56.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.041+0000 7f0032a00700 1 -- 172.21.15.123:0/4193939212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5626cb1736c0 msgr2=0x5626cb18dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:56.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.041+0000 7f0032a00700 1 --2- 172.21.15.123:0/4193939212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5626cb1736c0 0x5626cb18dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:56.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.041+0000 7f0032a00700 1 -- 172.21.15.123:0/4193939212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626cb172da0 msgr2=0x5626cb17fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:11:56.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.041+0000 7f0032a00700 1 --2- 172.21.15.123:0/4193939212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626cb172da0 0x5626cb17fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:56.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.041+0000 7f0032a00700 1 -- 172.21.15.123:0/4193939212 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626cb1733d0 con 0x5626cb16da50 2024-05-08T06:11:56.042 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.042+0000 7f0032a00700 1 --2- 172.21.15.123:0/4193939212 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626cb16da50 0x5626cb17f770 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0020005c50 tx=0x7f0020005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8dc392e60eec1e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:11:56.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.042+0000 7f001d600700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-05-08T06:11:56.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.042+0000 7f001d600700 1 -- 172.21.15.123:0/4193939212 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002002a9b0 con 0x5626cb16da50 2024-05-08T06:11:56.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.042+0000 7f001d600700 1 -- 172.21.15.123:0/4193939212 <== mon.2 v2:172.21.15.163:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f002002ab50 con 0x5626cb16da50 2024-05-08T06:11:56.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.042+0000 7f0044cb2b80 1 -- 172.21.15.123:0/4193939212 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0024005570 con 0x5626cb16da50 2024-05-08T06:11:56.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.043+0000 7f001d600700 1 -- 172.21.15.123:0/4193939212 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0020004830 con 0x5626cb16da50 2024-05-08T06:11:56.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.043+0000 7f0044cb2b80 10 client.25403 fetch_fsmap learned FSMap version 24 2024-05-08T06:11:56.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.043+0000 7f0044cb2b80 1 -- 172.21.15.123:0/4193939212 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5626cb1f13b0 con 0x5626cb16da50 2024-05-08T06:11:56.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.044+0000 7f001d600700 1 -- 172.21.15.123:0/4193939212 <== mon.2 v2:172.21.15.163:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0024005570 con 0x5626cb16da50 2024-05-08T06:11:56.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.044+0000 7f001d600700 1 -- 172.21.15.123:0/4193939212 <== mon.2 v2:172.21.15.163:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f00200383b0 con 0x5626cb16da50 2024-05-08T06:11:56.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.044+0000 7f0044cb2b80 10 client.25403 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:11:56.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.044+0000 7f0044cb2b80 10 client.25403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T06:11:56.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.044+0000 7f0044cb2b80 20 client.25403 populate_metadata read hostname 'smithi123' 2024-05-08T06:11:56.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.044+0000 7f0044cb2b80 1 --2- 172.21.15.123:0/4193939212 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5626cb1f18f0 0x5626cb1f3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:11:56.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.044+0000 7f0044cb2b80 4 client.25403 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T06:11:56.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.044+0000 7f0044cb2b80 1 -- 172.21.15.123:0/4193939212 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5626cb0cd400 con 0x5626cb1f18f0 2024-05-08T06:11:56.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.045+0000 7f0033400700 1 --2- 172.21.15.123:0/4193939212 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5626cb1f18f0 0x5626cb1f3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:11:56.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.046+0000 7f0033400700 1 --2- 172.21.15.123:0/4193939212 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5626cb1f18f0 0x5626cb1f3d10 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9aee05f215db3a10 server_cookie=169d4685848d0bf0 in_seq=0 out_seq=0 2024-05-08T06:11:56.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.046+0000 7f001d600700 10 client.25403 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:11:56.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.046+0000 7f001d600700 1 -- 172.21.15.123:0/4193939212 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5626cb0cd400 con 0x5626cb1f18f0 2024-05-08T06:11:56.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.046+0000 7f001d600700 10 client.25403 handle_command_reply: tid=0 2024-05-08T06:11:56.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.046+0000 7f0044cb2b80 1 client.25403 shutdown 2024-05-08T06:11:56.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.047+0000 7f0044cb2b80 1 -- 172.21.15.123:0/4193939212 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626cb16da50 msgr2=0x5626cb17f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:56.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.047+0000 7f0044cb2b80 1 --2- 172.21.15.123:0/4193939212 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626cb16da50 0x5626cb17f770 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0020005c50 tx=0x7f0020005be0 comp rx=0 tx=0).stop 2024-05-08T06:11:56.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.047+0000 7f0044cb2b80 1 -- 172.21.15.123:0/4193939212 shutdown_connections 2024-05-08T06:11:56.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.047+0000 7f0044cb2b80 1 --2- 172.21.15.123:0/4193939212 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5626cb1f18f0 0x5626cb1f3d10 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:56.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.047+0000 7f0044cb2b80 1 --2- 172.21.15.123:0/4193939212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5626cb1736c0 0x5626cb18dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:56.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.047+0000 7f0044cb2b80 1 --2- 172.21.15.123:0/4193939212 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5626cb16da50 0x5626cb17f770 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:56.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.047+0000 7f0044cb2b80 1 --2- 172.21.15.123:0/4193939212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626cb172da0 0x5626cb17fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:56.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.047+0000 7f0044cb2b80 1 -- 172.21.15.123:0/4193939212 >> 172.21.15.123:0/4193939212 conn(0x5626cb0e56d0 msgr2=0x5626cb173aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:56.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.047+0000 7f001d600700 0 client.25403 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:11:56.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.047+0000 7f0044cb2b80 1 -- 172.21.15.123:0/4193939212 shutdown_connections 2024-05-08T06:11:56.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.047+0000 7f0044cb2b80 1 -- 172.21.15.123:0/4193939212 wait complete. 2024-05-08T06:11:56.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.048+0000 7f0044cb2b80 20 client.25403 trim_cache size 0 max 16384 2024-05-08T06:11:56.049 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-05-08T06:11:56.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.048+0000 7f0038e00700 1 -- 172.21.15.123:0/294515168 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f00180b2900 msgr2=0x7f00180b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:56.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.048+0000 7f0038e00700 1 --2- 172.21.15.123:0/294515168 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f00180b2900 0x7f00180b4dc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f002c00ef90 tx=0x7f002c009480 comp rx=0 tx=0).stop 2024-05-08T06:11:56.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.049+0000 7f0038e00700 1 -- 172.21.15.123:0/294515168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0034109460 msgr2=0x7f00341a4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:11:56.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.049+0000 7f0038e00700 1 --2- 172.21.15.123:0/294515168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0034109460 0x7f00341a4920 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f002c00efc0 tx=0x7f002c00c3a0 comp rx=0 tx=0).stop 2024-05-08T06:11:56.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.049+0000 7f0038e00700 1 -- 172.21.15.123:0/294515168 shutdown_connections 2024-05-08T06:11:56.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.049+0000 7f0038e00700 1 --2- 172.21.15.123:0/294515168 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f00180b2900 0x7f00180b4dc0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:56.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.049+0000 7f0038e00700 1 --2- 172.21.15.123:0/294515168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0034109460 0x7f00341a4920 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:56.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.049+0000 7f0038e00700 1 --2- 172.21.15.123:0/294515168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00341033d0 0x7f003410f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:56.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.049+0000 7f0038e00700 1 --2- 172.21.15.123:0/294515168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0034103d90 0x7f003410fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:11:56.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.049+0000 7f0038e00700 1 -- 172.21.15.123:0/294515168 >> 172.21.15.123:0/294515168 conn(0x7f00340fef20 msgr2=0x7f00340ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:11:56.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.049+0000 7f0038e00700 1 -- 172.21.15.123:0/294515168 shutdown_connections 2024-05-08T06:11:56.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:11:56.049+0000 7f0038e00700 1 -- 172.21.15.123:0/294515168 wait complete. 2024-05-08T06:11:56.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:56 smithi163 ceph-mon[34520]: pgmap v504: 97 pgs: 97 active+clean; 676 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-08T06:11:56.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:56 smithi192 ceph-mon[34664]: pgmap v504: 97 pgs: 97 active+clean; 676 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-08T06:11:56.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:56 smithi123 ceph-mon[29787]: pgmap v504: 97 pgs: 97 active+clean; 676 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-08T06:11:58.061 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T06:11:58.061 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T06:11:58.475 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:11:58.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:11:58 smithi163 ceph-mon[34520]: pgmap v505: 97 pgs: 97 active+clean; 676 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:11:58.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:11:58 smithi192 ceph-mon[34664]: pgmap v505: 97 pgs: 97 active+clean; 676 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:11:58.842 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:11:58 smithi123 ceph-mon[29787]: pgmap v505: 97 pgs: 97 active+clean; 676 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:12:00.634 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:00 smithi123 ceph-mon[29787]: pgmap v506: 97 pgs: 97 active+clean; 676 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:12:00.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:00 smithi163 ceph-mon[34520]: pgmap v506: 97 pgs: 97 active+clean; 676 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:12:00.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:00 smithi192 ceph-mon[34664]: pgmap v506: 97 pgs: 97 active+clean; 676 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:12:01.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.033+0000 7f94aac00700 1 -- 172.21.15.123:0/4169064340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94a4108850 msgr2=0x7f94a41103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:01.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.033+0000 7f94aac00700 1 --2- 172.21.15.123:0/4169064340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94a4108850 0x7f94a41103d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f949c009fc0 tx=0x7f949c00b330 comp rx=0 tx=0).stop 2024-05-08T06:12:01.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.033+0000 7f94aac00700 1 -- 172.21.15.123:0/4169064340 shutdown_connections 2024-05-08T06:12:01.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.033+0000 7f94aac00700 1 --2- 172.21.15.123:0/4169064340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94a4103750 0x7f94a4103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:01.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.033+0000 7f94aac00700 1 --2- 172.21.15.123:0/4169064340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94a4108850 0x7f94a41103d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:01.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.033+0000 7f94aac00700 1 --2- 172.21.15.123:0/4169064340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94a4104100 0x7f94a4108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:01.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.033+0000 7f94aac00700 1 -- 172.21.15.123:0/4169064340 >> 172.21.15.123:0/4169064340 conn(0x7f94a407a600 msgr2=0x7f94a407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:01.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.033+0000 7f94aac00700 1 -- 172.21.15.123:0/4169064340 shutdown_connections 2024-05-08T06:12:01.034 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.033+0000 7f94aac00700 1 -- 172.21.15.123:0/4169064340 wait complete. 2024-05-08T06:12:01.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.034+0000 7f94aac00700 1 Processor -- start 2024-05-08T06:12:01.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.035+0000 7f94aac00700 1 -- start start 2024-05-08T06:12:01.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.035+0000 7f94aac00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94a4103750 0x7f94a410f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:01.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.035+0000 7f94aac00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94a4104100 0x7f94a410fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:01.035 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.035+0000 7f94aac00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94a4108850 0x7f94a41a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:01.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.035+0000 7f94aac00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f94a4113670 con 0x7f94a4108850 2024-05-08T06:12:01.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.035+0000 7f94aac00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f94a41134f0 con 0x7f94a4103750 2024-05-08T06:12:01.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.035+0000 7f94aac00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f94a41137f0 con 0x7f94a4104100 2024-05-08T06:12:01.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.036+0000 7f94a3e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94a4103750 0x7f94a410f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:01.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.036+0000 7f94a3400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94a4104100 0x7f94a410fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:01.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.036+0000 7f94a3e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94a4103750 0x7f94a410f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:57320/0 (socket says 172.21.15.123:57320) 2024-05-08T06:12:01.036 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.036+0000 7f94a3e00700 1 -- 172.21.15.123:0/3887503084 learned_addr learned my addr 172.21.15.123:0/3887503084 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:01.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.036+0000 7f94a8c00700 1 --2- 172.21.15.123:0/3887503084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94a4108850 0x7f94a41a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:01.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.037+0000 7f94a3e00700 1 -- 172.21.15.123:0/3887503084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94a4104100 msgr2=0x7f94a410fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:01.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.037+0000 7f94a3e00700 1 --2- 172.21.15.123:0/3887503084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94a4104100 0x7f94a410fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:01.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.037+0000 7f94a3e00700 1 -- 172.21.15.123:0/3887503084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94a4108850 msgr2=0x7f94a41a4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:01.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.037+0000 7f94a3e00700 1 --2- 172.21.15.123:0/3887503084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94a4108850 0x7f94a41a4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:01.037 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.037+0000 7f94a3e00700 1 -- 172.21.15.123:0/3887503084 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a41a5070 con 0x7f94a4103750 2024-05-08T06:12:01.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.037+0000 7f94a3e00700 1 --2- 172.21.15.123:0/3887503084 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94a4103750 0x7f94a410f620 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f949400e9c0 tx=0x7f949400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:01.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.038+0000 7f94a0c00700 1 -- 172.21.15.123:0/3887503084 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9494004a30 con 0x7f94a4103750 2024-05-08T06:12:01.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.038+0000 7f94aac00700 1 -- 172.21.15.123:0/3887503084 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a41a5360 con 0x7f94a4103750 2024-05-08T06:12:01.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.038+0000 7f94a0c00700 1 -- 172.21.15.123:0/3887503084 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f949400cc70 con 0x7f94a4103750 2024-05-08T06:12:01.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.038+0000 7f94aac00700 1 -- 172.21.15.123:0/3887503084 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a41acbe0 con 0x7f94a4103750 2024-05-08T06:12:01.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.038+0000 7f94a0c00700 1 -- 172.21.15.123:0/3887503084 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9494005490 con 0x7f94a4103750 2024-05-08T06:12:01.041 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.042+0000 7f94a0c00700 1 -- 172.21.15.123:0/3887503084 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7f9494021440 con 0x7f94a4103750 2024-05-08T06:12:01.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.043+0000 7f94a0c00700 1 --2- 172.21.15.123:0/3887503084 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f94880b2980 0x7f94880b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:01.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.043+0000 7f94a0c00700 1 -- 172.21.15.123:0/3887503084 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f94940e21d0 con 0x7f94a4103750 2024-05-08T06:12:01.043 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.044+0000 7f94a3400700 1 --2- 172.21.15.123:0/3887503084 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f94880b2980 0x7f94880b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:01.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.044+0000 7f94b5522b80 1 Processor -- start 2024-05-08T06:12:01.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.044+0000 7f94b5522b80 1 -- start start 2024-05-08T06:12:01.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.044+0000 7f94b5522b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf1f83c40 0x564cf1f84020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:01.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.044+0000 7f94b5522b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cf1f84560 0x564cf1f7e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:01.044 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.044+0000 7f94a3400700 1 --2- 172.21.15.123:0/3887503084 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f94880b2980 0x7f94880b4e40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f94a4077730 tx=0x7f9490009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:01.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.045+0000 7f94b5522b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564cf1f7e970 0x564cf1f7edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:01.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.045+0000 7f94b5522b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564cf1ed0aa0 con 0x564cf1f7e970 2024-05-08T06:12:01.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.045+0000 7f94b5522b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x564cf1e6e7e0 con 0x564cf1f84560 2024-05-08T06:12:01.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.045+0000 7f94b5522b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564cf1ecca60 con 0x564cf1f83c40 2024-05-08T06:12:01.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.045+0000 7f94a3400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cf1f84560 0x564cf1f7e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:01.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.045+0000 7f94a8c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564cf1f7e970 0x564cf1f7edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:01.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.045+0000 7f94a3400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cf1f84560 0x564cf1f7e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:57332/0 (socket says 172.21.15.123:57332) 2024-05-08T06:12:01.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.045+0000 7f94a3400700 1 -- 172.21.15.123:0/551627760 learned_addr learned my addr 172.21.15.123:0/551627760 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:01.045 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.045+0000 7f94a3e00700 1 --2- 172.21.15.123:0/551627760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf1f83c40 0x564cf1f84020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:01.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.045+0000 7f94a3400700 1 -- 172.21.15.123:0/551627760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf1f83c40 msgr2=0x564cf1f84020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:01.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.045+0000 7f94a3400700 1 --2- 172.21.15.123:0/551627760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf1f83c40 0x564cf1f84020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:01.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.045+0000 7f94a3400700 1 -- 172.21.15.123:0/551627760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564cf1f7e970 msgr2=0x564cf1f7edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:01.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.045+0000 7f94a3400700 1 --2- 172.21.15.123:0/551627760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564cf1f7e970 0x564cf1f7edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:01.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.045+0000 7f94a3400700 1 -- 172.21.15.123:0/551627760 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cf1e1d950 con 0x564cf1f84560 2024-05-08T06:12:01.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.046+0000 7f94a8c00700 1 --2- 172.21.15.123:0/551627760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564cf1f7e970 0x564cf1f7edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:12:01.046 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.046+0000 7f94a3400700 1 --2- 172.21.15.123:0/551627760 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cf1f84560 0x564cf1f7e350 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9490011d50 tx=0x7f9490014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93ca9b9b6633f097 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:01.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.046+0000 7f9499600700 1 -- 172.21.15.123:0/551627760 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949000ebd0 con 0x564cf1f84560 2024-05-08T06:12:01.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.046+0000 7f9499600700 1 -- 172.21.15.123:0/551627760 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f949000ed70 con 0x564cf1f84560 2024-05-08T06:12:01.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.046+0000 7f9499600700 1 -- 172.21.15.123:0/551627760 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949001d780 con 0x564cf1f84560 2024-05-08T06:12:01.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.047+0000 7f94b5522b80 1 -- 172.21.15.123:0/551627760 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cf1f84560 msgr2=0x564cf1f7e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:01.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.047+0000 7f94b5522b80 1 --2- 172.21.15.123:0/551627760 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cf1f84560 0x564cf1f7e350 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9490011d50 tx=0x7f9490014000 comp rx=0 tx=0).stop 2024-05-08T06:12:01.047 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.047+0000 7f94b5522b80 1 -- 172.21.15.123:0/551627760 shutdown_connections 2024-05-08T06:12:01.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.047+0000 7f94b5522b80 1 --2- 172.21.15.123:0/551627760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564cf1f7e970 0x564cf1f7edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:01.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.047+0000 7f94b5522b80 1 --2- 172.21.15.123:0/551627760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf1f83c40 0x564cf1f84020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:01.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.047+0000 7f94b5522b80 1 --2- 172.21.15.123:0/551627760 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cf1f84560 0x564cf1f7e350 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:01.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.047+0000 7f94b5522b80 1 -- 172.21.15.123:0/551627760 >> 172.21.15.123:0/551627760 conn(0x564cf1ef6750 msgr2=0x564cf1f78b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:01.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.047+0000 7f94b5522b80 1 -- 172.21.15.123:0/551627760 shutdown_connections 2024-05-08T06:12:01.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.048+0000 7f94b5522b80 1 -- 172.21.15.123:0/551627760 wait complete. 2024-05-08T06:12:01.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.048+0000 7f94b5522b80 1 -- start start 2024-05-08T06:12:01.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.048+0000 7f94b5522b80 1 Processor -- start 2024-05-08T06:12:01.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.049+0000 7f94b5522b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564cf1f7e970 0x564cf1f8a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:01.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.049+0000 7f94b5522b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cf1f83c40 0x564cf1f875c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:01.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.049+0000 7f94b5522b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf1f84560 0x564cf1f87b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:01.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.049+0000 7f94b5522b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564cf1f8afa0 con 0x564cf1f7e970 2024-05-08T06:12:01.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.049+0000 7f94b5522b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x564cf1e36ba0 con 0x564cf1f83c40 2024-05-08T06:12:01.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.049+0000 7f94b5522b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564cf1f8b120 con 0x564cf1f84560 2024-05-08T06:12:01.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.049+0000 7f94b5522b80 10 client.0 _ll_register_callbacks cb 0x564cf1e77cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:12:01.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.049+0000 7f94a3400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cf1f83c40 0x564cf1f875c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:01.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.049+0000 7f94a3400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cf1f83c40 0x564cf1f875c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:57340/0 (socket says 172.21.15.123:57340) 2024-05-08T06:12:01.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.049+0000 7f94a3400700 1 -- 172.21.15.123:0/2858771627 learned_addr learned my addr 172.21.15.123:0/2858771627 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:01.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.049+0000 7f94a3e00700 1 --2- 172.21.15.123:0/2858771627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564cf1f7e970 0x564cf1f8a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:01.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.050+0000 7f94a3400700 1 -- 172.21.15.123:0/2858771627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf1f84560 msgr2=0x564cf1f87b00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T06:12:01.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.050+0000 7f94a3400700 1 --2- 172.21.15.123:0/2858771627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf1f84560 0x564cf1f87b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:01.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.050+0000 7f94a3400700 1 -- 172.21.15.123:0/2858771627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564cf1f7e970 msgr2=0x564cf1f8a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:01.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.050+0000 7f94a3400700 1 --2- 172.21.15.123:0/2858771627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564cf1f7e970 0x564cf1f8a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:01.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.050+0000 7f94a3400700 1 -- 172.21.15.123:0/2858771627 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cf1e964a0 con 0x564cf1f83c40 2024-05-08T06:12:01.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.050+0000 7f94a3e00700 1 --2- 172.21.15.123:0/2858771627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564cf1f7e970 0x564cf1f8a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:12:01.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.050+0000 7f94a3400700 1 --2- 172.21.15.123:0/2858771627 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cf1f83c40 0x564cf1f875c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f94900117d0 tx=0x7f949000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=860e9351359b5d77 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:01.051 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.050+0000 7f948e000700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-05-08T06:12:01.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.050+0000 7f948e000700 1 -- 172.21.15.123:0/2858771627 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9490033060 con 0x564cf1f83c40 2024-05-08T06:12:01.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.050+0000 7f948e000700 1 -- 172.21.15.123:0/2858771627 <== mon.2 v2:172.21.15.163:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f9490005390 con 0x564cf1f83c40 2024-05-08T06:12:01.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.051+0000 7f94b5522b80 1 -- 172.21.15.123:0/2858771627 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f949000ed70 con 0x564cf1f83c40 2024-05-08T06:12:01.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.051+0000 7f948e000700 1 -- 172.21.15.123:0/2858771627 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9490005590 con 0x564cf1f83c40 2024-05-08T06:12:01.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.051+0000 7f94b5522b80 10 client.25415 fetch_fsmap learned FSMap version 24 2024-05-08T06:12:01.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.051+0000 7f94b5522b80 1 -- 172.21.15.123:0/2858771627 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564cf1f95830 con 0x564cf1f83c40 2024-05-08T06:12:01.052 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.051+0000 7f948e000700 1 -- 172.21.15.123:0/2858771627 <== mon.2 v2:172.21.15.163:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f949000ed70 con 0x564cf1f83c40 2024-05-08T06:12:01.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.052+0000 7f948e000700 1 -- 172.21.15.123:0/2858771627 <== mon.2 v2:172.21.15.163:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f949001c7b0 con 0x564cf1f83c40 2024-05-08T06:12:01.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.052+0000 7f94b5522b80 10 client.25415 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:12:01.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.052+0000 7f94b5522b80 10 client.25415 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T06:12:01.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.052+0000 7f94b5522b80 20 client.25415 populate_metadata read hostname 'smithi123' 2024-05-08T06:12:01.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.052+0000 7f94b5522b80 1 --2- 172.21.15.123:0/2858771627 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x564cf1f97d00 0x564cf1f9a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:01.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.052+0000 7f94b5522b80 4 client.25415 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:12:01.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.052+0000 7f94b5522b80 1 -- 172.21.15.123:0/2858771627 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564cf1f4b1c0 con 0x564cf1f97d00 2024-05-08T06:12:01.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.053+0000 7f94a3e00700 1 --2- 172.21.15.123:0/2858771627 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x564cf1f97d00 0x564cf1f9a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:01.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.053+0000 7f94a3e00700 1 --2- 172.21.15.123:0/2858771627 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x564cf1f97d00 0x564cf1f9a0f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b3031200513fdfa server_cookie=119337de6a25557f in_seq=0 out_seq=0 2024-05-08T06:12:01.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.053+0000 7f948e000700 10 client.25415 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:01.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.054+0000 7f948e000700 1 -- 172.21.15.123:0/2858771627 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564cf1f4b1c0 con 0x564cf1f97d00 2024-05-08T06:12:01.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.054+0000 7f948e000700 10 client.25415 handle_command_reply: tid=0 2024-05-08T06:12:01.054 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.054+0000 7f94b5522b80 1 client.25415 shutdown 2024-05-08T06:12:01.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.054+0000 7f94b5522b80 1 -- 172.21.15.123:0/2858771627 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cf1f83c40 msgr2=0x564cf1f875c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:01.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.054+0000 7f94b5522b80 1 --2- 172.21.15.123:0/2858771627 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cf1f83c40 0x564cf1f875c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f94900117d0 tx=0x7f949000ed00 comp rx=0 tx=0).stop 2024-05-08T06:12:01.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.055+0000 7f94b5522b80 1 -- 172.21.15.123:0/2858771627 shutdown_connections 2024-05-08T06:12:01.055 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.055+0000 7f94b5522b80 1 --2- 172.21.15.123:0/2858771627 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x564cf1f97d00 0x564cf1f9a0f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:01.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.055+0000 7f94b5522b80 1 --2- 172.21.15.123:0/2858771627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564cf1f7e970 0x564cf1f8a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:01.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.055+0000 7f948e000700 0 client.25415 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:01.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.055+0000 7f94b5522b80 1 --2- 172.21.15.123:0/2858771627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf1f84560 0x564cf1f87b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:01.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.055+0000 7f94b5522b80 1 --2- 172.21.15.123:0/2858771627 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cf1f83c40 0x564cf1f875c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:01.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.055+0000 7f94b5522b80 1 -- 172.21.15.123:0/2858771627 >> 172.21.15.123:0/2858771627 conn(0x564cf1ef6750 msgr2=0x564cf1f78210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:01.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.055+0000 7f94b5522b80 1 -- 172.21.15.123:0/2858771627 shutdown_connections 2024-05-08T06:12:01.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.055+0000 7f94b5522b80 1 -- 172.21.15.123:0/2858771627 wait complete. 2024-05-08T06:12:01.056 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.055+0000 7f94b5522b80 20 client.25415 trim_cache size 0 max 16384 2024-05-08T06:12:01.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.077+0000 7f94b5522b80 1 Processor -- start 2024-05-08T06:12:01.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.077+0000 7f94b5522b80 1 -- start start 2024-05-08T06:12:01.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.077+0000 7f94b5522b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564cf1f7e970 0x564cf1f7e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:01.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.077+0000 7f94b5522b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf1f83c40 0x564cf1f851d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:01.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.077+0000 7f94b5522b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cf1f84560 0x564cf1f78990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:01.077 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.077+0000 7f94b5522b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f948400fbe0 con 0x564cf1f7e970 2024-05-08T06:12:01.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.077+0000 7f94b5522b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f948400d470 con 0x564cf1f84560 2024-05-08T06:12:01.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.077+0000 7f94b5522b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f948400b2c0 con 0x564cf1f83c40 2024-05-08T06:12:01.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.078+0000 7f94a3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564cf1f7e970 0x564cf1f7e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:01.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.078+0000 7f94a3400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf1f83c40 0x564cf1f851d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:01.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.078+0000 7f94a3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564cf1f7e970 0x564cf1f7e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50616/0 (socket says 172.21.15.123:50616) 2024-05-08T06:12:01.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.078+0000 7f94a8c00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cf1f84560 0x564cf1f78990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:01.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.078+0000 7f94a3400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf1f83c40 0x564cf1f851d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:53214/0 (socket says 172.21.15.123:53214) 2024-05-08T06:12:01.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.078+0000 7f94a3400700 1 -- 172.21.15.123:0/3684814423 learned_addr learned my addr 172.21.15.123:0/3684814423 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:01.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.078+0000 7f94a3400700 1 -- 172.21.15.123:0/3684814423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cf1f84560 msgr2=0x564cf1f78990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:01.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.078+0000 7f94a3400700 1 --2- 172.21.15.123:0/3684814423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cf1f84560 0x564cf1f78990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:01.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.078+0000 7f94a3400700 1 -- 172.21.15.123:0/3684814423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564cf1f7e970 msgr2=0x564cf1f7e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:01.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.079+0000 7f94a3400700 1 --2- 172.21.15.123:0/3684814423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564cf1f7e970 0x564cf1f7e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:01.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.079+0000 7f94a3400700 1 -- 172.21.15.123:0/3684814423 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cf1f82050 con 0x564cf1f83c40 2024-05-08T06:12:01.079 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.079+0000 7f94a8c00700 1 --2- 172.21.15.123:0/3684814423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cf1f84560 0x564cf1f78990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:12:01.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.079+0000 7f94a3e00700 1 --2- 172.21.15.123:0/3684814423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564cf1f7e970 0x564cf1f7e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:12:01.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.079+0000 7f94a3400700 1 --2- 172.21.15.123:0/3684814423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf1f83c40 0x564cf1f851d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f94900118f0 tx=0x7f9490011e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f42e6d08573aaa90 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:01.080 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.079+0000 7f9498c00700 1 -- 172.21.15.123:0/3684814423 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9490033070 con 0x564cf1f83c40 2024-05-08T06:12:01.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.080+0000 7f9498c00700 1 -- 172.21.15.123:0/3684814423 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f949000ed70 con 0x564cf1f83c40 2024-05-08T06:12:01.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.080+0000 7f9498c00700 1 -- 172.21.15.123:0/3684814423 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949003d080 con 0x564cf1f83c40 2024-05-08T06:12:01.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.080+0000 7f94b5522b80 1 -- 172.21.15.123:0/3684814423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf1f83c40 msgr2=0x564cf1f851d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:01.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.080+0000 7f94b5522b80 1 --2- 172.21.15.123:0/3684814423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf1f83c40 0x564cf1f851d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f94900118f0 tx=0x7f9490011e30 comp rx=0 tx=0).stop 2024-05-08T06:12:01.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.081+0000 7f94b5522b80 1 -- 172.21.15.123:0/3684814423 shutdown_connections 2024-05-08T06:12:01.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.081+0000 7f94b5522b80 1 --2- 172.21.15.123:0/3684814423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564cf1f7e970 0x564cf1f7e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:01.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.081+0000 7f94b5522b80 1 --2- 172.21.15.123:0/3684814423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cf1f84560 0x564cf1f78990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:01.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.081+0000 7f94b5522b80 1 --2- 172.21.15.123:0/3684814423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf1f83c40 0x564cf1f851d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:01.081 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.081+0000 7f94b5522b80 1 -- 172.21.15.123:0/3684814423 >> 172.21.15.123:0/3684814423 conn(0x564cf1ef6750 msgr2=0x564cf1f826a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:01.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.081+0000 7f94b5522b80 1 -- 172.21.15.123:0/3684814423 shutdown_connections 2024-05-08T06:12:01.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.081+0000 7f94b5522b80 1 -- 172.21.15.123:0/3684814423 wait complete. 2024-05-08T06:12:01.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.082+0000 7f94b5522b80 1 -- start start 2024-05-08T06:12:01.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.082+0000 7f94b5522b80 1 Processor -- start 2024-05-08T06:12:01.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.083+0000 7f94b5522b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf1f7e970 0x564cf1f8e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:01.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.083+0000 7f94b5522b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564cf1f83c40 0x564cf1f8ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:01.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.083+0000 7f94b5522b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cf1f84560 0x564cf20012e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:01.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.083+0000 7f94b5522b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f948400b930 con 0x564cf1f83c40 2024-05-08T06:12:01.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.083+0000 7f94b5522b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f948400cda0 con 0x564cf1f84560 2024-05-08T06:12:01.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.083+0000 7f94b5522b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f948400c000 con 0x564cf1f7e970 2024-05-08T06:12:01.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.083+0000 7f94b5522b80 10 client.0 _ll_register_callbacks cb 0x564cf1f9a610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:12:01.083 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.083+0000 7f94a3400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564cf1f83c40 0x564cf1f8ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:01.084 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.083+0000 7f94a3400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564cf1f83c40 0x564cf1f8ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50624/0 (socket says 172.21.15.123:50624) 2024-05-08T06:12:01.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.083+0000 7f94a3400700 1 -- 172.21.15.123:0/3560069897 learned_addr learned my addr 172.21.15.123:0/3560069897 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:01.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.083+0000 7f94a8c00700 1 --2- 172.21.15.123:0/3560069897 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cf1f84560 0x564cf20012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:01.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.083+0000 7f94a3e00700 1 --2- 172.21.15.123:0/3560069897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf1f7e970 0x564cf1f8e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:01.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.084+0000 7f94a8c00700 1 -- 172.21.15.123:0/3560069897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf1f7e970 msgr2=0x564cf1f8e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:01.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.084+0000 7f94a8c00700 1 --2- 172.21.15.123:0/3560069897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf1f7e970 0x564cf1f8e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:01.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.084+0000 7f94a8c00700 1 -- 172.21.15.123:0/3560069897 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564cf1f83c40 msgr2=0x564cf1f8ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:01.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.084+0000 7f94a8c00700 1 --2- 172.21.15.123:0/3560069897 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564cf1f83c40 0x564cf1f8ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:01.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.084+0000 7f94a8c00700 1 -- 172.21.15.123:0/3560069897 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cf2001820 con 0x564cf1f84560 2024-05-08T06:12:01.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.084+0000 7f94a3400700 1 --2- 172.21.15.123:0/3560069897 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564cf1f83c40 0x564cf1f8ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:12:01.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.084+0000 7f94a8c00700 1 --2- 172.21.15.123:0/3560069897 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cf1f84560 0x564cf20012e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x564cf1f7f330 tx=0x7f949c009d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37e910466cb38487 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:01.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.084+0000 7f948e000700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-05-08T06:12:01.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.084+0000 7f948e000700 1 -- 172.21.15.123:0/3560069897 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c0075e0 con 0x564cf1f84560 2024-05-08T06:12:01.085 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.084+0000 7f94b5522b80 1 -- 172.21.15.123:0/3560069897 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f949000ed70 con 0x564cf1f84560 2024-05-08T06:12:01.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.084+0000 7f948e000700 1 -- 172.21.15.123:0/3560069897 <== mon.2 v2:172.21.15.163:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f949c007780 con 0x564cf1f84560 2024-05-08T06:12:01.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.085+0000 7f948e000700 1 -- 172.21.15.123:0/3560069897 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f949c007980 con 0x564cf1f84560 2024-05-08T06:12:01.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.085+0000 7f94b5522b80 10 client.25421 fetch_fsmap learned FSMap version 24 2024-05-08T06:12:01.086 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.085+0000 7f94b5522b80 1 -- 172.21.15.123:0/3560069897 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564cf2002b60 con 0x564cf1f84560 2024-05-08T06:12:01.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.085+0000 7f948e000700 1 -- 172.21.15.123:0/3560069897 <== mon.2 v2:172.21.15.163:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f949000ed70 con 0x564cf1f84560 2024-05-08T06:12:01.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.085+0000 7f948e000700 1 -- 172.21.15.123:0/3560069897 <== mon.2 v2:172.21.15.163:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f949c011070 con 0x564cf1f84560 2024-05-08T06:12:01.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.086+0000 7f94b5522b80 10 client.25421 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:12:01.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.086+0000 7f94b5522b80 10 client.25421 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T06:12:01.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.086+0000 7f94b5522b80 20 client.25421 populate_metadata read hostname 'smithi123' 2024-05-08T06:12:01.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.086+0000 7f94b5522b80 1 --2- 172.21.15.123:0/3560069897 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x564cf2003250 0x564cf2005670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:01.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.086+0000 7f94b5522b80 4 client.25421 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T06:12:01.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.086+0000 7f94b5522b80 1 -- 172.21.15.123:0/3560069897 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564cf1ede480 con 0x564cf2003250 2024-05-08T06:12:01.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.086+0000 7f94a3e00700 1 --2- 172.21.15.123:0/3560069897 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x564cf2003250 0x564cf2005670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:01.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.087+0000 7f948e000700 10 client.25421 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:01.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.087+0000 7f94a3e00700 1 --2- 172.21.15.123:0/3560069897 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x564cf2003250 0x564cf2005670 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fa99e99d331fed4 server_cookie=6d04d2207aed16f7 in_seq=0 out_seq=0 2024-05-08T06:12:01.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.088+0000 7f948e000700 1 -- 172.21.15.123:0/3560069897 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564cf1ede480 con 0x564cf2003250 2024-05-08T06:12:01.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.088+0000 7f948e000700 10 client.25421 handle_command_reply: tid=0 2024-05-08T06:12:01.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.088+0000 7f94b5522b80 1 client.25421 shutdown 2024-05-08T06:12:01.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.088+0000 7f94b5522b80 1 -- 172.21.15.123:0/3560069897 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cf1f84560 msgr2=0x564cf20012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:01.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.088+0000 7f94b5522b80 1 --2- 172.21.15.123:0/3560069897 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cf1f84560 0x564cf20012e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x564cf1f7f330 tx=0x7f949c009d50 comp rx=0 tx=0).stop 2024-05-08T06:12:01.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.088+0000 7f94b5522b80 1 -- 172.21.15.123:0/3560069897 shutdown_connections 2024-05-08T06:12:01.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.088+0000 7f94b5522b80 1 --2- 172.21.15.123:0/3560069897 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x564cf2003250 0x564cf2005670 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:01.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.088+0000 7f94b5522b80 1 --2- 172.21.15.123:0/3560069897 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564cf1f84560 0x564cf20012e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:01.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.088+0000 7f94b5522b80 1 --2- 172.21.15.123:0/3560069897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cf1f7e970 0x564cf1f8e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:01.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.088+0000 7f94b5522b80 1 --2- 172.21.15.123:0/3560069897 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564cf1f83c40 0x564cf1f8ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:01.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.088+0000 7f94b5522b80 1 -- 172.21.15.123:0/3560069897 >> 172.21.15.123:0/3560069897 conn(0x564cf1ef6750 msgr2=0x564cf1f82f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:01.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.089+0000 7f948e000700 0 client.25421 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:01.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.089+0000 7f94b5522b80 1 -- 172.21.15.123:0/3560069897 shutdown_connections 2024-05-08T06:12:01.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.089+0000 7f94b5522b80 1 -- 172.21.15.123:0/3560069897 wait complete. 2024-05-08T06:12:01.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.089+0000 7f94b5522b80 20 client.25421 trim_cache size 0 max 16384 2024-05-08T06:12:01.090 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-05-08T06:12:01.090 INFO:teuthology.orchestra.run.smithi123.stdout: "return_code": 0, 2024-05-08T06:12:01.090 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_tag": "10f81d43-3f91-49b4-9a2d-eff3b2b67b32", 2024-05-08T06:12:01.090 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": "asynchronous" 2024-05-08T06:12:01.090 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-05-08T06:12:01.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.090+0000 7f94aac00700 1 -- 172.21.15.123:0/3887503084 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f94880b2980 msgr2=0x7f94880b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:01.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.090+0000 7f94aac00700 1 --2- 172.21.15.123:0/3887503084 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f94880b2980 0x7f94880b4e40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f94a4077730 tx=0x7f9490009260 comp rx=0 tx=0).stop 2024-05-08T06:12:01.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.090+0000 7f94aac00700 1 -- 172.21.15.123:0/3887503084 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94a4103750 msgr2=0x7f94a410f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:01.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.090+0000 7f94aac00700 1 --2- 172.21.15.123:0/3887503084 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94a4103750 0x7f94a410f620 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f949400e9c0 tx=0x7f949400ecd0 comp rx=0 tx=0).stop 2024-05-08T06:12:01.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.091+0000 7f94aac00700 1 -- 172.21.15.123:0/3887503084 shutdown_connections 2024-05-08T06:12:01.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.091+0000 7f94aac00700 1 --2- 172.21.15.123:0/3887503084 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f94880b2980 0x7f94880b4e40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:01.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.091+0000 7f94aac00700 1 --2- 172.21.15.123:0/3887503084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94a4108850 0x7f94a41a4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:01.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.091+0000 7f94aac00700 1 --2- 172.21.15.123:0/3887503084 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94a4103750 0x7f94a410f620 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:01.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.091+0000 7f94aac00700 1 --2- 172.21.15.123:0/3887503084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94a4104100 0x7f94a410fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:01.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.091+0000 7f94aac00700 1 -- 172.21.15.123:0/3887503084 >> 172.21.15.123:0/3887503084 conn(0x7f94a407a600 msgr2=0x7f94a41018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:01.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.091+0000 7f94aac00700 1 -- 172.21.15.123:0/3887503084 shutdown_connections 2024-05-08T06:12:01.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:01.091+0000 7f94aac00700 1 -- 172.21.15.123:0/3887503084 wait complete. 2024-05-08T06:12:02.082 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.082+0000 7f5766e00640 1 -- 172.21.15.123:0/2399243160 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5760103a60 msgr2=0x7f5760103ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:02.082 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.082+0000 7f5766e00640 1 --2- 172.21.15.123:0/2399243160 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5760103a60 0x7f5760103ee0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f574c009860 tx=0x7f574c02a340 comp rx=0 tx=0).stop 2024-05-08T06:12:02.083 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.082+0000 7f5766e00640 1 -- 172.21.15.123:0/2399243160 shutdown_connections 2024-05-08T06:12:02.083 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.082+0000 7f5766e00640 1 --2- 172.21.15.123:0/2399243160 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5760104420 0x7f576010acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:02.083 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.082+0000 7f5766e00640 1 --2- 172.21.15.123:0/2399243160 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5760103a60 0x7f5760103ee0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:02.083 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.082+0000 7f5766e00640 1 --2- 172.21.15.123:0/2399243160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5760102870 0x7f5760102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:02.083 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.082+0000 7f5766e00640 1 -- 172.21.15.123:0/2399243160 >> 172.21.15.123:0/2399243160 conn(0x7f57600fdde0 msgr2=0x7f5760100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:02.083 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.082+0000 7f5766e00640 1 -- 172.21.15.123:0/2399243160 shutdown_connections 2024-05-08T06:12:02.083 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.082+0000 7f5766e00640 1 -- 172.21.15.123:0/2399243160 wait complete. 2024-05-08T06:12:02.083 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.084+0000 7f5766e00640 1 Processor -- start 2024-05-08T06:12:02.083 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.084+0000 7f5766e00640 1 -- start start 2024-05-08T06:12:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.084+0000 7f5766e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5760102870 0x7f576019ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.084+0000 7f5766e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5760103a60 0x7f576019b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.084+0000 7f5766e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5760104420 0x7f57601a2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.084+0000 7f5766e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f576010e3a0 con 0x7f5760103a60 2024-05-08T06:12:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.084+0000 7f5766e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f576010e220 con 0x7f5760104420 2024-05-08T06:12:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.084+0000 7f5766e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f576010e520 con 0x7f5760102870 2024-05-08T06:12:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.085+0000 7f5765a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5760102870 0x7f576019ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.085+0000 7f5765a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5760102870 0x7f576019ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:53236/0 (socket says 172.21.15.123:53236) 2024-05-08T06:12:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.085+0000 7f5765a00640 1 -- 172.21.15.123:0/3059775543 learned_addr learned my addr 172.21.15.123:0/3059775543 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.085+0000 7f5765000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5760103a60 0x7f576019b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.085+0000 7f5766400640 1 --2- 172.21.15.123:0/3059775543 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5760104420 0x7f57601a2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.085+0000 7f5765a00640 1 -- 172.21.15.123:0/3059775543 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5760104420 msgr2=0x7f57601a2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:02.086 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.085+0000 7f5765a00640 1 --2- 172.21.15.123:0/3059775543 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5760104420 0x7f57601a2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:02.086 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.085+0000 7f5765a00640 1 -- 172.21.15.123:0/3059775543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5760103a60 msgr2=0x7f576019b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:02.086 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.085+0000 7f5765a00640 1 --2- 172.21.15.123:0/3059775543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5760103a60 0x7f576019b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:02.086 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.085+0000 7f5765a00640 1 -- 172.21.15.123:0/3059775543 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57601a28c0 con 0x7f5760102870 2024-05-08T06:12:02.086 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.085+0000 7f5765000640 1 --2- 172.21.15.123:0/3059775543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5760103a60 0x7f576019b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:12:02.086 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.086+0000 7f5765a00640 1 --2- 172.21.15.123:0/3059775543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5760102870 0x7f576019ac80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f575400e9f0 tx=0x7f575400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:02.087 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.086+0000 7f575e000640 1 -- 172.21.15.123:0/3059775543 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575400ccc0 con 0x7f5760102870 2024-05-08T06:12:02.087 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.086+0000 7f575e000640 1 -- 172.21.15.123:0/3059775543 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57540035f0 con 0x7f5760102870 2024-05-08T06:12:02.087 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.086+0000 7f575e000640 1 -- 172.21.15.123:0/3059775543 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575400ce60 con 0x7f5760102870 2024-05-08T06:12:02.087 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.086+0000 7f5766e00640 1 -- 172.21.15.123:0/3059775543 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57601a2bb0 con 0x7f5760102870 2024-05-08T06:12:02.087 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.086+0000 7f5766e00640 1 -- 172.21.15.123:0/3059775543 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57601a3080 con 0x7f5760102870 2024-05-08T06:12:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.091+0000 7f576b21d740 1 Processor -- start 2024-05-08T06:12:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.091+0000 7f576b21d740 1 -- start start 2024-05-08T06:12:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.091+0000 7f576b21d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca4aa6d320 0x55ca4aa6d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.091+0000 7f576b21d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca4aa6dd10 0x55ca4aa72810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.091+0000 7f576b21d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca4aa72e80 0x55ca4aa75310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.091+0000 7f576b21d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ca4a9d1550 con 0x55ca4aa72e80 2024-05-08T06:12:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.091+0000 7f576b21d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55ca4a976bc0 con 0x55ca4aa6dd10 2024-05-08T06:12:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.091+0000 7f576b21d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55ca4a9cd510 con 0x55ca4aa6d320 2024-05-08T06:12:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.092+0000 7f5766400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca4aa6d320 0x55ca4aa6d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.092+0000 7f575e000640 1 -- 172.21.15.123:0/3059775543 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7f5754004000 con 0x7f5760102870 2024-05-08T06:12:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.092+0000 7f5765000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca4aa72e80 0x55ca4aa75310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.092+0000 7f5765a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca4aa6dd10 0x55ca4aa72810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.092+0000 7f5765a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca4aa6dd10 0x55ca4aa72810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:57392/0 (socket says 172.21.15.123:57392) 2024-05-08T06:12:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.092+0000 7f5765a00640 1 -- 172.21.15.123:0/2240904145 learned_addr learned my addr 172.21.15.123:0/2240904145 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.093+0000 7f5765a00640 1 -- 172.21.15.123:0/2240904145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca4aa6d320 msgr2=0x55ca4aa6d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.093+0000 7f5765a00640 1 --2- 172.21.15.123:0/2240904145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca4aa6d320 0x55ca4aa6d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.093+0000 7f5765a00640 1 -- 172.21.15.123:0/2240904145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca4aa72e80 msgr2=0x55ca4aa75310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.093+0000 7f5765a00640 1 --2- 172.21.15.123:0/2240904145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca4aa72e80 0x55ca4aa75310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.093+0000 7f5765a00640 1 -- 172.21.15.123:0/2240904145 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca4a6e3210 con 0x55ca4aa6dd10 2024-05-08T06:12:02.093 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.093+0000 7f5765000640 1 --2- 172.21.15.123:0/2240904145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca4aa72e80 0x55ca4aa75310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:12:02.094 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.093+0000 7f5765a00640 1 --2- 172.21.15.123:0/2240904145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca4aa6dd10 0x55ca4aa72810 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f57540ea410 tx=0x7f57540ea500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f78aaae583a0a78 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:02.094 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.094+0000 7f5752a00640 1 -- 172.21.15.123:0/2240904145 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57540f6070 con 0x55ca4aa6dd10 2024-05-08T06:12:02.094 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.094+0000 7f5752a00640 1 -- 172.21.15.123:0/2240904145 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57540e40d0 con 0x55ca4aa6dd10 2024-05-08T06:12:02.094 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.094+0000 7f5752a00640 1 -- 172.21.15.123:0/2240904145 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57540ee740 con 0x55ca4aa6dd10 2024-05-08T06:12:02.094 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.094+0000 7f575e000640 1 --2- 172.21.15.123:0/3059775543 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f57480b27e0 0x7f57480b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:02.095 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.095+0000 7f575e000640 1 -- 172.21.15.123:0/3059775543 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f57540e24a0 con 0x7f5760102870 2024-05-08T06:12:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.095+0000 7f5766400640 1 --2- 172.21.15.123:0/3059775543 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f57480b27e0 0x7f57480b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.095+0000 7f576b21d740 1 -- 172.21.15.123:0/2240904145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca4aa6dd10 msgr2=0x55ca4aa72810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.095+0000 7f576b21d740 1 --2- 172.21.15.123:0/2240904145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca4aa6dd10 0x55ca4aa72810 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f57540ea410 tx=0x7f57540ea500 comp rx=0 tx=0).stop 2024-05-08T06:12:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.095+0000 7f576b21d740 1 -- 172.21.15.123:0/2240904145 shutdown_connections 2024-05-08T06:12:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.095+0000 7f576b21d740 1 --2- 172.21.15.123:0/2240904145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca4aa72e80 0x55ca4aa75310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.095+0000 7f576b21d740 1 --2- 172.21.15.123:0/2240904145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca4aa6dd10 0x55ca4aa72810 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.095+0000 7f576b21d740 1 --2- 172.21.15.123:0/2240904145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca4aa6d320 0x55ca4aa6d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.095+0000 7f576b21d740 1 -- 172.21.15.123:0/2240904145 >> 172.21.15.123:0/2240904145 conn(0x55ca4aa35230 msgr2=0x55ca4aa68da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.095+0000 7f576b21d740 1 -- 172.21.15.123:0/2240904145 shutdown_connections 2024-05-08T06:12:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.095+0000 7f576b21d740 1 -- 172.21.15.123:0/2240904145 wait complete. 2024-05-08T06:12:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.096+0000 7f5766400640 1 --2- 172.21.15.123:0/3059775543 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f57480b27e0 0x7f57480b4ca0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x55ca4aa6e570 tx=0x7f5758009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.096+0000 7f576b21d740 1 -- start start 2024-05-08T06:12:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.096+0000 7f576b21d740 1 Processor -- start 2024-05-08T06:12:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.097+0000 7f576b21d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca4aa6d320 0x55ca4aa77350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.097+0000 7f576b21d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca4aa6dd10 0x55ca4aa77890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.097+0000 7f576b21d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca4aa72e80 0x55ca4aa7e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.097+0000 7f576b21d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ca4aa75e30 con 0x55ca4aa72e80 2024-05-08T06:12:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.097+0000 7f576b21d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55ca4a98a660 con 0x55ca4aa6d320 2024-05-08T06:12:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.097+0000 7f576b21d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55ca4aa75fb0 con 0x55ca4aa6dd10 2024-05-08T06:12:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.097+0000 7f576b21d740 10 client.0 _ll_register_callbacks cb 0x55ca4a954400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:12:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.097+0000 7f5765a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca4aa6d320 0x55ca4aa77350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.097+0000 7f5766400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca4aa72e80 0x55ca4aa7e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.097+0000 7f5765a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca4aa6d320 0x55ca4aa77350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:57400/0 (socket says 172.21.15.123:57400) 2024-05-08T06:12:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.097+0000 7f5765a00640 1 -- 172.21.15.123:0/384940553 learned_addr learned my addr 172.21.15.123:0/384940553 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.097+0000 7f5765000640 1 --2- 172.21.15.123:0/384940553 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca4aa6dd10 0x55ca4aa77890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.097+0000 7f5765a00640 1 -- 172.21.15.123:0/384940553 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca4aa6dd10 msgr2=0x55ca4aa77890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.097+0000 7f5765a00640 1 --2- 172.21.15.123:0/384940553 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca4aa6dd10 0x55ca4aa77890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.097+0000 7f5765a00640 1 -- 172.21.15.123:0/384940553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca4aa72e80 msgr2=0x55ca4aa7e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.097+0000 7f5765a00640 1 --2- 172.21.15.123:0/384940553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca4aa72e80 0x55ca4aa7e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.097+0000 7f5765a00640 1 -- 172.21.15.123:0/384940553 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca4a7af620 con 0x55ca4aa6d320 2024-05-08T06:12:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.098+0000 7f5765a00640 1 --2- 172.21.15.123:0/384940553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca4aa6d320 0x55ca4aa77350 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f57540a9a00 tx=0x7f57540e47b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db26ec7c444782c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.098+0000 7f5747400640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-05-08T06:12:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.098+0000 7f5747400640 1 -- 172.21.15.123:0/384940553 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57540a5930 con 0x55ca4aa6d320 2024-05-08T06:12:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.098+0000 7f576b21d740 1 -- 172.21.15.123:0/384940553 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57540e40d0 con 0x55ca4aa6d320 2024-05-08T06:12:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.098+0000 7f5747400640 1 -- 172.21.15.123:0/384940553 <== mon.2 v2:172.21.15.163:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f57540037a0 con 0x55ca4aa6d320 2024-05-08T06:12:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.098+0000 7f5747400640 1 -- 172.21.15.123:0/384940553 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f57540042f0 con 0x55ca4aa6d320 2024-05-08T06:12:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.098+0000 7f576b21d740 10 client.25433 fetch_fsmap learned FSMap version 24 2024-05-08T06:12:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.098+0000 7f576b21d740 1 -- 172.21.15.123:0/384940553 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ca4aa75e30 con 0x55ca4aa6d320 2024-05-08T06:12:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.099+0000 7f5747400640 1 -- 172.21.15.123:0/384940553 <== mon.2 v2:172.21.15.163:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57540e40d0 con 0x55ca4aa6d320 2024-05-08T06:12:02.101 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.100+0000 7f5747400640 1 -- 172.21.15.123:0/384940553 <== mon.2 v2:172.21.15.163:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f57540e2080 con 0x55ca4aa6d320 2024-05-08T06:12:02.101 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.100+0000 7f576b21d740 10 client.25433 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:12:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.100+0000 7f576b21d740 10 client.25433 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T06:12:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.100+0000 7f576b21d740 20 client.25433 populate_metadata read hostname 'smithi123' 2024-05-08T06:12:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.100+0000 7f576b21d740 1 --2- 172.21.15.123:0/384940553 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55ca4aa82a30 0x55ca4aa84e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.100+0000 7f576b21d740 4 client.25433 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:12:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.100+0000 7f576b21d740 1 -- 172.21.15.123:0/384940553 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca4a9df160 con 0x55ca4aa82a30 2024-05-08T06:12:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.100+0000 7f5765000640 1 --2- 172.21.15.123:0/384940553 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55ca4aa82a30 0x55ca4aa84e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.101+0000 7f5765000640 1 --2- 172.21.15.123:0/384940553 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55ca4aa82a30 0x55ca4aa84e20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f115e2e1e37c8ef server_cookie=8faab15ce01c3acb in_seq=0 out_seq=0 2024-05-08T06:12:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.101+0000 7f5747400640 10 client.25433 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.102+0000 7f5747400640 1 -- 172.21.15.123:0/384940553 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ca4a9df160 con 0x55ca4aa82a30 2024-05-08T06:12:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.102+0000 7f5747400640 10 client.25433 handle_command_reply: tid=0 2024-05-08T06:12:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.102+0000 7f576b21d740 1 client.25433 shutdown 2024-05-08T06:12:02.103 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.102+0000 7f576b21d740 1 -- 172.21.15.123:0/384940553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca4aa6d320 msgr2=0x55ca4aa77350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:02.103 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.102+0000 7f576b21d740 1 --2- 172.21.15.123:0/384940553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca4aa6d320 0x55ca4aa77350 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f57540a9a00 tx=0x7f57540e47b0 comp rx=0 tx=0).stop 2024-05-08T06:12:02.103 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.102+0000 7f576b21d740 1 -- 172.21.15.123:0/384940553 shutdown_connections 2024-05-08T06:12:02.103 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.102+0000 7f576b21d740 1 --2- 172.21.15.123:0/384940553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca4aa72e80 0x55ca4aa7e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:02.103 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.102+0000 7f576b21d740 1 --2- 172.21.15.123:0/384940553 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55ca4aa82a30 0x55ca4aa84e20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:02.103 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.103+0000 7f576b21d740 1 --2- 172.21.15.123:0/384940553 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca4aa6dd10 0x55ca4aa77890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.103+0000 7f576b21d740 1 --2- 172.21.15.123:0/384940553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca4aa6d320 0x55ca4aa77350 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.103+0000 7f576b21d740 1 -- 172.21.15.123:0/384940553 >> 172.21.15.123:0/384940553 conn(0x55ca4aa35230 msgr2=0x55ca4aa67e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.103+0000 7f5747400640 0 client.25433 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.103+0000 7f576b21d740 1 -- 172.21.15.123:0/384940553 shutdown_connections 2024-05-08T06:12:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.103+0000 7f576b21d740 1 -- 172.21.15.123:0/384940553 wait complete. 2024-05-08T06:12:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.104+0000 7f576b21d740 20 client.25433 trim_cache size 0 max 16384 2024-05-08T06:12:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.120+0000 7f576b21d740 1 Processor -- start 2024-05-08T06:12:02.120 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.120+0000 7f576b21d740 1 -- start start 2024-05-08T06:12:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.120+0000 7f576b21d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca4aa6d320 0x55ca4aa77670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.120+0000 7f576b21d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca4aa6dd10 0x55ca4aa896b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.120+0000 7f576b21d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca4aa72e80 0x55ca4aa89bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.120+0000 7f576b21d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f57601aa310 con 0x55ca4aa72e80 2024-05-08T06:12:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.120+0000 7f576b21d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f57601a7be0 con 0x55ca4aa6d320 2024-05-08T06:12:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.120+0000 7f576b21d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f57601a5a50 con 0x55ca4aa6dd10 2024-05-08T06:12:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.121+0000 7f5765a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca4aa6d320 0x55ca4aa77670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.121+0000 7f5765a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca4aa6d320 0x55ca4aa77670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:57414/0 (socket says 172.21.15.123:57414) 2024-05-08T06:12:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.121+0000 7f5765a00640 1 -- 172.21.15.123:0/1760708844 learned_addr learned my addr 172.21.15.123:0/1760708844 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.121+0000 7f5765000640 1 --2- 172.21.15.123:0/1760708844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca4aa6dd10 0x55ca4aa896b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.121+0000 7f5766400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca4aa72e80 0x55ca4aa89bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.121+0000 7f5765a00640 1 -- 172.21.15.123:0/1760708844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca4aa6dd10 msgr2=0x55ca4aa896b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.121+0000 7f5765a00640 1 --2- 172.21.15.123:0/1760708844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca4aa6dd10 0x55ca4aa896b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.121+0000 7f5765a00640 1 -- 172.21.15.123:0/1760708844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca4aa72e80 msgr2=0x55ca4aa89bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.121+0000 7f5765a00640 1 --2- 172.21.15.123:0/1760708844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca4aa72e80 0x55ca4aa89bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.121+0000 7f5765a00640 1 -- 172.21.15.123:0/1760708844 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca4aa76100 con 0x55ca4aa6d320 2024-05-08T06:12:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.121+0000 7f5766400640 1 --2- 172.21.15.123:0/1760708844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca4aa72e80 0x55ca4aa89bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:12:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.122+0000 7f5765a00640 1 --2- 172.21.15.123:0/1760708844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca4aa6d320 0x55ca4aa77670 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f57540a6ce0 tx=0x7f57540e5ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f48d019c589959b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.122+0000 7f5752000640 1 -- 172.21.15.123:0/1760708844 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57540a91e0 con 0x55ca4aa6d320 2024-05-08T06:12:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.122+0000 7f5752000640 1 -- 172.21.15.123:0/1760708844 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57540ea290 con 0x55ca4aa6d320 2024-05-08T06:12:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.122+0000 7f5752000640 1 -- 172.21.15.123:0/1760708844 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57540a93c0 con 0x55ca4aa6d320 2024-05-08T06:12:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.123+0000 7f576b21d740 1 -- 172.21.15.123:0/1760708844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca4aa6d320 msgr2=0x55ca4aa77670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.123+0000 7f576b21d740 1 --2- 172.21.15.123:0/1760708844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca4aa6d320 0x55ca4aa77670 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f57540a6ce0 tx=0x7f57540e5ee0 comp rx=0 tx=0).stop 2024-05-08T06:12:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.123+0000 7f576b21d740 1 -- 172.21.15.123:0/1760708844 shutdown_connections 2024-05-08T06:12:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.123+0000 7f576b21d740 1 --2- 172.21.15.123:0/1760708844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca4aa72e80 0x55ca4aa89bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.123+0000 7f576b21d740 1 --2- 172.21.15.123:0/1760708844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca4aa6dd10 0x55ca4aa896b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.123+0000 7f576b21d740 1 --2- 172.21.15.123:0/1760708844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca4aa6d320 0x55ca4aa77670 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.123+0000 7f576b21d740 1 -- 172.21.15.123:0/1760708844 >> 172.21.15.123:0/1760708844 conn(0x55ca4aa35230 msgr2=0x55ca4aa76f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.123+0000 7f576b21d740 1 -- 172.21.15.123:0/1760708844 shutdown_connections 2024-05-08T06:12:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.123+0000 7f576b21d740 1 -- 172.21.15.123:0/1760708844 wait complete. 2024-05-08T06:12:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.124+0000 7f576b21d740 1 -- start start 2024-05-08T06:12:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.124+0000 7f576b21d740 1 Processor -- start 2024-05-08T06:12:02.126 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.124+0000 7f576b21d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca4aa6d320 0x55ca4aa7b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:02.126 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.124+0000 7f576b21d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca4aa6dd10 0x55ca4aa7b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:02.126 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.124+0000 7f576b21d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca4aa72e80 0x55ca4aa7c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.124+0000 7f576b21d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f57601a60a0 con 0x55ca4aa6d320 2024-05-08T06:12:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.124+0000 7f576b21d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f57601a7510 con 0x55ca4aa6dd10 2024-05-08T06:12:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.124+0000 7f576b21d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f57601a6770 con 0x55ca4aa72e80 2024-05-08T06:12:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.124+0000 7f576b21d740 10 client.0 _ll_register_callbacks cb 0x7f57540f4070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:12:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.125+0000 7f5765a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca4aa6d320 0x55ca4aa7b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.125+0000 7f5765a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca4aa6d320 0x55ca4aa7b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50664/0 (socket says 172.21.15.123:50664) 2024-05-08T06:12:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.125+0000 7f5765a00640 1 -- 172.21.15.123:0/108675459 learned_addr learned my addr 172.21.15.123:0/108675459 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.125+0000 7f5765000640 1 --2- 172.21.15.123:0/108675459 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca4aa6dd10 0x55ca4aa7b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.125+0000 7f5766400640 1 --2- 172.21.15.123:0/108675459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca4aa72e80 0x55ca4aa7c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.125+0000 7f5765000640 1 -- 172.21.15.123:0/108675459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca4aa72e80 msgr2=0x55ca4aa7c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.125+0000 7f5765000640 1 --2- 172.21.15.123:0/108675459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca4aa72e80 0x55ca4aa7c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.125+0000 7f5765000640 1 -- 172.21.15.123:0/108675459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca4aa6d320 msgr2=0x55ca4aa7b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.126+0000 7f5765000640 1 --2- 172.21.15.123:0/108675459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca4aa6d320 0x55ca4aa7b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.126+0000 7f5765000640 1 -- 172.21.15.123:0/108675459 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca4aa7cf40 con 0x55ca4aa6dd10 2024-05-08T06:12:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.126+0000 7f5765a00640 1 --2- 172.21.15.123:0/108675459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca4aa6d320 0x55ca4aa7b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:12:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.126+0000 7f5765000640 1 --2- 172.21.15.123:0/108675459 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca4aa6dd10 0x55ca4aa7b9e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f574c035040 tx=0x7f574c005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a00e8695b338419 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.126+0000 7f5747400640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-05-08T06:12:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.126+0000 7f5747400640 1 -- 172.21.15.123:0/108675459 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574c037040 con 0x55ca4aa6dd10 2024-05-08T06:12:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.126+0000 7f576b21d740 1 -- 172.21.15.123:0/108675459 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57540ea290 con 0x55ca4aa6dd10 2024-05-08T06:12:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.126+0000 7f5747400640 1 -- 172.21.15.123:0/108675459 <== mon.2 v2:172.21.15.163:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f574c030750 con 0x55ca4aa6dd10 2024-05-08T06:12:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.127+0000 7f5747400640 1 -- 172.21.15.123:0/108675459 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f574c004040 con 0x55ca4aa6dd10 2024-05-08T06:12:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.127+0000 7f576b21d740 10 client.25439 fetch_fsmap learned FSMap version 24 2024-05-08T06:12:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.127+0000 7f576b21d740 1 -- 172.21.15.123:0/108675459 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ca4aa7e2c0 con 0x55ca4aa6dd10 2024-05-08T06:12:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.127+0000 7f5747400640 1 -- 172.21.15.123:0/108675459 <== mon.2 v2:172.21.15.163:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57540ea290 con 0x55ca4aa6dd10 2024-05-08T06:12:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.127+0000 7f5747400640 1 -- 172.21.15.123:0/108675459 <== mon.2 v2:172.21.15.163:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f574c049030 con 0x55ca4aa6dd10 2024-05-08T06:12:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.127+0000 7f576b21d740 10 client.25439 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:12:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.127+0000 7f576b21d740 10 client.25439 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T06:12:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.127+0000 7f576b21d740 20 client.25439 populate_metadata read hostname 'smithi123' 2024-05-08T06:12:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.127+0000 7f576b21d740 1 --2- 172.21.15.123:0/108675459 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55ca4aaed980 0x55ca4aaefda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.127+0000 7f576b21d740 4 client.25439 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T06:12:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.127+0000 7f576b21d740 1 -- 172.21.15.123:0/108675459 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ca4a9d8fd0 con 0x55ca4aaed980 2024-05-08T06:12:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.128+0000 7f5765a00640 1 --2- 172.21.15.123:0/108675459 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55ca4aaed980 0x55ca4aaefda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:02.130 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.128+0000 7f5765a00640 1 --2- 172.21.15.123:0/108675459 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55ca4aaed980 0x55ca4aaefda0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58e52b837784404b server_cookie=5f66dddd3745e2fc in_seq=0 out_seq=0 2024-05-08T06:12:02.130 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.128+0000 7f5747400640 10 client.25439 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:02.133 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.132+0000 7f5747400640 1 -- 172.21.15.123:0/108675459 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ca4a9d8fd0 con 0x55ca4aaed980 2024-05-08T06:12:02.133 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.132+0000 7f5747400640 10 client.25439 handle_command_reply: tid=0 2024-05-08T06:12:02.133 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.132+0000 7f576b21d740 1 client.25439 shutdown 2024-05-08T06:12:02.133 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.132+0000 7f576b21d740 1 -- 172.21.15.123:0/108675459 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca4aa6dd10 msgr2=0x55ca4aa7b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:02.133 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.132+0000 7f576b21d740 1 --2- 172.21.15.123:0/108675459 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca4aa6dd10 0x55ca4aa7b9e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f574c035040 tx=0x7f574c005c90 comp rx=0 tx=0).stop 2024-05-08T06:12:02.133 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.133+0000 7f576b21d740 1 -- 172.21.15.123:0/108675459 shutdown_connections 2024-05-08T06:12:02.133 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.133+0000 7f576b21d740 1 --2- 172.21.15.123:0/108675459 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55ca4aaed980 0x55ca4aaefda0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:02.133 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.133+0000 7f576b21d740 1 --2- 172.21.15.123:0/108675459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ca4aa72e80 0x55ca4aa7c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:02.133 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.133+0000 7f576b21d740 1 --2- 172.21.15.123:0/108675459 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ca4aa6dd10 0x55ca4aa7b9e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:02.133 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.133+0000 7f576b21d740 1 --2- 172.21.15.123:0/108675459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca4aa6d320 0x55ca4aa7b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:02.133 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.133+0000 7f576b21d740 1 -- 172.21.15.123:0/108675459 >> 172.21.15.123:0/108675459 conn(0x55ca4aa35230 msgr2=0x55ca4aa6ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:02.133 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.133+0000 7f5747400640 0 client.25439 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:02.134 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.133+0000 7f576b21d740 1 -- 172.21.15.123:0/108675459 shutdown_connections 2024-05-08T06:12:02.134 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.133+0000 7f576b21d740 1 -- 172.21.15.123:0/108675459 wait complete. 2024-05-08T06:12:02.134 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.133+0000 7f576b21d740 20 client.25439 trim_cache size 0 max 16384 2024-05-08T06:12:02.134 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout:{ 2024-05-08T06:12:02.134 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout: "message": "", 2024-05-08T06:12:02.134 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout: "return_code": 0 2024-05-08T06:12:02.134 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout:} 2024-05-08T06:12:02.135 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.134+0000 7f5766e00640 1 -- 172.21.15.123:0/3059775543 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f57480b27e0 msgr2=0x7f57480b4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:02.135 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.134+0000 7f5766e00640 1 --2- 172.21.15.123:0/3059775543 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f57480b27e0 0x7f57480b4ca0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x55ca4aa6e570 tx=0x7f5758009500 comp rx=0 tx=0).stop 2024-05-08T06:12:02.135 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.134+0000 7f5766e00640 1 -- 172.21.15.123:0/3059775543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5760102870 msgr2=0x7f576019ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:02.135 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.134+0000 7f5766e00640 1 --2- 172.21.15.123:0/3059775543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5760102870 0x7f576019ac80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f575400e9f0 tx=0x7f575400eeb0 comp rx=0 tx=0).stop 2024-05-08T06:12:02.135 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.134+0000 7f5766e00640 1 -- 172.21.15.123:0/3059775543 shutdown_connections 2024-05-08T06:12:02.135 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.134+0000 7f5766e00640 1 --2- 172.21.15.123:0/3059775543 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f57480b27e0 0x7f57480b4ca0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:02.135 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.134+0000 7f5766e00640 1 --2- 172.21.15.123:0/3059775543 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5760104420 0x7f57601a2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:02.135 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.134+0000 7f5766e00640 1 --2- 172.21.15.123:0/3059775543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5760103a60 0x7f576019b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:02.135 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.134+0000 7f5766e00640 1 --2- 172.21.15.123:0/3059775543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5760102870 0x7f576019ac80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:02.136 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.134+0000 7f5766e00640 1 -- 172.21.15.123:0/3059775543 >> 172.21.15.123:0/3059775543 conn(0x7f57600fdde0 msgr2=0x7f5760109200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:02.136 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.134+0000 7f5766e00640 1 -- 172.21.15.123:0/3059775543 shutdown_connections 2024-05-08T06:12:02.136 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:02.134+0000 7f5766e00640 1 -- 172.21.15.123:0/3059775543 wait complete. 2024-05-08T06:12:02.191 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-08T06:12:02.619 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:12:02.774 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:02 smithi123 ceph-mon[29787]: scrub queued for path: / 2024-05-08T06:12:02.774 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:02 smithi123 ceph-mon[29787]: scrub summary: idle+waiting paths [/] 2024-05-08T06:12:02.774 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:02 smithi123 ceph-mon[29787]: scrub summary: active paths [/] 2024-05-08T06:12:02.774 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:02 smithi123 ceph-mon[29787]: scrub summary: idle+waiting paths [/] 2024-05-08T06:12:02.774 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:02 smithi123 ceph-mon[29787]: pgmap v507: 97 pgs: 97 active+clean; 676 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:12:02.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:02 smithi163 ceph-mon[34520]: scrub queued for path: / 2024-05-08T06:12:02.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:02 smithi163 ceph-mon[34520]: scrub summary: idle+waiting paths [/] 2024-05-08T06:12:02.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:02 smithi163 ceph-mon[34520]: scrub summary: active paths [/] 2024-05-08T06:12:02.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:02 smithi163 ceph-mon[34520]: scrub summary: idle+waiting paths [/] 2024-05-08T06:12:02.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:02 smithi163 ceph-mon[34520]: pgmap v507: 97 pgs: 97 active+clean; 676 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:12:02.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:02 smithi192 ceph-mon[34664]: scrub queued for path: / 2024-05-08T06:12:02.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:02 smithi192 ceph-mon[34664]: scrub summary: idle+waiting paths [/] 2024-05-08T06:12:02.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:02 smithi192 ceph-mon[34664]: scrub summary: active paths [/] 2024-05-08T06:12:02.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:02 smithi192 ceph-mon[34664]: scrub summary: idle+waiting paths [/] 2024-05-08T06:12:02.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:02 smithi192 ceph-mon[34664]: pgmap v507: 97 pgs: 97 active+clean; 676 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:12:04.748 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:04 smithi123 ceph-mon[29787]: pgmap v508: 97 pgs: 97 active+clean; 676 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-08T06:12:04.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:04 smithi163 ceph-mon[34520]: pgmap v508: 97 pgs: 97 active+clean; 676 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-08T06:12:04.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:04 smithi192 ceph-mon[34664]: pgmap v508: 97 pgs: 97 active+clean; 676 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-08T06:12:05.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.087+0000 7f13d1800700 1 -- 172.21.15.123:0/1612769212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13cc10ff00 msgr2=0x7f13cc1122f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:05.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.087+0000 7f13d1800700 1 --2- 172.21.15.123:0/1612769212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13cc10ff00 0x7f13cc1122f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f13c400b0a0 tx=0x7f13c400b3b0 comp rx=0 tx=0).stop 2024-05-08T06:12:05.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.087+0000 7f13d1800700 1 -- 172.21.15.123:0/1612769212 shutdown_connections 2024-05-08T06:12:05.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.087+0000 7f13d1800700 1 --2- 172.21.15.123:0/1612769212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13cc10ff00 0x7f13cc1122f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:05.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.087+0000 7f13d1800700 1 --2- 172.21.15.123:0/1612769212 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13cc0756d0 0x7f13cc075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:05.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.087+0000 7f13d1800700 1 --2- 172.21.15.123:0/1612769212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13cc075ff0 0x7f13cc10f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:05.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.087+0000 7f13d1800700 1 -- 172.21.15.123:0/1612769212 >> 172.21.15.123:0/1612769212 conn(0x7f13cc07a600 msgr2=0x7f13cc07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:05.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.087+0000 7f13d1800700 1 -- 172.21.15.123:0/1612769212 shutdown_connections 2024-05-08T06:12:05.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.087+0000 7f13d1800700 1 -- 172.21.15.123:0/1612769212 wait complete. 2024-05-08T06:12:05.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.088+0000 7f13d1800700 1 Processor -- start 2024-05-08T06:12:05.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.088+0000 7f13d1800700 1 -- start start 2024-05-08T06:12:05.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.089+0000 7f13d1800700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13cc0756d0 0x7f13cc19bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:05.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.089+0000 7f13d1800700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13cc075ff0 0x7f13cc19c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:05.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.089+0000 7f13d1800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13cc10ff00 0x7f13cc1a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:05.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.089+0000 7f13d1800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc115890 con 0x7f13cc10ff00 2024-05-08T06:12:05.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.089+0000 7f13d1800700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc115710 con 0x7f13cc0756d0 2024-05-08T06:12:05.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.089+0000 7f13d1800700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc115a10 con 0x7f13cc075ff0 2024-05-08T06:12:05.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.089+0000 7f13cb400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13cc10ff00 0x7f13cc1a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:05.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.089+0000 7f13cb400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13cc10ff00 0x7f13cc1a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56138/0 (socket says 172.21.15.123:56138) 2024-05-08T06:12:05.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.089+0000 7f13cb400700 1 -- 172.21.15.123:0/2655622659 learned_addr learned my addr 172.21.15.123:0/2655622659 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:05.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.089+0000 7f13caa00700 1 --2- 172.21.15.123:0/2655622659 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13cc0756d0 0x7f13cc19bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:05.090 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.089+0000 7f13ca000700 1 --2- 172.21.15.123:0/2655622659 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13cc075ff0 0x7f13cc19c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:05.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.090+0000 7f13cb400700 1 -- 172.21.15.123:0/2655622659 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13cc075ff0 msgr2=0x7f13cc19c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:05.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.090+0000 7f13cb400700 1 --2- 172.21.15.123:0/2655622659 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13cc075ff0 0x7f13cc19c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:05.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.090+0000 7f13cb400700 1 -- 172.21.15.123:0/2655622659 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13cc0756d0 msgr2=0x7f13cc19bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:05.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.090+0000 7f13cb400700 1 --2- 172.21.15.123:0/2655622659 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13cc0756d0 0x7f13cc19bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:05.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.090+0000 7f13cb400700 1 -- 172.21.15.123:0/2655622659 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13cc1a0d40 con 0x7f13cc10ff00 2024-05-08T06:12:05.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.090+0000 7f13ca000700 1 --2- 172.21.15.123:0/2655622659 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13cc075ff0 0x7f13cc19c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:12:05.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.090+0000 7f13caa00700 1 --2- 172.21.15.123:0/2655622659 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13cc0756d0 0x7f13cc19bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:12:05.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.090+0000 7f13cb400700 1 --2- 172.21.15.123:0/2655622659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13cc10ff00 0x7f13cc1a0600 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f13c400b780 tx=0x7f13c4004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:05.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.090+0000 7f13c3400700 1 -- 172.21.15.123:0/2655622659 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c4021070 con 0x7f13cc10ff00 2024-05-08T06:12:05.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.090+0000 7f13d1800700 1 -- 172.21.15.123:0/2655622659 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13cc1a0fd0 con 0x7f13cc10ff00 2024-05-08T06:12:05.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.090+0000 7f13c3400700 1 -- 172.21.15.123:0/2655622659 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13c4007e30 con 0x7f13cc10ff00 2024-05-08T06:12:05.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.090+0000 7f13c3400700 1 -- 172.21.15.123:0/2655622659 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c40045b0 con 0x7f13cc10ff00 2024-05-08T06:12:05.092 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.091+0000 7f13d1800700 1 -- 172.21.15.123:0/2655622659 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13cc1a88b0 con 0x7f13cc10ff00 2024-05-08T06:12:05.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.094+0000 7f13c3400700 1 -- 172.21.15.123:0/2655622659 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7f13c4004750 con 0x7f13cc10ff00 2024-05-08T06:12:05.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.096+0000 7f13dc23bb80 1 Processor -- start 2024-05-08T06:12:05.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.096+0000 7f13dc23bb80 1 -- start start 2024-05-08T06:12:05.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.097+0000 7f13dc23bb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5445eabc0 0x55b5445eafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:05.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.097+0000 7f13dc23bb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5445eb4e0 0x55b5445e52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:05.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.097+0000 7f13dc23bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5445e58f0 0x55b5445e5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:05.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.097+0000 7f13dc23bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b544537a20 con 0x55b5445e58f0 2024-05-08T06:12:05.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.097+0000 7f13dc23bb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b5444d5760 con 0x55b5445eb4e0 2024-05-08T06:12:05.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.097+0000 7f13dc23bb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b5445339e0 con 0x55b5445eabc0 2024-05-08T06:12:05.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.097+0000 7f13c3400700 1 --2- 172.21.15.123:0/2655622659 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f13b00b2900 0x7f13b00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:05.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.097+0000 7f13c3400700 1 -- 172.21.15.123:0/2655622659 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f13c400f020 con 0x7f13cc10ff00 2024-05-08T06:12:05.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.097+0000 7f13ca000700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5445eabc0 0x55b5445eafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:05.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.097+0000 7f13ca000700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5445eabc0 0x55b5445eafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:41832/0 (socket says 172.21.15.123:41832) 2024-05-08T06:12:05.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.097+0000 7f13ca000700 1 -- 172.21.15.123:0/4226684448 learned_addr learned my addr 172.21.15.123:0/4226684448 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:05.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.097+0000 7f13caa00700 1 --2- 172.21.15.123:0/4226684448 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5445eb4e0 0x55b5445e52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:05.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.097+0000 7f13ca000700 1 --2- 172.21.15.123:0/4226684448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5445e58f0 0x55b5445e5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:05.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.097+0000 7f13cb400700 1 --2- 172.21.15.123:0/2655622659 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f13b00b2900 0x7f13b00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:05.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.098+0000 7f13ca000700 1 -- 172.21.15.123:0/4226684448 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5445eb4e0 msgr2=0x55b5445e52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:05.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.098+0000 7f13ca000700 1 --2- 172.21.15.123:0/4226684448 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5445eb4e0 0x55b5445e52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:05.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.098+0000 7f13ca000700 1 -- 172.21.15.123:0/4226684448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5445e58f0 msgr2=0x55b5445e5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:05.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.098+0000 7f13ca000700 1 --2- 172.21.15.123:0/4226684448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5445e58f0 0x55b5445e5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:05.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.098+0000 7f13ca000700 1 -- 172.21.15.123:0/4226684448 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5444848d0 con 0x55b5445eabc0 2024-05-08T06:12:05.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.098+0000 7f13caa00700 1 --2- 172.21.15.123:0/4226684448 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5445eb4e0 0x55b5445e52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:12:05.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.098+0000 7f13ca000700 1 --2- 172.21.15.123:0/4226684448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5445eabc0 0x55b5445eafa0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f13b8009fd0 tx=0x7f13b8009c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f64cb9f875a282 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:05.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.098+0000 7f13cb400700 1 --2- 172.21.15.123:0/2655622659 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f13b00b2900 0x7f13b00b4dc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f13c400b070 tx=0x7f13c4009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:05.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.098+0000 7f13b7e00700 1 -- 172.21.15.123:0/4226684448 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b8004030 con 0x55b5445eabc0 2024-05-08T06:12:05.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.098+0000 7f13b7e00700 1 -- 172.21.15.123:0/4226684448 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13b80116f0 con 0x55b5445eabc0 2024-05-08T06:12:05.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.098+0000 7f13b7e00700 1 -- 172.21.15.123:0/4226684448 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b8003680 con 0x55b5445eabc0 2024-05-08T06:12:05.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.099+0000 7f13dc23bb80 1 -- 172.21.15.123:0/4226684448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5445eabc0 msgr2=0x55b5445eafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:05.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.099+0000 7f13dc23bb80 1 --2- 172.21.15.123:0/4226684448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5445eabc0 0x55b5445eafa0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f13b8009fd0 tx=0x7f13b8009c10 comp rx=0 tx=0).stop 2024-05-08T06:12:05.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.099+0000 7f13dc23bb80 1 -- 172.21.15.123:0/4226684448 shutdown_connections 2024-05-08T06:12:05.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.099+0000 7f13dc23bb80 1 --2- 172.21.15.123:0/4226684448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5445e58f0 0x55b5445e5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:05.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.099+0000 7f13dc23bb80 1 --2- 172.21.15.123:0/4226684448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5445eabc0 0x55b5445eafa0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:05.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.099+0000 7f13dc23bb80 1 --2- 172.21.15.123:0/4226684448 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5445eb4e0 0x55b5445e52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:05.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.099+0000 7f13dc23bb80 1 -- 172.21.15.123:0/4226684448 >> 172.21.15.123:0/4226684448 conn(0x55b54455d6d0 msgr2=0x55b5445dfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:05.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.100+0000 7f13dc23bb80 1 -- 172.21.15.123:0/4226684448 shutdown_connections 2024-05-08T06:12:05.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.100+0000 7f13dc23bb80 1 -- 172.21.15.123:0/4226684448 wait complete. 2024-05-08T06:12:05.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.100+0000 7f13dc23bb80 1 -- start start 2024-05-08T06:12:05.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.100+0000 7f13dc23bb80 1 Processor -- start 2024-05-08T06:12:05.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.101+0000 7f13dc23bb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5445e58f0 0x55b5445f11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:05.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.101+0000 7f13dc23bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5445eabc0 0x55b5445ee570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:05.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.101+0000 7f13dc23bb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5445eb4e0 0x55b5445eeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:05.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.101+0000 7f13dc23bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b5445f1f20 con 0x55b5445eabc0 2024-05-08T06:12:05.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.101+0000 7f13dc23bb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b54449db20 con 0x55b5445eb4e0 2024-05-08T06:12:05.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.101+0000 7f13dc23bb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b5445f20a0 con 0x55b5445e58f0 2024-05-08T06:12:05.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.101+0000 7f13dc23bb80 10 client.0 _ll_register_callbacks cb 0x55b5444dec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:12:05.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.101+0000 7f13caa00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5445eabc0 0x55b5445ee570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:05.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.101+0000 7f13ca000700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5445e58f0 0x55b5445f11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:05.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.101+0000 7f13caa00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5445eabc0 0x55b5445ee570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56150/0 (socket says 172.21.15.123:56150) 2024-05-08T06:12:05.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.101+0000 7f13caa00700 1 -- 172.21.15.123:0/3894883223 learned_addr learned my addr 172.21.15.123:0/3894883223 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:05.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.102+0000 7f13ca000700 1 --2- 172.21.15.123:0/3894883223 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5445eb4e0 0x55b5445eeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:05.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.102+0000 7f13caa00700 1 -- 172.21.15.123:0/3894883223 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5445e58f0 msgr2=0x55b5445f11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:05.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.102+0000 7f13caa00700 1 --2- 172.21.15.123:0/3894883223 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5445e58f0 0x55b5445f11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:05.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.102+0000 7f13caa00700 1 -- 172.21.15.123:0/3894883223 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5445eb4e0 msgr2=0x55b5445eeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:05.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.102+0000 7f13caa00700 1 --2- 172.21.15.123:0/3894883223 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5445eb4e0 0x55b5445eeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:05.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.102+0000 7f13caa00700 1 -- 172.21.15.123:0/3894883223 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5444fd420 con 0x55b5445eabc0 2024-05-08T06:12:05.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.102+0000 7f13caa00700 1 --2- 172.21.15.123:0/3894883223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5445eabc0 0x55b5445ee570 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f13bc012d20 tx=0x7f13bc00f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e8680b326bf7c8a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:05.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.102+0000 7f13b4c00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-08T06:12:05.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.103+0000 7f13b4c00700 1 -- 172.21.15.123:0/3894883223 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13bc00fb80 con 0x55b5445eabc0 2024-05-08T06:12:05.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.103+0000 7f13b4c00700 1 -- 172.21.15.123:0/3894883223 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f13bc00fd20 con 0x55b5445eabc0 2024-05-08T06:12:05.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.103+0000 7f13dc23bb80 1 -- 172.21.15.123:0/3894883223 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13b80116f0 con 0x55b5445eabc0 2024-05-08T06:12:05.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.103+0000 7f13b4c00700 1 -- 172.21.15.123:0/3894883223 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f13bc004830 con 0x55b5445eabc0 2024-05-08T06:12:05.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.103+0000 7f13dc23bb80 10 client.15333 fetch_fsmap learned FSMap version 24 2024-05-08T06:12:05.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.103+0000 7f13dc23bb80 1 -- 172.21.15.123:0/3894883223 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b5445fc820 con 0x55b5445eabc0 2024-05-08T06:12:05.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.103+0000 7f13b4c00700 1 -- 172.21.15.123:0/3894883223 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13b80116f0 con 0x55b5445eabc0 2024-05-08T06:12:05.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.103+0000 7f13b4c00700 1 -- 172.21.15.123:0/3894883223 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f13bc0055c0 con 0x55b5445eabc0 2024-05-08T06:12:05.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.104+0000 7f13dc23bb80 10 client.15333 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:12:05.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.104+0000 7f13dc23bb80 10 client.15333 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T06:12:05.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.104+0000 7f13dc23bb80 20 client.15333 populate_metadata read hostname 'smithi123' 2024-05-08T06:12:05.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.104+0000 7f13dc23bb80 1 --2- 172.21.15.123:0/3894883223 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55b5445fee60 0x55b544601250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:05.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.104+0000 7f13dc23bb80 4 client.15333 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:12:05.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.104+0000 7f13dc23bb80 1 -- 172.21.15.123:0/3894883223 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5445b2140 con 0x55b5445fee60 2024-05-08T06:12:05.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.104+0000 7f13ca000700 1 --2- 172.21.15.123:0/3894883223 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55b5445fee60 0x55b544601250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:05.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.105+0000 7f13ca000700 1 --2- 172.21.15.123:0/3894883223 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55b5445fee60 0x55b544601250 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83f2bb4359ffbd41 server_cookie=ae1ec8bb95a0b87e in_seq=0 out_seq=0 2024-05-08T06:12:05.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.105+0000 7f13b4c00700 10 client.15333 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:05.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.105+0000 7f13b4c00700 1 -- 172.21.15.123:0/3894883223 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b5445b2140 con 0x55b5445fee60 2024-05-08T06:12:05.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.105+0000 7f13b4c00700 10 client.15333 handle_command_reply: tid=0 2024-05-08T06:12:05.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.105+0000 7f13dc23bb80 1 client.15333 shutdown 2024-05-08T06:12:05.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.106+0000 7f13dc23bb80 1 -- 172.21.15.123:0/3894883223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5445eabc0 msgr2=0x55b5445ee570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:05.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.106+0000 7f13dc23bb80 1 --2- 172.21.15.123:0/3894883223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5445eabc0 0x55b5445ee570 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f13bc012d20 tx=0x7f13bc00f6a0 comp rx=0 tx=0).stop 2024-05-08T06:12:05.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.106+0000 7f13dc23bb80 1 -- 172.21.15.123:0/3894883223 shutdown_connections 2024-05-08T06:12:05.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.106+0000 7f13dc23bb80 1 --2- 172.21.15.123:0/3894883223 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55b5445fee60 0x55b544601250 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:05.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.106+0000 7f13dc23bb80 1 --2- 172.21.15.123:0/3894883223 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5445eb4e0 0x55b5445eeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:05.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.106+0000 7f13dc23bb80 1 --2- 172.21.15.123:0/3894883223 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5445e58f0 0x55b5445f11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:05.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.106+0000 7f13dc23bb80 1 --2- 172.21.15.123:0/3894883223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5445eabc0 0x55b5445ee570 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:05.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.106+0000 7f13dc23bb80 1 -- 172.21.15.123:0/3894883223 >> 172.21.15.123:0/3894883223 conn(0x55b54455d6d0 msgr2=0x55b5445df190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:05.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.106+0000 7f13b4c00700 0 client.15333 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:05.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.106+0000 7f13dc23bb80 1 -- 172.21.15.123:0/3894883223 shutdown_connections 2024-05-08T06:12:05.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.106+0000 7f13dc23bb80 1 -- 172.21.15.123:0/3894883223 wait complete. 2024-05-08T06:12:05.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.106+0000 7f13dc23bb80 20 client.15333 trim_cache size 0 max 16384 2024-05-08T06:12:05.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.140+0000 7f13dc23bb80 1 Processor -- start 2024-05-08T06:12:05.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.140+0000 7f13dc23bb80 1 -- start start 2024-05-08T06:12:05.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.140+0000 7f13dc23bb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5445e58f0 0x55b5445e52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:05.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.140+0000 7f13dc23bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5445eb4e0 0x55b5445fafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:05.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.140+0000 7f13dc23bb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5445df910 0x55b5445e9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:05.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.140+0000 7f13dc23bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc1b2510 con 0x55b5445eb4e0 2024-05-08T06:12:05.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.140+0000 7f13dc23bb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc1afda0 con 0x55b5445e58f0 2024-05-08T06:12:05.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.140+0000 7f13dc23bb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc1adbf0 con 0x55b5445df910 2024-05-08T06:12:05.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.141+0000 7f13ca000700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5445e58f0 0x55b5445e52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:05.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.141+0000 7f13caa00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5445eb4e0 0x55b5445fafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:05.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.141+0000 7f13ca000700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5445e58f0 0x55b5445e52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:49084/0 (socket says 172.21.15.123:49084) 2024-05-08T06:12:05.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.141+0000 7f13ca000700 1 -- 172.21.15.123:0/2824554986 learned_addr learned my addr 172.21.15.123:0/2824554986 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:05.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.141+0000 7f13ca000700 1 --2- 172.21.15.123:0/2824554986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5445df910 0x55b5445e9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:05.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.141+0000 7f13ca000700 1 -- 172.21.15.123:0/2824554986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5445df910 msgr2=0x55b5445e9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:05.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.141+0000 7f13ca000700 1 --2- 172.21.15.123:0/2824554986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5445df910 0x55b5445e9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:05.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.141+0000 7f13ca000700 1 -- 172.21.15.123:0/2824554986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5445eb4e0 msgr2=0x55b5445fafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:05.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.141+0000 7f13ca000700 1 --2- 172.21.15.123:0/2824554986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5445eb4e0 0x55b5445fafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:05.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.141+0000 7f13ca000700 1 -- 172.21.15.123:0/2824554986 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5446015c0 con 0x55b5445e58f0 2024-05-08T06:12:05.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.141+0000 7f13ca000700 1 --2- 172.21.15.123:0/2824554986 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5445e58f0 0x55b5445e52d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f13b8009fd0 tx=0x7f13b8022040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a12ad41a80ed91ab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:05.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.142+0000 7f13b7400700 1 -- 172.21.15.123:0/2824554986 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b8035070 con 0x55b5445e58f0 2024-05-08T06:12:05.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.142+0000 7f13b7400700 1 -- 172.21.15.123:0/2824554986 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13b80050d0 con 0x55b5445e58f0 2024-05-08T06:12:05.142 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.142+0000 7f13b7400700 1 -- 172.21.15.123:0/2824554986 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b8030530 con 0x55b5445e58f0 2024-05-08T06:12:05.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.142+0000 7f13dc23bb80 1 -- 172.21.15.123:0/2824554986 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5445e58f0 msgr2=0x55b5445e52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:05.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.142+0000 7f13dc23bb80 1 --2- 172.21.15.123:0/2824554986 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5445e58f0 0x55b5445e52d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f13b8009fd0 tx=0x7f13b8022040 comp rx=0 tx=0).stop 2024-05-08T06:12:05.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.143+0000 7f13dc23bb80 1 -- 172.21.15.123:0/2824554986 shutdown_connections 2024-05-08T06:12:05.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.143+0000 7f13dc23bb80 1 --2- 172.21.15.123:0/2824554986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5445df910 0x55b5445e9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:05.143 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.143+0000 7f13dc23bb80 1 --2- 172.21.15.123:0/2824554986 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5445e58f0 0x55b5445e52d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:05.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.143+0000 7f13dc23bb80 1 --2- 172.21.15.123:0/2824554986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5445eb4e0 0x55b5445fafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:05.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.143+0000 7f13dc23bb80 1 -- 172.21.15.123:0/2824554986 >> 172.21.15.123:0/2824554986 conn(0x55b54455d6d0 msgr2=0x55b5445e96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:05.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.143+0000 7f13dc23bb80 1 -- 172.21.15.123:0/2824554986 shutdown_connections 2024-05-08T06:12:05.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.143+0000 7f13dc23bb80 1 -- 172.21.15.123:0/2824554986 wait complete. 2024-05-08T06:12:05.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.143+0000 7f13dc23bb80 1 -- start start 2024-05-08T06:12:05.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.144+0000 7f13dc23bb80 1 Processor -- start 2024-05-08T06:12:05.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.144+0000 7f13dc23bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5445df910 0x55b5445f7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:05.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.144+0000 7f13dc23bb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5445e58f0 0x55b5445f78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:05.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.144+0000 7f13dc23bb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5445eb4e0 0x55b5445f4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:05.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.144+0000 7f13dc23bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc1ae260 con 0x55b5445df910 2024-05-08T06:12:05.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.144+0000 7f13dc23bb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc1af6d0 con 0x55b5445eb4e0 2024-05-08T06:12:05.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.144+0000 7f13dc23bb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc1ae930 con 0x55b5445e58f0 2024-05-08T06:12:05.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.144+0000 7f13dc23bb80 10 client.0 _ll_register_callbacks cb 0x55b5444dec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:12:05.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.144+0000 7f13ca000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5445df910 0x55b5445f7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:05.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.145+0000 7f13ca000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5445df910 0x55b5445f7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56168/0 (socket says 172.21.15.123:56168) 2024-05-08T06:12:05.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.145+0000 7f13ca000700 1 -- 172.21.15.123:0/4171398988 learned_addr learned my addr 172.21.15.123:0/4171398988 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:05.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.145+0000 7f13caa00700 1 --2- 172.21.15.123:0/4171398988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5445e58f0 0x55b5445f78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:05.146 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.145+0000 7f13ca000700 1 --2- 172.21.15.123:0/4171398988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5445eb4e0 0x55b5445f4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:05.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.145+0000 7f13ca000700 1 -- 172.21.15.123:0/4171398988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5445e58f0 msgr2=0x55b5445f78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:05.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.145+0000 7f13ca000700 1 --2- 172.21.15.123:0/4171398988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5445e58f0 0x55b5445f78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:05.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.145+0000 7f13ca000700 1 -- 172.21.15.123:0/4171398988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5445eb4e0 msgr2=0x55b5445f4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:05.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.145+0000 7f13ca000700 1 --2- 172.21.15.123:0/4171398988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5445eb4e0 0x55b5445f4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:05.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.145+0000 7f13ca000700 1 -- 172.21.15.123:0/4171398988 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5445f49f0 con 0x55b5445df910 2024-05-08T06:12:05.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.145+0000 7f13caa00700 1 --2- 172.21.15.123:0/4171398988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5445e58f0 0x55b5445f78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:12:05.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.145+0000 7f13ca000700 1 --2- 172.21.15.123:0/4171398988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5445df910 0x55b5445f7360 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f13b8009d90 tx=0x7f13b8009730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a53277ca9663bb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:05.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.145+0000 7f13b4c00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-08T06:12:05.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.146+0000 7f13b4c00700 1 -- 172.21.15.123:0/4171398988 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b8035040 con 0x55b5445df910 2024-05-08T06:12:05.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.146+0000 7f13b4c00700 1 -- 172.21.15.123:0/4171398988 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f13b801c040 con 0x55b5445df910 2024-05-08T06:12:05.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.146+0000 7f13dc23bb80 1 -- 172.21.15.123:0/4171398988 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13b80050d0 con 0x55b5445df910 2024-05-08T06:12:05.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.146+0000 7f13b4c00700 1 -- 172.21.15.123:0/4171398988 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f13b80117a0 con 0x55b5445df910 2024-05-08T06:12:05.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.146+0000 7f13dc23bb80 10 client.15339 fetch_fsmap learned FSMap version 24 2024-05-08T06:12:05.147 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.146+0000 7f13dc23bb80 1 -- 172.21.15.123:0/4171398988 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b54466a0b0 con 0x55b5445df910 2024-05-08T06:12:05.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.146+0000 7f13b4c00700 1 -- 172.21.15.123:0/4171398988 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13b80050d0 con 0x55b5445df910 2024-05-08T06:12:05.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.146+0000 7f13b4c00700 1 -- 172.21.15.123:0/4171398988 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f13b8046020 con 0x55b5445df910 2024-05-08T06:12:05.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.146+0000 7f13dc23bb80 10 client.15339 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:12:05.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.146+0000 7f13dc23bb80 10 client.15339 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T06:12:05.149 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.146+0000 7f13dc23bb80 20 client.15339 populate_metadata read hostname 'smithi123' 2024-05-08T06:12:05.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.146+0000 7f13dc23bb80 1 --2- 172.21.15.123:0/4171398988 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55b54466a4c0 0x55b54466c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:05.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.147+0000 7f13dc23bb80 4 client.15339 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T06:12:05.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.147+0000 7f13dc23bb80 1 -- 172.21.15.123:0/4171398988 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b544545400 con 0x55b54466a4c0 2024-05-08T06:12:05.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.147+0000 7f13caa00700 1 --2- 172.21.15.123:0/4171398988 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55b54466a4c0 0x55b54466c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:05.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.147+0000 7f13caa00700 1 --2- 172.21.15.123:0/4171398988 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55b54466a4c0 0x55b54466c8b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fce8e03c4637836 server_cookie=1b6e4606dbb630d0 in_seq=0 out_seq=0 2024-05-08T06:12:05.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.147+0000 7f13b4c00700 10 client.15339 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:05.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.148+0000 7f13b4c00700 1 -- 172.21.15.123:0/4171398988 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b544545400 con 0x55b54466a4c0 2024-05-08T06:12:05.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.148+0000 7f13b4c00700 10 client.15339 handle_command_reply: tid=0 2024-05-08T06:12:05.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.148+0000 7f13dc23bb80 1 client.15339 shutdown 2024-05-08T06:12:05.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.148+0000 7f13dc23bb80 1 -- 172.21.15.123:0/4171398988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5445df910 msgr2=0x55b5445f7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:05.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.148+0000 7f13dc23bb80 1 --2- 172.21.15.123:0/4171398988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5445df910 0x55b5445f7360 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f13b8009d90 tx=0x7f13b8009730 comp rx=0 tx=0).stop 2024-05-08T06:12:05.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.148+0000 7f13dc23bb80 1 -- 172.21.15.123:0/4171398988 shutdown_connections 2024-05-08T06:12:05.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.148+0000 7f13dc23bb80 1 --2- 172.21.15.123:0/4171398988 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55b54466a4c0 0x55b54466c8b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:05.150 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.148+0000 7f13dc23bb80 1 --2- 172.21.15.123:0/4171398988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5445df910 0x55b5445f7360 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:05.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.148+0000 7f13dc23bb80 1 --2- 172.21.15.123:0/4171398988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b5445eb4e0 0x55b5445f4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:05.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.148+0000 7f13dc23bb80 1 --2- 172.21.15.123:0/4171398988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5445e58f0 0x55b5445f78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:05.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.148+0000 7f13dc23bb80 1 -- 172.21.15.123:0/4171398988 >> 172.21.15.123:0/4171398988 conn(0x55b54455d6d0 msgr2=0x55b5445df190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:05.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.148+0000 7f13b4c00700 0 client.15339 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:05.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.148+0000 7f13dc23bb80 1 -- 172.21.15.123:0/4171398988 shutdown_connections 2024-05-08T06:12:05.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.148+0000 7f13dc23bb80 1 -- 172.21.15.123:0/4171398988 wait complete. 2024-05-08T06:12:05.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.148+0000 7f13dc23bb80 20 client.15339 trim_cache size 0 max 16384 2024-05-08T06:12:05.151 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-05-08T06:12:05.151 INFO:teuthology.orchestra.run.smithi123.stdout: "status": "no active scrubs running", 2024-05-08T06:12:05.151 INFO:teuthology.orchestra.run.smithi123.stdout: "scrubs": {} 2024-05-08T06:12:05.151 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-05-08T06:12:05.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.149+0000 7f13d1800700 1 -- 172.21.15.123:0/2655622659 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f13b00b2900 msgr2=0x7f13b00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:05.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.149+0000 7f13d1800700 1 --2- 172.21.15.123:0/2655622659 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f13b00b2900 0x7f13b00b4dc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f13c400b070 tx=0x7f13c4009040 comp rx=0 tx=0).stop 2024-05-08T06:12:05.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.149+0000 7f13d1800700 1 -- 172.21.15.123:0/2655622659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13cc10ff00 msgr2=0x7f13cc1a0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:05.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.149+0000 7f13d1800700 1 --2- 172.21.15.123:0/2655622659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13cc10ff00 0x7f13cc1a0600 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f13c400b780 tx=0x7f13c4004d00 comp rx=0 tx=0).stop 2024-05-08T06:12:05.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.149+0000 7f13d1800700 1 -- 172.21.15.123:0/2655622659 shutdown_connections 2024-05-08T06:12:05.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.149+0000 7f13d1800700 1 --2- 172.21.15.123:0/2655622659 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f13b00b2900 0x7f13b00b4dc0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:05.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.149+0000 7f13d1800700 1 --2- 172.21.15.123:0/2655622659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13cc10ff00 0x7f13cc1a0600 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:05.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.149+0000 7f13d1800700 1 --2- 172.21.15.123:0/2655622659 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13cc0756d0 0x7f13cc19bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:05.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.149+0000 7f13d1800700 1 --2- 172.21.15.123:0/2655622659 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13cc075ff0 0x7f13cc19c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:05.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.149+0000 7f13d1800700 1 -- 172.21.15.123:0/2655622659 >> 172.21.15.123:0/2655622659 conn(0x7f13cc07a600 msgr2=0x7f13cc1018e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:05.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.149+0000 7f13d1800700 1 -- 172.21.15.123:0/2655622659 shutdown_connections 2024-05-08T06:12:05.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:05.149+0000 7f13d1800700 1 -- 172.21.15.123:0/2655622659 wait complete. 2024-05-08T06:12:05.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:05 smithi163 ceph-mon[34520]: scrub summary: idle 2024-05-08T06:12:05.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:05 smithi192 ceph-mon[34664]: scrub summary: idle 2024-05-08T06:12:05.867 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:05 smithi123 ceph-mon[29787]: scrub summary: idle 2024-05-08T06:12:06.743 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:06 smithi123 ceph-mon[29787]: pgmap v509: 97 pgs: 97 active+clean; 677 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-08T06:12:06.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:06 smithi163 ceph-mon[34520]: pgmap v509: 97 pgs: 97 active+clean; 677 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-08T06:12:06.791 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T06:12:06.792 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T06:12:06.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:06 smithi192 ceph-mon[34664]: pgmap v509: 97 pgs: 97 active+clean; 677 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-08T06:12:07.210 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:12:08.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:08 smithi123 ceph-mon[29787]: pgmap v510: 97 pgs: 97 active+clean; 677 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-08T06:12:08.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:08 smithi163 ceph-mon[34520]: pgmap v510: 97 pgs: 97 active+clean; 677 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-08T06:12:08.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:08 smithi192 ceph-mon[34664]: pgmap v510: 97 pgs: 97 active+clean; 677 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-08T06:12:08.803 INFO:tasks.workunit.client.0.smithi123.stderr:Updating files: 59% (7243/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 96% (11867/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-08T06:12:09.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.786+0000 7f8643200700 1 -- 172.21.15.123:0/2030118585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8644109460 msgr2=0x7f8644109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:09.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.786+0000 7f8643200700 1 --2- 172.21.15.123:0/2030118585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8644109460 0x7f8644109840 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8634009990 tx=0x7f8634009ca0 comp rx=0 tx=0).stop 2024-05-08T06:12:09.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.786+0000 7f8643200700 1 -- 172.21.15.123:0/2030118585 shutdown_connections 2024-05-08T06:12:09.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.786+0000 7f8643200700 1 --2- 172.21.15.123:0/2030118585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8644109460 0x7f8644109840 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:09.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.786+0000 7f8643200700 1 --2- 172.21.15.123:0/2030118585 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8644103d90 0x7f86441102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:09.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.786+0000 7f8643200700 1 --2- 172.21.15.123:0/2030118585 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86441033d0 0x7f8644103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:09.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.786+0000 7f8643200700 1 -- 172.21.15.123:0/2030118585 >> 172.21.15.123:0/2030118585 conn(0x7f86440fef20 msgr2=0x7f8644101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:09.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.786+0000 7f8643200700 1 -- 172.21.15.123:0/2030118585 shutdown_connections 2024-05-08T06:12:09.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.786+0000 7f8643200700 1 -- 172.21.15.123:0/2030118585 wait complete. 2024-05-08T06:12:09.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.787+0000 7f8643200700 1 Processor -- start 2024-05-08T06:12:09.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.787+0000 7f8643200700 1 -- start start 2024-05-08T06:12:09.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.788+0000 7f8643200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86441033d0 0x7f864419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:09.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.788+0000 7f8643200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8644103d90 0x7f864419fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:09.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.788+0000 7f8643200700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8644109460 0x7f86441998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:09.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.788+0000 7f8643200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f86441135c0 con 0x7f8644103d90 2024-05-08T06:12:09.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.788+0000 7f8643200700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8644113440 con 0x7f86441033d0 2024-05-08T06:12:09.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.788+0000 7f8643200700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8644113740 con 0x7f8644109460 2024-05-08T06:12:09.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.788+0000 7f8641400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8644103d90 0x7f864419fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:09.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.788+0000 7f8641e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86441033d0 0x7f864419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:09.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.788+0000 7f8641400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8644103d90 0x7f864419fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56178/0 (socket says 172.21.15.123:56178) 2024-05-08T06:12:09.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.788+0000 7f8641e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86441033d0 0x7f864419f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:49108/0 (socket says 172.21.15.123:49108) 2024-05-08T06:12:09.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.788+0000 7f8641400700 1 -- 172.21.15.123:0/528783464 learned_addr learned my addr 172.21.15.123:0/528783464 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:09.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.788+0000 7f8642800700 1 --2- 172.21.15.123:0/528783464 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8644109460 0x7f86441998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:09.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.789+0000 7f8641400700 1 -- 172.21.15.123:0/528783464 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8644109460 msgr2=0x7f86441998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:09.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.789+0000 7f8641400700 1 --2- 172.21.15.123:0/528783464 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8644109460 0x7f86441998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:09.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.789+0000 7f8641400700 1 -- 172.21.15.123:0/528783464 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86441033d0 msgr2=0x7f864419f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:09.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.789+0000 7f8641400700 1 --2- 172.21.15.123:0/528783464 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86441033d0 0x7f864419f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:09.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.789+0000 7f8641400700 1 -- 172.21.15.123:0/528783464 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f864419a020 con 0x7f8644103d90 2024-05-08T06:12:09.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.789+0000 7f8641e00700 1 --2- 172.21.15.123:0/528783464 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86441033d0 0x7f864419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:12:09.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.789+0000 7f8641400700 1 --2- 172.21.15.123:0/528783464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8644103d90 0x7f864419fc40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f863000eb30 tx=0x7f863000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:09.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.789+0000 7f863aa00700 1 -- 172.21.15.123:0/528783464 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863000cbd0 con 0x7f8644103d90 2024-05-08T06:12:09.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.789+0000 7f8643200700 1 -- 172.21.15.123:0/528783464 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f864419a310 con 0x7f8644103d90 2024-05-08T06:12:09.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.789+0000 7f863aa00700 1 -- 172.21.15.123:0/528783464 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86300044d0 con 0x7f8644103d90 2024-05-08T06:12:09.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.789+0000 7f863aa00700 1 -- 172.21.15.123:0/528783464 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8630003e30 con 0x7f8644103d90 2024-05-08T06:12:09.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.790+0000 7f8643200700 1 -- 172.21.15.123:0/528783464 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86441a65d0 con 0x7f8644103d90 2024-05-08T06:12:09.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.791+0000 7f8643200700 1 -- 172.21.15.123:0/528783464 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f864406a190 con 0x7f8644103d90 2024-05-08T06:12:09.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.794+0000 7f863aa00700 1 -- 172.21.15.123:0/528783464 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7f8630004000 con 0x7f8644103d90 2024-05-08T06:12:09.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.795+0000 7f863aa00700 1 --2- 172.21.15.123:0/528783464 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f86280b2900 0x7f86280b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:09.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.795+0000 7f863aa00700 1 -- 172.21.15.123:0/528783464 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f86300e2550 con 0x7f8644103d90 2024-05-08T06:12:09.795 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.795+0000 7f8641e00700 1 --2- 172.21.15.123:0/528783464 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f86280b2900 0x7f86280b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:09.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.796+0000 7f8641e00700 1 --2- 172.21.15.123:0/528783464 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f86280b2900 0x7f86280b4dc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f864419aa00 tx=0x7f863400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:09.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:09.800+0000 7f863aa00700 1 -- 172.21.15.123:0/528783464 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f86300042f0 con 0x7f8644103d90 2024-05-08T06:12:09.868 INFO:tasks.workunit.client.0.smithi123.stderr:Note: switching to 'bee299f4ac344e4c45e550d7a80abcadd7bc3d5a'. 2024-05-08T06:12:09.869 INFO:tasks.workunit.client.0.smithi123.stderr: 2024-05-08T06:12:09.869 INFO:tasks.workunit.client.0.smithi123.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-08T06:12:09.869 INFO:tasks.workunit.client.0.smithi123.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-08T06:12:09.869 INFO:tasks.workunit.client.0.smithi123.stderr:state without impacting any branches by switching back to a branch. 2024-05-08T06:12:09.869 INFO:tasks.workunit.client.0.smithi123.stderr: 2024-05-08T06:12:09.869 INFO:tasks.workunit.client.0.smithi123.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-08T06:12:09.869 INFO:tasks.workunit.client.0.smithi123.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-08T06:12:09.869 INFO:tasks.workunit.client.0.smithi123.stderr: 2024-05-08T06:12:09.869 INFO:tasks.workunit.client.0.smithi123.stderr: git switch -c 2024-05-08T06:12:09.869 INFO:tasks.workunit.client.0.smithi123.stderr: 2024-05-08T06:12:09.869 INFO:tasks.workunit.client.0.smithi123.stderr:Or undo this operation with: 2024-05-08T06:12:09.869 INFO:tasks.workunit.client.0.smithi123.stderr: 2024-05-08T06:12:09.869 INFO:tasks.workunit.client.0.smithi123.stderr: git switch - 2024-05-08T06:12:09.869 INFO:tasks.workunit.client.0.smithi123.stderr: 2024-05-08T06:12:09.870 INFO:tasks.workunit.client.0.smithi123.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-08T06:12:09.870 INFO:tasks.workunit.client.0.smithi123.stderr: 2024-05-08T06:12:09.870 INFO:tasks.workunit.client.0.smithi123.stderr:HEAD is now at bee299f4ac3 mds: relax divergent backtrace scrub failures for replicated ancestor inodes 2024-05-08T06:12:09.884 DEBUG:teuthology.orchestra.run.smithi123:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-08T06:12:09.939 INFO:tasks.workunit.client.0.smithi123.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-08T06:12:09.945 INFO:tasks.workunit.client.0.smithi123.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-08T06:12:09.945 INFO:tasks.workunit.client.0.smithi123.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-08T06:12:10.087 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:10.087+0000 7f8643200700 1 -- 172.21.15.123:0/528783464 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f864419adf0 con 0x7f8644103d90 2024-05-08T06:12:10.089 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:10.089+0000 7f863aa00700 1 -- 172.21.15.123:0/528783464 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11885 (secure 0 0 0) 0x7f86300aa220 con 0x7f8644103d90 2024-05-08T06:12:10.090 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:12:10.090 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3646476522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3646476522},{"type":"v1","addr":"172.21.15.123:6835","nonce":3646476522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2763706497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2763706497},{"type":"v1","addr":"172.21.15.123:6837","nonce":2763706497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/274635265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":274635265},{"type":"v1","addr":"172.21.15.123:6839","nonce":274635265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/175423115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":175423115},{"type":"v1","addr":"172.21.15.192:6833","nonce":175423115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/3263009472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3263009472},{"type":"v1","addr":"172.21.15.163:6833","nonce":3263009472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3308612826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3308612826},{"type":"v1","addr":"172.21.15.163:6835","nonce":3308612826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T06:05:54.755550+0000","modified":"2024-05-08T06:10:12.600417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24512,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/202513680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":202513680},{"type":"v1","addr":"172.21.15.192:6835","nonce":202513680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/2483545969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2483545969},{"type":"v1","addr":"172.21.15.192:6837","nonce":2483545969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/660511838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":660511838},{"type":"v1","addr":"172.21.15.163:6837","nonce":660511838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.192:6839/3598682360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":3598682360},{"type":"v1","addr":"172.21.15.192:6839","nonce":3598682360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6841/1571726231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":1571726231},{"type":"v1","addr":"172.21.15.123:6841","nonce":1571726231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.163:6839/1568161515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":1568161515},{"type":"v1","addr":"172.21.15.163:6839","nonce":1568161515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24512,24496,24524]},"id":1}]} 2024-05-08T06:12:10.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:10.092+0000 7f8643200700 1 -- 172.21.15.123:0/528783464 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f86280b2900 msgr2=0x7f86280b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:10.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:10.092+0000 7f8643200700 1 --2- 172.21.15.123:0/528783464 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f86280b2900 0x7f86280b4dc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f864419aa00 tx=0x7f863400b260 comp rx=0 tx=0).stop 2024-05-08T06:12:10.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:10.092+0000 7f8643200700 1 -- 172.21.15.123:0/528783464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8644103d90 msgr2=0x7f864419fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:10.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:10.092+0000 7f8643200700 1 --2- 172.21.15.123:0/528783464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8644103d90 0x7f864419fc40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f863000eb30 tx=0x7f863000ee40 comp rx=0 tx=0).stop 2024-05-08T06:12:10.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:10.093+0000 7f8643200700 1 -- 172.21.15.123:0/528783464 shutdown_connections 2024-05-08T06:12:10.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:10.093+0000 7f8643200700 1 --2- 172.21.15.123:0/528783464 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f86280b2900 0x7f86280b4dc0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:10.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:10.093+0000 7f8643200700 1 --2- 172.21.15.123:0/528783464 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8644109460 0x7f86441998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:10.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:10.093+0000 7f8643200700 1 --2- 172.21.15.123:0/528783464 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86441033d0 0x7f864419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:10.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:10.093+0000 7f8643200700 1 --2- 172.21.15.123:0/528783464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8644103d90 0x7f864419fc40 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:10.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:10.093+0000 7f8643200700 1 -- 172.21.15.123:0/528783464 >> 172.21.15.123:0/528783464 conn(0x7f86440fef20 msgr2=0x7f86440ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:10.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:10.093+0000 7f8643200700 1 -- 172.21.15.123:0/528783464 shutdown_connections 2024-05-08T06:12:10.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:10.093+0000 7f8643200700 1 -- 172.21.15.123:0/528783464 wait complete. 2024-05-08T06:12:10.095 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 24 2024-05-08T06:12:10.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:10 smithi163 ceph-mon[34520]: pgmap v511: 97 pgs: 97 active+clean; 677 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-08T06:12:10.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:10 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/528783464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:12:10.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:10 smithi192 ceph-mon[34664]: pgmap v511: 97 pgs: 97 active+clean; 677 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-08T06:12:10.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:10 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/528783464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:12:10.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:10 smithi123 ceph-mon[29787]: pgmap v511: 97 pgs: 97 active+clean; 677 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-08T06:12:10.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:10 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/528783464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:12:11.321 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-08T06:12:11.668 INFO:tasks.workunit.client.0.smithi123.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-08T06:12:11.788 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:12:11.811 INFO:tasks.workunit.client.0.smithi123.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-08T06:12:11.886 INFO:tasks.workunit.client.0.smithi123.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-08T06:12:11.891 INFO:tasks.workunit.client.0.smithi123.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-08T06:12:11.891 INFO:tasks.workunit.client.0.smithi123.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-08T06:12:11.972 INFO:tasks.workunit.client.0.smithi123.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-08T06:12:11.983 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-05-08T06:12:11.983 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-08T06:12:12.041 INFO:tasks.workunit:Running workunits matching direct_io on client.0... 2024-05-08T06:12:12.042 INFO:tasks.workunit:Running workunit direct_io/big.sh... 2024-05-08T06:12:12.043 DEBUG:teuthology.orchestra.run.smithi123:workunit test direct_io/big.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/big.sh 2024-05-08T06:12:12.103 INFO:tasks.workunit.client.0.smithi123.stdout:test large (16MB) dio write 2024-05-08T06:12:12.103 INFO:tasks.workunit.client.0.smithi123.stderr:+ echo 'test large (16MB) dio write' 2024-05-08T06:12:12.104 INFO:tasks.workunit.client.0.smithi123.stderr:+ dd if=/dev/zero of=foo.big bs=16M count=1 oflag=direct 2024-05-08T06:12:12.390 INFO:tasks.workunit.client.0.smithi123.stderr:1+0 records in 2024-05-08T06:12:12.390 INFO:tasks.workunit.client.0.smithi123.stderr:1+0 records out 2024-05-08T06:12:12.390 INFO:tasks.workunit.client.0.smithi123.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.281517 s, 59.6 MB/s 2024-05-08T06:12:12.393 INFO:tasks.workunit.client.0.smithi123.stderr:+ echo OK 2024-05-08T06:12:12.394 INFO:tasks.workunit.client.0.smithi123.stdout:OK 2024-05-08T06:12:12.395 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-08T06:12:12.395 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-08T06:12:12.476 INFO:tasks.workunit:Running workunit direct_io/direct_io_test... 2024-05-08T06:12:12.476 DEBUG:teuthology.orchestra.run.smithi123:workunit test direct_io/direct_io_test> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/direct_io_test 2024-05-08T06:12:12.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:12 smithi123 ceph-mon[29787]: pgmap v512: 97 pgs: 97 active+clean; 677 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-08T06:12:12.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:12 smithi163 ceph-mon[34520]: pgmap v512: 97 pgs: 97 active+clean; 677 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-08T06:12:12.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:12 smithi192 ceph-mon[34664]: pgmap v512: 97 pgs: 97 active+clean; 677 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-08T06:12:14.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.346+0000 7f0662e00700 1 -- 172.21.15.123:0/311277000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f065c102ad0 msgr2=0x7f065c10f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:14.347 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.346+0000 7f0662e00700 1 --2- 172.21.15.123:0/311277000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f065c102ad0 0x7f065c10f790 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f06480098f0 tx=0x7f0648009c00 comp rx=0 tx=0).stop 2024-05-08T06:12:14.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.347+0000 7f0662e00700 1 -- 172.21.15.123:0/311277000 shutdown_connections 2024-05-08T06:12:14.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.347+0000 7f0662e00700 1 --2- 172.21.15.123:0/311277000 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f065c10fcd0 0x7f065c1120c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:14.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.347+0000 7f0662e00700 1 --2- 172.21.15.123:0/311277000 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f065c102120 0x7f065c102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:14.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.347+0000 7f0662e00700 1 --2- 172.21.15.123:0/311277000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f065c102ad0 0x7f065c10f790 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:14.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.347+0000 7f0662e00700 1 -- 172.21.15.123:0/311277000 >> 172.21.15.123:0/311277000 conn(0x7f065c0fddc0 msgr2=0x7f065c1001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:14.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.347+0000 7f0662e00700 1 -- 172.21.15.123:0/311277000 shutdown_connections 2024-05-08T06:12:14.348 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.347+0000 7f0662e00700 1 -- 172.21.15.123:0/311277000 wait complete. 2024-05-08T06:12:14.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.348+0000 7f0662e00700 1 Processor -- start 2024-05-08T06:12:14.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.348+0000 7f0662e00700 1 -- start start 2024-05-08T06:12:14.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.348+0000 7f0662e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f065c102120 0x7f065c1a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:14.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.349+0000 7f0662e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f065c102ad0 0x7f065c1a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:14.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.349+0000 7f0662e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f065c10fcd0 0x7f065c19baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:14.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.349+0000 7f0662e00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f065c1157c0 con 0x7f065c10fcd0 2024-05-08T06:12:14.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.349+0000 7f0662e00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f065c115640 con 0x7f065c102ad0 2024-05-08T06:12:14.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.349+0000 7f0662e00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f065c115940 con 0x7f065c102120 2024-05-08T06:12:14.349 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.349+0000 7f0661a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f065c102120 0x7f065c1a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:14.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.349+0000 7f0661a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f065c102120 0x7f065c1a18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:41914/0 (socket says 172.21.15.123:41914) 2024-05-08T06:12:14.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.349+0000 7f0661a00700 1 -- 172.21.15.123:0/2788354731 learned_addr learned my addr 172.21.15.123:0/2788354731 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:14.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.349+0000 7f0662400700 1 --2- 172.21.15.123:0/2788354731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f065c10fcd0 0x7f065c19baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:14.350 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.349+0000 7f0661000700 1 --2- 172.21.15.123:0/2788354731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f065c102ad0 0x7f065c1a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:14.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.349+0000 7f0661a00700 1 -- 172.21.15.123:0/2788354731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f065c102ad0 msgr2=0x7f065c1a1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:14.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.349+0000 7f0661a00700 1 --2- 172.21.15.123:0/2788354731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f065c102ad0 0x7f065c1a1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:14.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.349+0000 7f0661a00700 1 -- 172.21.15.123:0/2788354731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f065c10fcd0 msgr2=0x7f065c19baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:14.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.349+0000 7f0661a00700 1 --2- 172.21.15.123:0/2788354731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f065c10fcd0 0x7f065c19baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:14.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.349+0000 7f0661a00700 1 -- 172.21.15.123:0/2788354731 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f065c19c200 con 0x7f065c102120 2024-05-08T06:12:14.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.350+0000 7f0661a00700 1 --2- 172.21.15.123:0/2788354731 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f065c102120 0x7f065c1a18e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f065000c9b0 tx=0x7f065000ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:14.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.350+0000 7f065a000700 1 -- 172.21.15.123:0/2788354731 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650007b00 con 0x7f065c102120 2024-05-08T06:12:14.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.350+0000 7f065a000700 1 -- 172.21.15.123:0/2788354731 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f065000ce50 con 0x7f065c102120 2024-05-08T06:12:14.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.350+0000 7f065a000700 1 -- 172.21.15.123:0/2788354731 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650018660 con 0x7f065c102120 2024-05-08T06:12:14.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.350+0000 7f0662e00700 1 -- 172.21.15.123:0/2788354731 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f065c19c4f0 con 0x7f065c102120 2024-05-08T06:12:14.351 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.350+0000 7f0662e00700 1 -- 172.21.15.123:0/2788354731 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f065c1a87b0 con 0x7f065c102120 2024-05-08T06:12:14.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.354+0000 7f065a000700 1 -- 172.21.15.123:0/2788354731 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7f065000f450 con 0x7f065c102120 2024-05-08T06:12:14.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.355+0000 7f065a000700 1 --2- 172.21.15.123:0/2788354731 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f06440b2900 0x7f06440b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:14.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.356+0000 7f065a000700 1 -- 172.21.15.123:0/2788354731 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f06500e2080 con 0x7f065c102120 2024-05-08T06:12:14.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.356+0000 7f0661000700 1 --2- 172.21.15.123:0/2788354731 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f06440b2900 0x7f06440b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:14.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.356+0000 7f066ecccb80 1 Processor -- start 2024-05-08T06:12:14.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.356+0000 7f066ecccb80 1 -- start start 2024-05-08T06:12:14.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.357+0000 7f066ecccb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56454f4f09e0 0x56454f4f0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:14.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.357+0000 7f066ecccb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56454f4f1300 0x56454f4eafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:14.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.357+0000 7f066ecccb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56454f4eb600 0x56454f4eba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:14.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.357+0000 7f0661000700 1 --2- 172.21.15.123:0/2788354731 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f06440b2900 0x7f06440b4dc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0648009fd0 tx=0x7f0648019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:14.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.357+0000 7f066ecccb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56454f43da20 con 0x56454f4eb600 2024-05-08T06:12:14.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.357+0000 7f066ecccb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x56454f3db760 con 0x56454f4f1300 2024-05-08T06:12:14.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.357+0000 7f066ecccb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56454f4399e0 con 0x56454f4f09e0 2024-05-08T06:12:14.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.357+0000 7f0661a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56454f4f09e0 0x56454f4f0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:14.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.357+0000 7f0661a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56454f4f09e0 0x56454f4f0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:41930/0 (socket says 172.21.15.123:41930) 2024-05-08T06:12:14.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.357+0000 7f0661a00700 1 -- 172.21.15.123:0/2721951260 learned_addr learned my addr 172.21.15.123:0/2721951260 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:14.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.357+0000 7f0662400700 1 --2- 172.21.15.123:0/2721951260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56454f4eb600 0x56454f4eba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:14.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.357+0000 7f0661000700 1 --2- 172.21.15.123:0/2721951260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56454f4f1300 0x56454f4eafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:14.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.357+0000 7f0661a00700 1 -- 172.21.15.123:0/2721951260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56454f4f1300 msgr2=0x56454f4eafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:14.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.357+0000 7f0661a00700 1 --2- 172.21.15.123:0/2721951260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56454f4f1300 0x56454f4eafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:14.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.357+0000 7f0661a00700 1 -- 172.21.15.123:0/2721951260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56454f4eb600 msgr2=0x56454f4eba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:14.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.357+0000 7f0661a00700 1 --2- 172.21.15.123:0/2721951260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56454f4eb600 0x56454f4eba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:14.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.358+0000 7f0661a00700 1 -- 172.21.15.123:0/2721951260 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56454f38a8d0 con 0x56454f4f09e0 2024-05-08T06:12:14.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.358+0000 7f0662400700 1 --2- 172.21.15.123:0/2721951260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56454f4eb600 0x56454f4eba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:12:14.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.358+0000 7f0661a00700 1 --2- 172.21.15.123:0/2721951260 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56454f4f09e0 0x56454f4f0dc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f06500a5730 tx=0x7f06500a8ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25eaeaaeabffcc22 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:14.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.358+0000 7f064ea00700 1 -- 172.21.15.123:0/2721951260 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06500a96b0 con 0x56454f4f09e0 2024-05-08T06:12:14.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.358+0000 7f064ea00700 1 -- 172.21.15.123:0/2721951260 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06500ada60 con 0x56454f4f09e0 2024-05-08T06:12:14.359 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.358+0000 7f064ea00700 1 -- 172.21.15.123:0/2721951260 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06500a9910 con 0x56454f4f09e0 2024-05-08T06:12:14.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.359+0000 7f066ecccb80 1 -- 172.21.15.123:0/2721951260 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56454f4f09e0 msgr2=0x56454f4f0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:14.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.359+0000 7f066ecccb80 1 --2- 172.21.15.123:0/2721951260 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56454f4f09e0 0x56454f4f0dc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f06500a5730 tx=0x7f06500a8ef0 comp rx=0 tx=0).stop 2024-05-08T06:12:14.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.359+0000 7f066ecccb80 1 -- 172.21.15.123:0/2721951260 shutdown_connections 2024-05-08T06:12:14.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.359+0000 7f066ecccb80 1 --2- 172.21.15.123:0/2721951260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56454f4eb600 0x56454f4eba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:14.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.359+0000 7f066ecccb80 1 --2- 172.21.15.123:0/2721951260 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56454f4f09e0 0x56454f4f0dc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:14.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.359+0000 7f066ecccb80 1 --2- 172.21.15.123:0/2721951260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56454f4f1300 0x56454f4eafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:14.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.359+0000 7f066ecccb80 1 -- 172.21.15.123:0/2721951260 >> 172.21.15.123:0/2721951260 conn(0x56454f419b60 msgr2=0x56454f4e59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:14.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.359+0000 7f066ecccb80 1 -- 172.21.15.123:0/2721951260 shutdown_connections 2024-05-08T06:12:14.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.359+0000 7f066ecccb80 1 -- 172.21.15.123:0/2721951260 wait complete. 2024-05-08T06:12:14.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.360+0000 7f066ecccb80 1 -- start start 2024-05-08T06:12:14.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.360+0000 7f066ecccb80 1 Processor -- start 2024-05-08T06:12:14.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.361+0000 7f066ecccb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56454f4eb600 0x56454f4f6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:14.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.361+0000 7f066ecccb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56454f4f09e0 0x56454f4f42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:14.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.361+0000 7f066ecccb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56454f4f1300 0x56454f4f47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:14.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.361+0000 7f066ecccb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56454f4f7db0 con 0x56454f4f1300 2024-05-08T06:12:14.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.361+0000 7f066ecccb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x56454f3a3b20 con 0x56454f4f09e0 2024-05-08T06:12:14.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.361+0000 7f066ecccb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56454f4f7f30 con 0x56454f4eb600 2024-05-08T06:12:14.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.361+0000 7f066ecccb80 10 client.0 _ll_register_callbacks cb 0x56454f3e4c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:12:14.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.361+0000 7f0661a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56454f4eb600 0x56454f4f6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:14.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.361+0000 7f0661a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56454f4eb600 0x56454f4f6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:41938/0 (socket says 172.21.15.123:41938) 2024-05-08T06:12:14.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.361+0000 7f0661a00700 1 -- 172.21.15.123:0/2792300014 learned_addr learned my addr 172.21.15.123:0/2792300014 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:14.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.361+0000 7f0662400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56454f4f1300 0x56454f4f47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:14.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.361+0000 7f0661000700 1 --2- 172.21.15.123:0/2792300014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56454f4f09e0 0x56454f4f42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:14.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.362+0000 7f0661a00700 1 -- 172.21.15.123:0/2792300014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56454f4f09e0 msgr2=0x56454f4f42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:14.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.362+0000 7f0661a00700 1 --2- 172.21.15.123:0/2792300014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56454f4f09e0 0x56454f4f42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:14.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.362+0000 7f0661a00700 1 -- 172.21.15.123:0/2792300014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56454f4f1300 msgr2=0x56454f4f47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:14.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.362+0000 7f0661a00700 1 --2- 172.21.15.123:0/2792300014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56454f4f1300 0x56454f4f47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:14.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.362+0000 7f0661a00700 1 -- 172.21.15.123:0/2792300014 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56454f403420 con 0x56454f4eb600 2024-05-08T06:12:14.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.362+0000 7f0661000700 1 --2- 172.21.15.123:0/2792300014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56454f4f09e0 0x56454f4f42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:12:14.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.362+0000 7f0661a00700 1 --2- 172.21.15.123:0/2792300014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56454f4eb600 0x56454f4f6f10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f06500a56e0 tx=0x7f0650007f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65cb37416a062e18 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:14.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.362+0000 7f0643400700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-05-08T06:12:14.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.362+0000 7f0643400700 1 -- 172.21.15.123:0/2792300014 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06500c7070 con 0x56454f4eb600 2024-05-08T06:12:14.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.362+0000 7f0643400700 1 -- 172.21.15.123:0/2792300014 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f06500a5ca0 con 0x56454f4eb600 2024-05-08T06:12:14.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.362+0000 7f066ecccb80 1 -- 172.21.15.123:0/2792300014 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06500ada60 con 0x56454f4eb600 2024-05-08T06:12:14.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.363+0000 7f0643400700 1 -- 172.21.15.123:0/2792300014 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f06500c2070 con 0x56454f4eb600 2024-05-08T06:12:14.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.363+0000 7f066ecccb80 10 client.25462 fetch_fsmap learned FSMap version 24 2024-05-08T06:12:14.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.363+0000 7f066ecccb80 1 -- 172.21.15.123:0/2792300014 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56454f4f7db0 con 0x56454f4eb600 2024-05-08T06:12:14.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.363+0000 7f0643400700 1 -- 172.21.15.123:0/2792300014 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06500ada60 con 0x56454f4eb600 2024-05-08T06:12:14.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.363+0000 7f0643400700 1 -- 172.21.15.123:0/2792300014 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f06500beb00 con 0x56454f4eb600 2024-05-08T06:12:14.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.363+0000 7f066ecccb80 10 client.25462 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:12:14.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.363+0000 7f066ecccb80 10 client.25462 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T06:12:14.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.363+0000 7f066ecccb80 20 client.25462 populate_metadata read hostname 'smithi123' 2024-05-08T06:12:14.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.363+0000 7f066ecccb80 1 --2- 172.21.15.123:0/2792300014 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x56454f5047d0 0x56454f506bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:14.366 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.363+0000 7f066ecccb80 4 client.25462 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:12:14.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.363+0000 7f066ecccb80 1 -- 172.21.15.123:0/2792300014 --> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56454f4b80a0 con 0x56454f5047d0 2024-05-08T06:12:14.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.364+0000 7f0661000700 1 --2- 172.21.15.123:0/2792300014 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x56454f5047d0 0x56454f506bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:14.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.364+0000 7f0661000700 1 --2- 172.21.15.123:0/2792300014 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x56454f5047d0 0x56454f506bc0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=334a3387cbd23539 server_cookie=98f147892d5c72ac in_seq=0 out_seq=0 2024-05-08T06:12:14.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.364+0000 7f0643400700 10 client.25462 ms_handle_connect on v2:172.21.15.192:6838/3598682360 2024-05-08T06:12:14.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.365+0000 7f0643400700 1 -- 172.21.15.123:0/2792300014 <== mds.2 v2:172.21.15.192:6838/3598682360 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56454f4b80a0 con 0x56454f5047d0 2024-05-08T06:12:14.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.365+0000 7f0643400700 10 client.25462 handle_command_reply: tid=0 2024-05-08T06:12:14.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.365+0000 7f066ecccb80 1 client.25462 shutdown 2024-05-08T06:12:14.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.365+0000 7f066ecccb80 1 -- 172.21.15.123:0/2792300014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56454f4eb600 msgr2=0x56454f4f6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:14.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.365+0000 7f066ecccb80 1 --2- 172.21.15.123:0/2792300014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56454f4eb600 0x56454f4f6f10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f06500a56e0 tx=0x7f0650007f50 comp rx=0 tx=0).stop 2024-05-08T06:12:14.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.365+0000 7f066ecccb80 1 -- 172.21.15.123:0/2792300014 shutdown_connections 2024-05-08T06:12:14.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.365+0000 7f066ecccb80 1 --2- 172.21.15.123:0/2792300014 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x56454f5047d0 0x56454f506bc0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:14.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.365+0000 7f066ecccb80 1 --2- 172.21.15.123:0/2792300014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56454f4f1300 0x56454f4f47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:14.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.365+0000 7f066ecccb80 1 --2- 172.21.15.123:0/2792300014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56454f4eb600 0x56454f4f6f10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:14.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.365+0000 7f066ecccb80 1 --2- 172.21.15.123:0/2792300014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56454f4f09e0 0x56454f4f42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:14.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.365+0000 7f0643400700 0 client.25462 ms_handle_reset on v2:172.21.15.192:6838/3598682360 2024-05-08T06:12:14.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.365+0000 7f066ecccb80 1 -- 172.21.15.123:0/2792300014 >> 172.21.15.123:0/2792300014 conn(0x56454f419b60 msgr2=0x56454f4e0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:14.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.366+0000 7f066ecccb80 1 -- 172.21.15.123:0/2792300014 shutdown_connections 2024-05-08T06:12:14.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.366+0000 7f066ecccb80 1 -- 172.21.15.123:0/2792300014 wait complete. 2024-05-08T06:12:14.368 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.366+0000 7f066ecccb80 20 client.25462 trim_cache size 0 max 16384 2024-05-08T06:12:14.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.382+0000 7f066ecccb80 1 Processor -- start 2024-05-08T06:12:14.382 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.382+0000 7f066ecccb80 1 -- start start 2024-05-08T06:12:14.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.383+0000 7f066ecccb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56454f4eb600 0x56454f500e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:14.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.383+0000 7f066ecccb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56454f4f09e0 0x56454f4f1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:14.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.383+0000 7f066ecccb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56454f4f1300 0x56454f4f32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:14.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.383+0000 7f066ecccb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f065c1b2450 con 0x56454f4eb600 2024-05-08T06:12:14.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.383+0000 7f066ecccb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f065c1afce0 con 0x56454f4f09e0 2024-05-08T06:12:14.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.383+0000 7f066ecccb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f065c1adb30 con 0x56454f4f1300 2024-05-08T06:12:14.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.383+0000 7f0662400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56454f4f1300 0x56454f4f32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:14.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.383+0000 7f0662400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56454f4f1300 0x56454f4f32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:41952/0 (socket says 172.21.15.123:41952) 2024-05-08T06:12:14.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.383+0000 7f0662400700 1 -- 172.21.15.123:0/1452562074 learned_addr learned my addr 172.21.15.123:0/1452562074 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:14.383 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.383+0000 7f0661a00700 1 --2- 172.21.15.123:0/1452562074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56454f4eb600 0x56454f500e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:14.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.383+0000 7f0661000700 1 --2- 172.21.15.123:0/1452562074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56454f4f09e0 0x56454f4f1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:14.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.383+0000 7f0662400700 1 -- 172.21.15.123:0/1452562074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56454f4f09e0 msgr2=0x56454f4f1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:14.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.383+0000 7f0662400700 1 --2- 172.21.15.123:0/1452562074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56454f4f09e0 0x56454f4f1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:14.384 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.383+0000 7f0662400700 1 -- 172.21.15.123:0/1452562074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56454f4eb600 msgr2=0x56454f500e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:14.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.383+0000 7f0662400700 1 --2- 172.21.15.123:0/1452562074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56454f4eb600 0x56454f500e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:14.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.383+0000 7f0662400700 1 -- 172.21.15.123:0/1452562074 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56454f4f2800 con 0x56454f4f1300 2024-05-08T06:12:14.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.383+0000 7f0661a00700 1 --2- 172.21.15.123:0/1452562074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56454f4eb600 0x56454f500e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:12:14.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.383+0000 7f0661000700 1 --2- 172.21.15.123:0/1452562074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56454f4f09e0 0x56454f4f1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:12:14.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.384+0000 7f0662400700 1 --2- 172.21.15.123:0/1452562074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56454f4f1300 0x56454f4f32d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f06540094d0 tx=0x7f06540099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abeecb45650e1024 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:14.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.384+0000 7f064e000700 1 -- 172.21.15.123:0/1452562074 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0654003a00 con 0x56454f4f1300 2024-05-08T06:12:14.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.384+0000 7f064e000700 1 -- 172.21.15.123:0/1452562074 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0654003ba0 con 0x56454f4f1300 2024-05-08T06:12:14.385 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.384+0000 7f064e000700 1 -- 172.21.15.123:0/1452562074 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0654017700 con 0x56454f4f1300 2024-05-08T06:12:14.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.385+0000 7f066ecccb80 1 -- 172.21.15.123:0/1452562074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56454f4f1300 msgr2=0x56454f4f32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:14.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.385+0000 7f066ecccb80 1 --2- 172.21.15.123:0/1452562074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56454f4f1300 0x56454f4f32d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f06540094d0 tx=0x7f06540099e0 comp rx=0 tx=0).stop 2024-05-08T06:12:14.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.385+0000 7f066ecccb80 1 -- 172.21.15.123:0/1452562074 shutdown_connections 2024-05-08T06:12:14.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.385+0000 7f066ecccb80 1 --2- 172.21.15.123:0/1452562074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56454f4eb600 0x56454f500e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:14.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.385+0000 7f066ecccb80 1 --2- 172.21.15.123:0/1452562074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56454f4f1300 0x56454f4f32d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:14.386 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.385+0000 7f066ecccb80 1 --2- 172.21.15.123:0/1452562074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56454f4f09e0 0x56454f4f1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:14.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.385+0000 7f066ecccb80 1 -- 172.21.15.123:0/1452562074 >> 172.21.15.123:0/1452562074 conn(0x56454f419b60 msgr2=0x56454f4f2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:14.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.385+0000 7f066ecccb80 1 -- 172.21.15.123:0/1452562074 shutdown_connections 2024-05-08T06:12:14.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.385+0000 7f066ecccb80 1 -- 172.21.15.123:0/1452562074 wait complete. 2024-05-08T06:12:14.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.386+0000 7f066ecccb80 1 -- start start 2024-05-08T06:12:14.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.386+0000 7f066ecccb80 1 Processor -- start 2024-05-08T06:12:14.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.386+0000 7f066ecccb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56454f4eb600 0x56454f4fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:14.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.386+0000 7f066ecccb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56454f4f09e0 0x56454f4fd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:14.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.386+0000 7f066ecccb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56454f4f1300 0x56454f4fa190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:14.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.386+0000 7f066ecccb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f065c1ae1a0 con 0x56454f4f1300 2024-05-08T06:12:14.387 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.386+0000 7f066ecccb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f065c1af610 con 0x56454f4f09e0 2024-05-08T06:12:14.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.386+0000 7f066ecccb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f065c1ae870 con 0x56454f4eb600 2024-05-08T06:12:14.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.387+0000 7f066ecccb80 10 client.0 _ll_register_callbacks cb 0x56454f3e4c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:12:14.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.387+0000 7f0661a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56454f4eb600 0x56454f4fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:14.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.387+0000 7f0662400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56454f4f1300 0x56454f4fa190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:14.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.387+0000 7f0661a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56454f4eb600 0x56454f4fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:41966/0 (socket says 172.21.15.123:41966) 2024-05-08T06:12:14.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.387+0000 7f0661a00700 1 -- 172.21.15.123:0/3053389007 learned_addr learned my addr 172.21.15.123:0/3053389007 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:14.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.387+0000 7f0661000700 1 --2- 172.21.15.123:0/3053389007 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56454f4f09e0 0x56454f4fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:14.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.387+0000 7f0661a00700 1 -- 172.21.15.123:0/3053389007 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56454f4f09e0 msgr2=0x56454f4fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:14.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.387+0000 7f0661a00700 1 --2- 172.21.15.123:0/3053389007 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56454f4f09e0 0x56454f4fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:14.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.387+0000 7f0661a00700 1 -- 172.21.15.123:0/3053389007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56454f4f1300 msgr2=0x56454f4fa190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:14.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.387+0000 7f0661a00700 1 --2- 172.21.15.123:0/3053389007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56454f4f1300 0x56454f4fa190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:14.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.387+0000 7f0661a00700 1 -- 172.21.15.123:0/3053389007 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56454f4fa7d0 con 0x56454f4eb600 2024-05-08T06:12:14.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.387+0000 7f0661000700 1 --2- 172.21.15.123:0/3053389007 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56454f4f09e0 0x56454f4fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:12:14.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.388+0000 7f0661a00700 1 --2- 172.21.15.123:0/3053389007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56454f4eb600 0x56454f4fd1c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f06500a6920 tx=0x7f06500912c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65b4fcb783a4a41d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:14.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.388+0000 7f0643400700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-05-08T06:12:14.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.388+0000 7f0643400700 1 -- 172.21.15.123:0/3053389007 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06500b6030 con 0x56454f4eb600 2024-05-08T06:12:14.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.388+0000 7f0643400700 1 -- 172.21.15.123:0/3053389007 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f06500a5ca0 con 0x56454f4eb600 2024-05-08T06:12:14.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.388+0000 7f066ecccb80 1 -- 172.21.15.123:0/3053389007 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0654003ba0 con 0x56454f4eb600 2024-05-08T06:12:14.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.388+0000 7f0643400700 1 -- 172.21.15.123:0/3053389007 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f06500c7040 con 0x56454f4eb600 2024-05-08T06:12:14.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.388+0000 7f066ecccb80 10 client.25468 fetch_fsmap learned FSMap version 24 2024-05-08T06:12:14.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.388+0000 7f066ecccb80 1 -- 172.21.15.123:0/3053389007 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56454f56f770 con 0x56454f4eb600 2024-05-08T06:12:14.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.388+0000 7f0643400700 1 -- 172.21.15.123:0/3053389007 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0654003ba0 con 0x56454f4eb600 2024-05-08T06:12:14.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.388+0000 7f0643400700 1 -- 172.21.15.123:0/3053389007 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f06500a8380 con 0x56454f4eb600 2024-05-08T06:12:14.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.389+0000 7f066ecccb80 10 client.25468 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:12:14.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.389+0000 7f066ecccb80 10 client.25468 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T06:12:14.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.389+0000 7f066ecccb80 20 client.25468 populate_metadata read hostname 'smithi123' 2024-05-08T06:12:14.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.389+0000 7f066ecccb80 1 --2- 172.21.15.123:0/3053389007 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x56454f56fbd0 0x56454f571fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:14.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.389+0000 7f066ecccb80 4 client.25468 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T06:12:14.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.389+0000 7f066ecccb80 1 -- 172.21.15.123:0/3053389007 --> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56454f44b400 con 0x56454f56fbd0 2024-05-08T06:12:14.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.389+0000 7f0661000700 1 --2- 172.21.15.123:0/3053389007 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x56454f56fbd0 0x56454f571fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:14.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.389+0000 7f0661000700 1 --2- 172.21.15.123:0/3053389007 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x56454f56fbd0 0x56454f571fc0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3235fb1f231c2ac3 server_cookie=79f3cfa803bec3e6 in_seq=0 out_seq=0 2024-05-08T06:12:14.395 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.389+0000 7f0643400700 10 client.25468 ms_handle_connect on v2:172.21.15.192:6838/3598682360 2024-05-08T06:12:14.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.390+0000 7f0643400700 1 -- 172.21.15.123:0/3053389007 <== mds.2 v2:172.21.15.192:6838/3598682360 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56454f44b400 con 0x56454f56fbd0 2024-05-08T06:12:14.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.390+0000 7f0643400700 10 client.25468 handle_command_reply: tid=0 2024-05-08T06:12:14.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.390+0000 7f066ecccb80 1 client.25468 shutdown 2024-05-08T06:12:14.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.390+0000 7f066ecccb80 1 -- 172.21.15.123:0/3053389007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56454f4eb600 msgr2=0x56454f4fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:14.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.390+0000 7f066ecccb80 1 --2- 172.21.15.123:0/3053389007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56454f4eb600 0x56454f4fd1c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f06500a6920 tx=0x7f06500912c0 comp rx=0 tx=0).stop 2024-05-08T06:12:14.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.390+0000 7f066ecccb80 1 -- 172.21.15.123:0/3053389007 shutdown_connections 2024-05-08T06:12:14.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.390+0000 7f066ecccb80 1 --2- 172.21.15.123:0/3053389007 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x56454f56fbd0 0x56454f571fc0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:14.396 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.390+0000 7f066ecccb80 1 --2- 172.21.15.123:0/3053389007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56454f4f1300 0x56454f4fa190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:14.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.390+0000 7f066ecccb80 1 --2- 172.21.15.123:0/3053389007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56454f4eb600 0x56454f4fd1c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:14.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.390+0000 7f066ecccb80 1 --2- 172.21.15.123:0/3053389007 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56454f4f09e0 0x56454f4fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:14.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.390+0000 7f0643400700 0 client.25468 ms_handle_reset on v2:172.21.15.192:6838/3598682360 2024-05-08T06:12:14.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.390+0000 7f066ecccb80 1 -- 172.21.15.123:0/3053389007 >> 172.21.15.123:0/3053389007 conn(0x56454f419b60 msgr2=0x56454f4dfbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:14.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.390+0000 7f066ecccb80 1 -- 172.21.15.123:0/3053389007 shutdown_connections 2024-05-08T06:12:14.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.390+0000 7f066ecccb80 1 -- 172.21.15.123:0/3053389007 wait complete. 2024-05-08T06:12:14.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.391+0000 7f066ecccb80 20 client.25468 trim_cache size 0 max 16384 2024-05-08T06:12:14.397 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-05-08T06:12:14.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.391+0000 7f0662e00700 1 -- 172.21.15.123:0/2788354731 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f06440b2900 msgr2=0x7f06440b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:14.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.391+0000 7f0662e00700 1 --2- 172.21.15.123:0/2788354731 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f06440b2900 0x7f06440b4dc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0648009fd0 tx=0x7f0648019040 comp rx=0 tx=0).stop 2024-05-08T06:12:14.398 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.391+0000 7f0662e00700 1 -- 172.21.15.123:0/2788354731 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f065c102120 msgr2=0x7f065c1a18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:14.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.391+0000 7f0662e00700 1 --2- 172.21.15.123:0/2788354731 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f065c102120 0x7f065c1a18e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f065000c9b0 tx=0x7f065000ccc0 comp rx=0 tx=0).stop 2024-05-08T06:12:14.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.391+0000 7f0662e00700 1 -- 172.21.15.123:0/2788354731 shutdown_connections 2024-05-08T06:12:14.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.391+0000 7f0662e00700 1 --2- 172.21.15.123:0/2788354731 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f06440b2900 0x7f06440b4dc0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:14.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.391+0000 7f0662e00700 1 --2- 172.21.15.123:0/2788354731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f065c10fcd0 0x7f065c19baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:14.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.391+0000 7f0662e00700 1 --2- 172.21.15.123:0/2788354731 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f065c102120 0x7f065c1a18e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:14.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.391+0000 7f0662e00700 1 --2- 172.21.15.123:0/2788354731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f065c102ad0 0x7f065c1a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:14.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.391+0000 7f0662e00700 1 -- 172.21.15.123:0/2788354731 >> 172.21.15.123:0/2788354731 conn(0x7f065c0fddc0 msgr2=0x7f065c1001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:14.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.391+0000 7f0662e00700 1 -- 172.21.15.123:0/2788354731 shutdown_connections 2024-05-08T06:12:14.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:14.391+0000 7f0662e00700 1 -- 172.21.15.123:0/2788354731 wait complete. 2024-05-08T06:12:14.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:14 smithi123 ceph-mon[29787]: pgmap v513: 97 pgs: 97 active+clean; 8.7 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 683 KiB/s wr, 6 op/s 2024-05-08T06:12:15.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:14 smithi163 ceph-mon[34520]: pgmap v513: 97 pgs: 97 active+clean; 8.7 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 683 KiB/s wr, 6 op/s 2024-05-08T06:12:15.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:14 smithi192 ceph-mon[34664]: pgmap v513: 97 pgs: 97 active+clean; 8.7 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 683 KiB/s wr, 6 op/s 2024-05-08T06:12:15.291 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.291+0000 7f1d7fe00640 1 -- 172.21.15.123:0/2140234544 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d781025e0 msgr2=0x7f1d78102a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:15.291 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.291+0000 7f1d7fe00640 1 --2- 172.21.15.123:0/2140234544 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d781025e0 0x7f1d78102a60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1d70009ca0 tx=0x7f1d7002a8c0 comp rx=0 tx=0).stop 2024-05-08T06:12:15.291 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.291+0000 7f1d7fe00640 1 -- 172.21.15.123:0/2140234544 shutdown_connections 2024-05-08T06:12:15.291 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.291+0000 7f1d7fe00640 1 --2- 172.21.15.123:0/2140234544 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d781025e0 0x7f1d78102a60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:15.291 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.291+0000 7f1d7fe00640 1 --2- 172.21.15.123:0/2140234544 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d78101c40 0x7f1d781020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:15.292 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.291+0000 7f1d7fe00640 1 --2- 172.21.15.123:0/2140234544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d781012e0 0x7f1d78101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:15.292 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.291+0000 7f1d7fe00640 1 -- 172.21.15.123:0/2140234544 >> 172.21.15.123:0/2140234544 conn(0x7f1d780fce80 msgr2=0x7f1d780ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:15.292 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.291+0000 7f1d7fe00640 1 -- 172.21.15.123:0/2140234544 shutdown_connections 2024-05-08T06:12:15.292 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.291+0000 7f1d7fe00640 1 -- 172.21.15.123:0/2140234544 wait complete. 2024-05-08T06:12:15.294 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.292+0000 7f1d7fe00640 1 Processor -- start 2024-05-08T06:12:15.294 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.292+0000 7f1d7fe00640 1 -- start start 2024-05-08T06:12:15.294 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.292+0000 7f1d7fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d781012e0 0x7f1d7819ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:15.294 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.293+0000 7f1d7fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d78101c40 0x7f1d7819b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:15.294 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.293+0000 7f1d7fe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d781025e0 0x7f1d781a22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:15.294 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.293+0000 7f1d7fe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1d78110780 con 0x7f1d781012e0 2024-05-08T06:12:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.293+0000 7f1d7fe00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1d78110600 con 0x7f1d78101c40 2024-05-08T06:12:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.293+0000 7f1d7fe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1d78110900 con 0x7f1d781025e0 2024-05-08T06:12:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.293+0000 7f1d7d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d781012e0 0x7f1d7819ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.293+0000 7f1d7d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d781012e0 0x7f1d7819ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51248/0 (socket says 172.21.15.123:51248) 2024-05-08T06:12:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.293+0000 7f1d7d400640 1 -- 172.21.15.123:0/3465148505 learned_addr learned my addr 172.21.15.123:0/3465148505 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.293+0000 7f1d7ca00640 1 --2- 172.21.15.123:0/3465148505 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d78101c40 0x7f1d7819b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.293+0000 7f1d7de00640 1 --2- 172.21.15.123:0/3465148505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d781025e0 0x7f1d781a22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.293+0000 7f1d7d400640 1 -- 172.21.15.123:0/3465148505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d781025e0 msgr2=0x7f1d781a22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.293+0000 7f1d7d400640 1 --2- 172.21.15.123:0/3465148505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d781025e0 0x7f1d781a22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.293+0000 7f1d7d400640 1 -- 172.21.15.123:0/3465148505 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d78101c40 msgr2=0x7f1d7819b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.293+0000 7f1d7d400640 1 --2- 172.21.15.123:0/3465148505 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d78101c40 0x7f1d7819b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.293+0000 7f1d7d400640 1 -- 172.21.15.123:0/3465148505 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d781a2960 con 0x7f1d781012e0 2024-05-08T06:12:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.293+0000 7f1d7de00640 1 --2- 172.21.15.123:0/3465148505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d781025e0 0x7f1d781a22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:12:15.296 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.294+0000 7f1d7d400640 1 --2- 172.21.15.123:0/3465148505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d781012e0 0x7f1d7819ad90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c00ea60 tx=0x7f1d6c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:15.296 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.294+0000 7f1d76000640 1 -- 172.21.15.123:0/3465148505 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c00cde0 con 0x7f1d781012e0 2024-05-08T06:12:15.296 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.294+0000 7f1d7fe00640 1 -- 172.21.15.123:0/3465148505 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d781a2c50 con 0x7f1d781012e0 2024-05-08T06:12:15.296 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.294+0000 7f1d76000640 1 -- 172.21.15.123:0/3465148505 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d6c0035f0 con 0x7f1d781012e0 2024-05-08T06:12:15.296 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.294+0000 7f1d76000640 1 -- 172.21.15.123:0/3465148505 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c010690 con 0x7f1d781012e0 2024-05-08T06:12:15.296 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.294+0000 7f1d7fe00640 1 -- 172.21.15.123:0/3465148505 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d781a31a0 con 0x7f1d781012e0 2024-05-08T06:12:15.298 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.298+0000 7f1d82c63740 1 Processor -- start 2024-05-08T06:12:15.298 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.299+0000 7f1d82c63740 1 -- start start 2024-05-08T06:12:15.300 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.299+0000 7f1d82c63740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564917824120 0x564917824540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:15.300 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.299+0000 7f1d82c63740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564917824b10 0x564917829610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:15.300 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.299+0000 7f1d82c63740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564917829c80 0x56491782c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:15.300 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.299+0000 7f1d82c63740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564917788550 con 0x564917824120 2024-05-08T06:12:15.300 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.299+0000 7f1d82c63740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x56491772dbc0 con 0x564917829c80 2024-05-08T06:12:15.300 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.299+0000 7f1d82c63740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564917784510 con 0x564917824b10 2024-05-08T06:12:15.300 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.299+0000 7f1d76000640 1 -- 172.21.15.123:0/3465148505 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7f1d6c0037a0 con 0x7f1d781012e0 2024-05-08T06:12:15.300 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.300+0000 7f1d7de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564917824120 0x564917824540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:15.300 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.300+0000 7f1d7ca00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564917829c80 0x56491782c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:15.300 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.300+0000 7f1d7ca00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564917829c80 0x56491782c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:34300/0 (socket says 172.21.15.123:34300) 2024-05-08T06:12:15.301 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.300+0000 7f1d7ca00640 1 -- 172.21.15.123:0/648056654 learned_addr learned my addr 172.21.15.123:0/648056654 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:15.301 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.300+0000 7f1d7d400640 1 --2- 172.21.15.123:0/648056654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564917824b10 0x564917829610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:15.301 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.300+0000 7f1d7ca00640 1 -- 172.21.15.123:0/648056654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564917824b10 msgr2=0x564917829610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:15.301 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.300+0000 7f1d7ca00640 1 --2- 172.21.15.123:0/648056654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564917824b10 0x564917829610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:15.301 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.300+0000 7f1d7ca00640 1 -- 172.21.15.123:0/648056654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564917824120 msgr2=0x564917824540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:15.301 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.300+0000 7f1d7ca00640 1 --2- 172.21.15.123:0/648056654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564917824120 0x564917824540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:15.301 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.300+0000 7f1d7ca00640 1 -- 172.21.15.123:0/648056654 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56491749a210 con 0x564917829c80 2024-05-08T06:12:15.301 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.300+0000 7f1d7d400640 1 --2- 172.21.15.123:0/648056654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564917824b10 0x564917829610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:12:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.300+0000 7f1d7ca00640 1 --2- 172.21.15.123:0/648056654 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564917829c80 0x56491782c110 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1d64009ca0 tx=0x7f1d64011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e7ff9c9edd24bfb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.301+0000 7f1d6a600640 1 -- 172.21.15.123:0/648056654 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6401e070 con 0x564917829c80 2024-05-08T06:12:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.301+0000 7f1d6a600640 1 -- 172.21.15.123:0/648056654 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d64019ae0 con 0x564917829c80 2024-05-08T06:12:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.301+0000 7f1d6a600640 1 -- 172.21.15.123:0/648056654 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6400e9e0 con 0x564917829c80 2024-05-08T06:12:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.302+0000 7f1d82c63740 1 -- 172.21.15.123:0/648056654 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564917829c80 msgr2=0x56491782c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.302+0000 7f1d82c63740 1 --2- 172.21.15.123:0/648056654 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564917829c80 0x56491782c110 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1d64009ca0 tx=0x7f1d64011810 comp rx=0 tx=0).stop 2024-05-08T06:12:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.302+0000 7f1d76000640 1 --2- 172.21.15.123:0/3465148505 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f1d600b2920 0x7f1d600b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.302+0000 7f1d82c63740 1 -- 172.21.15.123:0/648056654 shutdown_connections 2024-05-08T06:12:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.302+0000 7f1d82c63740 1 --2- 172.21.15.123:0/648056654 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564917829c80 0x56491782c110 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.302+0000 7f1d82c63740 1 --2- 172.21.15.123:0/648056654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564917824b10 0x564917829610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.302+0000 7f1d82c63740 1 --2- 172.21.15.123:0/648056654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564917824120 0x564917824540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.302+0000 7f1d82c63740 1 -- 172.21.15.123:0/648056654 >> 172.21.15.123:0/648056654 conn(0x5649177ec230 msgr2=0x56491781fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.302+0000 7f1d76000640 1 -- 172.21.15.123:0/3465148505 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f1d6c0e2550 con 0x7f1d781012e0 2024-05-08T06:12:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.302+0000 7f1d7de00640 1 --2- 172.21.15.123:0/3465148505 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f1d600b2920 0x7f1d600b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.302+0000 7f1d82c63740 1 -- 172.21.15.123:0/648056654 shutdown_connections 2024-05-08T06:12:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.302+0000 7f1d82c63740 1 -- 172.21.15.123:0/648056654 wait complete. 2024-05-08T06:12:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.303+0000 7f1d82c63740 1 -- start start 2024-05-08T06:12:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.303+0000 7f1d82c63740 1 Processor -- start 2024-05-08T06:12:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.303+0000 7f1d7de00640 1 --2- 172.21.15.123:0/3465148505 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f1d600b2920 0x7f1d600b4de0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1d70005e30 tx=0x7f1d70005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.304+0000 7f1d82c63740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564917824120 0x56491782e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.304+0000 7f1d82c63740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564917824b10 0x56491782e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.304+0000 7f1d82c63740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564917829c80 0x564917835680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.304+0000 7f1d82c63740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56491782cc30 con 0x564917824120 2024-05-08T06:12:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.304+0000 7f1d82c63740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x564917741660 con 0x564917829c80 2024-05-08T06:12:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.304+0000 7f1d82c63740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56491782cdb0 con 0x564917824b10 2024-05-08T06:12:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.304+0000 7f1d82c63740 10 client.0 _ll_register_callbacks cb 0x56491770b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:12:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.304+0000 7f1d7ca00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564917824b10 0x56491782e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.304+0000 7f1d7ca00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564917824b10 0x56491782e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:34520/0 (socket says 172.21.15.123:34520) 2024-05-08T06:12:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.304+0000 7f1d7ca00640 1 -- 172.21.15.123:0/3295628534 learned_addr learned my addr 172.21.15.123:0/3295628534 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.304+0000 7f1d7d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564917824120 0x56491782e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.304+0000 7f1d7de00640 1 --2- 172.21.15.123:0/3295628534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564917829c80 0x564917835680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.304+0000 7f1d7ca00640 1 -- 172.21.15.123:0/3295628534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564917829c80 msgr2=0x564917835680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.304+0000 7f1d7ca00640 1 --2- 172.21.15.123:0/3295628534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564917829c80 0x564917835680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.304+0000 7f1d7ca00640 1 -- 172.21.15.123:0/3295628534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564917824120 msgr2=0x56491782e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.304+0000 7f1d7ca00640 1 --2- 172.21.15.123:0/3295628534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564917824120 0x56491782e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.304+0000 7f1d7ca00640 1 -- 172.21.15.123:0/3295628534 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564917566620 con 0x564917824b10 2024-05-08T06:12:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.304+0000 7f1d7de00640 1 --2- 172.21.15.123:0/3295628534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564917829c80 0x564917835680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:12:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.305+0000 7f1d7ca00640 1 --2- 172.21.15.123:0/3295628534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564917824b10 0x56491782e640 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1d64009710 tx=0x7f1d6400e430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4b43ecb383fb3ff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.305+0000 7f1d5ea00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-05-08T06:12:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.305+0000 7f1d5ea00640 1 -- 172.21.15.123:0/3295628534 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d64027020 con 0x564917824b10 2024-05-08T06:12:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.305+0000 7f1d82c63740 1 -- 172.21.15.123:0/3295628534 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d64019ae0 con 0x564917824b10 2024-05-08T06:12:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.305+0000 7f1d5ea00640 1 -- 172.21.15.123:0/3295628534 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f1d64005460 con 0x564917824b10 2024-05-08T06:12:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.305+0000 7f1d5ea00640 1 -- 172.21.15.123:0/3295628534 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1d6401e070 con 0x564917824b10 2024-05-08T06:12:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.305+0000 7f1d82c63740 10 client.25480 fetch_fsmap learned FSMap version 24 2024-05-08T06:12:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.305+0000 7f1d82c63740 1 -- 172.21.15.123:0/3295628534 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564917838660 con 0x564917824b10 2024-05-08T06:12:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.305+0000 7f1d5ea00640 1 -- 172.21.15.123:0/3295628534 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d64019ae0 con 0x564917824b10 2024-05-08T06:12:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.306+0000 7f1d5ea00640 1 -- 172.21.15.123:0/3295628534 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f1d64004ad0 con 0x564917824b10 2024-05-08T06:12:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.306+0000 7f1d82c63740 10 client.25480 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:12:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.306+0000 7f1d82c63740 10 client.25480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T06:12:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.306+0000 7f1d82c63740 20 client.25480 populate_metadata read hostname 'smithi123' 2024-05-08T06:12:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.306+0000 7f1d82c63740 1 --2- 172.21.15.123:0/3295628534 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x564917839be0 0x56491783bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.306+0000 7f1d82c63740 4 client.25480 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:12:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.306+0000 7f1d82c63740 1 -- 172.21.15.123:0/3295628534 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564917796160 con 0x564917839be0 2024-05-08T06:12:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.306+0000 7f1d7d400640 1 --2- 172.21.15.123:0/3295628534 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x564917839be0 0x56491783bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.307+0000 7f1d7d400640 1 --2- 172.21.15.123:0/3295628534 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x564917839be0 0x56491783bfd0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=993f9dd3da298405 server_cookie=4b9cf791ae1058b7 in_seq=0 out_seq=0 2024-05-08T06:12:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.307+0000 7f1d5ea00640 10 client.25480 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.307+0000 7f1d5ea00640 1 -- 172.21.15.123:0/3295628534 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564917796160 con 0x564917839be0 2024-05-08T06:12:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.307+0000 7f1d5ea00640 10 client.25480 handle_command_reply: tid=0 2024-05-08T06:12:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.308+0000 7f1d82c63740 1 client.25480 shutdown 2024-05-08T06:12:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.308+0000 7f1d82c63740 1 -- 172.21.15.123:0/3295628534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564917824b10 msgr2=0x56491782e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.308+0000 7f1d82c63740 1 --2- 172.21.15.123:0/3295628534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564917824b10 0x56491782e640 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1d64009710 tx=0x7f1d6400e430 comp rx=0 tx=0).stop 2024-05-08T06:12:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.308+0000 7f1d82c63740 1 -- 172.21.15.123:0/3295628534 shutdown_connections 2024-05-08T06:12:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.308+0000 7f1d82c63740 1 --2- 172.21.15.123:0/3295628534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564917829c80 0x564917835680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.308+0000 7f1d82c63740 1 --2- 172.21.15.123:0/3295628534 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x564917839be0 0x56491783bfd0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.308+0000 7f1d82c63740 1 --2- 172.21.15.123:0/3295628534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564917824b10 0x56491782e640 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.308+0000 7f1d82c63740 1 --2- 172.21.15.123:0/3295628534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564917824120 0x56491782e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.308+0000 7f1d5ea00640 0 client.25480 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.308+0000 7f1d82c63740 1 -- 172.21.15.123:0/3295628534 >> 172.21.15.123:0/3295628534 conn(0x5649177ec230 msgr2=0x56491781ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:15.312 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.308+0000 7f1d82c63740 1 -- 172.21.15.123:0/3295628534 shutdown_connections 2024-05-08T06:12:15.312 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.308+0000 7f1d82c63740 1 -- 172.21.15.123:0/3295628534 wait complete. 2024-05-08T06:12:15.312 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.308+0000 7f1d82c63740 20 client.25480 trim_cache size 0 max 16384 2024-05-08T06:12:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.332+0000 7f1d82c63740 1 Processor -- start 2024-05-08T06:12:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.332+0000 7f1d82c63740 1 -- start start 2024-05-08T06:12:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.332+0000 7f1d82c63740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564917824120 0x564917822a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.332+0000 7f1d82c63740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564917824b10 0x564917835640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.332+0000 7f1d82c63740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564917829c80 0x56491781f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:15.333 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.332+0000 7f1d82c63740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1d781ae770 con 0x564917824120 2024-05-08T06:12:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.332+0000 7f1d82c63740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1d781ac040 con 0x564917829c80 2024-05-08T06:12:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.332+0000 7f1d82c63740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1d781a9e90 con 0x564917824b10 2024-05-08T06:12:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.333+0000 7f1d7ca00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564917824b10 0x564917835640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.333+0000 7f1d7ca00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564917824b10 0x564917835640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:34532/0 (socket says 172.21.15.123:34532) 2024-05-08T06:12:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.333+0000 7f1d7ca00640 1 -- 172.21.15.123:0/814671717 learned_addr learned my addr 172.21.15.123:0/814671717 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.333+0000 7f1d7d400640 1 --2- 172.21.15.123:0/814671717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564917824120 0x564917822a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.333+0000 7f1d7de00640 1 --2- 172.21.15.123:0/814671717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564917829c80 0x56491781f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.333+0000 7f1d7ca00640 1 -- 172.21.15.123:0/814671717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564917829c80 msgr2=0x56491781f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.333+0000 7f1d7ca00640 1 --2- 172.21.15.123:0/814671717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564917829c80 0x56491781f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.333+0000 7f1d7ca00640 1 -- 172.21.15.123:0/814671717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564917824120 msgr2=0x564917822a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.333+0000 7f1d7ca00640 1 --2- 172.21.15.123:0/814671717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564917824120 0x564917822a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:15.334 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.333+0000 7f1d7ca00640 1 -- 172.21.15.123:0/814671717 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564917820850 con 0x564917824b10 2024-05-08T06:12:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.333+0000 7f1d7ca00640 1 --2- 172.21.15.123:0/814671717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564917824b10 0x564917835640 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1d64011d10 tx=0x7f1d64012f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acf3dca248676b6c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.333+0000 7f1d69c00640 1 -- 172.21.15.123:0/814671717 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d64027030 con 0x564917824b10 2024-05-08T06:12:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.334+0000 7f1d69c00640 1 -- 172.21.15.123:0/814671717 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d6401f040 con 0x564917824b10 2024-05-08T06:12:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.334+0000 7f1d69c00640 1 -- 172.21.15.123:0/814671717 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d640035f0 con 0x564917824b10 2024-05-08T06:12:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.334+0000 7f1d82c63740 1 -- 172.21.15.123:0/814671717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564917824b10 msgr2=0x564917835640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.334+0000 7f1d82c63740 1 --2- 172.21.15.123:0/814671717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564917824b10 0x564917835640 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1d64011d10 tx=0x7f1d64012f90 comp rx=0 tx=0).stop 2024-05-08T06:12:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.334+0000 7f1d82c63740 1 -- 172.21.15.123:0/814671717 shutdown_connections 2024-05-08T06:12:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.334+0000 7f1d82c63740 1 --2- 172.21.15.123:0/814671717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564917829c80 0x56491781f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.334+0000 7f1d82c63740 1 --2- 172.21.15.123:0/814671717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564917824b10 0x564917835640 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.334+0000 7f1d82c63740 1 --2- 172.21.15.123:0/814671717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564917824120 0x564917822a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.334+0000 7f1d82c63740 1 -- 172.21.15.123:0/814671717 >> 172.21.15.123:0/814671717 conn(0x5649177ec230 msgr2=0x564917825ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.334+0000 7f1d82c63740 1 -- 172.21.15.123:0/814671717 shutdown_connections 2024-05-08T06:12:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.335+0000 7f1d82c63740 1 -- 172.21.15.123:0/814671717 wait complete. 2024-05-08T06:12:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.335+0000 7f1d82c63740 1 -- start start 2024-05-08T06:12:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.335+0000 7f1d82c63740 1 Processor -- start 2024-05-08T06:12:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.335+0000 7f1d82c63740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564917824120 0x564917833030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.336+0000 7f1d82c63740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564917824b10 0x564917833570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.336+0000 7f1d82c63740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564917829c80 0x564917834530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.336+0000 7f1d82c63740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1d781aa500 con 0x564917829c80 2024-05-08T06:12:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.336+0000 7f1d82c63740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1d781ab970 con 0x564917824b10 2024-05-08T06:12:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.336+0000 7f1d82c63740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1d781aabd0 con 0x564917824120 2024-05-08T06:12:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.336+0000 7f1d82c63740 10 client.0 _ll_register_callbacks cb 0x7f1d6401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:12:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.336+0000 7f1d7d400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564917824120 0x564917833030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.336+0000 7f1d7d400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564917824120 0x564917833030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:34540/0 (socket says 172.21.15.123:34540) 2024-05-08T06:12:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.336+0000 7f1d7d400640 1 -- 172.21.15.123:0/1397824402 learned_addr learned my addr 172.21.15.123:0/1397824402 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.336+0000 7f1d7de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564917829c80 0x564917834530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.336+0000 7f1d7ca00640 1 --2- 172.21.15.123:0/1397824402 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564917824b10 0x564917833570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.336+0000 7f1d7d400640 1 -- 172.21.15.123:0/1397824402 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564917824b10 msgr2=0x564917833570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.336+0000 7f1d7d400640 1 --2- 172.21.15.123:0/1397824402 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564917824b10 0x564917833570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.336+0000 7f1d7d400640 1 -- 172.21.15.123:0/1397824402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564917829c80 msgr2=0x564917834530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.336+0000 7f1d7d400640 1 --2- 172.21.15.123:0/1397824402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564917829c80 0x564917834530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.336+0000 7f1d7d400640 1 -- 172.21.15.123:0/1397824402 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564917834b20 con 0x564917824120 2024-05-08T06:12:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.336+0000 7f1d7ca00640 1 --2- 172.21.15.123:0/1397824402 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564917824b10 0x564917833570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:12:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.336+0000 7f1d7d400640 1 --2- 172.21.15.123:0/1397824402 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564917824120 0x564917833030 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c0a6d90 tx=0x7f1d6c003e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a49454e37a5a138 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.336+0000 7f1d5ea00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-05-08T06:12:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.337+0000 7f1d5ea00640 1 -- 172.21.15.123:0/1397824402 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c0a9630 con 0x564917824120 2024-05-08T06:12:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.337+0000 7f1d5ea00640 1 -- 172.21.15.123:0/1397824402 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f1d6c004000 con 0x564917824120 2024-05-08T06:12:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.337+0000 7f1d82c63740 1 -- 172.21.15.123:0/1397824402 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d6401f040 con 0x564917824120 2024-05-08T06:12:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.337+0000 7f1d5ea00640 1 -- 172.21.15.123:0/1397824402 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1d6c0aa220 con 0x564917824120 2024-05-08T06:12:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.337+0000 7f1d82c63740 10 client.25486 fetch_fsmap learned FSMap version 24 2024-05-08T06:12:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.337+0000 7f1d82c63740 1 -- 172.21.15.123:0/1397824402 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5649178a47b0 con 0x564917824120 2024-05-08T06:12:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.337+0000 7f1d5ea00640 1 -- 172.21.15.123:0/1397824402 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d6c0aae50 con 0x564917824120 2024-05-08T06:12:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.337+0000 7f1d5ea00640 1 -- 172.21.15.123:0/1397824402 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f1d6c0e20b0 con 0x564917824120 2024-05-08T06:12:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.337+0000 7f1d82c63740 10 client.25486 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:12:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.337+0000 7f1d82c63740 10 client.25486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T06:12:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.337+0000 7f1d82c63740 20 client.25486 populate_metadata read hostname 'smithi123' 2024-05-08T06:12:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.337+0000 7f1d82c63740 1 --2- 172.21.15.123:0/1397824402 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5649178a5130 0x5649178a7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.337+0000 7f1d82c63740 4 client.25486 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T06:12:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.337+0000 7f1d82c63740 1 -- 172.21.15.123:0/1397824402 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56491778ffd0 con 0x5649178a5130 2024-05-08T06:12:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.338+0000 7f1d7ca00640 1 --2- 172.21.15.123:0/1397824402 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5649178a5130 0x5649178a7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:15.343 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.338+0000 7f1d7ca00640 1 --2- 172.21.15.123:0/1397824402 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5649178a5130 0x5649178a7550 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d8730ef571f65bb server_cookie=94346b672a1f2456 in_seq=0 out_seq=0 2024-05-08T06:12:15.343 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.338+0000 7f1d5ea00640 10 client.25486 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.351+0000 7f1d5ea00640 1 -- 172.21.15.123:0/1397824402 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56491778ffd0 con 0x5649178a5130 2024-05-08T06:12:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.351+0000 7f1d5ea00640 10 client.25486 handle_command_reply: tid=0 2024-05-08T06:12:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.351+0000 7f1d82c63740 1 client.25486 shutdown 2024-05-08T06:12:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.351+0000 7f1d82c63740 1 -- 172.21.15.123:0/1397824402 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564917824120 msgr2=0x564917833030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.351+0000 7f1d82c63740 1 --2- 172.21.15.123:0/1397824402 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564917824120 0x564917833030 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c0a6d90 tx=0x7f1d6c003e90 comp rx=0 tx=0).stop 2024-05-08T06:12:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.351+0000 7f1d82c63740 1 -- 172.21.15.123:0/1397824402 shutdown_connections 2024-05-08T06:12:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.351+0000 7f1d82c63740 1 --2- 172.21.15.123:0/1397824402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564917829c80 0x564917834530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.351+0000 7f1d82c63740 1 --2- 172.21.15.123:0/1397824402 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564917824b10 0x564917833570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.351+0000 7f1d82c63740 1 --2- 172.21.15.123:0/1397824402 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5649178a5130 0x5649178a7550 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.351+0000 7f1d82c63740 1 --2- 172.21.15.123:0/1397824402 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564917824120 0x564917833030 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.351+0000 7f1d5ea00640 0 client.25486 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.351+0000 7f1d82c63740 1 -- 172.21.15.123:0/1397824402 >> 172.21.15.123:0/1397824402 conn(0x5649177ec230 msgr2=0x56491781f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.351+0000 7f1d82c63740 1 -- 172.21.15.123:0/1397824402 shutdown_connections 2024-05-08T06:12:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.351+0000 7f1d82c63740 1 -- 172.21.15.123:0/1397824402 wait complete. 2024-05-08T06:12:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.351+0000 7f1d82c63740 20 client.25486 trim_cache size 0 max 16384 2024-05-08T06:12:15.353 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout:{ 2024-05-08T06:12:15.353 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout: "message": "", 2024-05-08T06:12:15.353 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout: "return_code": 0 2024-05-08T06:12:15.353 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout:} 2024-05-08T06:12:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.352+0000 7f1d7fe00640 1 -- 172.21.15.123:0/3465148505 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f1d600b2920 msgr2=0x7f1d600b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.352+0000 7f1d7fe00640 1 --2- 172.21.15.123:0/3465148505 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f1d600b2920 0x7f1d600b4de0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1d70005e30 tx=0x7f1d70005da0 comp rx=0 tx=0).stop 2024-05-08T06:12:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.352+0000 7f1d7fe00640 1 -- 172.21.15.123:0/3465148505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d781012e0 msgr2=0x7f1d7819ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.352+0000 7f1d7fe00640 1 --2- 172.21.15.123:0/3465148505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d781012e0 0x7f1d7819ad90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c00ea60 tx=0x7f1d6c00ef20 comp rx=0 tx=0).stop 2024-05-08T06:12:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.352+0000 7f1d7fe00640 1 -- 172.21.15.123:0/3465148505 shutdown_connections 2024-05-08T06:12:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.352+0000 7f1d7fe00640 1 --2- 172.21.15.123:0/3465148505 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f1d600b2920 0x7f1d600b4de0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.352+0000 7f1d7fe00640 1 --2- 172.21.15.123:0/3465148505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d781025e0 0x7f1d781a22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.352+0000 7f1d7fe00640 1 --2- 172.21.15.123:0/3465148505 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d78101c40 0x7f1d7819b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.352+0000 7f1d7fe00640 1 --2- 172.21.15.123:0/3465148505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d781012e0 0x7f1d7819ad90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.352+0000 7f1d7fe00640 1 -- 172.21.15.123:0/3465148505 >> 172.21.15.123:0/3465148505 conn(0x7f1d780fce80 msgr2=0x7f1d780fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:15.355 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.352+0000 7f1d7fe00640 1 -- 172.21.15.123:0/3465148505 shutdown_connections 2024-05-08T06:12:15.355 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:15.352+0000 7f1d7fe00640 1 -- 172.21.15.123:0/3465148505 wait complete. 2024-05-08T06:12:15.472 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-08T06:12:15.906 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:12:16.587 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:16 smithi123 ceph-mon[29787]: pgmap v514: 97 pgs: 97 active+clean; 9.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 275 op/s 2024-05-08T06:12:17.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:16 smithi163 ceph-mon[34520]: pgmap v514: 97 pgs: 97 active+clean; 9.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 275 op/s 2024-05-08T06:12:17.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:16 smithi192 ceph-mon[34664]: pgmap v514: 97 pgs: 97 active+clean; 9.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 275 op/s 2024-05-08T06:12:18.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.488+0000 7f0cccc00700 1 -- 172.21.15.123:0/3383192404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0cc8108850 msgr2=0x7f0cc81103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:18.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.488+0000 7f0cccc00700 1 --2- 172.21.15.123:0/3383192404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0cc8108850 0x7f0cc81103d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0cc000b0d0 tx=0x7f0cc000b3e0 comp rx=0 tx=0).stop 2024-05-08T06:12:18.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.488+0000 7f0cccc00700 1 -- 172.21.15.123:0/3383192404 shutdown_connections 2024-05-08T06:12:18.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.488+0000 7f0cccc00700 1 --2- 172.21.15.123:0/3383192404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0cc8108850 0x7f0cc81103d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:18.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.488+0000 7f0cccc00700 1 --2- 172.21.15.123:0/3383192404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cc8103750 0x7f0cc8103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:18.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.488+0000 7f0cccc00700 1 --2- 172.21.15.123:0/3383192404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0cc8104100 0x7f0cc8108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:18.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.488+0000 7f0cccc00700 1 -- 172.21.15.123:0/3383192404 >> 172.21.15.123:0/3383192404 conn(0x7f0cc807a600 msgr2=0x7f0cc807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:18.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.489+0000 7f0cccc00700 1 -- 172.21.15.123:0/3383192404 shutdown_connections 2024-05-08T06:12:18.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.489+0000 7f0cccc00700 1 -- 172.21.15.123:0/3383192404 wait complete. 2024-05-08T06:12:18.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.490+0000 7f0cccc00700 1 Processor -- start 2024-05-08T06:12:18.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.490+0000 7f0cccc00700 1 -- start start 2024-05-08T06:12:18.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.490+0000 7f0cccc00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0cc8103750 0x7f0cc81a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:18.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.490+0000 7f0cccc00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cc8104100 0x7f0cc81a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:18.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.490+0000 7f0cccc00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0cc8108850 0x7f0cc819bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:18.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.490+0000 7f0cccc00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc8113670 con 0x7f0cc8103750 2024-05-08T06:12:18.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.490+0000 7f0cccc00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc81134f0 con 0x7f0cc8104100 2024-05-08T06:12:18.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.490+0000 7f0cccc00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc81137f0 con 0x7f0cc8108850 2024-05-08T06:12:18.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.491+0000 7f0cc5e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0cc8103750 0x7f0cc81a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:18.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.491+0000 7f0cc5e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0cc8103750 0x7f0cc81a1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51284/0 (socket says 172.21.15.123:51284) 2024-05-08T06:12:18.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.491+0000 7f0cc5400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cc8104100 0x7f0cc81a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:18.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.491+0000 7f0cc5e00700 1 -- 172.21.15.123:0/2444113826 learned_addr learned my addr 172.21.15.123:0/2444113826 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:18.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.491+0000 7f0cc6800700 1 --2- 172.21.15.123:0/2444113826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0cc8108850 0x7f0cc819bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:18.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.491+0000 7f0cc6800700 1 -- 172.21.15.123:0/2444113826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cc8104100 msgr2=0x7f0cc81a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:18.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.491+0000 7f0cc6800700 1 --2- 172.21.15.123:0/2444113826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cc8104100 0x7f0cc81a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:18.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.491+0000 7f0cc6800700 1 -- 172.21.15.123:0/2444113826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0cc8103750 msgr2=0x7f0cc81a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:18.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.491+0000 7f0cc6800700 1 --2- 172.21.15.123:0/2444113826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0cc8103750 0x7f0cc81a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:18.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.491+0000 7f0cc6800700 1 -- 172.21.15.123:0/2444113826 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cc819c270 con 0x7f0cc8108850 2024-05-08T06:12:18.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.491+0000 7f0cc5400700 1 --2- 172.21.15.123:0/2444113826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cc8104100 0x7f0cc81a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:12:18.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.491+0000 7f0cc5e00700 1 --2- 172.21.15.123:0/2444113826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0cc8103750 0x7f0cc81a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:12:18.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.492+0000 7f0cc6800700 1 --2- 172.21.15.123:0/2444113826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0cc8108850 0x7f0cc819bb10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0cc000b770 tx=0x7f0cc0007980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:18.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.492+0000 7f0cbea00700 1 -- 172.21.15.123:0/2444113826 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc001f070 con 0x7f0cc8108850 2024-05-08T06:12:18.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.492+0000 7f0cccc00700 1 -- 172.21.15.123:0/2444113826 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cc819c500 con 0x7f0cc8108850 2024-05-08T06:12:18.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.492+0000 7f0cbea00700 1 -- 172.21.15.123:0/2444113826 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cc0007dc0 con 0x7f0cc8108850 2024-05-08T06:12:18.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.492+0000 7f0cbea00700 1 -- 172.21.15.123:0/2444113826 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc001b430 con 0x7f0cc8108850 2024-05-08T06:12:18.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.492+0000 7f0cccc00700 1 -- 172.21.15.123:0/2444113826 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cc81a87d0 con 0x7f0cc8108850 2024-05-08T06:12:18.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.496+0000 7f0cbea00700 1 -- 172.21.15.123:0/2444113826 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7f0cc0022070 con 0x7f0cc8108850 2024-05-08T06:12:18.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.497+0000 7f0cd7601b80 1 Processor -- start 2024-05-08T06:12:18.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.498+0000 7f0cd7601b80 1 -- start start 2024-05-08T06:12:18.498 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.498+0000 7f0cbea00700 1 --2- 172.21.15.123:0/2444113826 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f0cac0b2900 0x7f0cac0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:18.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.498+0000 7f0cd7601b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a12c718060 0x55a12c718440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:18.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.498+0000 7f0cd7601b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a12c718980 0x55a12c71c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:18.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.498+0000 7f0cbea00700 1 -- 172.21.15.123:0/2444113826 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f0cc00e2c60 con 0x7f0cc8108850 2024-05-08T06:12:18.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.498+0000 7f0cd7601b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a12c71c9d0 0x55a12c71ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:18.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.498+0000 7f0cd7601b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a12c66aa20 con 0x55a12c718060 2024-05-08T06:12:18.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.498+0000 7f0cd7601b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55a12c608760 con 0x55a12c718980 2024-05-08T06:12:18.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.498+0000 7f0cd7601b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a12c6669e0 con 0x55a12c71c9d0 2024-05-08T06:12:18.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.498+0000 7f0cc5400700 1 --2- 172.21.15.123:0/2444113826 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f0cac0b2900 0x7f0cac0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:18.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.498+0000 7f0cc6800700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a12c71c9d0 0x55a12c71ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:18.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.499+0000 7f0cc6800700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a12c71c9d0 0x55a12c71ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:34566/0 (socket says 172.21.15.123:34566) 2024-05-08T06:12:18.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.499+0000 7f0cc5400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a12c718980 0x55a12c71c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:18.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.499+0000 7f0cc6800700 1 -- 172.21.15.123:0/353723085 learned_addr learned my addr 172.21.15.123:0/353723085 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:18.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.499+0000 7f0cc5e00700 1 --2- 172.21.15.123:0/353723085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a12c718060 0x55a12c718440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:18.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.499+0000 7f0cc6800700 1 -- 172.21.15.123:0/353723085 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a12c718980 msgr2=0x55a12c71c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:18.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.499+0000 7f0cc6800700 1 --2- 172.21.15.123:0/353723085 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a12c718980 0x55a12c71c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:18.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.499+0000 7f0cc6800700 1 -- 172.21.15.123:0/353723085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a12c718060 msgr2=0x55a12c718440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:18.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.499+0000 7f0cc6800700 1 --2- 172.21.15.123:0/353723085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a12c718060 0x55a12c718440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:18.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.499+0000 7f0cc6800700 1 -- 172.21.15.123:0/353723085 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a12c5b78d0 con 0x55a12c71c9d0 2024-05-08T06:12:18.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.500+0000 7f0cc5400700 1 --2- 172.21.15.123:0/2444113826 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f0cac0b2900 0x7f0cac0b4dc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0cb40097c0 tx=0x7f0cb4009f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:18.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.500+0000 7f0cc6800700 1 --2- 172.21.15.123:0/353723085 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a12c71c9d0 0x55a12c71ce30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0cc00a7530 tx=0x7f0cc00099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3593954f821191bc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:18.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.500+0000 7f0cb3400700 1 -- 172.21.15.123:0/353723085 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0029070 con 0x55a12c71c9d0 2024-05-08T06:12:18.502 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.500+0000 7f0cb3400700 1 -- 172.21.15.123:0/353723085 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cc00aa9c0 con 0x55a12c71c9d0 2024-05-08T06:12:18.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.500+0000 7f0cb3400700 1 -- 172.21.15.123:0/353723085 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc00aa520 con 0x55a12c71c9d0 2024-05-08T06:12:18.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.501+0000 7f0cd7601b80 1 -- 172.21.15.123:0/353723085 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a12c71c9d0 msgr2=0x55a12c71ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:18.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.501+0000 7f0cd7601b80 1 --2- 172.21.15.123:0/353723085 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a12c71c9d0 0x55a12c71ce30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0cc00a7530 tx=0x7f0cc00099e0 comp rx=0 tx=0).stop 2024-05-08T06:12:18.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.501+0000 7f0cd7601b80 1 -- 172.21.15.123:0/353723085 shutdown_connections 2024-05-08T06:12:18.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.501+0000 7f0cd7601b80 1 --2- 172.21.15.123:0/353723085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a12c718060 0x55a12c718440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:18.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.501+0000 7f0cd7601b80 1 --2- 172.21.15.123:0/353723085 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a12c71c9d0 0x55a12c71ce30 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:18.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.501+0000 7f0cd7601b80 1 --2- 172.21.15.123:0/353723085 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a12c718980 0x55a12c71c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:18.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.501+0000 7f0cd7601b80 1 -- 172.21.15.123:0/353723085 >> 172.21.15.123:0/353723085 conn(0x55a12c646b60 msgr2=0x55a12c7127a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:18.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.501+0000 7f0cd7601b80 1 -- 172.21.15.123:0/353723085 shutdown_connections 2024-05-08T06:12:18.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.501+0000 7f0cd7601b80 1 -- 172.21.15.123:0/353723085 wait complete. 2024-05-08T06:12:18.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.501+0000 7f0cd7601b80 1 -- start start 2024-05-08T06:12:18.503 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.502+0000 7f0cd7601b80 1 Processor -- start 2024-05-08T06:12:18.505 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.502+0000 7f0cd7601b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a12c718060 0x55a12c723e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:18.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.502+0000 7f0cd7601b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a12c718980 0x55a12c7211c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:18.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.502+0000 7f0cd7601b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a12c71c9d0 0x55a12c721700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:18.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.502+0000 7f0cd7601b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a12c7249a0 con 0x55a12c718980 2024-05-08T06:12:18.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.502+0000 7f0cd7601b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55a12c5d0b20 con 0x55a12c718060 2024-05-08T06:12:18.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.502+0000 7f0cd7601b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a12c724b20 con 0x55a12c71c9d0 2024-05-08T06:12:18.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.502+0000 7f0cd7601b80 10 client.0 _ll_register_callbacks cb 0x55a12c611c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:12:18.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.503+0000 7f0cc6800700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a12c71c9d0 0x55a12c721700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:18.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.503+0000 7f0cc6800700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a12c71c9d0 0x55a12c721700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:34576/0 (socket says 172.21.15.123:34576) 2024-05-08T06:12:18.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.503+0000 7f0cc6800700 1 -- 172.21.15.123:0/1736880973 learned_addr learned my addr 172.21.15.123:0/1736880973 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:18.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.503+0000 7f0cc5400700 1 --2- 172.21.15.123:0/1736880973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a12c718980 0x55a12c7211c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:18.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.503+0000 7f0cc5e00700 1 --2- 172.21.15.123:0/1736880973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a12c718060 0x55a12c723e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:18.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.503+0000 7f0cc6800700 1 -- 172.21.15.123:0/1736880973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a12c718060 msgr2=0x55a12c723e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:18.506 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.503+0000 7f0cc6800700 1 --2- 172.21.15.123:0/1736880973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a12c718060 0x55a12c723e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:18.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.503+0000 7f0cc6800700 1 -- 172.21.15.123:0/1736880973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a12c718980 msgr2=0x55a12c7211c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:18.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.503+0000 7f0cc6800700 1 --2- 172.21.15.123:0/1736880973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a12c718980 0x55a12c7211c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:18.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.503+0000 7f0cc6800700 1 -- 172.21.15.123:0/1736880973 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a12c630420 con 0x55a12c71c9d0 2024-05-08T06:12:18.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.503+0000 7f0cc6800700 1 --2- 172.21.15.123:0/1736880973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a12c71c9d0 0x55a12c721700 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0cc00090a0 tx=0x7f0cc0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22115b0aba9b56c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:18.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.503+0000 7f0ca7e00700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-05-08T06:12:18.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.504+0000 7f0ca7e00700 1 -- 172.21.15.123:0/1736880973 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0029030 con 0x55a12c71c9d0 2024-05-08T06:12:18.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.504+0000 7f0ca7e00700 1 -- 172.21.15.123:0/1736880973 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f0cc00e0de0 con 0x55a12c71c9d0 2024-05-08T06:12:18.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.504+0000 7f0cd7601b80 1 -- 172.21.15.123:0/1736880973 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cc00aa9c0 con 0x55a12c71c9d0 2024-05-08T06:12:18.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.504+0000 7f0ca7e00700 1 -- 172.21.15.123:0/1736880973 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0cc00aa2c0 con 0x55a12c71c9d0 2024-05-08T06:12:18.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.504+0000 7f0cd7601b80 10 client.25498 fetch_fsmap learned FSMap version 24 2024-05-08T06:12:18.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.504+0000 7f0cd7601b80 1 -- 172.21.15.123:0/1736880973 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a12c7249a0 con 0x55a12c71c9d0 2024-05-08T06:12:18.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.504+0000 7f0ca7e00700 1 -- 172.21.15.123:0/1736880973 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cc00aa9c0 con 0x55a12c71c9d0 2024-05-08T06:12:18.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.504+0000 7f0ca7e00700 1 -- 172.21.15.123:0/1736880973 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f0cc0011070 con 0x55a12c71c9d0 2024-05-08T06:12:18.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.504+0000 7f0cd7601b80 10 client.25498 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:12:18.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.504+0000 7f0cd7601b80 10 client.25498 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-08T06:12:18.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.505+0000 7f0cd7601b80 20 client.25498 populate_metadata read hostname 'smithi123' 2024-05-08T06:12:18.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.505+0000 7f0cd7601b80 1 --2- 172.21.15.123:0/1736880973 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x55a12c731510 0x55a12c733930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:18.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.505+0000 7f0cd7601b80 4 client.25498 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:12:18.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.505+0000 7f0cd7601b80 1 -- 172.21.15.123:0/1736880973 --> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a12c6e50a0 con 0x55a12c731510 2024-05-08T06:12:18.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.505+0000 7f0cc5e00700 1 --2- 172.21.15.123:0/1736880973 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x55a12c731510 0x55a12c733930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:18.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.506+0000 7f0ca7e00700 10 client.25498 ms_handle_connect on v2:172.21.15.123:6840/1571726231 2024-05-08T06:12:18.508 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.506+0000 7f0cc5e00700 1 --2- 172.21.15.123:0/1736880973 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x55a12c731510 0x55a12c733930 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fbaebb432a598050 server_cookie=d45eac5d559515db in_seq=0 out_seq=0 2024-05-08T06:12:18.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.506+0000 7f0ca7e00700 1 -- 172.21.15.123:0/1736880973 <== mds.1 v2:172.21.15.123:6840/1571726231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a12c6e50a0 con 0x55a12c731510 2024-05-08T06:12:18.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.506+0000 7f0ca7e00700 10 client.25498 handle_command_reply: tid=0 2024-05-08T06:12:18.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.507+0000 7f0cd7601b80 1 client.25498 shutdown 2024-05-08T06:12:18.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.507+0000 7f0cd7601b80 1 -- 172.21.15.123:0/1736880973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a12c71c9d0 msgr2=0x55a12c721700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:18.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.507+0000 7f0cd7601b80 1 --2- 172.21.15.123:0/1736880973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a12c71c9d0 0x55a12c721700 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0cc00090a0 tx=0x7f0cc0009c00 comp rx=0 tx=0).stop 2024-05-08T06:12:18.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.507+0000 7f0cd7601b80 1 -- 172.21.15.123:0/1736880973 shutdown_connections 2024-05-08T06:12:18.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.507+0000 7f0cd7601b80 1 --2- 172.21.15.123:0/1736880973 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x55a12c731510 0x55a12c733930 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:18.509 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.507+0000 7f0cd7601b80 1 --2- 172.21.15.123:0/1736880973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a12c71c9d0 0x55a12c721700 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:18.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.507+0000 7f0cd7601b80 1 --2- 172.21.15.123:0/1736880973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a12c718060 0x55a12c723e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:18.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.507+0000 7f0cd7601b80 1 --2- 172.21.15.123:0/1736880973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a12c718980 0x55a12c7211c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:18.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.507+0000 7f0ca7e00700 0 client.25498 ms_handle_reset on v2:172.21.15.123:6840/1571726231 2024-05-08T06:12:18.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.507+0000 7f0cd7601b80 1 -- 172.21.15.123:0/1736880973 >> 172.21.15.123:0/1736880973 conn(0x55a12c646b60 msgr2=0x55a12c70d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:18.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.507+0000 7f0cd7601b80 1 -- 172.21.15.123:0/1736880973 shutdown_connections 2024-05-08T06:12:18.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.507+0000 7f0cd7601b80 1 -- 172.21.15.123:0/1736880973 wait complete. 2024-05-08T06:12:18.510 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.507+0000 7f0cd7601b80 20 client.25498 trim_cache size 0 max 16384 2024-05-08T06:12:18.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.524+0000 7f0cd7601b80 1 Processor -- start 2024-05-08T06:12:18.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.524+0000 7f0cd7601b80 1 -- start start 2024-05-08T06:12:18.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.525+0000 7f0cd7601b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a12c718060 0x55a12c72d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:18.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.525+0000 7f0cd7601b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a12c718980 0x55a12c70d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:18.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.525+0000 7f0cd7601b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a12c71c9d0 0x55a12c7391c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:18.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.525+0000 7f0cd7601b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc81ae050 con 0x55a12c718980 2024-05-08T06:12:18.526 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.525+0000 7f0cd7601b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc81ab8e0 con 0x55a12c718060 2024-05-08T06:12:18.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.525+0000 7f0cd7601b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc810e2a0 con 0x55a12c71c9d0 2024-05-08T06:12:18.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.525+0000 7f0cc6800700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a12c71c9d0 0x55a12c7391c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:18.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.525+0000 7f0cc6800700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a12c71c9d0 0x55a12c7391c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:34578/0 (socket says 172.21.15.123:34578) 2024-05-08T06:12:18.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.525+0000 7f0cc6800700 1 -- 172.21.15.123:0/4094025682 learned_addr learned my addr 172.21.15.123:0/4094025682 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:18.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.525+0000 7f0cc5e00700 1 --2- 172.21.15.123:0/4094025682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a12c718060 0x55a12c72d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:18.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.525+0000 7f0cc5400700 1 --2- 172.21.15.123:0/4094025682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a12c718980 0x55a12c70d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:18.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.525+0000 7f0cc6800700 1 -- 172.21.15.123:0/4094025682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a12c718060 msgr2=0x55a12c72d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:18.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.525+0000 7f0cc6800700 1 --2- 172.21.15.123:0/4094025682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a12c718060 0x55a12c72d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:18.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.525+0000 7f0cc6800700 1 -- 172.21.15.123:0/4094025682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a12c718980 msgr2=0x55a12c70d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:18.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.525+0000 7f0cc6800700 1 --2- 172.21.15.123:0/4094025682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a12c718980 0x55a12c70d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:18.527 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.525+0000 7f0cc6800700 1 -- 172.21.15.123:0/4094025682 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a12c70f3f0 con 0x55a12c71c9d0 2024-05-08T06:12:18.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.526+0000 7f0cc6800700 1 --2- 172.21.15.123:0/4094025682 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a12c71c9d0 0x55a12c7391c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0cc00a9f70 tx=0x7f0cc0006230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10d18af116abcf91 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:18.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.526+0000 7f0cb2a00700 1 -- 172.21.15.123:0/4094025682 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc00ae670 con 0x55a12c71c9d0 2024-05-08T06:12:18.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.526+0000 7f0cb2a00700 1 -- 172.21.15.123:0/4094025682 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cc00b2c00 con 0x55a12c71c9d0 2024-05-08T06:12:18.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.526+0000 7f0cb2a00700 1 -- 172.21.15.123:0/4094025682 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0005bd0 con 0x55a12c71c9d0 2024-05-08T06:12:18.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.527+0000 7f0cd7601b80 1 -- 172.21.15.123:0/4094025682 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a12c71c9d0 msgr2=0x55a12c7391c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:18.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.527+0000 7f0cd7601b80 1 --2- 172.21.15.123:0/4094025682 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a12c71c9d0 0x55a12c7391c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0cc00a9f70 tx=0x7f0cc0006230 comp rx=0 tx=0).stop 2024-05-08T06:12:18.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.527+0000 7f0cd7601b80 1 -- 172.21.15.123:0/4094025682 shutdown_connections 2024-05-08T06:12:18.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.527+0000 7f0cd7601b80 1 --2- 172.21.15.123:0/4094025682 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a12c71c9d0 0x55a12c7391c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:18.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.527+0000 7f0cd7601b80 1 --2- 172.21.15.123:0/4094025682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a12c718060 0x55a12c72d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:18.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.527+0000 7f0cd7601b80 1 --2- 172.21.15.123:0/4094025682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a12c718980 0x55a12c70d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:18.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.527+0000 7f0cd7601b80 1 -- 172.21.15.123:0/4094025682 >> 172.21.15.123:0/4094025682 conn(0x55a12c646b60 msgr2=0x55a12c70ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:18.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.527+0000 7f0cd7601b80 1 -- 172.21.15.123:0/4094025682 shutdown_connections 2024-05-08T06:12:18.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.527+0000 7f0cd7601b80 1 -- 172.21.15.123:0/4094025682 wait complete. 2024-05-08T06:12:18.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.527+0000 7f0cd7601b80 1 -- start start 2024-05-08T06:12:18.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.528+0000 7f0cd7601b80 1 Processor -- start 2024-05-08T06:12:18.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.528+0000 7f0cd7601b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a12c718060 0x55a12c729d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:18.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.528+0000 7f0cd7601b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a12c718980 0x55a12c72a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:18.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.528+0000 7f0cd7601b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a12c71c9d0 0x55a12c724d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:18.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.528+0000 7f0cd7601b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc81a9da0 con 0x55a12c718980 2024-05-08T06:12:18.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.528+0000 7f0cd7601b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc81ab210 con 0x55a12c71c9d0 2024-05-08T06:12:18.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.528+0000 7f0cd7601b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc81aa470 con 0x55a12c718060 2024-05-08T06:12:18.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.528+0000 7f0cd7601b80 10 client.0 _ll_register_callbacks cb 0x55a12c611c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:12:18.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.528+0000 7f0cc5e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a12c718060 0x55a12c729d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:18.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.528+0000 7f0cc5e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a12c718060 0x55a12c729d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:34580/0 (socket says 172.21.15.123:34580) 2024-05-08T06:12:18.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.528+0000 7f0cc5e00700 1 -- 172.21.15.123:0/404308503 learned_addr learned my addr 172.21.15.123:0/404308503 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:18.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.529+0000 7f0cc6800700 1 --2- 172.21.15.123:0/404308503 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a12c71c9d0 0x55a12c724d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:18.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.529+0000 7f0cc5400700 1 --2- 172.21.15.123:0/404308503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a12c718980 0x55a12c72a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:18.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.529+0000 7f0cc5e00700 1 -- 172.21.15.123:0/404308503 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a12c71c9d0 msgr2=0x55a12c724d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:18.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.529+0000 7f0cc5e00700 1 --2- 172.21.15.123:0/404308503 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a12c71c9d0 0x55a12c724d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:18.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.529+0000 7f0cc5e00700 1 -- 172.21.15.123:0/404308503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a12c718980 msgr2=0x55a12c72a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:18.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.529+0000 7f0cc5e00700 1 --2- 172.21.15.123:0/404308503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a12c718980 0x55a12c72a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:18.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.529+0000 7f0cc5e00700 1 -- 172.21.15.123:0/404308503 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a12c725270 con 0x55a12c718060 2024-05-08T06:12:18.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.529+0000 7f0cc5400700 1 --2- 172.21.15.123:0/404308503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a12c718980 0x55a12c72a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:12:18.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.529+0000 7f0cc5e00700 1 --2- 172.21.15.123:0/404308503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a12c718060 0x55a12c729d80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x55a12c734160 tx=0x7f0cb8004530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20bddd1c1f07e2fa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:18.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.529+0000 7f0ca7e00700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-05-08T06:12:18.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.529+0000 7f0ca7e00700 1 -- 172.21.15.123:0/404308503 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb8026070 con 0x55a12c718060 2024-05-08T06:12:18.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.529+0000 7f0ca7e00700 1 -- 172.21.15.123:0/404308503 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f0cb8013da0 con 0x55a12c718060 2024-05-08T06:12:18.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.529+0000 7f0cd7601b80 1 -- 172.21.15.123:0/404308503 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cc00b2c00 con 0x55a12c718060 2024-05-08T06:12:18.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.530+0000 7f0ca7e00700 1 -- 172.21.15.123:0/404308503 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0cb8004830 con 0x55a12c718060 2024-05-08T06:12:18.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.530+0000 7f0cd7601b80 10 client.25504 fetch_fsmap learned FSMap version 24 2024-05-08T06:12:18.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.530+0000 7f0cd7601b80 1 -- 172.21.15.123:0/404308503 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a12c79c550 con 0x55a12c718060 2024-05-08T06:12:18.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.530+0000 7f0ca7e00700 1 -- 172.21.15.123:0/404308503 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cc00b2c00 con 0x55a12c718060 2024-05-08T06:12:18.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.530+0000 7f0ca7e00700 1 -- 172.21.15.123:0/404308503 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f0cb8003680 con 0x55a12c718060 2024-05-08T06:12:18.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.530+0000 7f0cd7601b80 10 client.25504 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:12:18.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.530+0000 7f0cd7601b80 10 client.25504 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-08T06:12:18.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.530+0000 7f0cd7601b80 20 client.25504 populate_metadata read hostname 'smithi123' 2024-05-08T06:12:18.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.530+0000 7f0cd7601b80 1 --2- 172.21.15.123:0/404308503 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x55a12c79cbe0 0x55a12c79f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:18.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.530+0000 7f0cd7601b80 4 client.25504 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T06:12:18.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.530+0000 7f0cd7601b80 1 -- 172.21.15.123:0/404308503 --> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a12c678400 con 0x55a12c79cbe0 2024-05-08T06:12:18.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.531+0000 7f0cc5400700 1 --2- 172.21.15.123:0/404308503 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x55a12c79cbe0 0x55a12c79f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:18.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.531+0000 7f0cc5400700 1 --2- 172.21.15.123:0/404308503 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x55a12c79cbe0 0x55a12c79f000 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a898fbef05296ac server_cookie=9f4d3bca11ee56fc in_seq=0 out_seq=0 2024-05-08T06:12:18.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.531+0000 7f0ca7e00700 10 client.25504 ms_handle_connect on v2:172.21.15.123:6840/1571726231 2024-05-08T06:12:18.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.532+0000 7f0ca7e00700 1 -- 172.21.15.123:0/404308503 <== mds.1 v2:172.21.15.123:6840/1571726231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a12c678400 con 0x55a12c79cbe0 2024-05-08T06:12:18.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.532+0000 7f0ca7e00700 10 client.25504 handle_command_reply: tid=0 2024-05-08T06:12:18.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.532+0000 7f0cd7601b80 1 client.25504 shutdown 2024-05-08T06:12:18.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.532+0000 7f0cd7601b80 1 -- 172.21.15.123:0/404308503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a12c718060 msgr2=0x55a12c729d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:18.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.532+0000 7f0cd7601b80 1 --2- 172.21.15.123:0/404308503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a12c718060 0x55a12c729d80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x55a12c734160 tx=0x7f0cb8004530 comp rx=0 tx=0).stop 2024-05-08T06:12:18.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.532+0000 7f0cd7601b80 1 -- 172.21.15.123:0/404308503 shutdown_connections 2024-05-08T06:12:18.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.532+0000 7f0cd7601b80 1 --2- 172.21.15.123:0/404308503 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x55a12c79cbe0 0x55a12c79f000 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:18.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.532+0000 7f0cd7601b80 1 --2- 172.21.15.123:0/404308503 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a12c71c9d0 0x55a12c724d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:18.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.532+0000 7f0cd7601b80 1 --2- 172.21.15.123:0/404308503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a12c718060 0x55a12c729d80 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:18.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.532+0000 7f0cd7601b80 1 --2- 172.21.15.123:0/404308503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a12c718980 0x55a12c72a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:18.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.532+0000 7f0ca7e00700 0 client.25504 ms_handle_reset on v2:172.21.15.123:6840/1571726231 2024-05-08T06:12:18.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.532+0000 7f0cd7601b80 1 -- 172.21.15.123:0/404308503 >> 172.21.15.123:0/404308503 conn(0x55a12c646b60 msgr2=0x55a12c70d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:18.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.532+0000 7f0cd7601b80 1 -- 172.21.15.123:0/404308503 shutdown_connections 2024-05-08T06:12:18.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.532+0000 7f0cd7601b80 1 -- 172.21.15.123:0/404308503 wait complete. 2024-05-08T06:12:18.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.533+0000 7f0cd7601b80 20 client.25504 trim_cache size 0 max 16384 2024-05-08T06:12:18.538 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-05-08T06:12:18.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.533+0000 7f0cccc00700 1 -- 172.21.15.123:0/2444113826 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f0cac0b2900 msgr2=0x7f0cac0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:18.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.533+0000 7f0cccc00700 1 --2- 172.21.15.123:0/2444113826 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f0cac0b2900 0x7f0cac0b4dc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0cb40097c0 tx=0x7f0cb4009f70 comp rx=0 tx=0).stop 2024-05-08T06:12:18.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.533+0000 7f0cccc00700 1 -- 172.21.15.123:0/2444113826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0cc8108850 msgr2=0x7f0cc819bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:18.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.533+0000 7f0cccc00700 1 --2- 172.21.15.123:0/2444113826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0cc8108850 0x7f0cc819bb10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0cc000b770 tx=0x7f0cc0007980 comp rx=0 tx=0).stop 2024-05-08T06:12:18.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.534+0000 7f0cccc00700 1 -- 172.21.15.123:0/2444113826 shutdown_connections 2024-05-08T06:12:18.540 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.534+0000 7f0cccc00700 1 --2- 172.21.15.123:0/2444113826 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f0cac0b2900 0x7f0cac0b4dc0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:18.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.534+0000 7f0cccc00700 1 --2- 172.21.15.123:0/2444113826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0cc8103750 0x7f0cc81a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:18.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.534+0000 7f0cccc00700 1 --2- 172.21.15.123:0/2444113826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0cc8108850 0x7f0cc819bb10 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:18.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.534+0000 7f0cccc00700 1 --2- 172.21.15.123:0/2444113826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cc8104100 0x7f0cc81a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:18.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.534+0000 7f0cccc00700 1 -- 172.21.15.123:0/2444113826 >> 172.21.15.123:0/2444113826 conn(0x7f0cc807a600 msgr2=0x7f0cc81017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:18.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.534+0000 7f0cccc00700 1 -- 172.21.15.123:0/2444113826 shutdown_connections 2024-05-08T06:12:18.541 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:18.534+0000 7f0cccc00700 1 -- 172.21.15.123:0/2444113826 wait complete. 2024-05-08T06:12:18.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:18 smithi123 ceph-mon[29787]: pgmap v515: 97 pgs: 97 active+clean; 9.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 275 op/s 2024-05-08T06:12:19.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:18 smithi163 ceph-mon[34520]: pgmap v515: 97 pgs: 97 active+clean; 9.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 275 op/s 2024-05-08T06:12:19.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:18 smithi192 ceph-mon[34664]: pgmap v515: 97 pgs: 97 active+clean; 9.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 275 op/s 2024-05-08T06:12:19.633 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-08T06:12:20.074 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:12:20.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:20 smithi123 ceph-mon[29787]: pgmap v516: 97 pgs: 97 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 277 op/s 2024-05-08T06:12:21.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:20 smithi163 ceph-mon[34520]: pgmap v516: 97 pgs: 97 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 277 op/s 2024-05-08T06:12:21.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:20 smithi192 ceph-mon[34664]: pgmap v516: 97 pgs: 97 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 277 op/s 2024-05-08T06:12:22.000 INFO:tasks.workunit.client.0.smithi123.stdout:.......... 2024-05-08T06:12:22.000 INFO:tasks.workunit.client.0.smithi123.stdout:do_o_direct_reads: SUCCESS 2024-05-08T06:12:22.005 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-08T06:12:22.005 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-08T06:12:22.096 INFO:tasks.workunit:Running workunit direct_io/misc.sh... 2024-05-08T06:12:22.096 DEBUG:teuthology.orchestra.run.smithi123:workunit test direct_io/misc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/misc.sh 2024-05-08T06:12:22.163 INFO:tasks.workunit.client.0.smithi123.stderr:+ echo 'test read from hole' 2024-05-08T06:12:22.164 INFO:tasks.workunit.client.0.smithi123.stdout:test read from hole 2024-05-08T06:12:22.164 INFO:tasks.workunit.client.0.smithi123.stderr:+ dd if=/dev/zero of=dd3 bs=1 seek=1048576 count=0 2024-05-08T06:12:22.169 INFO:tasks.workunit.client.0.smithi123.stderr:0+0 records in 2024-05-08T06:12:22.169 INFO:tasks.workunit.client.0.smithi123.stderr:0+0 records out 2024-05-08T06:12:22.169 INFO:tasks.workunit.client.0.smithi123.stderr:0 bytes copied, 0.000108986 s, 0.0 kB/s 2024-05-08T06:12:22.169 INFO:tasks.workunit.client.0.smithi123.stderr:+ dd if=dd3 of=/tmp/ddout1 skip=8 bs=512 count=2 iflag=direct 2024-05-08T06:12:22.175 INFO:tasks.workunit.client.0.smithi123.stderr:2+0 records in 2024-05-08T06:12:22.175 INFO:tasks.workunit.client.0.smithi123.stderr:2+0 records out 2024-05-08T06:12:22.175 INFO:tasks.workunit.client.0.smithi123.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00297726 s, 344 kB/s 2024-05-08T06:12:22.175 INFO:tasks.workunit.client.0.smithi123.stderr:+ dd if=/dev/zero of=/tmp/dd3 bs=512 count=2 2024-05-08T06:12:22.178 INFO:tasks.workunit.client.0.smithi123.stderr:2+0 records in 2024-05-08T06:12:22.178 INFO:tasks.workunit.client.0.smithi123.stderr:2+0 records out 2024-05-08T06:12:22.178 INFO:tasks.workunit.client.0.smithi123.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 0.000129263 s, 7.9 MB/s 2024-05-08T06:12:22.178 INFO:tasks.workunit.client.0.smithi123.stderr:+ cmp /tmp/dd3 /tmp/ddout1 2024-05-08T06:12:22.181 INFO:tasks.workunit.client.0.smithi123.stdout:other thing 2024-05-08T06:12:22.182 INFO:tasks.workunit.client.0.smithi123.stderr:+ echo 'other thing' 2024-05-08T06:12:22.182 INFO:tasks.workunit.client.0.smithi123.stderr:+ dd if=/dev/urandom of=/tmp/dd10 bs=500 count=1 2024-05-08T06:12:22.184 INFO:tasks.workunit.client.0.smithi123.stderr:1+0 records in 2024-05-08T06:12:22.185 INFO:tasks.workunit.client.0.smithi123.stderr:1+0 records out 2024-05-08T06:12:22.185 INFO:tasks.workunit.client.0.smithi123.stderr:500 bytes copied, 0.000133663 s, 3.7 MB/s 2024-05-08T06:12:22.185 INFO:tasks.workunit.client.0.smithi123.stderr:+ dd if=/tmp/dd10 of=dd10 bs=512 seek=8388 count=1 2024-05-08T06:12:22.201 INFO:tasks.workunit.client.0.smithi123.stderr:0+1 records in 2024-05-08T06:12:22.201 INFO:tasks.workunit.client.0.smithi123.stderr:0+1 records out 2024-05-08T06:12:22.201 INFO:tasks.workunit.client.0.smithi123.stderr:500 bytes copied, 0.0121389 s, 41.2 kB/s 2024-05-08T06:12:22.201 INFO:tasks.workunit.client.0.smithi123.stderr:+ dd if=dd10 of=/tmp/dd10out bs=512 skip=8388 count=1 iflag=direct 2024-05-08T06:12:22.208 INFO:tasks.workunit.client.0.smithi123.stderr:0+1 records in 2024-05-08T06:12:22.208 INFO:tasks.workunit.client.0.smithi123.stderr:0+1 records out 2024-05-08T06:12:22.208 INFO:tasks.workunit.client.0.smithi123.stderr:500 bytes copied, 0.00401187 s, 125 kB/s 2024-05-08T06:12:22.208 INFO:tasks.workunit.client.0.smithi123.stderr:+ cmp /tmp/dd10 /tmp/dd10out 2024-05-08T06:12:22.211 INFO:tasks.workunit.client.0.smithi123.stdout:OK 2024-05-08T06:12:22.211 INFO:tasks.workunit.client.0.smithi123.stderr:+ echo OK 2024-05-08T06:12:22.212 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-08T06:12:22.212 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-08T06:12:22.298 INFO:tasks.workunit:Running workunit direct_io/test_short_dio_read... 2024-05-08T06:12:22.299 DEBUG:teuthology.orchestra.run.smithi123:workunit test direct_io/test_short_dio_read> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/test_short_dio_read 2024-05-08T06:12:22.380 INFO:tasks.workunit.client.0.smithi123.stdout:writing first 3 bytes of 10k file 2024-05-08T06:12:22.380 INFO:tasks.workunit.client.0.smithi123.stdout:reading O_DIRECT 2024-05-08T06:12:22.380 INFO:tasks.workunit.client.0.smithi123.stdout:got 10000 2024-05-08T06:12:22.381 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-08T06:12:22.381 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-08T06:12:22.477 INFO:tasks.workunit:Running workunit direct_io/test_sync_io... 2024-05-08T06:12:22.477 DEBUG:teuthology.orchestra.run.smithi123:workunit test direct_io/test_sync_io> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/test_sync_io 2024-05-08T06:12:22.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.561+0000 7f107f800700 1 -- 172.21.15.123:0/2218688406 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1078108850 msgr2=0x7f10781103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:22.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.561+0000 7f107f800700 1 --2- 172.21.15.123:0/2218688406 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1078108850 0x7f10781103d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f107000b0d0 tx=0x7f107000b3e0 comp rx=0 tx=0).stop 2024-05-08T06:12:22.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.561+0000 7f107f800700 1 -- 172.21.15.123:0/2218688406 shutdown_connections 2024-05-08T06:12:22.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.561+0000 7f107f800700 1 --2- 172.21.15.123:0/2218688406 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1078108850 0x7f10781103d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:22.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.561+0000 7f107f800700 1 --2- 172.21.15.123:0/2218688406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1078103750 0x7f1078103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:22.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.561+0000 7f107f800700 1 --2- 172.21.15.123:0/2218688406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1078104100 0x7f1078108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:22.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.561+0000 7f107f800700 1 -- 172.21.15.123:0/2218688406 >> 172.21.15.123:0/2218688406 conn(0x7f107807a600 msgr2=0x7f107807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:22.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.561+0000 7f107f800700 1 -- 172.21.15.123:0/2218688406 shutdown_connections 2024-05-08T06:12:22.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.561+0000 7f107f800700 1 -- 172.21.15.123:0/2218688406 wait complete. 2024-05-08T06:12:22.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.563+0000 7f107f800700 1 Processor -- start 2024-05-08T06:12:22.563 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.563+0000 7f107f800700 1 -- start start 2024-05-08T06:12:22.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.563+0000 7f107f800700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1078103750 0x7f10781a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:22.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.563+0000 7f107f800700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1078104100 0x7f10781a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:22.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.563+0000 7f107f800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1078108850 0x7f107819baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:22.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.563+0000 7f107f800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1078113690 con 0x7f1078108850 2024-05-08T06:12:22.564 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.563+0000 7f107f800700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1078113510 con 0x7f1078103750 2024-05-08T06:12:22.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.563+0000 7f107f800700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1078113810 con 0x7f1078104100 2024-05-08T06:12:22.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.564+0000 7f107ce00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1078103750 0x7f10781a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:22.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.564+0000 7f1077e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1078104100 0x7f10781a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:22.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.564+0000 7f107ce00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1078103750 0x7f10781a1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:34414/0 (socket says 172.21.15.123:34414) 2024-05-08T06:12:22.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.564+0000 7f107ce00700 1 -- 172.21.15.123:0/1142321413 learned_addr learned my addr 172.21.15.123:0/1142321413 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:22.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.564+0000 7f1077e00700 1 --2- 172.21.15.123:0/1142321413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1078104100 0x7f10781a1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:34612/0 (socket says 172.21.15.123:34612) 2024-05-08T06:12:22.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.564+0000 7f107d800700 1 --2- 172.21.15.123:0/1142321413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1078108850 0x7f107819baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:22.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.564+0000 7f107ce00700 1 -- 172.21.15.123:0/1142321413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1078104100 msgr2=0x7f10781a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:22.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.564+0000 7f107ce00700 1 --2- 172.21.15.123:0/1142321413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1078104100 0x7f10781a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:22.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.564+0000 7f107ce00700 1 -- 172.21.15.123:0/1142321413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1078108850 msgr2=0x7f107819baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:22.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.564+0000 7f107ce00700 1 --2- 172.21.15.123:0/1142321413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1078108850 0x7f107819baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:22.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.564+0000 7f107ce00700 1 -- 172.21.15.123:0/1142321413 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f107819c250 con 0x7f1078103750 2024-05-08T06:12:22.565 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.565+0000 7f107ce00700 1 --2- 172.21.15.123:0/1142321413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1078103750 0x7f10781a1930 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f106c00e9c0 tx=0x7f106c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:22.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.565+0000 7f1075600700 1 -- 172.21.15.123:0/1142321413 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c00cbd0 con 0x7f1078103750 2024-05-08T06:12:22.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.565+0000 7f1075600700 1 -- 172.21.15.123:0/1142321413 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f106c00ee60 con 0x7f1078103750 2024-05-08T06:12:22.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.565+0000 7f107f800700 1 -- 172.21.15.123:0/1142321413 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f107819c540 con 0x7f1078103750 2024-05-08T06:12:22.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.565+0000 7f1075600700 1 -- 172.21.15.123:0/1142321413 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c0055c0 con 0x7f1078103750 2024-05-08T06:12:22.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.565+0000 7f107f800700 1 -- 172.21.15.123:0/1142321413 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10781a8800 con 0x7f1078103750 2024-05-08T06:12:22.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.570+0000 7f108a0e1b80 1 Processor -- start 2024-05-08T06:12:22.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.570+0000 7f108a0e1b80 1 -- start start 2024-05-08T06:12:22.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.570+0000 7f108a0e1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ec45e5bc0 0x556ec45e5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:22.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.570+0000 7f108a0e1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec45e64e0 0x556ec45e02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:22.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.570+0000 7f108a0e1b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec45e08f0 0x556ec45e0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:22.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.570+0000 7f108a0e1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556ec4532a20 con 0x556ec45e5bc0 2024-05-08T06:12:22.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.570+0000 7f108a0e1b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x556ec44d0760 con 0x556ec45e08f0 2024-05-08T06:12:22.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.570+0000 7f108a0e1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x556ec452e9e0 con 0x556ec45e64e0 2024-05-08T06:12:22.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.570+0000 7f107d800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ec45e5bc0 0x556ec45e5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:22.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.570+0000 7f1075600700 1 -- 172.21.15.123:0/1142321413 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7f106c004750 con 0x7f1078103750 2024-05-08T06:12:22.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.571+0000 7f1077e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec45e08f0 0x556ec45e0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:22.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.571+0000 7f107d800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ec45e5bc0 0x556ec45e5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51346/0 (socket says 172.21.15.123:51346) 2024-05-08T06:12:22.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.571+0000 7f107d800700 1 -- 172.21.15.123:0/2624620198 learned_addr learned my addr 172.21.15.123:0/2624620198 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:22.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.571+0000 7f107ce00700 1 --2- 172.21.15.123:0/2624620198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec45e64e0 0x556ec45e02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:22.571 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.571+0000 7f107d800700 1 -- 172.21.15.123:0/2624620198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec45e64e0 msgr2=0x556ec45e02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:22.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.571+0000 7f107d800700 1 --2- 172.21.15.123:0/2624620198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec45e64e0 0x556ec45e02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:22.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.571+0000 7f107d800700 1 -- 172.21.15.123:0/2624620198 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec45e08f0 msgr2=0x556ec45e0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:22.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.571+0000 7f107d800700 1 --2- 172.21.15.123:0/2624620198 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec45e08f0 0x556ec45e0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:22.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.571+0000 7f107d800700 1 -- 172.21.15.123:0/2624620198 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ec447f8d0 con 0x556ec45e5bc0 2024-05-08T06:12:22.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.571+0000 7f1077e00700 1 --2- 172.21.15.123:0/2624620198 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec45e08f0 0x556ec45e0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:12:22.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.571+0000 7f107ce00700 1 --2- 172.21.15.123:0/2624620198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec45e64e0 0x556ec45e02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:12:22.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.572+0000 7f107d800700 1 --2- 172.21.15.123:0/2624620198 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ec45e5bc0 0x556ec45e5fa0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1070009bb0 tx=0x7f1070009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df9993b638fe8c1a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:22.572 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.572+0000 7f106a000700 1 -- 172.21.15.123:0/2624620198 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107000baa0 con 0x556ec45e5bc0 2024-05-08T06:12:22.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.572+0000 7f106a000700 1 -- 172.21.15.123:0/2624620198 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1070020070 con 0x556ec45e5bc0 2024-05-08T06:12:22.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.572+0000 7f106a000700 1 -- 172.21.15.123:0/2624620198 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10700054d0 con 0x556ec45e5bc0 2024-05-08T06:12:22.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.573+0000 7f1075600700 1 --2- 172.21.15.123:0/1142321413 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f10600b2880 0x7f10600b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:22.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.573+0000 7f1075600700 1 -- 172.21.15.123:0/1142321413 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f106c0e24e0 con 0x7f1078103750 2024-05-08T06:12:22.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.573+0000 7f107ce00700 1 --2- 172.21.15.123:0/1142321413 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f10600b2880 0x7f10600b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:22.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.573+0000 7f108a0e1b80 1 -- 172.21.15.123:0/2624620198 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ec45e5bc0 msgr2=0x556ec45e5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:22.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.573+0000 7f108a0e1b80 1 --2- 172.21.15.123:0/2624620198 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ec45e5bc0 0x556ec45e5fa0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1070009bb0 tx=0x7f1070009be0 comp rx=0 tx=0).stop 2024-05-08T06:12:22.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.573+0000 7f108a0e1b80 1 -- 172.21.15.123:0/2624620198 shutdown_connections 2024-05-08T06:12:22.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.573+0000 7f108a0e1b80 1 --2- 172.21.15.123:0/2624620198 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ec45e5bc0 0x556ec45e5fa0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:22.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.573+0000 7f108a0e1b80 1 --2- 172.21.15.123:0/2624620198 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec45e08f0 0x556ec45e0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:22.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.573+0000 7f108a0e1b80 1 --2- 172.21.15.123:0/2624620198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec45e64e0 0x556ec45e02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:22.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.573+0000 7f108a0e1b80 1 -- 172.21.15.123:0/2624620198 >> 172.21.15.123:0/2624620198 conn(0x556ec45586d0 msgr2=0x556ec45dab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:22.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.574+0000 7f108a0e1b80 1 -- 172.21.15.123:0/2624620198 shutdown_connections 2024-05-08T06:12:22.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.574+0000 7f108a0e1b80 1 -- 172.21.15.123:0/2624620198 wait complete. 2024-05-08T06:12:22.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.574+0000 7f107ce00700 1 --2- 172.21.15.123:0/1142321413 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f10600b2880 0x7f10600b4d40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f106c0a6d20 tx=0x7f106c005020 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:22.575 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.574+0000 7f108a0e1b80 1 -- start start 2024-05-08T06:12:22.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.575+0000 7f108a0e1b80 1 Processor -- start 2024-05-08T06:12:22.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.575+0000 7f108a0e1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec45e08f0 0x556ec45ec1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:22.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.575+0000 7f108a0e1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ec45e5bc0 0x556ec45e9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:22.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.575+0000 7f108a0e1b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec45e64e0 0x556ec45e9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:22.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.575+0000 7f108a0e1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556ec45ecec0 con 0x556ec45e5bc0 2024-05-08T06:12:22.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.575+0000 7f108a0e1b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x556ec4498b20 con 0x556ec45e64e0 2024-05-08T06:12:22.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.575+0000 7f108a0e1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x556ec45ed040 con 0x556ec45e08f0 2024-05-08T06:12:22.576 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.575+0000 7f108a0e1b80 10 client.0 _ll_register_callbacks cb 0x556ec44d9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:12:22.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.575+0000 7f1077e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ec45e5bc0 0x556ec45e9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:22.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.575+0000 7f107d800700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec45e08f0 0x556ec45ec1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:22.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.576+0000 7f1077e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ec45e5bc0 0x556ec45e9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51358/0 (socket says 172.21.15.123:51358) 2024-05-08T06:12:22.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.576+0000 7f1077e00700 1 -- 172.21.15.123:0/1703660385 learned_addr learned my addr 172.21.15.123:0/1703660385 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:22.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.576+0000 7f107d800700 1 --2- 172.21.15.123:0/1703660385 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec45e64e0 0x556ec45e9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:22.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.576+0000 7f107d800700 1 -- 172.21.15.123:0/1703660385 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec45e64e0 msgr2=0x556ec45e9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:22.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.576+0000 7f107d800700 1 --2- 172.21.15.123:0/1703660385 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec45e64e0 0x556ec45e9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:22.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.576+0000 7f107d800700 1 -- 172.21.15.123:0/1703660385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ec45e5bc0 msgr2=0x556ec45e9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:22.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.576+0000 7f107d800700 1 --2- 172.21.15.123:0/1703660385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ec45e5bc0 0x556ec45e9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:22.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.576+0000 7f107d800700 1 -- 172.21.15.123:0/1703660385 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ec44f8420 con 0x556ec45e08f0 2024-05-08T06:12:22.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.576+0000 7f1077e00700 1 --2- 172.21.15.123:0/1703660385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ec45e5bc0 0x556ec45e9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:12:22.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.576+0000 7f107d800700 1 --2- 172.21.15.123:0/1703660385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec45e08f0 0x556ec45ec1b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1070009010 tx=0x7f107001e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e444ed6b4172fd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:22.577 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.576+0000 7f105ea00700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-05-08T06:12:22.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.577+0000 7f105ea00700 1 -- 172.21.15.123:0/1703660385 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10700054e0 con 0x556ec45e08f0 2024-05-08T06:12:22.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.577+0000 7f105ea00700 1 -- 172.21.15.123:0/1703660385 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f1070030070 con 0x556ec45e08f0 2024-05-08T06:12:22.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.577+0000 7f108a0e1b80 1 -- 172.21.15.123:0/1703660385 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1070020070 con 0x556ec45e08f0 2024-05-08T06:12:22.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.577+0000 7f105ea00700 1 -- 172.21.15.123:0/1703660385 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f107000bcb0 con 0x556ec45e08f0 2024-05-08T06:12:22.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.577+0000 7f108a0e1b80 10 client.25513 fetch_fsmap learned FSMap version 24 2024-05-08T06:12:22.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.577+0000 7f108a0e1b80 1 -- 172.21.15.123:0/1703660385 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556ec45ecec0 con 0x556ec45e08f0 2024-05-08T06:12:22.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.577+0000 7f105ea00700 1 -- 172.21.15.123:0/1703660385 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1070020070 con 0x556ec45e08f0 2024-05-08T06:12:22.579 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.578+0000 7f105ea00700 1 -- 172.21.15.123:0/1703660385 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f1070011070 con 0x556ec45e08f0 2024-05-08T06:12:22.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.578+0000 7f108a0e1b80 10 client.25513 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:12:22.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.578+0000 7f108a0e1b80 10 client.25513 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T06:12:22.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.578+0000 7f108a0e1b80 20 client.25513 populate_metadata read hostname 'smithi123' 2024-05-08T06:12:22.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.578+0000 7f108a0e1b80 1 --2- 172.21.15.123:0/1703660385 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x556ec45f99c0 0x556ec45fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:22.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.578+0000 7f108a0e1b80 4 client.25513 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:12:22.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.578+0000 7f108a0e1b80 1 -- 172.21.15.123:0/1703660385 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ec45ad140 con 0x556ec45f99c0 2024-05-08T06:12:22.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.578+0000 7f1077e00700 1 --2- 172.21.15.123:0/1703660385 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x556ec45f99c0 0x556ec45fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:22.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.579+0000 7f105ea00700 10 client.25513 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:22.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.579+0000 7f1077e00700 1 --2- 172.21.15.123:0/1703660385 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x556ec45f99c0 0x556ec45fbdb0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddf48c4acc7e62d7 server_cookie=c1b816e432f0a11a in_seq=0 out_seq=0 2024-05-08T06:12:22.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.580+0000 7f105ea00700 1 -- 172.21.15.123:0/1703660385 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556ec45ad140 con 0x556ec45f99c0 2024-05-08T06:12:22.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.580+0000 7f105ea00700 10 client.25513 handle_command_reply: tid=0 2024-05-08T06:12:22.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.580+0000 7f108a0e1b80 1 client.25513 shutdown 2024-05-08T06:12:22.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.580+0000 7f108a0e1b80 1 -- 172.21.15.123:0/1703660385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec45e08f0 msgr2=0x556ec45ec1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:22.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.580+0000 7f108a0e1b80 1 --2- 172.21.15.123:0/1703660385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec45e08f0 0x556ec45ec1b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1070009010 tx=0x7f107001e040 comp rx=0 tx=0).stop 2024-05-08T06:12:22.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.580+0000 7f108a0e1b80 1 -- 172.21.15.123:0/1703660385 shutdown_connections 2024-05-08T06:12:22.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.580+0000 7f108a0e1b80 1 --2- 172.21.15.123:0/1703660385 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x556ec45f99c0 0x556ec45fbdb0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:22.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.580+0000 7f108a0e1b80 1 --2- 172.21.15.123:0/1703660385 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec45e64e0 0x556ec45e9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:22.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.580+0000 7f108a0e1b80 1 --2- 172.21.15.123:0/1703660385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec45e08f0 0x556ec45ec1b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:22.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.580+0000 7f105ea00700 0 client.25513 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:22.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.580+0000 7f108a0e1b80 1 --2- 172.21.15.123:0/1703660385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ec45e5bc0 0x556ec45e9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:22.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.580+0000 7f108a0e1b80 1 -- 172.21.15.123:0/1703660385 >> 172.21.15.123:0/1703660385 conn(0x556ec45586d0 msgr2=0x556ec45da190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:22.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.581+0000 7f108a0e1b80 1 -- 172.21.15.123:0/1703660385 shutdown_connections 2024-05-08T06:12:22.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.581+0000 7f108a0e1b80 1 -- 172.21.15.123:0/1703660385 wait complete. 2024-05-08T06:12:22.582 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.581+0000 7f108a0e1b80 20 client.25513 trim_cache size 0 max 16384 2024-05-08T06:12:22.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.597+0000 7f108a0e1b80 1 Processor -- start 2024-05-08T06:12:22.597 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.597+0000 7f108a0e1b80 1 -- start start 2024-05-08T06:12:22.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.598+0000 7f108a0e1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec45e08f0 0x556ec45e02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:22.598 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.598+0000 7f108a0e1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ec45e5bc0 0x556ec45f5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:22.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.598+0000 7f108a0e1b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec45e64e0 0x556ec45da910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:22.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.598+0000 7f108a0e1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f10781b23d0 con 0x556ec45e5bc0 2024-05-08T06:12:22.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.598+0000 7f108a0e1b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f10781afc60 con 0x556ec45e64e0 2024-05-08T06:12:22.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.598+0000 7f108a0e1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f10781adab0 con 0x556ec45e08f0 2024-05-08T06:12:22.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.598+0000 7f1077e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ec45e5bc0 0x556ec45f5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:22.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.598+0000 7f107d800700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec45e08f0 0x556ec45e02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:22.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.598+0000 7f107d800700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec45e08f0 0x556ec45e02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:34630/0 (socket says 172.21.15.123:34630) 2024-05-08T06:12:22.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.598+0000 7f1077e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ec45e5bc0 0x556ec45f5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51360/0 (socket says 172.21.15.123:51360) 2024-05-08T06:12:22.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.598+0000 7f107d800700 1 -- 172.21.15.123:0/1685258237 learned_addr learned my addr 172.21.15.123:0/1685258237 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:22.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.598+0000 7f107d800700 1 --2- 172.21.15.123:0/1685258237 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec45e64e0 0x556ec45da910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:22.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.599+0000 7f1077e00700 1 -- 172.21.15.123:0/1685258237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec45e08f0 msgr2=0x556ec45e02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:22.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.599+0000 7f1077e00700 1 --2- 172.21.15.123:0/1685258237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec45e08f0 0x556ec45e02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:22.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.599+0000 7f1077e00700 1 -- 172.21.15.123:0/1685258237 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec45e64e0 msgr2=0x556ec45da910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:22.599 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.599+0000 7f1077e00700 1 --2- 172.21.15.123:0/1685258237 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec45e64e0 0x556ec45da910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:22.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.599+0000 7f1077e00700 1 -- 172.21.15.123:0/1685258237 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ec45dc3b0 con 0x556ec45e5bc0 2024-05-08T06:12:22.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.599+0000 7f107d800700 1 --2- 172.21.15.123:0/1685258237 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec45e64e0 0x556ec45da910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:12:22.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.599+0000 7f1077e00700 1 --2- 172.21.15.123:0/1685258237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ec45e5bc0 0x556ec45f5f80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x556ec45fc320 tx=0x7f1064009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61032004a44b3a8e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:22.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.599+0000 7f1069600700 1 -- 172.21.15.123:0/1685258237 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1064012310 con 0x556ec45e5bc0 2024-05-08T06:12:22.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.599+0000 7f1069600700 1 -- 172.21.15.123:0/1685258237 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1064004830 con 0x556ec45e5bc0 2024-05-08T06:12:22.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.599+0000 7f1069600700 1 -- 172.21.15.123:0/1685258237 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10640053a0 con 0x556ec45e5bc0 2024-05-08T06:12:22.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.600+0000 7f108a0e1b80 1 -- 172.21.15.123:0/1685258237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ec45e5bc0 msgr2=0x556ec45f5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:22.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.600+0000 7f108a0e1b80 1 --2- 172.21.15.123:0/1685258237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ec45e5bc0 0x556ec45f5f80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x556ec45fc320 tx=0x7f1064009260 comp rx=0 tx=0).stop 2024-05-08T06:12:22.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.600+0000 7f108a0e1b80 1 -- 172.21.15.123:0/1685258237 shutdown_connections 2024-05-08T06:12:22.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.600+0000 7f108a0e1b80 1 --2- 172.21.15.123:0/1685258237 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec45e64e0 0x556ec45da910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:22.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.600+0000 7f108a0e1b80 1 --2- 172.21.15.123:0/1685258237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec45e08f0 0x556ec45e02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:22.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.600+0000 7f108a0e1b80 1 --2- 172.21.15.123:0/1685258237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ec45e5bc0 0x556ec45f5f80 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:22.601 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.600+0000 7f108a0e1b80 1 -- 172.21.15.123:0/1685258237 >> 172.21.15.123:0/1685258237 conn(0x556ec45586d0 msgr2=0x556ec45e8e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:22.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.600+0000 7f108a0e1b80 1 -- 172.21.15.123:0/1685258237 shutdown_connections 2024-05-08T06:12:22.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.601+0000 7f108a0e1b80 1 -- 172.21.15.123:0/1685258237 wait complete. 2024-05-08T06:12:22.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.601+0000 7f108a0e1b80 1 -- start start 2024-05-08T06:12:22.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.601+0000 7f108a0e1b80 1 Processor -- start 2024-05-08T06:12:22.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.602+0000 7f108a0e1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ec45e08f0 0x556ec4600e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:22.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.602+0000 7f108a0e1b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec45e5bc0 0x556ec4601390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:22.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.602+0000 7f108a0e1b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec45e64e0 0x556ec46018d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:22.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.602+0000 7f108a0e1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f10781ae120 con 0x556ec45e08f0 2024-05-08T06:12:22.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.602+0000 7f108a0e1b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f10781af590 con 0x556ec45e64e0 2024-05-08T06:12:22.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.602+0000 7f108a0e1b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f10781ae7f0 con 0x556ec45e5bc0 2024-05-08T06:12:22.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.602+0000 7f108a0e1b80 10 client.0 _ll_register_callbacks cb 0x556ec44d9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:12:22.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.602+0000 7f107d800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ec45e08f0 0x556ec4600e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:22.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.602+0000 7f1077e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec45e5bc0 0x556ec4601390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:22.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.602+0000 7f1077e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec45e5bc0 0x556ec4601390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:34632/0 (socket says 172.21.15.123:34632) 2024-05-08T06:12:22.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.602+0000 7f1077e00700 1 -- 172.21.15.123:0/2994909358 learned_addr learned my addr 172.21.15.123:0/2994909358 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:22.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.602+0000 7f107d800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ec45e08f0 0x556ec4600e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51374/0 (socket says 172.21.15.123:51374) 2024-05-08T06:12:22.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.602+0000 7f107d800700 1 --2- 172.21.15.123:0/2994909358 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec45e64e0 0x556ec46018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:22.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.603+0000 7f107d800700 1 -- 172.21.15.123:0/2994909358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec45e5bc0 msgr2=0x556ec4601390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:22.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.603+0000 7f107d800700 1 --2- 172.21.15.123:0/2994909358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec45e5bc0 0x556ec4601390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:22.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.603+0000 7f107d800700 1 -- 172.21.15.123:0/2994909358 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec45e64e0 msgr2=0x556ec46018d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:22.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.603+0000 7f107d800700 1 --2- 172.21.15.123:0/2994909358 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec45e64e0 0x556ec46018d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:22.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.603+0000 7f107d800700 1 -- 172.21.15.123:0/2994909358 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ec4663220 con 0x556ec45e08f0 2024-05-08T06:12:22.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.603+0000 7f107d800700 1 --2- 172.21.15.123:0/2994909358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ec45e08f0 0x556ec4600e50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x556ec45e59b0 tx=0x7f10700254f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4348a22d9f8dc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:22.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.603+0000 7f105ea00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-08T06:12:22.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.603+0000 7f105ea00700 1 -- 172.21.15.123:0/2994909358 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10700074a0 con 0x556ec45e08f0 2024-05-08T06:12:22.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.603+0000 7f105ea00700 1 -- 172.21.15.123:0/2994909358 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f1070030040 con 0x556ec45e08f0 2024-05-08T06:12:22.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.603+0000 7f108a0e1b80 1 -- 172.21.15.123:0/2994909358 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1064004830 con 0x556ec45e08f0 2024-05-08T06:12:22.606 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.604+0000 7f105ea00700 1 -- 172.21.15.123:0/2994909358 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f107003c020 con 0x556ec45e08f0 2024-05-08T06:12:22.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.604+0000 7f108a0e1b80 10 client.15417 fetch_fsmap learned FSMap version 24 2024-05-08T06:12:22.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.604+0000 7f108a0e1b80 1 -- 172.21.15.123:0/2994909358 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556ec46644e0 con 0x556ec45e08f0 2024-05-08T06:12:22.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.604+0000 7f105ea00700 1 -- 172.21.15.123:0/2994909358 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1064004830 con 0x556ec45e08f0 2024-05-08T06:12:22.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.604+0000 7f105ea00700 1 -- 172.21.15.123:0/2994909358 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f1070011040 con 0x556ec45e08f0 2024-05-08T06:12:22.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.604+0000 7f108a0e1b80 10 client.15417 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:12:22.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.604+0000 7f108a0e1b80 10 client.15417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T06:12:22.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.604+0000 7f108a0e1b80 20 client.15417 populate_metadata read hostname 'smithi123' 2024-05-08T06:12:22.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.604+0000 7f108a0e1b80 1 --2- 172.21.15.123:0/2994909358 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x556ec4664ad0 0x556ec4666ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:22.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.604+0000 7f108a0e1b80 4 client.15417 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T06:12:22.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.604+0000 7f108a0e1b80 1 -- 172.21.15.123:0/2994909358 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556ec4540400 con 0x556ec4664ad0 2024-05-08T06:12:22.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.605+0000 7f1077e00700 1 --2- 172.21.15.123:0/2994909358 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x556ec4664ad0 0x556ec4666ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:22.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.605+0000 7f1077e00700 1 --2- 172.21.15.123:0/2994909358 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x556ec4664ad0 0x556ec4666ef0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd1edc6995b915e3 server_cookie=b75f0374203e24d0 in_seq=0 out_seq=0 2024-05-08T06:12:22.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.605+0000 7f105ea00700 10 client.15417 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:22.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.605+0000 7f105ea00700 1 -- 172.21.15.123:0/2994909358 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556ec4540400 con 0x556ec4664ad0 2024-05-08T06:12:22.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.605+0000 7f105ea00700 10 client.15417 handle_command_reply: tid=0 2024-05-08T06:12:22.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.606+0000 7f108a0e1b80 1 client.15417 shutdown 2024-05-08T06:12:22.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.606+0000 7f108a0e1b80 1 -- 172.21.15.123:0/2994909358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ec45e08f0 msgr2=0x556ec4600e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:22.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.606+0000 7f108a0e1b80 1 --2- 172.21.15.123:0/2994909358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ec45e08f0 0x556ec4600e50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x556ec45e59b0 tx=0x7f10700254f0 comp rx=0 tx=0).stop 2024-05-08T06:12:22.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.606+0000 7f108a0e1b80 1 -- 172.21.15.123:0/2994909358 shutdown_connections 2024-05-08T06:12:22.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.606+0000 7f108a0e1b80 1 --2- 172.21.15.123:0/2994909358 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x556ec4664ad0 0x556ec4666ef0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:22.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.606+0000 7f105ea00700 0 client.15417 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:22.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.606+0000 7f108a0e1b80 1 --2- 172.21.15.123:0/2994909358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ec45e08f0 0x556ec4600e50 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:22.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.606+0000 7f108a0e1b80 1 --2- 172.21.15.123:0/2994909358 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556ec45e64e0 0x556ec46018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:22.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.606+0000 7f108a0e1b80 1 --2- 172.21.15.123:0/2994909358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ec45e5bc0 0x556ec4601390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:22.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.606+0000 7f108a0e1b80 1 -- 172.21.15.123:0/2994909358 >> 172.21.15.123:0/2994909358 conn(0x556ec45586d0 msgr2=0x556ec45e9690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:22.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.606+0000 7f108a0e1b80 1 -- 172.21.15.123:0/2994909358 shutdown_connections 2024-05-08T06:12:22.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.606+0000 7f108a0e1b80 1 -- 172.21.15.123:0/2994909358 wait complete. 2024-05-08T06:12:22.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.606+0000 7f108a0e1b80 20 client.15417 trim_cache size 0 max 16384 2024-05-08T06:12:22.609 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-05-08T06:12:22.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.607+0000 7f107f800700 1 -- 172.21.15.123:0/1142321413 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f10600b2880 msgr2=0x7f10600b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:22.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.607+0000 7f107f800700 1 --2- 172.21.15.123:0/1142321413 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f10600b2880 0x7f10600b4d40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f106c0a6d20 tx=0x7f106c005020 comp rx=0 tx=0).stop 2024-05-08T06:12:22.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.607+0000 7f107f800700 1 -- 172.21.15.123:0/1142321413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1078103750 msgr2=0x7f10781a1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:22.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.607+0000 7f107f800700 1 --2- 172.21.15.123:0/1142321413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1078103750 0x7f10781a1930 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f106c00e9c0 tx=0x7f106c00ecd0 comp rx=0 tx=0).stop 2024-05-08T06:12:22.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.607+0000 7f107f800700 1 -- 172.21.15.123:0/1142321413 shutdown_connections 2024-05-08T06:12:22.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.607+0000 7f107f800700 1 --2- 172.21.15.123:0/1142321413 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f10600b2880 0x7f10600b4d40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:22.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.607+0000 7f107f800700 1 --2- 172.21.15.123:0/1142321413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1078108850 0x7f107819baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:22.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.607+0000 7f107f800700 1 --2- 172.21.15.123:0/1142321413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1078103750 0x7f10781a1930 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:22.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.607+0000 7f107f800700 1 --2- 172.21.15.123:0/1142321413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1078104100 0x7f10781a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:22.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.607+0000 7f107f800700 1 -- 172.21.15.123:0/1142321413 >> 172.21.15.123:0/1142321413 conn(0x7f107807a600 msgr2=0x7f1078101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:22.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.608+0000 7f107f800700 1 -- 172.21.15.123:0/1142321413 shutdown_connections 2024-05-08T06:12:22.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:22.608+0000 7f107f800700 1 -- 172.21.15.123:0/1142321413 wait complete. 2024-05-08T06:12:22.840 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:22 smithi163 ceph-mon[34520]: pgmap v517: 97 pgs: 97 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.4 MiB/s wr, 986 op/s 2024-05-08T06:12:22.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:22 smithi123 ceph-mon[29787]: pgmap v517: 97 pgs: 97 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.4 MiB/s wr, 986 op/s 2024-05-08T06:12:23.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:22 smithi192 ceph-mon[34664]: pgmap v517: 97 pgs: 97 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.4 MiB/s wr, 986 op/s 2024-05-08T06:12:24.662 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T06:12:24.663 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T06:12:24.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:24 smithi123 ceph-mon[29787]: pgmap v518: 97 pgs: 97 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.4 MiB/s wr, 985 op/s 2024-05-08T06:12:25.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:24 smithi163 ceph-mon[34520]: pgmap v518: 97 pgs: 97 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.4 MiB/s wr, 985 op/s 2024-05-08T06:12:25.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:24 smithi192 ceph-mon[34664]: pgmap v518: 97 pgs: 97 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.4 MiB/s wr, 985 op/s 2024-05-08T06:12:25.074 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:12:26.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:26 smithi123 ceph-mon[29787]: pgmap v519: 97 pgs: 97 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 725 KiB/s wr, 1.36k op/s 2024-05-08T06:12:27.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:26 smithi163 ceph-mon[34520]: pgmap v519: 97 pgs: 97 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 725 KiB/s wr, 1.36k op/s 2024-05-08T06:12:27.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:26 smithi192 ceph-mon[34664]: pgmap v519: 97 pgs: 97 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 725 KiB/s wr, 1.36k op/s 2024-05-08T06:12:27.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.505+0000 7f328ba00700 1 -- 172.21.15.123:0/2303017784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3284109510 msgr2=0x7f32841098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:27.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.505+0000 7f328ba00700 1 --2- 172.21.15.123:0/2303017784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3284109510 0x7f32841098f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3278009990 tx=0x7f3278009ca0 comp rx=0 tx=0).stop 2024-05-08T06:12:27.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.505+0000 7f328ba00700 1 -- 172.21.15.123:0/2303017784 shutdown_connections 2024-05-08T06:12:27.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.505+0000 7f328ba00700 1 --2- 172.21.15.123:0/2303017784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3284109510 0x7f32841098f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:27.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.505+0000 7f328ba00700 1 --2- 172.21.15.123:0/2303017784 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f328406bb40 0x7f3284110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:27.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.505+0000 7f328ba00700 1 --2- 172.21.15.123:0/2303017784 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f328406b180 0x7f328406b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:27.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.505+0000 7f328ba00700 1 -- 172.21.15.123:0/2303017784 >> 172.21.15.123:0/2303017784 conn(0x7f3284079350 msgr2=0x7f3284079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:27.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.505+0000 7f328ba00700 1 -- 172.21.15.123:0/2303017784 shutdown_connections 2024-05-08T06:12:27.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.505+0000 7f328ba00700 1 -- 172.21.15.123:0/2303017784 wait complete. 2024-05-08T06:12:27.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.507+0000 7f328ba00700 1 Processor -- start 2024-05-08T06:12:27.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.507+0000 7f328ba00700 1 -- start start 2024-05-08T06:12:27.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.507+0000 7f328ba00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f328406b180 0x7f328419f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:27.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.507+0000 7f328ba00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f328406bb40 0x7f328419fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:27.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.507+0000 7f328ba00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3284109510 0x7f32841999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:27.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.507+0000 7f328ba00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3284113640 con 0x7f3284109510 2024-05-08T06:12:27.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.507+0000 7f328ba00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f32841134c0 con 0x7f328406bb40 2024-05-08T06:12:27.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.507+0000 7f328ba00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f32841137c0 con 0x7f328406b180 2024-05-08T06:12:27.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.508+0000 7f3289a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3284109510 0x7f32841999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:27.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.508+0000 7f3289a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3284109510 0x7f32841999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41822/0 (socket says 172.21.15.123:41822) 2024-05-08T06:12:27.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.508+0000 7f3283e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f328406bb40 0x7f328419fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:27.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.508+0000 7f3289a00700 1 -- 172.21.15.123:0/2505210425 learned_addr learned my addr 172.21.15.123:0/2505210425 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:27.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.508+0000 7f3289000700 1 --2- 172.21.15.123:0/2505210425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f328406b180 0x7f328419f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:27.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.508+0000 7f3289a00700 1 -- 172.21.15.123:0/2505210425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f328406b180 msgr2=0x7f328419f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:27.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.508+0000 7f3289a00700 1 --2- 172.21.15.123:0/2505210425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f328406b180 0x7f328419f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:27.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.508+0000 7f3289a00700 1 -- 172.21.15.123:0/2505210425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f328406bb40 msgr2=0x7f328419fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:27.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.508+0000 7f3289a00700 1 --2- 172.21.15.123:0/2505210425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f328406bb40 0x7f328419fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:27.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.508+0000 7f3289a00700 1 -- 172.21.15.123:0/2505210425 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f328419a120 con 0x7f3284109510 2024-05-08T06:12:27.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.508+0000 7f3289a00700 1 --2- 172.21.15.123:0/2505210425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3284109510 0x7f32841999c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f327c007ee0 tx=0x7f327c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:27.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.509+0000 7f3281600700 1 -- 172.21.15.123:0/2505210425 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c004bb0 con 0x7f3284109510 2024-05-08T06:12:27.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.509+0000 7f3281600700 1 -- 172.21.15.123:0/2505210425 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f327c004d50 con 0x7f3284109510 2024-05-08T06:12:27.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.509+0000 7f328ba00700 1 -- 172.21.15.123:0/2505210425 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f328419a410 con 0x7f3284109510 2024-05-08T06:12:27.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.509+0000 7f3281600700 1 -- 172.21.15.123:0/2505210425 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c005920 con 0x7f3284109510 2024-05-08T06:12:27.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.509+0000 7f328ba00700 1 -- 172.21.15.123:0/2505210425 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32841a6680 con 0x7f3284109510 2024-05-08T06:12:27.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.513+0000 7f32963e9b80 1 Processor -- start 2024-05-08T06:12:27.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.514+0000 7f3281600700 1 -- 172.21.15.123:0/2505210425 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7f327c01a020 con 0x7f3284109510 2024-05-08T06:12:27.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.514+0000 7f32963e9b80 1 -- start start 2024-05-08T06:12:27.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.514+0000 7f32963e9b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559524afdf50 0x559524afe330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:27.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.514+0000 7f32963e9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559524af9760 0x559524af9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:27.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.514+0000 7f32963e9b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559524afa120 0x559524b05e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:27.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.514+0000 7f32963e9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559524a47aa0 con 0x559524af9760 2024-05-08T06:12:27.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.514+0000 7f32963e9b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5595249e57e0 con 0x559524afa120 2024-05-08T06:12:27.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.514+0000 7f32963e9b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x559524a43a60 con 0x559524afdf50 2024-05-08T06:12:27.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.515+0000 7f3289000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559524af9760 0x559524af9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:27.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.515+0000 7f3289000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559524af9760 0x559524af9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41836/0 (socket says 172.21.15.123:41836) 2024-05-08T06:12:27.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.515+0000 7f3289000700 1 -- 172.21.15.123:0/848248671 learned_addr learned my addr 172.21.15.123:0/848248671 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:27.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.515+0000 7f3283e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559524afa120 0x559524b05e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:27.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.515+0000 7f3283e00700 1 --2- 172.21.15.123:0/848248671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559524afdf50 0x559524afe330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:27.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.515+0000 7f3289000700 1 -- 172.21.15.123:0/848248671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559524afdf50 msgr2=0x559524afe330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:27.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.515+0000 7f3289000700 1 --2- 172.21.15.123:0/848248671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559524afdf50 0x559524afe330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:27.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.515+0000 7f3289000700 1 -- 172.21.15.123:0/848248671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559524afa120 msgr2=0x559524b05e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:27.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.515+0000 7f3289000700 1 --2- 172.21.15.123:0/848248671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559524afa120 0x559524b05e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:27.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.515+0000 7f3289000700 1 -- 172.21.15.123:0/848248671 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559524994950 con 0x559524af9760 2024-05-08T06:12:27.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.515+0000 7f3289000700 1 --2- 172.21.15.123:0/848248671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559524af9760 0x559524af9be0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3278009990 tx=0x7f3278005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595a4e330a2015fe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:27.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.516+0000 7f3276000700 1 -- 172.21.15.123:0/848248671 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278027070 con 0x559524af9760 2024-05-08T06:12:27.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.516+0000 7f3276000700 1 -- 172.21.15.123:0/848248671 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32780056a0 con 0x559524af9760 2024-05-08T06:12:27.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.516+0000 7f3276000700 1 -- 172.21.15.123:0/848248671 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278003b80 con 0x559524af9760 2024-05-08T06:12:27.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.516+0000 7f3281600700 1 --2- 172.21.15.123:0/2505210425 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f326c0b2880 0x7f326c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:27.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.516+0000 7f32963e9b80 1 -- 172.21.15.123:0/848248671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559524af9760 msgr2=0x559524af9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:27.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.516+0000 7f32963e9b80 1 --2- 172.21.15.123:0/848248671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559524af9760 0x559524af9be0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3278009990 tx=0x7f3278005bc0 comp rx=0 tx=0).stop 2024-05-08T06:12:27.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.517+0000 7f3281600700 1 -- 172.21.15.123:0/2505210425 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f327c0e1880 con 0x7f3284109510 2024-05-08T06:12:27.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.517+0000 7f3283e00700 1 --2- 172.21.15.123:0/2505210425 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f326c0b2880 0x7f326c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:27.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.517+0000 7f32963e9b80 1 -- 172.21.15.123:0/848248671 shutdown_connections 2024-05-08T06:12:27.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.517+0000 7f32963e9b80 1 --2- 172.21.15.123:0/848248671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559524afa120 0x559524b05e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:27.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.517+0000 7f32963e9b80 1 --2- 172.21.15.123:0/848248671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559524afdf50 0x559524afe330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:27.534 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.517+0000 7f32963e9b80 1 --2- 172.21.15.123:0/848248671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559524af9760 0x559524af9be0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:27.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.517+0000 7f32963e9b80 1 -- 172.21.15.123:0/848248671 >> 172.21.15.123:0/848248671 conn(0x559524a6d750 msgr2=0x559524aefb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:27.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.517+0000 7f32963e9b80 1 -- 172.21.15.123:0/848248671 shutdown_connections 2024-05-08T06:12:27.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.517+0000 7f32963e9b80 1 -- 172.21.15.123:0/848248671 wait complete. 2024-05-08T06:12:27.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.517+0000 7f3283e00700 1 --2- 172.21.15.123:0/2505210425 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f326c0b2880 0x7f326c0b4d40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x559524af07a0 tx=0x7f32700047c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:27.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.518+0000 7f32963e9b80 1 -- start start 2024-05-08T06:12:27.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.518+0000 7f32963e9b80 1 Processor -- start 2024-05-08T06:12:27.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.518+0000 7f32963e9b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559524afa120 0x559524b05be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:27.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.518+0000 7f32963e9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559524afdf50 0x559524b02ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:27.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.518+0000 7f32963e9b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559524b03420 0x559524b038a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:27.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.518+0000 7f32963e9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559524b063b0 con 0x559524afdf50 2024-05-08T06:12:27.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.518+0000 7f32963e9b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5595249adba0 con 0x559524afa120 2024-05-08T06:12:27.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.518+0000 7f32963e9b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x559524b06530 con 0x559524b03420 2024-05-08T06:12:27.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.518+0000 7f32963e9b80 10 client.0 _ll_register_callbacks cb 0x5595249eecf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:12:27.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.519+0000 7f3283e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559524afdf50 0x559524b02ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:27.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.519+0000 7f3289000700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559524afa120 0x559524b05be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:27.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.519+0000 7f3283e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559524afdf50 0x559524b02ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41846/0 (socket says 172.21.15.123:41846) 2024-05-08T06:12:27.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.519+0000 7f3283e00700 1 -- 172.21.15.123:0/1137295640 learned_addr learned my addr 172.21.15.123:0/1137295640 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:27.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.519+0000 7f3289a00700 1 --2- 172.21.15.123:0/1137295640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559524b03420 0x559524b038a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:27.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.519+0000 7f3283e00700 1 -- 172.21.15.123:0/1137295640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559524b03420 msgr2=0x559524b038a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:27.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.519+0000 7f3283e00700 1 --2- 172.21.15.123:0/1137295640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559524b03420 0x559524b038a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:27.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.519+0000 7f3283e00700 1 -- 172.21.15.123:0/1137295640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559524afa120 msgr2=0x559524b05be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:27.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.519+0000 7f3283e00700 1 --2- 172.21.15.123:0/1137295640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559524afa120 0x559524b05be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:27.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.519+0000 7f3283e00700 1 -- 172.21.15.123:0/1137295640 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559524a0d4a0 con 0x559524afdf50 2024-05-08T06:12:27.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.519+0000 7f3289000700 1 --2- 172.21.15.123:0/1137295640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559524afa120 0x559524b05be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:12:27.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.519+0000 7f3283e00700 1 --2- 172.21.15.123:0/1137295640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559524afdf50 0x559524b02ee0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3270004560 tx=0x7f3270009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a71ed89897c35e32 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:27.536 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.519+0000 7f326aa00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-08T06:12:27.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.520+0000 7f326aa00700 1 -- 172.21.15.123:0/1137295640 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327000ed80 con 0x559524afdf50 2024-05-08T06:12:27.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.520+0000 7f326aa00700 1 -- 172.21.15.123:0/1137295640 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f3270022430 con 0x559524afdf50 2024-05-08T06:12:27.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.520+0000 7f32963e9b80 1 -- 172.21.15.123:0/1137295640 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32780056a0 con 0x559524afdf50 2024-05-08T06:12:27.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.520+0000 7f326aa00700 1 -- 172.21.15.123:0/1137295640 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3270022670 con 0x559524afdf50 2024-05-08T06:12:27.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.520+0000 7f32963e9b80 10 client.15429 fetch_fsmap learned FSMap version 24 2024-05-08T06:12:27.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.520+0000 7f32963e9b80 1 -- 172.21.15.123:0/1137295640 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559524b11180 con 0x559524afdf50 2024-05-08T06:12:27.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.520+0000 7f326aa00700 1 -- 172.21.15.123:0/1137295640 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32780056a0 con 0x559524afdf50 2024-05-08T06:12:27.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.520+0000 7f326aa00700 1 -- 172.21.15.123:0/1137295640 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f3270010840 con 0x559524afdf50 2024-05-08T06:12:27.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.521+0000 7f32963e9b80 10 client.15429 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:12:27.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.521+0000 7f32963e9b80 10 client.15429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T06:12:27.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.521+0000 7f32963e9b80 20 client.15429 populate_metadata read hostname 'smithi123' 2024-05-08T06:12:27.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.521+0000 7f32963e9b80 1 --2- 172.21.15.123:0/1137295640 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x559524b137c0 0x559524b15bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:27.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.521+0000 7f32963e9b80 4 client.15429 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:12:27.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.521+0000 7f32963e9b80 1 -- 172.21.15.123:0/1137295640 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559524ac21c0 con 0x559524b137c0 2024-05-08T06:12:27.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.521+0000 7f3289000700 1 --2- 172.21.15.123:0/1137295640 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x559524b137c0 0x559524b15bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:27.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.522+0000 7f326aa00700 10 client.15429 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:27.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.522+0000 7f3289000700 1 --2- 172.21.15.123:0/1137295640 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x559524b137c0 0x559524b15bb0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9f4c57161cde96d server_cookie=8b34f4c3d69166fe in_seq=0 out_seq=0 2024-05-08T06:12:27.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.522+0000 7f326aa00700 1 -- 172.21.15.123:0/1137295640 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559524ac21c0 con 0x559524b137c0 2024-05-08T06:12:27.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.522+0000 7f326aa00700 10 client.15429 handle_command_reply: tid=0 2024-05-08T06:12:27.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.523+0000 7f32963e9b80 1 client.15429 shutdown 2024-05-08T06:12:27.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.523+0000 7f32963e9b80 1 -- 172.21.15.123:0/1137295640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559524afdf50 msgr2=0x559524b02ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:27.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.523+0000 7f32963e9b80 1 --2- 172.21.15.123:0/1137295640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559524afdf50 0x559524b02ee0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3270004560 tx=0x7f3270009260 comp rx=0 tx=0).stop 2024-05-08T06:12:27.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.523+0000 7f32963e9b80 1 -- 172.21.15.123:0/1137295640 shutdown_connections 2024-05-08T06:12:27.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.523+0000 7f32963e9b80 1 --2- 172.21.15.123:0/1137295640 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x559524b137c0 0x559524b15bb0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:27.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.523+0000 7f32963e9b80 1 --2- 172.21.15.123:0/1137295640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559524b03420 0x559524b038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:27.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.523+0000 7f32963e9b80 1 --2- 172.21.15.123:0/1137295640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559524afa120 0x559524b05be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:27.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.523+0000 7f32963e9b80 1 --2- 172.21.15.123:0/1137295640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559524afdf50 0x559524b02ee0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:27.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.523+0000 7f32963e9b80 1 -- 172.21.15.123:0/1137295640 >> 172.21.15.123:0/1137295640 conn(0x559524a6d750 msgr2=0x559524aff5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:27.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.523+0000 7f326aa00700 0 client.15429 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:27.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.523+0000 7f32963e9b80 1 -- 172.21.15.123:0/1137295640 shutdown_connections 2024-05-08T06:12:27.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.523+0000 7f32963e9b80 1 -- 172.21.15.123:0/1137295640 wait complete. 2024-05-08T06:12:27.539 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.523+0000 7f32963e9b80 20 client.15429 trim_cache size 0 max 16384 2024-05-08T06:12:27.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.545+0000 7f32963e9b80 1 Processor -- start 2024-05-08T06:12:27.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.545+0000 7f32963e9b80 1 -- start start 2024-05-08T06:12:27.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.546+0000 7f32963e9b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559524afa120 0x559524afd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:27.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.546+0000 7f32963e9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559524afdf50 0x559524b1b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:27.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.546+0000 7f32963e9b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559524b03420 0x559524b1ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:27.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.546+0000 7f32963e9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f32841ae150 con 0x559524afdf50 2024-05-08T06:12:27.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.546+0000 7f32963e9b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f32841ab9e0 con 0x559524afa120 2024-05-08T06:12:27.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.546+0000 7f32963e9b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f32841a9830 con 0x559524b03420 2024-05-08T06:12:27.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.546+0000 7f3283e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559524afdf50 0x559524b1b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:27.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.546+0000 7f3283e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559524afdf50 0x559524b1b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41848/0 (socket says 172.21.15.123:41848) 2024-05-08T06:12:27.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.546+0000 7f3283e00700 1 -- 172.21.15.123:0/4208154393 learned_addr learned my addr 172.21.15.123:0/4208154393 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:27.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.546+0000 7f3289a00700 1 --2- 172.21.15.123:0/4208154393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559524b03420 0x559524b1ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:27.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.546+0000 7f3289000700 1 --2- 172.21.15.123:0/4208154393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559524afa120 0x559524afd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:27.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.546+0000 7f3289a00700 1 -- 172.21.15.123:0/4208154393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559524afa120 msgr2=0x559524afd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:27.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.546+0000 7f3289a00700 1 --2- 172.21.15.123:0/4208154393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559524afa120 0x559524afd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:27.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.546+0000 7f3289a00700 1 -- 172.21.15.123:0/4208154393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559524afdf50 msgr2=0x559524b1b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:27.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.547+0000 7f3289a00700 1 --2- 172.21.15.123:0/4208154393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559524afdf50 0x559524b1b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:27.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.547+0000 7f3289a00700 1 -- 172.21.15.123:0/4208154393 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559524b01450 con 0x559524b03420 2024-05-08T06:12:27.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.547+0000 7f3283e00700 1 --2- 172.21.15.123:0/4208154393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559524afdf50 0x559524b1b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:12:27.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.547+0000 7f3289a00700 1 --2- 172.21.15.123:0/4208154393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559524b03420 0x559524b1ba00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f327c0a60c0 tx=0x7f327c007f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b5f768de344a56e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:27.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.547+0000 7f3275600700 1 -- 172.21.15.123:0/4208154393 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c0a8960 con 0x559524b03420 2024-05-08T06:12:27.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.547+0000 7f3275600700 1 -- 172.21.15.123:0/4208154393 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f327c0ab070 con 0x559524b03420 2024-05-08T06:12:27.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.547+0000 7f3275600700 1 -- 172.21.15.123:0/4208154393 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c0a90b0 con 0x559524b03420 2024-05-08T06:12:27.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.548+0000 7f32963e9b80 1 -- 172.21.15.123:0/4208154393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559524b03420 msgr2=0x559524b1ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:27.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.548+0000 7f32963e9b80 1 --2- 172.21.15.123:0/4208154393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559524b03420 0x559524b1ba00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f327c0a60c0 tx=0x7f327c007f90 comp rx=0 tx=0).stop 2024-05-08T06:12:27.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.548+0000 7f32963e9b80 1 -- 172.21.15.123:0/4208154393 shutdown_connections 2024-05-08T06:12:27.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.548+0000 7f32963e9b80 1 --2- 172.21.15.123:0/4208154393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559524b03420 0x559524b1ba00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:27.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.548+0000 7f32963e9b80 1 --2- 172.21.15.123:0/4208154393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559524afa120 0x559524afd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:27.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.548+0000 7f32963e9b80 1 --2- 172.21.15.123:0/4208154393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559524afdf50 0x559524b1b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:27.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.548+0000 7f32963e9b80 1 -- 172.21.15.123:0/4208154393 >> 172.21.15.123:0/4208154393 conn(0x559524a6d750 msgr2=0x559524af4bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:27.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.548+0000 7f32963e9b80 1 -- 172.21.15.123:0/4208154393 shutdown_connections 2024-05-08T06:12:27.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.548+0000 7f32963e9b80 1 -- 172.21.15.123:0/4208154393 wait complete. 2024-05-08T06:12:27.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.549+0000 7f32963e9b80 1 -- start start 2024-05-08T06:12:27.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.549+0000 7f32963e9b80 1 Processor -- start 2024-05-08T06:12:27.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.549+0000 7f32963e9b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559524afa120 0x559524b087c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:27.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.549+0000 7f32963e9b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559524afdf50 0x559524b08d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:27.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.549+0000 7f32963e9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559524b03420 0x559524b09240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:27.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.549+0000 7f32963e9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f32841a9ea0 con 0x559524b03420 2024-05-08T06:12:27.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.549+0000 7f32963e9b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f32841ab310 con 0x559524afdf50 2024-05-08T06:12:27.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.549+0000 7f32963e9b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f32841aa570 con 0x559524afa120 2024-05-08T06:12:27.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.549+0000 7f32963e9b80 10 client.0 _ll_register_callbacks cb 0x5595249eecf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:12:27.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.550+0000 7f3289a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559524b03420 0x559524b09240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:27.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.550+0000 7f3289a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559524b03420 0x559524b09240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41858/0 (socket says 172.21.15.123:41858) 2024-05-08T06:12:27.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.550+0000 7f3289a00700 1 -- 172.21.15.123:0/1363930537 learned_addr learned my addr 172.21.15.123:0/1363930537 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:27.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.550+0000 7f3283e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559524afdf50 0x559524b08d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:27.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.550+0000 7f3289000700 1 --2- 172.21.15.123:0/1363930537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559524afa120 0x559524b087c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:27.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.550+0000 7f3289a00700 1 -- 172.21.15.123:0/1363930537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559524afa120 msgr2=0x559524b087c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:27.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.550+0000 7f3289a00700 1 --2- 172.21.15.123:0/1363930537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559524afa120 0x559524b087c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:27.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.550+0000 7f3289a00700 1 -- 172.21.15.123:0/1363930537 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559524afdf50 msgr2=0x559524b08d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:27.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.550+0000 7f3289a00700 1 --2- 172.21.15.123:0/1363930537 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559524afdf50 0x559524b08d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:27.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.550+0000 7f3289a00700 1 -- 172.21.15.123:0/1363930537 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559524b7e220 con 0x559524b03420 2024-05-08T06:12:27.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.550+0000 7f3289a00700 1 --2- 172.21.15.123:0/1363930537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559524b03420 0x559524b09240 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f327c019ff0 tx=0x7f327c0a9360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a714fd0a09a68c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:27.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.550+0000 7f326aa00700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-08T06:12:27.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.551+0000 7f326aa00700 1 -- 172.21.15.123:0/1363930537 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c06c070 con 0x559524b03420 2024-05-08T06:12:27.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.551+0000 7f326aa00700 1 -- 172.21.15.123:0/1363930537 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f327c0b1a30 con 0x559524b03420 2024-05-08T06:12:27.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.551+0000 7f32963e9b80 1 -- 172.21.15.123:0/1363930537 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f327c0ab070 con 0x559524b03420 2024-05-08T06:12:27.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.551+0000 7f326aa00700 1 -- 172.21.15.123:0/1363930537 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f327c0a8960 con 0x559524b03420 2024-05-08T06:12:27.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.551+0000 7f32963e9b80 10 client.15435 fetch_fsmap learned FSMap version 24 2024-05-08T06:12:27.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.551+0000 7f32963e9b80 1 -- 172.21.15.123:0/1363930537 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559524b7f520 con 0x559524b03420 2024-05-08T06:12:27.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.551+0000 7f326aa00700 1 -- 172.21.15.123:0/1363930537 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f327c0ab070 con 0x559524b03420 2024-05-08T06:12:27.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.551+0000 7f326aa00700 1 -- 172.21.15.123:0/1363930537 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f327c0a47c0 con 0x559524b03420 2024-05-08T06:12:27.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.551+0000 7f32963e9b80 10 client.15435 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:12:27.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.551+0000 7f32963e9b80 10 client.15435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T06:12:27.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.551+0000 7f32963e9b80 20 client.15435 populate_metadata read hostname 'smithi123' 2024-05-08T06:12:27.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.551+0000 7f32963e9b80 1 --2- 172.21.15.123:0/1363930537 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x559524b7fb10 0x559524b81f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:27.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.551+0000 7f32963e9b80 4 client.15435 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T06:12:27.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.551+0000 7f32963e9b80 1 -- 172.21.15.123:0/1363930537 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559524a55480 con 0x559524b7fb10 2024-05-08T06:12:27.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.552+0000 7f3289000700 1 --2- 172.21.15.123:0/1363930537 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x559524b7fb10 0x559524b81f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:27.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.553+0000 7f3289000700 1 --2- 172.21.15.123:0/1363930537 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x559524b7fb10 0x559524b81f00 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9786e735d551de95 server_cookie=6e389daed50fdae4 in_seq=0 out_seq=0 2024-05-08T06:12:27.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.553+0000 7f326aa00700 10 client.15435 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:27.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.554+0000 7f326aa00700 1 -- 172.21.15.123:0/1363930537 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559524a55480 con 0x559524b7fb10 2024-05-08T06:12:27.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.554+0000 7f326aa00700 10 client.15435 handle_command_reply: tid=0 2024-05-08T06:12:27.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.554+0000 7f32963e9b80 1 client.15435 shutdown 2024-05-08T06:12:27.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.554+0000 7f32963e9b80 1 -- 172.21.15.123:0/1363930537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559524b03420 msgr2=0x559524b09240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:27.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.554+0000 7f32963e9b80 1 --2- 172.21.15.123:0/1363930537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559524b03420 0x559524b09240 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f327c019ff0 tx=0x7f327c0a9360 comp rx=0 tx=0).stop 2024-05-08T06:12:27.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.554+0000 7f32963e9b80 1 -- 172.21.15.123:0/1363930537 shutdown_connections 2024-05-08T06:12:27.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.554+0000 7f32963e9b80 1 --2- 172.21.15.123:0/1363930537 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x559524b7fb10 0x559524b81f00 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:27.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.554+0000 7f32963e9b80 1 --2- 172.21.15.123:0/1363930537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559524b03420 0x559524b09240 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:27.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.554+0000 7f32963e9b80 1 --2- 172.21.15.123:0/1363930537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559524afa120 0x559524b087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:27.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.554+0000 7f32963e9b80 1 --2- 172.21.15.123:0/1363930537 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559524afdf50 0x559524b08d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:27.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.554+0000 7f32963e9b80 1 -- 172.21.15.123:0/1363930537 >> 172.21.15.123:0/1363930537 conn(0x559524a6d750 msgr2=0x559524b04710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:27.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.554+0000 7f326aa00700 0 client.15435 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:27.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.555+0000 7f32963e9b80 1 -- 172.21.15.123:0/1363930537 shutdown_connections 2024-05-08T06:12:27.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.555+0000 7f32963e9b80 1 -- 172.21.15.123:0/1363930537 wait complete. 2024-05-08T06:12:27.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.555+0000 7f32963e9b80 20 client.15435 trim_cache size 0 max 16384 2024-05-08T06:12:27.558 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-05-08T06:12:27.558 INFO:teuthology.orchestra.run.smithi123.stdout: "return_code": 0, 2024-05-08T06:12:27.558 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_tag": "86f1be5f-fd3f-4b11-88c5-cd5e28b29c18", 2024-05-08T06:12:27.559 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": "asynchronous" 2024-05-08T06:12:27.559 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-05-08T06:12:27.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.555+0000 7f328ba00700 1 -- 172.21.15.123:0/2505210425 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f326c0b2880 msgr2=0x7f326c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:27.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.556+0000 7f328ba00700 1 --2- 172.21.15.123:0/2505210425 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f326c0b2880 0x7f326c0b4d40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x559524af07a0 tx=0x7f32700047c0 comp rx=0 tx=0).stop 2024-05-08T06:12:27.559 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.556+0000 7f328ba00700 1 -- 172.21.15.123:0/2505210425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3284109510 msgr2=0x7f32841999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:27.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.556+0000 7f328ba00700 1 --2- 172.21.15.123:0/2505210425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3284109510 0x7f32841999c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f327c007ee0 tx=0x7f327c00d3b0 comp rx=0 tx=0).stop 2024-05-08T06:12:27.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.556+0000 7f328ba00700 1 -- 172.21.15.123:0/2505210425 shutdown_connections 2024-05-08T06:12:27.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.556+0000 7f328ba00700 1 --2- 172.21.15.123:0/2505210425 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f326c0b2880 0x7f326c0b4d40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:27.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.556+0000 7f328ba00700 1 --2- 172.21.15.123:0/2505210425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3284109510 0x7f32841999c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:27.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.556+0000 7f328ba00700 1 --2- 172.21.15.123:0/2505210425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f328406b180 0x7f328419f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:27.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.556+0000 7f328ba00700 1 --2- 172.21.15.123:0/2505210425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f328406bb40 0x7f328419fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:27.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.556+0000 7f328ba00700 1 -- 172.21.15.123:0/2505210425 >> 172.21.15.123:0/2505210425 conn(0x7f3284079350 msgr2=0x7f32841010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:27.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.556+0000 7f328ba00700 1 -- 172.21.15.123:0/2505210425 shutdown_connections 2024-05-08T06:12:27.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:27.556+0000 7f328ba00700 1 -- 172.21.15.123:0/2505210425 wait complete. 2024-05-08T06:12:28.034 INFO:tasks.workunit.client.0.smithi123.stdout:writing pattern 2024-05-08T06:12:28.034 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4190208 len 1024 2024-05-08T06:12:28.035 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4190208 len 1024 2024-05-08T06:12:28.035 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4190208 len 1024 2024-05-08T06:12:28.035 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4190208 len 1024 2024-05-08T06:12:28.035 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4190208 len 2048 2024-05-08T06:12:28.035 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4190208 len 2048 2024-05-08T06:12:28.035 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4190208 len 2048 2024-05-08T06:12:28.035 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4190208 len 2048 2024-05-08T06:12:28.035 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4190208 len 4096 2024-05-08T06:12:28.035 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4190208 len 4096 2024-05-08T06:12:28.035 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4190208 len 4096 2024-05-08T06:12:28.035 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4190208 len 4096 2024-05-08T06:12:28.035 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4190208 len 8192 2024-05-08T06:12:28.035 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4190208 len 8192 2024-05-08T06:12:28.035 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4190208 len 8192 2024-05-08T06:12:28.035 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4190208 len 8192 2024-05-08T06:12:28.036 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4190208 len 16384 2024-05-08T06:12:28.036 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4190208 len 16384 2024-05-08T06:12:28.036 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4190208 len 16384 2024-05-08T06:12:28.036 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4190208 len 16384 2024-05-08T06:12:28.036 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4190720 len 1024 2024-05-08T06:12:28.036 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4190720 len 1024 2024-05-08T06:12:28.036 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4190720 len 1024 2024-05-08T06:12:28.036 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4190720 len 1024 2024-05-08T06:12:28.036 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4190720 len 2048 2024-05-08T06:12:28.036 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4190720 len 2048 2024-05-08T06:12:28.036 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4190720 len 2048 2024-05-08T06:12:28.036 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4190720 len 2048 2024-05-08T06:12:28.036 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4190720 len 4096 2024-05-08T06:12:28.036 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4190720 len 4096 2024-05-08T06:12:28.036 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4190720 len 4096 2024-05-08T06:12:28.037 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4190720 len 4096 2024-05-08T06:12:28.037 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4190720 len 8192 2024-05-08T06:12:28.037 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4190720 len 8192 2024-05-08T06:12:28.037 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4190720 len 8192 2024-05-08T06:12:28.037 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4190720 len 8192 2024-05-08T06:12:28.037 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4190720 len 16384 2024-05-08T06:12:28.037 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4190720 len 16384 2024-05-08T06:12:28.037 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4190720 len 16384 2024-05-08T06:12:28.037 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4190720 len 16384 2024-05-08T06:12:28.037 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4191232 len 1024 2024-05-08T06:12:28.037 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4191232 len 1024 2024-05-08T06:12:28.037 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4191232 len 1024 2024-05-08T06:12:28.037 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4191232 len 1024 2024-05-08T06:12:28.037 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4191232 len 2048 2024-05-08T06:12:28.037 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4191232 len 2048 2024-05-08T06:12:28.037 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4191232 len 2048 2024-05-08T06:12:28.038 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4191232 len 2048 2024-05-08T06:12:28.038 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4191232 len 4096 2024-05-08T06:12:28.038 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4191232 len 4096 2024-05-08T06:12:28.038 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4191232 len 4096 2024-05-08T06:12:28.038 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4191232 len 4096 2024-05-08T06:12:28.038 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4191232 len 8192 2024-05-08T06:12:28.038 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4191232 len 8192 2024-05-08T06:12:28.038 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4191232 len 8192 2024-05-08T06:12:28.038 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4191232 len 8192 2024-05-08T06:12:28.038 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4191232 len 16384 2024-05-08T06:12:28.038 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4191232 len 16384 2024-05-08T06:12:28.038 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4191232 len 16384 2024-05-08T06:12:28.038 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4191232 len 16384 2024-05-08T06:12:28.038 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4191744 len 1024 2024-05-08T06:12:28.038 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4191744 len 1024 2024-05-08T06:12:28.039 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4191744 len 1024 2024-05-08T06:12:28.039 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4191744 len 1024 2024-05-08T06:12:28.039 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4191744 len 2048 2024-05-08T06:12:28.039 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4191744 len 2048 2024-05-08T06:12:28.039 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4191744 len 2048 2024-05-08T06:12:28.039 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4191744 len 2048 2024-05-08T06:12:28.039 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4191744 len 4096 2024-05-08T06:12:28.039 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4191744 len 4096 2024-05-08T06:12:28.039 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4191744 len 4096 2024-05-08T06:12:28.039 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4191744 len 4096 2024-05-08T06:12:28.039 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4191744 len 8192 2024-05-08T06:12:28.039 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4191744 len 8192 2024-05-08T06:12:28.039 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4191744 len 8192 2024-05-08T06:12:28.039 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4191744 len 8192 2024-05-08T06:12:28.039 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4191744 len 16384 2024-05-08T06:12:28.040 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4191744 len 16384 2024-05-08T06:12:28.040 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4191744 len 16384 2024-05-08T06:12:28.040 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4191744 len 16384 2024-05-08T06:12:28.040 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4192256 len 1024 2024-05-08T06:12:28.040 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4192256 len 1024 2024-05-08T06:12:28.040 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4192256 len 1024 2024-05-08T06:12:28.040 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4192256 len 1024 2024-05-08T06:12:28.040 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4192256 len 2048 2024-05-08T06:12:28.040 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4192256 len 2048 2024-05-08T06:12:28.040 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4192256 len 2048 2024-05-08T06:12:28.107 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4192256 len 2048 2024-05-08T06:12:28.108 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4192256 len 4096 2024-05-08T06:12:28.108 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4192256 len 4096 2024-05-08T06:12:28.108 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4192256 len 4096 2024-05-08T06:12:28.108 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4192256 len 4096 2024-05-08T06:12:28.108 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4192256 len 8192 2024-05-08T06:12:28.108 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4192256 len 8192 2024-05-08T06:12:28.108 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4192256 len 8192 2024-05-08T06:12:28.108 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4192256 len 8192 2024-05-08T06:12:28.108 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4192256 len 16384 2024-05-08T06:12:28.108 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4192256 len 16384 2024-05-08T06:12:28.108 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4192256 len 16384 2024-05-08T06:12:28.108 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4192256 len 16384 2024-05-08T06:12:28.108 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4192768 len 1024 2024-05-08T06:12:28.108 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4192768 len 1024 2024-05-08T06:12:28.108 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4192768 len 1024 2024-05-08T06:12:28.108 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4192768 len 1024 2024-05-08T06:12:28.109 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4192768 len 2048 2024-05-08T06:12:28.109 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4192768 len 2048 2024-05-08T06:12:28.109 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4192768 len 2048 2024-05-08T06:12:28.109 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4192768 len 2048 2024-05-08T06:12:28.109 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4192768 len 4096 2024-05-08T06:12:28.109 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4192768 len 4096 2024-05-08T06:12:28.109 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4192768 len 4096 2024-05-08T06:12:28.109 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4192768 len 4096 2024-05-08T06:12:28.109 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4192768 len 8192 2024-05-08T06:12:28.109 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4192768 len 8192 2024-05-08T06:12:28.109 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4192768 len 8192 2024-05-08T06:12:28.109 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4192768 len 8192 2024-05-08T06:12:28.109 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4192768 len 16384 2024-05-08T06:12:28.109 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4192768 len 16384 2024-05-08T06:12:28.109 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4192768 len 16384 2024-05-08T06:12:28.109 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4192768 len 16384 2024-05-08T06:12:28.110 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4193280 len 1024 2024-05-08T06:12:28.110 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4193280 len 1024 2024-05-08T06:12:28.110 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4193280 len 1024 2024-05-08T06:12:28.110 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4193280 len 1024 2024-05-08T06:12:28.110 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4193280 len 2048 2024-05-08T06:12:28.110 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4193280 len 2048 2024-05-08T06:12:28.110 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4193280 len 2048 2024-05-08T06:12:28.110 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4193280 len 2048 2024-05-08T06:12:28.110 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4193280 len 4096 2024-05-08T06:12:28.110 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4193280 len 4096 2024-05-08T06:12:28.110 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4193280 len 4096 2024-05-08T06:12:28.110 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4193280 len 4096 2024-05-08T06:12:28.110 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4193280 len 8192 2024-05-08T06:12:28.111 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4193280 len 8192 2024-05-08T06:12:28.111 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4193280 len 8192 2024-05-08T06:12:28.111 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4193280 len 8192 2024-05-08T06:12:28.111 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4193280 len 16384 2024-05-08T06:12:28.111 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4193280 len 16384 2024-05-08T06:12:28.111 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4193280 len 16384 2024-05-08T06:12:28.111 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4193280 len 16384 2024-05-08T06:12:28.111 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4193792 len 1024 2024-05-08T06:12:28.111 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4193792 len 1024 2024-05-08T06:12:28.111 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4193792 len 1024 2024-05-08T06:12:28.111 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4193792 len 1024 2024-05-08T06:12:28.111 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4193792 len 2048 2024-05-08T06:12:28.111 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4193792 len 2048 2024-05-08T06:12:28.111 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4193792 len 2048 2024-05-08T06:12:28.111 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4193792 len 2048 2024-05-08T06:12:28.111 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4193792 len 4096 2024-05-08T06:12:28.112 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4193792 len 4096 2024-05-08T06:12:28.112 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4193792 len 4096 2024-05-08T06:12:28.112 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4193792 len 4096 2024-05-08T06:12:28.112 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4193792 len 8192 2024-05-08T06:12:28.112 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4193792 len 8192 2024-05-08T06:12:28.112 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4193792 len 8192 2024-05-08T06:12:28.112 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4193792 len 8192 2024-05-08T06:12:28.112 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4193792 len 16384 2024-05-08T06:12:28.112 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4193792 len 16384 2024-05-08T06:12:28.112 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4193792 len 16384 2024-05-08T06:12:28.112 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4193792 len 16384 2024-05-08T06:12:28.112 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4194304 len 1024 2024-05-08T06:12:28.112 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4194304 len 1024 2024-05-08T06:12:28.112 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4194304 len 1024 2024-05-08T06:12:28.112 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4194304 len 1024 2024-05-08T06:12:28.112 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4194304 len 2048 2024-05-08T06:12:28.112 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4194304 len 2048 2024-05-08T06:12:28.113 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4194304 len 2048 2024-05-08T06:12:28.113 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4194304 len 2048 2024-05-08T06:12:28.113 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4194304 len 4096 2024-05-08T06:12:28.113 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4194304 len 4096 2024-05-08T06:12:28.113 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4194304 len 4096 2024-05-08T06:12:28.113 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4194304 len 4096 2024-05-08T06:12:28.113 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4194304 len 8192 2024-05-08T06:12:28.113 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4194304 len 8192 2024-05-08T06:12:28.113 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4194304 len 8192 2024-05-08T06:12:28.158 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4194304 len 8192 2024-05-08T06:12:28.159 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4194304 len 16384 2024-05-08T06:12:28.159 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4194304 len 16384 2024-05-08T06:12:28.159 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4194304 len 16384 2024-05-08T06:12:28.159 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4194304 len 16384 2024-05-08T06:12:28.159 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4194816 len 1024 2024-05-08T06:12:28.159 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4194816 len 1024 2024-05-08T06:12:28.159 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4194816 len 1024 2024-05-08T06:12:28.159 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4194816 len 1024 2024-05-08T06:12:28.159 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4194816 len 2048 2024-05-08T06:12:28.159 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4194816 len 2048 2024-05-08T06:12:28.159 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4194816 len 2048 2024-05-08T06:12:28.159 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4194816 len 2048 2024-05-08T06:12:28.159 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4194816 len 4096 2024-05-08T06:12:28.160 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4194816 len 4096 2024-05-08T06:12:28.160 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4194816 len 4096 2024-05-08T06:12:28.160 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4194816 len 4096 2024-05-08T06:12:28.160 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4194816 len 8192 2024-05-08T06:12:28.160 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4194816 len 8192 2024-05-08T06:12:28.160 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4194816 len 8192 2024-05-08T06:12:28.160 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4194816 len 8192 2024-05-08T06:12:28.160 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4194816 len 16384 2024-05-08T06:12:28.160 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4194816 len 16384 2024-05-08T06:12:28.160 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4194816 len 16384 2024-05-08T06:12:28.160 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4194816 len 16384 2024-05-08T06:12:28.160 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4195328 len 1024 2024-05-08T06:12:28.160 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4195328 len 1024 2024-05-08T06:12:28.160 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4195328 len 1024 2024-05-08T06:12:28.161 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4195328 len 1024 2024-05-08T06:12:28.161 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4195328 len 2048 2024-05-08T06:12:28.161 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4195328 len 2048 2024-05-08T06:12:28.161 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4195328 len 2048 2024-05-08T06:12:28.161 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4195328 len 2048 2024-05-08T06:12:28.161 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4195328 len 4096 2024-05-08T06:12:28.161 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4195328 len 4096 2024-05-08T06:12:28.161 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4195328 len 4096 2024-05-08T06:12:28.161 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4195328 len 4096 2024-05-08T06:12:28.161 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4195328 len 8192 2024-05-08T06:12:28.161 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4195328 len 8192 2024-05-08T06:12:28.162 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4195328 len 8192 2024-05-08T06:12:28.162 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4195328 len 8192 2024-05-08T06:12:28.162 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4195328 len 16384 2024-05-08T06:12:28.162 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4195328 len 16384 2024-05-08T06:12:28.162 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4195328 len 16384 2024-05-08T06:12:28.162 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4195328 len 16384 2024-05-08T06:12:28.162 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4195840 len 1024 2024-05-08T06:12:28.162 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4195840 len 1024 2024-05-08T06:12:28.162 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4195840 len 1024 2024-05-08T06:12:28.162 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4195840 len 1024 2024-05-08T06:12:28.162 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4195840 len 2048 2024-05-08T06:12:28.162 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4195840 len 2048 2024-05-08T06:12:28.162 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4195840 len 2048 2024-05-08T06:12:28.162 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4195840 len 2048 2024-05-08T06:12:28.163 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4195840 len 4096 2024-05-08T06:12:28.163 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4195840 len 4096 2024-05-08T06:12:28.163 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4195840 len 4096 2024-05-08T06:12:28.163 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4195840 len 4096 2024-05-08T06:12:28.163 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4195840 len 8192 2024-05-08T06:12:28.163 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4195840 len 8192 2024-05-08T06:12:28.163 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4195840 len 8192 2024-05-08T06:12:28.163 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4195840 len 8192 2024-05-08T06:12:28.163 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4195840 len 16384 2024-05-08T06:12:28.163 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4195840 len 16384 2024-05-08T06:12:28.163 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4195840 len 16384 2024-05-08T06:12:28.163 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4195840 len 16384 2024-05-08T06:12:28.163 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4196352 len 1024 2024-05-08T06:12:28.163 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4196352 len 1024 2024-05-08T06:12:28.164 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4196352 len 1024 2024-05-08T06:12:28.164 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4196352 len 1024 2024-05-08T06:12:28.164 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4196352 len 2048 2024-05-08T06:12:28.164 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4196352 len 2048 2024-05-08T06:12:28.164 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4196352 len 2048 2024-05-08T06:12:28.164 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4196352 len 2048 2024-05-08T06:12:28.164 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4196352 len 4096 2024-05-08T06:12:28.164 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4196352 len 4096 2024-05-08T06:12:28.164 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4196352 len 4096 2024-05-08T06:12:28.164 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4196352 len 4096 2024-05-08T06:12:28.164 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4196352 len 8192 2024-05-08T06:12:28.164 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4196352 len 8192 2024-05-08T06:12:28.164 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4196352 len 8192 2024-05-08T06:12:28.164 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4196352 len 8192 2024-05-08T06:12:28.165 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4196352 len 16384 2024-05-08T06:12:28.165 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4196352 len 16384 2024-05-08T06:12:28.165 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4196352 len 16384 2024-05-08T06:12:28.165 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4196352 len 16384 2024-05-08T06:12:28.165 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4196864 len 1024 2024-05-08T06:12:28.165 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4196864 len 1024 2024-05-08T06:12:28.206 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4196864 len 1024 2024-05-08T06:12:28.206 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4196864 len 1024 2024-05-08T06:12:28.206 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4196864 len 2048 2024-05-08T06:12:28.206 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4196864 len 2048 2024-05-08T06:12:28.206 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4196864 len 2048 2024-05-08T06:12:28.206 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4196864 len 2048 2024-05-08T06:12:28.206 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4196864 len 4096 2024-05-08T06:12:28.206 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4196864 len 4096 2024-05-08T06:12:28.206 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4196864 len 4096 2024-05-08T06:12:28.206 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4196864 len 4096 2024-05-08T06:12:28.207 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4196864 len 8192 2024-05-08T06:12:28.207 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4196864 len 8192 2024-05-08T06:12:28.207 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4196864 len 8192 2024-05-08T06:12:28.207 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4196864 len 8192 2024-05-08T06:12:28.207 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4196864 len 16384 2024-05-08T06:12:28.207 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4196864 len 16384 2024-05-08T06:12:28.207 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4196864 len 16384 2024-05-08T06:12:28.207 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4196864 len 16384 2024-05-08T06:12:28.207 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4197376 len 1024 2024-05-08T06:12:28.207 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4197376 len 1024 2024-05-08T06:12:28.207 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4197376 len 1024 2024-05-08T06:12:28.207 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4197376 len 1024 2024-05-08T06:12:28.207 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4197376 len 2048 2024-05-08T06:12:28.207 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4197376 len 2048 2024-05-08T06:12:28.207 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4197376 len 2048 2024-05-08T06:12:28.207 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4197376 len 2048 2024-05-08T06:12:28.208 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4197376 len 4096 2024-05-08T06:12:28.208 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4197376 len 4096 2024-05-08T06:12:28.208 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4197376 len 4096 2024-05-08T06:12:28.208 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4197376 len 4096 2024-05-08T06:12:28.208 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4197376 len 8192 2024-05-08T06:12:28.208 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4197376 len 8192 2024-05-08T06:12:28.208 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4197376 len 8192 2024-05-08T06:12:28.208 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4197376 len 8192 2024-05-08T06:12:28.208 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4197376 len 16384 2024-05-08T06:12:28.208 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4197376 len 16384 2024-05-08T06:12:28.208 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4197376 len 16384 2024-05-08T06:12:28.208 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4197376 len 16384 2024-05-08T06:12:28.208 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4197888 len 1024 2024-05-08T06:12:28.208 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4197888 len 1024 2024-05-08T06:12:28.208 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4197888 len 1024 2024-05-08T06:12:28.209 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4197888 len 1024 2024-05-08T06:12:28.209 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4197888 len 2048 2024-05-08T06:12:28.209 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4197888 len 2048 2024-05-08T06:12:28.209 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4197888 len 2048 2024-05-08T06:12:28.209 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4197888 len 2048 2024-05-08T06:12:28.209 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4197888 len 4096 2024-05-08T06:12:28.209 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4197888 len 4096 2024-05-08T06:12:28.209 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4197888 len 4096 2024-05-08T06:12:28.209 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4197888 len 4096 2024-05-08T06:12:28.209 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4197888 len 8192 2024-05-08T06:12:28.209 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4197888 len 8192 2024-05-08T06:12:28.209 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4197888 len 8192 2024-05-08T06:12:28.209 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4197888 len 8192 2024-05-08T06:12:28.209 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 0 offset 4197888 len 16384 2024-05-08T06:12:28.209 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4197888 len 16384 2024-05-08T06:12:28.210 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 0 offset 4197888 len 16384 2024-05-08T06:12:28.210 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 0 offset 4197888 len 16384 2024-05-08T06:12:28.210 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4190208 len 1024 2024-05-08T06:12:28.210 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4190208 len 1024 2024-05-08T06:12:28.210 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4190208 len 1024 2024-05-08T06:12:28.210 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4190208 len 1024 2024-05-08T06:12:28.210 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4190208 len 2048 2024-05-08T06:12:28.210 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4190208 len 2048 2024-05-08T06:12:28.210 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4190208 len 2048 2024-05-08T06:12:28.210 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4190208 len 2048 2024-05-08T06:12:28.210 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4190208 len 4096 2024-05-08T06:12:28.210 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4190208 len 4096 2024-05-08T06:12:28.210 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4190208 len 4096 2024-05-08T06:12:28.210 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4190208 len 4096 2024-05-08T06:12:28.210 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4190208 len 8192 2024-05-08T06:12:28.211 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4190208 len 8192 2024-05-08T06:12:28.211 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4190208 len 8192 2024-05-08T06:12:28.211 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4190208 len 8192 2024-05-08T06:12:28.211 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4190208 len 16384 2024-05-08T06:12:28.211 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4190208 len 16384 2024-05-08T06:12:28.211 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4190208 len 16384 2024-05-08T06:12:28.211 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4190208 len 16384 2024-05-08T06:12:28.211 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4190720 len 1024 2024-05-08T06:12:28.211 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4190720 len 1024 2024-05-08T06:12:28.211 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4190720 len 1024 2024-05-08T06:12:28.211 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4190720 len 1024 2024-05-08T06:12:28.211 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4190720 len 2048 2024-05-08T06:12:28.211 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4190720 len 2048 2024-05-08T06:12:28.211 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4190720 len 2048 2024-05-08T06:12:28.211 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4190720 len 2048 2024-05-08T06:12:28.212 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4190720 len 4096 2024-05-08T06:12:28.272 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4190720 len 4096 2024-05-08T06:12:28.272 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4190720 len 4096 2024-05-08T06:12:28.272 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4190720 len 4096 2024-05-08T06:12:28.272 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4190720 len 8192 2024-05-08T06:12:28.272 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4190720 len 8192 2024-05-08T06:12:28.273 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4190720 len 8192 2024-05-08T06:12:28.273 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4190720 len 8192 2024-05-08T06:12:28.273 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4190720 len 16384 2024-05-08T06:12:28.273 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4190720 len 16384 2024-05-08T06:12:28.273 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4190720 len 16384 2024-05-08T06:12:28.273 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4190720 len 16384 2024-05-08T06:12:28.273 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4191232 len 1024 2024-05-08T06:12:28.273 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4191232 len 1024 2024-05-08T06:12:28.273 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4191232 len 1024 2024-05-08T06:12:28.273 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4191232 len 1024 2024-05-08T06:12:28.273 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4191232 len 2048 2024-05-08T06:12:28.273 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4191232 len 2048 2024-05-08T06:12:28.273 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4191232 len 2048 2024-05-08T06:12:28.273 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4191232 len 2048 2024-05-08T06:12:28.273 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4191232 len 4096 2024-05-08T06:12:28.273 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4191232 len 4096 2024-05-08T06:12:28.274 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4191232 len 4096 2024-05-08T06:12:28.274 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4191232 len 4096 2024-05-08T06:12:28.274 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4191232 len 8192 2024-05-08T06:12:28.274 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4191232 len 8192 2024-05-08T06:12:28.274 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4191232 len 8192 2024-05-08T06:12:28.274 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4191232 len 8192 2024-05-08T06:12:28.274 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4191232 len 16384 2024-05-08T06:12:28.274 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4191232 len 16384 2024-05-08T06:12:28.274 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4191232 len 16384 2024-05-08T06:12:28.274 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4191232 len 16384 2024-05-08T06:12:28.274 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4191744 len 1024 2024-05-08T06:12:28.274 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4191744 len 1024 2024-05-08T06:12:28.274 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4191744 len 1024 2024-05-08T06:12:28.274 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4191744 len 1024 2024-05-08T06:12:28.274 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4191744 len 2048 2024-05-08T06:12:28.274 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4191744 len 2048 2024-05-08T06:12:28.275 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4191744 len 2048 2024-05-08T06:12:28.275 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4191744 len 2048 2024-05-08T06:12:28.275 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4191744 len 4096 2024-05-08T06:12:28.275 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4191744 len 4096 2024-05-08T06:12:28.275 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4191744 len 4096 2024-05-08T06:12:28.275 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4191744 len 4096 2024-05-08T06:12:28.275 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4191744 len 8192 2024-05-08T06:12:28.275 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4191744 len 8192 2024-05-08T06:12:28.275 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4191744 len 8192 2024-05-08T06:12:28.275 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4191744 len 8192 2024-05-08T06:12:28.275 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4191744 len 16384 2024-05-08T06:12:28.275 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4191744 len 16384 2024-05-08T06:12:28.275 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4191744 len 16384 2024-05-08T06:12:28.275 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4191744 len 16384 2024-05-08T06:12:28.275 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4192256 len 1024 2024-05-08T06:12:28.276 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4192256 len 1024 2024-05-08T06:12:28.276 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4192256 len 1024 2024-05-08T06:12:28.276 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4192256 len 1024 2024-05-08T06:12:28.276 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4192256 len 2048 2024-05-08T06:12:28.276 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4192256 len 2048 2024-05-08T06:12:28.276 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4192256 len 2048 2024-05-08T06:12:28.276 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4192256 len 2048 2024-05-08T06:12:28.276 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4192256 len 4096 2024-05-08T06:12:28.276 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4192256 len 4096 2024-05-08T06:12:28.276 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4192256 len 4096 2024-05-08T06:12:28.276 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4192256 len 4096 2024-05-08T06:12:28.276 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4192256 len 8192 2024-05-08T06:12:28.276 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4192256 len 8192 2024-05-08T06:12:28.276 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4192256 len 8192 2024-05-08T06:12:28.276 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4192256 len 8192 2024-05-08T06:12:28.277 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4192256 len 16384 2024-05-08T06:12:28.277 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4192256 len 16384 2024-05-08T06:12:28.277 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4192256 len 16384 2024-05-08T06:12:28.277 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4192256 len 16384 2024-05-08T06:12:28.277 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4192768 len 1024 2024-05-08T06:12:28.277 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4192768 len 1024 2024-05-08T06:12:28.277 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4192768 len 1024 2024-05-08T06:12:28.277 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4192768 len 1024 2024-05-08T06:12:28.277 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4192768 len 2048 2024-05-08T06:12:28.277 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4192768 len 2048 2024-05-08T06:12:28.277 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4192768 len 2048 2024-05-08T06:12:28.277 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4192768 len 2048 2024-05-08T06:12:28.277 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4192768 len 4096 2024-05-08T06:12:28.277 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4192768 len 4096 2024-05-08T06:12:28.278 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4192768 len 4096 2024-05-08T06:12:28.278 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4192768 len 4096 2024-05-08T06:12:28.278 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4192768 len 8192 2024-05-08T06:12:28.338 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4192768 len 8192 2024-05-08T06:12:28.338 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4192768 len 8192 2024-05-08T06:12:28.339 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4192768 len 8192 2024-05-08T06:12:28.339 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4192768 len 16384 2024-05-08T06:12:28.339 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4192768 len 16384 2024-05-08T06:12:28.339 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4192768 len 16384 2024-05-08T06:12:28.339 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4192768 len 16384 2024-05-08T06:12:28.339 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4193280 len 1024 2024-05-08T06:12:28.339 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4193280 len 1024 2024-05-08T06:12:28.339 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4193280 len 1024 2024-05-08T06:12:28.339 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4193280 len 1024 2024-05-08T06:12:28.339 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4193280 len 2048 2024-05-08T06:12:28.339 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4193280 len 2048 2024-05-08T06:12:28.339 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4193280 len 2048 2024-05-08T06:12:28.339 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4193280 len 2048 2024-05-08T06:12:28.339 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4193280 len 4096 2024-05-08T06:12:28.339 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4193280 len 4096 2024-05-08T06:12:28.340 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4193280 len 4096 2024-05-08T06:12:28.340 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4193280 len 4096 2024-05-08T06:12:28.340 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4193280 len 8192 2024-05-08T06:12:28.340 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4193280 len 8192 2024-05-08T06:12:28.340 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4193280 len 8192 2024-05-08T06:12:28.340 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4193280 len 8192 2024-05-08T06:12:28.340 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4193280 len 16384 2024-05-08T06:12:28.340 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4193280 len 16384 2024-05-08T06:12:28.340 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4193280 len 16384 2024-05-08T06:12:28.340 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4193280 len 16384 2024-05-08T06:12:28.340 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4193792 len 1024 2024-05-08T06:12:28.340 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4193792 len 1024 2024-05-08T06:12:28.340 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4193792 len 1024 2024-05-08T06:12:28.340 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4193792 len 1024 2024-05-08T06:12:28.341 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4193792 len 2048 2024-05-08T06:12:28.341 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4193792 len 2048 2024-05-08T06:12:28.341 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4193792 len 2048 2024-05-08T06:12:28.341 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4193792 len 2048 2024-05-08T06:12:28.341 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4193792 len 4096 2024-05-08T06:12:28.341 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4193792 len 4096 2024-05-08T06:12:28.341 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4193792 len 4096 2024-05-08T06:12:28.341 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4193792 len 4096 2024-05-08T06:12:28.341 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4193792 len 8192 2024-05-08T06:12:28.341 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4193792 len 8192 2024-05-08T06:12:28.341 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4193792 len 8192 2024-05-08T06:12:28.341 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4193792 len 8192 2024-05-08T06:12:28.341 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4193792 len 16384 2024-05-08T06:12:28.341 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4193792 len 16384 2024-05-08T06:12:28.341 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4193792 len 16384 2024-05-08T06:12:28.341 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4193792 len 16384 2024-05-08T06:12:28.342 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4194304 len 1024 2024-05-08T06:12:28.342 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4194304 len 1024 2024-05-08T06:12:28.342 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4194304 len 1024 2024-05-08T06:12:28.342 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4194304 len 1024 2024-05-08T06:12:28.342 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4194304 len 2048 2024-05-08T06:12:28.342 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4194304 len 2048 2024-05-08T06:12:28.342 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4194304 len 2048 2024-05-08T06:12:28.342 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4194304 len 2048 2024-05-08T06:12:28.342 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4194304 len 4096 2024-05-08T06:12:28.342 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4194304 len 4096 2024-05-08T06:12:28.342 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4194304 len 4096 2024-05-08T06:12:28.342 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4194304 len 4096 2024-05-08T06:12:28.342 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4194304 len 8192 2024-05-08T06:12:28.342 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4194304 len 8192 2024-05-08T06:12:28.342 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4194304 len 8192 2024-05-08T06:12:28.343 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4194304 len 8192 2024-05-08T06:12:28.343 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4194304 len 16384 2024-05-08T06:12:28.343 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4194304 len 16384 2024-05-08T06:12:28.343 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4194304 len 16384 2024-05-08T06:12:28.343 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4194304 len 16384 2024-05-08T06:12:28.343 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4194816 len 1024 2024-05-08T06:12:28.343 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4194816 len 1024 2024-05-08T06:12:28.343 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4194816 len 1024 2024-05-08T06:12:28.343 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4194816 len 1024 2024-05-08T06:12:28.343 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4194816 len 2048 2024-05-08T06:12:28.343 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4194816 len 2048 2024-05-08T06:12:28.343 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4194816 len 2048 2024-05-08T06:12:28.343 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4194816 len 2048 2024-05-08T06:12:28.343 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4194816 len 4096 2024-05-08T06:12:28.343 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4194816 len 4096 2024-05-08T06:12:28.344 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4194816 len 4096 2024-05-08T06:12:28.344 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4194816 len 4096 2024-05-08T06:12:28.344 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4194816 len 8192 2024-05-08T06:12:28.344 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4194816 len 8192 2024-05-08T06:12:28.344 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4194816 len 8192 2024-05-08T06:12:28.344 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4194816 len 8192 2024-05-08T06:12:28.344 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4194816 len 16384 2024-05-08T06:12:28.391 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4194816 len 16384 2024-05-08T06:12:28.391 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4194816 len 16384 2024-05-08T06:12:28.391 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4194816 len 16384 2024-05-08T06:12:28.391 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4195328 len 1024 2024-05-08T06:12:28.392 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4195328 len 1024 2024-05-08T06:12:28.392 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4195328 len 1024 2024-05-08T06:12:28.392 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4195328 len 1024 2024-05-08T06:12:28.392 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4195328 len 2048 2024-05-08T06:12:28.392 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4195328 len 2048 2024-05-08T06:12:28.392 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4195328 len 2048 2024-05-08T06:12:28.392 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4195328 len 2048 2024-05-08T06:12:28.392 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4195328 len 4096 2024-05-08T06:12:28.393 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4195328 len 4096 2024-05-08T06:12:28.393 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4195328 len 4096 2024-05-08T06:12:28.393 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4195328 len 4096 2024-05-08T06:12:28.393 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4195328 len 8192 2024-05-08T06:12:28.393 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4195328 len 8192 2024-05-08T06:12:28.393 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4195328 len 8192 2024-05-08T06:12:28.393 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4195328 len 8192 2024-05-08T06:12:28.393 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4195328 len 16384 2024-05-08T06:12:28.393 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4195328 len 16384 2024-05-08T06:12:28.394 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4195328 len 16384 2024-05-08T06:12:28.394 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4195328 len 16384 2024-05-08T06:12:28.394 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4195840 len 1024 2024-05-08T06:12:28.394 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4195840 len 1024 2024-05-08T06:12:28.394 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4195840 len 1024 2024-05-08T06:12:28.394 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4195840 len 1024 2024-05-08T06:12:28.394 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4195840 len 2048 2024-05-08T06:12:28.394 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4195840 len 2048 2024-05-08T06:12:28.395 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4195840 len 2048 2024-05-08T06:12:28.395 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4195840 len 2048 2024-05-08T06:12:28.395 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4195840 len 4096 2024-05-08T06:12:28.395 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4195840 len 4096 2024-05-08T06:12:28.395 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4195840 len 4096 2024-05-08T06:12:28.395 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4195840 len 4096 2024-05-08T06:12:28.395 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4195840 len 8192 2024-05-08T06:12:28.395 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4195840 len 8192 2024-05-08T06:12:28.395 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4195840 len 8192 2024-05-08T06:12:28.395 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4195840 len 8192 2024-05-08T06:12:28.396 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4195840 len 16384 2024-05-08T06:12:28.396 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4195840 len 16384 2024-05-08T06:12:28.396 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4195840 len 16384 2024-05-08T06:12:28.396 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4195840 len 16384 2024-05-08T06:12:28.396 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4196352 len 1024 2024-05-08T06:12:28.397 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4196352 len 1024 2024-05-08T06:12:28.397 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4196352 len 1024 2024-05-08T06:12:28.397 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4196352 len 1024 2024-05-08T06:12:28.397 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4196352 len 2048 2024-05-08T06:12:28.397 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4196352 len 2048 2024-05-08T06:12:28.397 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4196352 len 2048 2024-05-08T06:12:28.397 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4196352 len 2048 2024-05-08T06:12:28.397 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4196352 len 4096 2024-05-08T06:12:28.398 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4196352 len 4096 2024-05-08T06:12:28.398 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4196352 len 4096 2024-05-08T06:12:28.398 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4196352 len 4096 2024-05-08T06:12:28.398 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4196352 len 8192 2024-05-08T06:12:28.398 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4196352 len 8192 2024-05-08T06:12:28.398 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4196352 len 8192 2024-05-08T06:12:28.398 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4196352 len 8192 2024-05-08T06:12:28.398 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4196352 len 16384 2024-05-08T06:12:28.399 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4196352 len 16384 2024-05-08T06:12:28.399 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4196352 len 16384 2024-05-08T06:12:28.399 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4196352 len 16384 2024-05-08T06:12:28.399 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4196864 len 1024 2024-05-08T06:12:28.399 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4196864 len 1024 2024-05-08T06:12:28.399 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4196864 len 1024 2024-05-08T06:12:28.399 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4196864 len 1024 2024-05-08T06:12:28.399 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4196864 len 2048 2024-05-08T06:12:28.400 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4196864 len 2048 2024-05-08T06:12:28.400 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4196864 len 2048 2024-05-08T06:12:28.400 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4196864 len 2048 2024-05-08T06:12:28.400 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4196864 len 4096 2024-05-08T06:12:28.400 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4196864 len 4096 2024-05-08T06:12:28.400 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4196864 len 4096 2024-05-08T06:12:28.400 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4196864 len 4096 2024-05-08T06:12:28.400 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4196864 len 8192 2024-05-08T06:12:28.400 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4196864 len 8192 2024-05-08T06:12:28.400 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4196864 len 8192 2024-05-08T06:12:28.401 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4196864 len 8192 2024-05-08T06:12:28.401 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4196864 len 16384 2024-05-08T06:12:28.401 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4196864 len 16384 2024-05-08T06:12:28.401 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4196864 len 16384 2024-05-08T06:12:28.401 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4196864 len 16384 2024-05-08T06:12:28.401 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4197376 len 1024 2024-05-08T06:12:28.437 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4197376 len 1024 2024-05-08T06:12:28.437 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4197376 len 1024 2024-05-08T06:12:28.437 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4197376 len 1024 2024-05-08T06:12:28.438 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4197376 len 2048 2024-05-08T06:12:28.438 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4197376 len 2048 2024-05-08T06:12:28.438 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4197376 len 2048 2024-05-08T06:12:28.438 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4197376 len 2048 2024-05-08T06:12:28.438 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4197376 len 4096 2024-05-08T06:12:28.438 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4197376 len 4096 2024-05-08T06:12:28.438 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4197376 len 4096 2024-05-08T06:12:28.438 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4197376 len 4096 2024-05-08T06:12:28.438 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4197376 len 8192 2024-05-08T06:12:28.438 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4197376 len 8192 2024-05-08T06:12:28.439 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4197376 len 8192 2024-05-08T06:12:28.439 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4197376 len 8192 2024-05-08T06:12:28.439 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4197376 len 16384 2024-05-08T06:12:28.439 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4197376 len 16384 2024-05-08T06:12:28.439 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4197376 len 16384 2024-05-08T06:12:28.439 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4197376 len 16384 2024-05-08T06:12:28.439 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4197888 len 1024 2024-05-08T06:12:28.440 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4197888 len 1024 2024-05-08T06:12:28.440 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4197888 len 1024 2024-05-08T06:12:28.440 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4197888 len 1024 2024-05-08T06:12:28.440 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4197888 len 2048 2024-05-08T06:12:28.440 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4197888 len 2048 2024-05-08T06:12:28.440 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4197888 len 2048 2024-05-08T06:12:28.441 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4197888 len 2048 2024-05-08T06:12:28.441 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4197888 len 4096 2024-05-08T06:12:28.441 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4197888 len 4096 2024-05-08T06:12:28.441 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4197888 len 4096 2024-05-08T06:12:28.441 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4197888 len 4096 2024-05-08T06:12:28.441 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4197888 len 8192 2024-05-08T06:12:28.441 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4197888 len 8192 2024-05-08T06:12:28.441 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4197888 len 8192 2024-05-08T06:12:28.442 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4197888 len 8192 2024-05-08T06:12:28.442 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 512 offset 4197888 len 16384 2024-05-08T06:12:28.442 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4197888 len 16384 2024-05-08T06:12:28.442 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 512 offset 4197888 len 16384 2024-05-08T06:12:28.442 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 512 offset 4197888 len 16384 2024-05-08T06:12:28.442 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4190208 len 1024 2024-05-08T06:12:28.442 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4190208 len 1024 2024-05-08T06:12:28.443 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4190208 len 1024 2024-05-08T06:12:28.443 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4190208 len 1024 2024-05-08T06:12:28.443 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4190208 len 2048 2024-05-08T06:12:28.443 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4190208 len 2048 2024-05-08T06:12:28.443 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4190208 len 2048 2024-05-08T06:12:28.443 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4190208 len 2048 2024-05-08T06:12:28.443 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4190208 len 4096 2024-05-08T06:12:28.444 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4190208 len 4096 2024-05-08T06:12:28.444 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4190208 len 4096 2024-05-08T06:12:28.444 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4190208 len 4096 2024-05-08T06:12:28.444 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4190208 len 8192 2024-05-08T06:12:28.444 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4190208 len 8192 2024-05-08T06:12:28.444 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4190208 len 8192 2024-05-08T06:12:28.445 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4190208 len 8192 2024-05-08T06:12:28.445 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4190208 len 16384 2024-05-08T06:12:28.445 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4190208 len 16384 2024-05-08T06:12:28.445 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4190208 len 16384 2024-05-08T06:12:28.445 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4190208 len 16384 2024-05-08T06:12:28.445 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4190720 len 1024 2024-05-08T06:12:28.446 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4190720 len 1024 2024-05-08T06:12:28.446 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4190720 len 1024 2024-05-08T06:12:28.446 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4190720 len 1024 2024-05-08T06:12:28.446 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4190720 len 2048 2024-05-08T06:12:28.446 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4190720 len 2048 2024-05-08T06:12:28.446 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4190720 len 2048 2024-05-08T06:12:28.446 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4190720 len 2048 2024-05-08T06:12:28.446 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4190720 len 4096 2024-05-08T06:12:28.446 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4190720 len 4096 2024-05-08T06:12:28.446 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4190720 len 4096 2024-05-08T06:12:28.446 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4190720 len 4096 2024-05-08T06:12:28.447 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4190720 len 8192 2024-05-08T06:12:28.447 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4190720 len 8192 2024-05-08T06:12:28.447 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4190720 len 8192 2024-05-08T06:12:28.447 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4190720 len 8192 2024-05-08T06:12:28.447 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4190720 len 16384 2024-05-08T06:12:28.447 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4190720 len 16384 2024-05-08T06:12:28.447 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4190720 len 16384 2024-05-08T06:12:28.447 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4190720 len 16384 2024-05-08T06:12:28.448 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4191232 len 1024 2024-05-08T06:12:28.448 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4191232 len 1024 2024-05-08T06:12:28.448 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4191232 len 1024 2024-05-08T06:12:28.448 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4191232 len 1024 2024-05-08T06:12:28.498 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4191232 len 2048 2024-05-08T06:12:28.498 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4191232 len 2048 2024-05-08T06:12:28.498 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4191232 len 2048 2024-05-08T06:12:28.498 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4191232 len 2048 2024-05-08T06:12:28.498 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4191232 len 4096 2024-05-08T06:12:28.498 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4191232 len 4096 2024-05-08T06:12:28.498 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4191232 len 4096 2024-05-08T06:12:28.499 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4191232 len 4096 2024-05-08T06:12:28.499 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4191232 len 8192 2024-05-08T06:12:28.499 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4191232 len 8192 2024-05-08T06:12:28.499 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4191232 len 8192 2024-05-08T06:12:28.499 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4191232 len 8192 2024-05-08T06:12:28.499 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4191232 len 16384 2024-05-08T06:12:28.499 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4191232 len 16384 2024-05-08T06:12:28.499 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4191232 len 16384 2024-05-08T06:12:28.499 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4191232 len 16384 2024-05-08T06:12:28.499 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4191744 len 1024 2024-05-08T06:12:28.499 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4191744 len 1024 2024-05-08T06:12:28.499 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4191744 len 1024 2024-05-08T06:12:28.499 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4191744 len 1024 2024-05-08T06:12:28.500 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4191744 len 2048 2024-05-08T06:12:28.500 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4191744 len 2048 2024-05-08T06:12:28.500 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4191744 len 2048 2024-05-08T06:12:28.500 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4191744 len 2048 2024-05-08T06:12:28.500 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4191744 len 4096 2024-05-08T06:12:28.500 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4191744 len 4096 2024-05-08T06:12:28.500 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4191744 len 4096 2024-05-08T06:12:28.500 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4191744 len 4096 2024-05-08T06:12:28.501 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4191744 len 8192 2024-05-08T06:12:28.501 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4191744 len 8192 2024-05-08T06:12:28.501 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4191744 len 8192 2024-05-08T06:12:28.501 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4191744 len 8192 2024-05-08T06:12:28.501 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4191744 len 16384 2024-05-08T06:12:28.501 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4191744 len 16384 2024-05-08T06:12:28.501 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4191744 len 16384 2024-05-08T06:12:28.502 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4191744 len 16384 2024-05-08T06:12:28.503 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4192256 len 1024 2024-05-08T06:12:28.503 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4192256 len 1024 2024-05-08T06:12:28.503 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4192256 len 1024 2024-05-08T06:12:28.503 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4192256 len 1024 2024-05-08T06:12:28.503 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4192256 len 2048 2024-05-08T06:12:28.503 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4192256 len 2048 2024-05-08T06:12:28.504 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4192256 len 2048 2024-05-08T06:12:28.504 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4192256 len 2048 2024-05-08T06:12:28.504 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4192256 len 4096 2024-05-08T06:12:28.504 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4192256 len 4096 2024-05-08T06:12:28.504 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4192256 len 4096 2024-05-08T06:12:28.504 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4192256 len 4096 2024-05-08T06:12:28.504 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4192256 len 8192 2024-05-08T06:12:28.505 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4192256 len 8192 2024-05-08T06:12:28.505 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4192256 len 8192 2024-05-08T06:12:28.505 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4192256 len 8192 2024-05-08T06:12:28.505 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4192256 len 16384 2024-05-08T06:12:28.505 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4192256 len 16384 2024-05-08T06:12:28.505 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4192256 len 16384 2024-05-08T06:12:28.505 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4192256 len 16384 2024-05-08T06:12:28.506 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4192768 len 1024 2024-05-08T06:12:28.506 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4192768 len 1024 2024-05-08T06:12:28.506 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4192768 len 1024 2024-05-08T06:12:28.506 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4192768 len 1024 2024-05-08T06:12:28.506 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4192768 len 2048 2024-05-08T06:12:28.506 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4192768 len 2048 2024-05-08T06:12:28.507 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4192768 len 2048 2024-05-08T06:12:28.507 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4192768 len 2048 2024-05-08T06:12:28.507 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4192768 len 4096 2024-05-08T06:12:28.507 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4192768 len 4096 2024-05-08T06:12:28.507 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4192768 len 4096 2024-05-08T06:12:28.507 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4192768 len 4096 2024-05-08T06:12:28.507 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4192768 len 8192 2024-05-08T06:12:28.508 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4192768 len 8192 2024-05-08T06:12:28.508 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4192768 len 8192 2024-05-08T06:12:28.508 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4192768 len 8192 2024-05-08T06:12:28.508 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4192768 len 16384 2024-05-08T06:12:28.508 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4192768 len 16384 2024-05-08T06:12:28.508 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4192768 len 16384 2024-05-08T06:12:28.508 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4192768 len 16384 2024-05-08T06:12:28.509 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4193280 len 1024 2024-05-08T06:12:28.509 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4193280 len 1024 2024-05-08T06:12:28.509 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4193280 len 1024 2024-05-08T06:12:28.509 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4193280 len 1024 2024-05-08T06:12:28.509 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4193280 len 2048 2024-05-08T06:12:28.509 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4193280 len 2048 2024-05-08T06:12:28.509 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4193280 len 2048 2024-05-08T06:12:28.523 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.500+0000 7fda05200640 1 -- 172.21.15.123:0/2904022039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda00104420 msgr2=0x7fda0010acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:28.523 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.500+0000 7fda05200640 1 --2- 172.21.15.123:0/2904022039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda00104420 0x7fda0010acb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0009900 tx=0x7fd9f002a3f0 comp rx=0 tx=0).stop 2024-05-08T06:12:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.500+0000 7fda05200640 1 -- 172.21.15.123:0/2904022039 shutdown_connections 2024-05-08T06:12:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.500+0000 7fda05200640 1 --2- 172.21.15.123:0/2904022039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda00104420 0x7fda0010acb0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.500+0000 7fda05200640 1 --2- 172.21.15.123:0/2904022039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda00103a60 0x7fda00103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.500+0000 7fda05200640 1 --2- 172.21.15.123:0/2904022039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda00102870 0x7fda00102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.500+0000 7fda05200640 1 -- 172.21.15.123:0/2904022039 >> 172.21.15.123:0/2904022039 conn(0x7fda000fdde0 msgr2=0x7fda00100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.500+0000 7fda05200640 1 -- 172.21.15.123:0/2904022039 shutdown_connections 2024-05-08T06:12:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.500+0000 7fda05200640 1 -- 172.21.15.123:0/2904022039 wait complete. 2024-05-08T06:12:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.501+0000 7fda05200640 1 Processor -- start 2024-05-08T06:12:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.501+0000 7fda05200640 1 -- start start 2024-05-08T06:12:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.501+0000 7fda05200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda00102870 0x7fda0019ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.501+0000 7fda05200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda00103a60 0x7fda0019b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.501+0000 7fda05200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda00104420 0x7fda001a2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.501+0000 7fda05200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fda0010e3a0 con 0x7fda00103a60 2024-05-08T06:12:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.501+0000 7fda05200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fda0010e220 con 0x7fda00104420 2024-05-08T06:12:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.501+0000 7fda05200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fda0010e520 con 0x7fda00102870 2024-05-08T06:12:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.502+0000 7fd9ff400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda00102870 0x7fda0019ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.502+0000 7fd9ff400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda00102870 0x7fda0019ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:46844/0 (socket says 172.21.15.123:46844) 2024-05-08T06:12:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.502+0000 7fd9ff400640 1 -- 172.21.15.123:0/4094049630 learned_addr learned my addr 172.21.15.123:0/4094049630 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.502+0000 7fd9fea00640 1 --2- 172.21.15.123:0/4094049630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda00103a60 0x7fda0019b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.502+0000 7fd9ffe00640 1 --2- 172.21.15.123:0/4094049630 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda00104420 0x7fda001a2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.502+0000 7fd9ff400640 1 -- 172.21.15.123:0/4094049630 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda00104420 msgr2=0x7fda001a2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.502+0000 7fd9ff400640 1 --2- 172.21.15.123:0/4094049630 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda00104420 0x7fda001a2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.502+0000 7fd9ff400640 1 -- 172.21.15.123:0/4094049630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda00103a60 msgr2=0x7fda0019b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.502+0000 7fd9ff400640 1 --2- 172.21.15.123:0/4094049630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda00103a60 0x7fda0019b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.502+0000 7fd9ff400640 1 -- 172.21.15.123:0/4094049630 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda001a28b0 con 0x7fda00102870 2024-05-08T06:12:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.502+0000 7fd9fea00640 1 --2- 172.21.15.123:0/4094049630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda00103a60 0x7fda0019b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:12:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.502+0000 7fd9ff400640 1 --2- 172.21.15.123:0/4094049630 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda00102870 0x7fda0019ac70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009fd0 tx=0x7fd9f800c270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.503+0000 7fd9f7e00640 1 -- 172.21.15.123:0/4094049630 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8011470 con 0x7fda00102870 2024-05-08T06:12:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.503+0000 7fd9f7e00640 1 -- 172.21.15.123:0/4094049630 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9f8011ac0 con 0x7fda00102870 2024-05-08T06:12:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.503+0000 7fd9f7e00640 1 -- 172.21.15.123:0/4094049630 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f801aab0 con 0x7fda00102870 2024-05-08T06:12:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.503+0000 7fda05200640 1 -- 172.21.15.123:0/4094049630 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda001a2ba0 con 0x7fda00102870 2024-05-08T06:12:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.503+0000 7fda05200640 1 -- 172.21.15.123:0/4094049630 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda001a30a0 con 0x7fda00102870 2024-05-08T06:12:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.507+0000 7fda0967f740 1 Processor -- start 2024-05-08T06:12:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.507+0000 7fda0967f740 1 -- start start 2024-05-08T06:12:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.508+0000 7fda0967f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e44cad3320 0x55e44cad3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.508+0000 7fda0967f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e44cad3d10 0x55e44cad8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.508+0000 7fda0967f740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e44cad8e80 0x55e44cadb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.508+0000 7fda0967f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e44ca37550 con 0x55e44cad3d10 2024-05-08T06:12:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.508+0000 7fd9f7e00640 1 -- 172.21.15.123:0/4094049630 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7fd9f801ac50 con 0x7fda00102870 2024-05-08T06:12:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.508+0000 7fda0967f740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55e44c9dcbc0 con 0x55e44cad8e80 2024-05-08T06:12:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.508+0000 7fda0967f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55e44ca33510 con 0x55e44cad3320 2024-05-08T06:12:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.508+0000 7fd9ffe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e44cad3320 0x55e44cad3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.508+0000 7fd9ffe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e44cad3320 0x55e44cad3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:46858/0 (socket says 172.21.15.123:46858) 2024-05-08T06:12:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.508+0000 7fd9ff400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e44cad3d10 0x55e44cad8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.508+0000 7fd9ffe00640 1 -- 172.21.15.123:0/200480819 learned_addr learned my addr 172.21.15.123:0/200480819 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.508+0000 7fd9fea00640 1 --2- 172.21.15.123:0/200480819 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e44cad8e80 0x55e44cadb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.508+0000 7fd9ffe00640 1 -- 172.21.15.123:0/200480819 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e44cad8e80 msgr2=0x55e44cadb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.508+0000 7fd9ffe00640 1 --2- 172.21.15.123:0/200480819 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e44cad8e80 0x55e44cadb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:28.530 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.508+0000 7fd9ffe00640 1 -- 172.21.15.123:0/200480819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e44cad3d10 msgr2=0x55e44cad8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.508+0000 7fd9ffe00640 1 --2- 172.21.15.123:0/200480819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e44cad3d10 0x55e44cad8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.508+0000 7fd9ffe00640 1 -- 172.21.15.123:0/200480819 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e44c749210 con 0x55e44cad3320 2024-05-08T06:12:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.509+0000 7fd9ffe00640 1 --2- 172.21.15.123:0/200480819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e44cad3320 0x55e44cad3740 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0033000 tx=0x7fd9f0032ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d81494b57e0f4b19 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.509+0000 7fd9f4c00640 1 -- 172.21.15.123:0/200480819 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f003f070 con 0x55e44cad3320 2024-05-08T06:12:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.509+0000 7fd9f4c00640 1 -- 172.21.15.123:0/200480819 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9f0004040 con 0x55e44cad3320 2024-05-08T06:12:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.509+0000 7fd9f4c00640 1 -- 172.21.15.123:0/200480819 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0035070 con 0x55e44cad3320 2024-05-08T06:12:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.510+0000 7fd9f7e00640 1 --2- 172.21.15.123:0/4094049630 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd9e40b2860 0x7fd9e40b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.510+0000 7fd9f7e00640 1 -- 172.21.15.123:0/4094049630 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fd9f80e27e0 con 0x7fda00102870 2024-05-08T06:12:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.510+0000 7fd9ff400640 1 --2- 172.21.15.123:0/4094049630 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd9e40b2860 0x7fd9e40b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.510+0000 7fda0967f740 1 -- 172.21.15.123:0/200480819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e44cad3320 msgr2=0x55e44cad3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.510+0000 7fda0967f740 1 --2- 172.21.15.123:0/200480819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e44cad3320 0x55e44cad3740 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0033000 tx=0x7fd9f0032ba0 comp rx=0 tx=0).stop 2024-05-08T06:12:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.510+0000 7fda0967f740 1 -- 172.21.15.123:0/200480819 shutdown_connections 2024-05-08T06:12:28.532 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.510+0000 7fda0967f740 1 --2- 172.21.15.123:0/200480819 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e44cad8e80 0x55e44cadb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.510+0000 7fda0967f740 1 --2- 172.21.15.123:0/200480819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e44cad3d10 0x55e44cad8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.510+0000 7fda0967f740 1 --2- 172.21.15.123:0/200480819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e44cad3320 0x55e44cad3740 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.510+0000 7fda0967f740 1 -- 172.21.15.123:0/200480819 >> 172.21.15.123:0/200480819 conn(0x55e44ca9b230 msgr2=0x55e44caceda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.511+0000 7fda0967f740 1 -- 172.21.15.123:0/200480819 shutdown_connections 2024-05-08T06:12:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.511+0000 7fda0967f740 1 -- 172.21.15.123:0/200480819 wait complete. 2024-05-08T06:12:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.511+0000 7fd9ff400640 1 --2- 172.21.15.123:0/4094049630 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd9e40b2860 0x7fd9e40b4d20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd9f80a9d40 tx=0x7fd9f80e8530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.511+0000 7fda0967f740 1 -- start start 2024-05-08T06:12:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.511+0000 7fda0967f740 1 Processor -- start 2024-05-08T06:12:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.512+0000 7fda0967f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e44cad3320 0x55e44cadd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.512+0000 7fda0967f740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e44cad3d10 0x55e44cadd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.512+0000 7fda0967f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e44cad8e80 0x55e44cae4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.512+0000 7fda0967f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e44cadbdd0 con 0x55e44cad3320 2024-05-08T06:12:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.512+0000 7fda0967f740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55e44c9f0660 con 0x55e44cad3d10 2024-05-08T06:12:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.512+0000 7fda0967f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55e44cadbf50 con 0x55e44cad8e80 2024-05-08T06:12:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.512+0000 7fda0967f740 10 client.0 _ll_register_callbacks cb 0x55e44c9ba400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:12:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.512+0000 7fd9ffe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e44cad3320 0x55e44cadd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.512+0000 7fd9ffe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e44cad3320 0x55e44cadd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41894/0 (socket says 172.21.15.123:41894) 2024-05-08T06:12:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.512+0000 7fd9ffe00640 1 -- 172.21.15.123:0/3318252831 learned_addr learned my addr 172.21.15.123:0/3318252831 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.512+0000 7fd9fea00640 1 --2- 172.21.15.123:0/3318252831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e44cad3d10 0x55e44cadd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.512+0000 7fd9ffe00640 1 --2- 172.21.15.123:0/3318252831 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e44cad8e80 0x55e44cae4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.513+0000 7fd9fea00640 1 -- 172.21.15.123:0/3318252831 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e44cad8e80 msgr2=0x55e44cae4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.513+0000 7fd9fea00640 1 --2- 172.21.15.123:0/3318252831 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e44cad8e80 0x55e44cae4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.513+0000 7fd9fea00640 1 -- 172.21.15.123:0/3318252831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e44cad3320 msgr2=0x55e44cadd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.513+0000 7fd9fea00640 1 --2- 172.21.15.123:0/3318252831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e44cad3320 0x55e44cadd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.513+0000 7fd9fea00640 1 -- 172.21.15.123:0/3318252831 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e44c815620 con 0x55e44cad3d10 2024-05-08T06:12:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.513+0000 7fd9ffe00640 1 --2- 172.21.15.123:0/3318252831 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e44cad8e80 0x55e44cae4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:12:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.513+0000 7fd9fea00640 1 --2- 172.21.15.123:0/3318252831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e44cad3d10 0x55e44cadd840 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fda0019c170 tx=0x7fd9e8012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f35737b1f8b9fb32 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.513+0000 7fd9ed600640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-05-08T06:12:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.513+0000 7fd9ed600640 1 -- 172.21.15.123:0/3318252831 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e8011c70 con 0x55e44cad3d10 2024-05-08T06:12:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.513+0000 7fda0967f740 1 -- 172.21.15.123:0/3318252831 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9f0004040 con 0x55e44cad3d10 2024-05-08T06:12:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.514+0000 7fd9ed600640 1 -- 172.21.15.123:0/3318252831 <== mon.2 v2:172.21.15.163:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fd9e8011e10 con 0x55e44cad3d10 2024-05-08T06:12:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.514+0000 7fd9ed600640 1 -- 172.21.15.123:0/3318252831 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd9e8005020 con 0x55e44cad3d10 2024-05-08T06:12:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.514+0000 7fda0967f740 10 client.25562 fetch_fsmap learned FSMap version 24 2024-05-08T06:12:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.514+0000 7fda0967f740 1 -- 172.21.15.123:0/3318252831 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e44cae7920 con 0x55e44cad3d10 2024-05-08T06:12:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.514+0000 7fd9ed600640 1 -- 172.21.15.123:0/3318252831 <== mon.2 v2:172.21.15.163:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9f0004040 con 0x55e44cad3d10 2024-05-08T06:12:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.514+0000 7fd9ed600640 1 -- 172.21.15.123:0/3318252831 <== mon.2 v2:172.21.15.163:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fd9e801c470 con 0x55e44cad3d10 2024-05-08T06:12:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.515+0000 7fda0967f740 10 client.25562 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:12:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.515+0000 7fda0967f740 10 client.25562 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T06:12:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.515+0000 7fda0967f740 20 client.25562 populate_metadata read hostname 'smithi123' 2024-05-08T06:12:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.515+0000 7fda0967f740 1 --2- 172.21.15.123:0/3318252831 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55e44cae8f70 0x55e44caeb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:28.539 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.515+0000 7fda0967f740 4 client.25562 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:12:28.539 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.515+0000 7fda0967f740 1 -- 172.21.15.123:0/3318252831 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e44ca45160 con 0x55e44cae8f70 2024-05-08T06:12:28.539 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.515+0000 7fd9ffe00640 1 --2- 172.21.15.123:0/3318252831 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55e44cae8f70 0x55e44caeb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:28.539 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.516+0000 7fd9ffe00640 1 --2- 172.21.15.123:0/3318252831 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55e44cae8f70 0x55e44caeb3f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a86a158377af3c74 server_cookie=551c0d80f5064830 in_seq=0 out_seq=0 2024-05-08T06:12:28.539 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.516+0000 7fd9ed600640 10 client.25562 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:28.539 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.516+0000 7fd9ed600640 1 -- 172.21.15.123:0/3318252831 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e44ca45160 con 0x55e44cae8f70 2024-05-08T06:12:28.539 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.516+0000 7fd9ed600640 10 client.25562 handle_command_reply: tid=0 2024-05-08T06:12:28.540 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.516+0000 7fda0967f740 1 client.25562 shutdown 2024-05-08T06:12:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.517+0000 7fda0967f740 1 -- 172.21.15.123:0/3318252831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e44cad3d10 msgr2=0x55e44cadd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.517+0000 7fda0967f740 1 --2- 172.21.15.123:0/3318252831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e44cad3d10 0x55e44cadd840 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fda0019c170 tx=0x7fd9e8012b60 comp rx=0 tx=0).stop 2024-05-08T06:12:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.517+0000 7fda0967f740 1 -- 172.21.15.123:0/3318252831 shutdown_connections 2024-05-08T06:12:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.517+0000 7fda0967f740 1 --2- 172.21.15.123:0/3318252831 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55e44cae8f70 0x55e44caeb3f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.517+0000 7fda0967f740 1 --2- 172.21.15.123:0/3318252831 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e44cad8e80 0x55e44cae4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.517+0000 7fda0967f740 1 --2- 172.21.15.123:0/3318252831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e44cad3d10 0x55e44cadd840 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.517+0000 7fd9ed600640 0 client.25562 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.517+0000 7fda0967f740 1 --2- 172.21.15.123:0/3318252831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e44cad3320 0x55e44cadd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.517+0000 7fda0967f740 1 -- 172.21.15.123:0/3318252831 >> 172.21.15.123:0/3318252831 conn(0x55e44ca9b230 msgr2=0x55e44cacde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.517+0000 7fda0967f740 1 -- 172.21.15.123:0/3318252831 shutdown_connections 2024-05-08T06:12:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.517+0000 7fda0967f740 1 -- 172.21.15.123:0/3318252831 wait complete. 2024-05-08T06:12:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.517+0000 7fda0967f740 20 client.25562 trim_cache size 0 max 16384 2024-05-08T06:12:28.543 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4193280 len 2048 2024-05-08T06:12:28.543 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4193280 len 4096 2024-05-08T06:12:28.544 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4193280 len 4096 2024-05-08T06:12:28.544 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4193280 len 4096 2024-05-08T06:12:28.544 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4193280 len 4096 2024-05-08T06:12:28.544 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4193280 len 8192 2024-05-08T06:12:28.544 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4193280 len 8192 2024-05-08T06:12:28.544 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4193280 len 8192 2024-05-08T06:12:28.544 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4193280 len 8192 2024-05-08T06:12:28.545 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4193280 len 16384 2024-05-08T06:12:28.545 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4193280 len 16384 2024-05-08T06:12:28.545 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4193280 len 16384 2024-05-08T06:12:28.545 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4193280 len 16384 2024-05-08T06:12:28.545 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4193792 len 1024 2024-05-08T06:12:28.545 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4193792 len 1024 2024-05-08T06:12:28.545 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4193792 len 1024 2024-05-08T06:12:28.546 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4193792 len 1024 2024-05-08T06:12:28.546 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4193792 len 2048 2024-05-08T06:12:28.546 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4193792 len 2048 2024-05-08T06:12:28.546 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4193792 len 2048 2024-05-08T06:12:28.546 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4193792 len 2048 2024-05-08T06:12:28.546 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4193792 len 4096 2024-05-08T06:12:28.547 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4193792 len 4096 2024-05-08T06:12:28.547 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4193792 len 4096 2024-05-08T06:12:28.547 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4193792 len 4096 2024-05-08T06:12:28.547 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4193792 len 8192 2024-05-08T06:12:28.547 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4193792 len 8192 2024-05-08T06:12:28.547 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4193792 len 8192 2024-05-08T06:12:28.547 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4193792 len 8192 2024-05-08T06:12:28.548 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4193792 len 16384 2024-05-08T06:12:28.548 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4193792 len 16384 2024-05-08T06:12:28.548 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4193792 len 16384 2024-05-08T06:12:28.548 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4193792 len 16384 2024-05-08T06:12:28.548 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4194304 len 1024 2024-05-08T06:12:28.548 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4194304 len 1024 2024-05-08T06:12:28.548 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4194304 len 1024 2024-05-08T06:12:28.549 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4194304 len 1024 2024-05-08T06:12:28.549 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4194304 len 2048 2024-05-08T06:12:28.549 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4194304 len 2048 2024-05-08T06:12:28.549 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4194304 len 2048 2024-05-08T06:12:28.549 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4194304 len 2048 2024-05-08T06:12:28.549 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4194304 len 4096 2024-05-08T06:12:28.549 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4194304 len 4096 2024-05-08T06:12:28.550 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4194304 len 4096 2024-05-08T06:12:28.550 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4194304 len 4096 2024-05-08T06:12:28.550 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4194304 len 8192 2024-05-08T06:12:28.550 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4194304 len 8192 2024-05-08T06:12:28.550 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4194304 len 8192 2024-05-08T06:12:28.550 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4194304 len 8192 2024-05-08T06:12:28.550 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4194304 len 16384 2024-05-08T06:12:28.551 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4194304 len 16384 2024-05-08T06:12:28.551 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4194304 len 16384 2024-05-08T06:12:28.551 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4194304 len 16384 2024-05-08T06:12:28.551 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4194816 len 1024 2024-05-08T06:12:28.551 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4194816 len 1024 2024-05-08T06:12:28.551 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4194816 len 1024 2024-05-08T06:12:28.551 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4194816 len 1024 2024-05-08T06:12:28.552 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4194816 len 2048 2024-05-08T06:12:28.552 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4194816 len 2048 2024-05-08T06:12:28.552 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4194816 len 2048 2024-05-08T06:12:28.552 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4194816 len 2048 2024-05-08T06:12:28.552 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4194816 len 4096 2024-05-08T06:12:28.552 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4194816 len 4096 2024-05-08T06:12:28.552 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4194816 len 4096 2024-05-08T06:12:28.553 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4194816 len 4096 2024-05-08T06:12:28.553 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4194816 len 8192 2024-05-08T06:12:28.553 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4194816 len 8192 2024-05-08T06:12:28.553 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4194816 len 8192 2024-05-08T06:12:28.553 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4194816 len 8192 2024-05-08T06:12:28.553 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4194816 len 16384 2024-05-08T06:12:28.553 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4194816 len 16384 2024-05-08T06:12:28.554 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4194816 len 16384 2024-05-08T06:12:28.554 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4194816 len 16384 2024-05-08T06:12:28.554 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4195328 len 1024 2024-05-08T06:12:28.554 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4195328 len 1024 2024-05-08T06:12:28.554 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4195328 len 1024 2024-05-08T06:12:28.554 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4195328 len 1024 2024-05-08T06:12:28.554 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4195328 len 2048 2024-05-08T06:12:28.555 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4195328 len 2048 2024-05-08T06:12:28.555 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4195328 len 2048 2024-05-08T06:12:28.555 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4195328 len 2048 2024-05-08T06:12:28.555 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4195328 len 4096 2024-05-08T06:12:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.539+0000 7fda0967f740 1 Processor -- start 2024-05-08T06:12:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.539+0000 7fda0967f740 1 -- start start 2024-05-08T06:12:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.539+0000 7fda0967f740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e44cad3320 0x55e44cacef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.539+0000 7fda0967f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e44cad3d10 0x55e44cae4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.539+0000 7fda0967f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e44cad8e80 0x55e44caefcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.539+0000 7fda0967f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fda001ae6d0 con 0x55e44cad8e80 2024-05-08T06:12:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.539+0000 7fda0967f740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fda001abfa0 con 0x55e44cad3320 2024-05-08T06:12:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.539+0000 7fda0967f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fda001a9df0 con 0x55e44cad3d10 2024-05-08T06:12:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.540+0000 7fd9fea00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e44cad3d10 0x55e44cae4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.540+0000 7fd9fea00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e44cad3d10 0x55e44cae4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:46868/0 (socket says 172.21.15.123:46868) 2024-05-08T06:12:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.540+0000 7fd9fea00640 1 -- 172.21.15.123:0/3156618117 learned_addr learned my addr 172.21.15.123:0/3156618117 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.540+0000 7fd9ffe00640 1 --2- 172.21.15.123:0/3156618117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e44cad8e80 0x55e44caefcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:28.558 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4195328 len 4096 2024-05-08T06:12:28.558 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4195328 len 4096 2024-05-08T06:12:28.559 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4195328 len 4096 2024-05-08T06:12:28.559 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4195328 len 8192 2024-05-08T06:12:28.559 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4195328 len 8192 2024-05-08T06:12:28.559 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4195328 len 8192 2024-05-08T06:12:28.559 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4195328 len 8192 2024-05-08T06:12:28.559 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4195328 len 16384 2024-05-08T06:12:28.560 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4195328 len 16384 2024-05-08T06:12:28.560 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4195328 len 16384 2024-05-08T06:12:28.560 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4195328 len 16384 2024-05-08T06:12:28.560 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4195840 len 1024 2024-05-08T06:12:28.560 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4195840 len 1024 2024-05-08T06:12:28.560 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4195840 len 1024 2024-05-08T06:12:28.561 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4195840 len 1024 2024-05-08T06:12:28.561 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4195840 len 2048 2024-05-08T06:12:28.561 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4195840 len 2048 2024-05-08T06:12:28.561 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4195840 len 2048 2024-05-08T06:12:28.561 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4195840 len 2048 2024-05-08T06:12:28.561 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4195840 len 4096 2024-05-08T06:12:28.562 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4195840 len 4096 2024-05-08T06:12:28.562 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4195840 len 4096 2024-05-08T06:12:28.562 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4195840 len 4096 2024-05-08T06:12:28.562 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4195840 len 8192 2024-05-08T06:12:28.562 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4195840 len 8192 2024-05-08T06:12:28.562 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4195840 len 8192 2024-05-08T06:12:28.562 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4195840 len 8192 2024-05-08T06:12:28.563 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4195840 len 16384 2024-05-08T06:12:28.563 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4195840 len 16384 2024-05-08T06:12:28.563 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4195840 len 16384 2024-05-08T06:12:28.563 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4195840 len 16384 2024-05-08T06:12:28.563 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4196352 len 1024 2024-05-08T06:12:28.563 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4196352 len 1024 2024-05-08T06:12:28.564 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4196352 len 1024 2024-05-08T06:12:28.564 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4196352 len 1024 2024-05-08T06:12:28.564 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4196352 len 2048 2024-05-08T06:12:28.564 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4196352 len 2048 2024-05-08T06:12:28.564 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4196352 len 2048 2024-05-08T06:12:28.564 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4196352 len 2048 2024-05-08T06:12:28.564 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4196352 len 4096 2024-05-08T06:12:28.565 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4196352 len 4096 2024-05-08T06:12:28.565 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4196352 len 4096 2024-05-08T06:12:28.565 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4196352 len 4096 2024-05-08T06:12:28.565 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4196352 len 8192 2024-05-08T06:12:28.565 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4196352 len 8192 2024-05-08T06:12:28.565 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4196352 len 8192 2024-05-08T06:12:28.566 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4196352 len 8192 2024-05-08T06:12:28.566 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4196352 len 16384 2024-05-08T06:12:28.566 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4196352 len 16384 2024-05-08T06:12:28.566 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4196352 len 16384 2024-05-08T06:12:28.566 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4196352 len 16384 2024-05-08T06:12:28.566 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4196864 len 1024 2024-05-08T06:12:28.566 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4196864 len 1024 2024-05-08T06:12:28.567 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4196864 len 1024 2024-05-08T06:12:28.567 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4196864 len 1024 2024-05-08T06:12:28.567 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4196864 len 2048 2024-05-08T06:12:28.567 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4196864 len 2048 2024-05-08T06:12:28.567 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4196864 len 2048 2024-05-08T06:12:28.567 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4196864 len 2048 2024-05-08T06:12:28.567 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4196864 len 4096 2024-05-08T06:12:28.568 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4196864 len 4096 2024-05-08T06:12:28.568 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4196864 len 4096 2024-05-08T06:12:28.568 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4196864 len 4096 2024-05-08T06:12:28.568 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4196864 len 8192 2024-05-08T06:12:28.568 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4196864 len 8192 2024-05-08T06:12:28.568 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4196864 len 8192 2024-05-08T06:12:28.569 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4196864 len 8192 2024-05-08T06:12:28.569 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4196864 len 16384 2024-05-08T06:12:28.569 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4196864 len 16384 2024-05-08T06:12:28.569 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4196864 len 16384 2024-05-08T06:12:28.569 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4196864 len 16384 2024-05-08T06:12:28.569 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4197376 len 1024 2024-05-08T06:12:28.569 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4197376 len 1024 2024-05-08T06:12:28.570 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4197376 len 1024 2024-05-08T06:12:28.570 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4197376 len 1024 2024-05-08T06:12:28.570 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4197376 len 2048 2024-05-08T06:12:28.570 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4197376 len 2048 2024-05-08T06:12:28.570 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4197376 len 2048 2024-05-08T06:12:28.570 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4197376 len 2048 2024-05-08T06:12:28.570 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4197376 len 4096 2024-05-08T06:12:28.571 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4197376 len 4096 2024-05-08T06:12:28.571 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4197376 len 4096 2024-05-08T06:12:28.571 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4197376 len 4096 2024-05-08T06:12:28.579 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.540+0000 7fd9ffe00640 1 --2- 172.21.15.123:0/3156618117 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e44cad3320 0x55e44cacef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:28.579 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.540+0000 7fd9fea00640 1 -- 172.21.15.123:0/3156618117 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e44cad3320 msgr2=0x55e44cacef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:28.579 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.540+0000 7fd9fea00640 1 --2- 172.21.15.123:0/3156618117 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e44cad3320 0x55e44cacef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:28.579 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.540+0000 7fd9fea00640 1 -- 172.21.15.123:0/3156618117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e44cad8e80 msgr2=0x55e44caefcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:28.579 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.540+0000 7fd9fea00640 1 --2- 172.21.15.123:0/3156618117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e44cad8e80 0x55e44caefcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:28.579 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.540+0000 7fd9fea00640 1 -- 172.21.15.123:0/3156618117 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e44cacfa50 con 0x55e44cad3d10 2024-05-08T06:12:28.579 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.540+0000 7fd9fea00640 1 --2- 172.21.15.123:0/3156618117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e44cad3d10 0x55e44cae4840 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd9e801c8e0 tx=0x7fd9e8009fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=772ae68c66624c98 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:28.579 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.541+0000 7fd9efe00640 1 -- 172.21.15.123:0/3156618117 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e801cc90 con 0x55e44cad3d10 2024-05-08T06:12:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.541+0000 7fd9efe00640 1 -- 172.21.15.123:0/3156618117 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9e8005670 con 0x55e44cad3d10 2024-05-08T06:12:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.541+0000 7fd9efe00640 1 -- 172.21.15.123:0/3156618117 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e80048c0 con 0x55e44cad3d10 2024-05-08T06:12:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.542+0000 7fda0967f740 1 -- 172.21.15.123:0/3156618117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e44cad3d10 msgr2=0x55e44cae4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.542+0000 7fda0967f740 1 --2- 172.21.15.123:0/3156618117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e44cad3d10 0x55e44cae4840 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd9e801c8e0 tx=0x7fd9e8009fb0 comp rx=0 tx=0).stop 2024-05-08T06:12:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.542+0000 7fda0967f740 1 -- 172.21.15.123:0/3156618117 shutdown_connections 2024-05-08T06:12:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.542+0000 7fda0967f740 1 --2- 172.21.15.123:0/3156618117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e44cad8e80 0x55e44caefcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.542+0000 7fda0967f740 1 --2- 172.21.15.123:0/3156618117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e44cad3d10 0x55e44cae4840 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.542+0000 7fda0967f740 1 --2- 172.21.15.123:0/3156618117 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e44cad3320 0x55e44cacef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.542+0000 7fda0967f740 1 -- 172.21.15.123:0/3156618117 >> 172.21.15.123:0/3156618117 conn(0x55e44ca9b230 msgr2=0x55e44cace820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.542+0000 7fda0967f740 1 -- 172.21.15.123:0/3156618117 shutdown_connections 2024-05-08T06:12:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.542+0000 7fda0967f740 1 -- 172.21.15.123:0/3156618117 wait complete. 2024-05-08T06:12:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.543+0000 7fda0967f740 1 -- start start 2024-05-08T06:12:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.543+0000 7fda0967f740 1 Processor -- start 2024-05-08T06:12:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.543+0000 7fda0967f740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e44cad3320 0x55e44cae24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.543+0000 7fda0967f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e44cad3d10 0x55e44cae29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:28.581 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.543+0000 7fda0967f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e44cad8e80 0x55e44cae39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:28.581 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.543+0000 7fda0967f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fda001aa460 con 0x55e44cad8e80 2024-05-08T06:12:28.581 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.543+0000 7fda0967f740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fda001ab8d0 con 0x55e44cad3320 2024-05-08T06:12:28.581 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.543+0000 7fda0967f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fda001aab30 con 0x55e44cad3d10 2024-05-08T06:12:28.581 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.543+0000 7fda0967f740 10 client.0 _ll_register_callbacks cb 0x7fd9e80097e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:12:28.581 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.544+0000 7fd9fea00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e44cad3d10 0x55e44cae29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:28.581 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.544+0000 7fd9ffe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e44cad3320 0x55e44cae24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:28.581 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.544+0000 7fd9fea00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e44cad3d10 0x55e44cae29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:46874/0 (socket says 172.21.15.123:46874) 2024-05-08T06:12:28.581 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.544+0000 7fd9fea00640 1 -- 172.21.15.123:0/66154425 learned_addr learned my addr 172.21.15.123:0/66154425 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:28.581 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.544+0000 7fd9ffe00640 1 --2- 172.21.15.123:0/66154425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e44cad8e80 0x55e44cae39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:28.581 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.544+0000 7fd9fea00640 1 -- 172.21.15.123:0/66154425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e44cad3320 msgr2=0x55e44cae24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:28.581 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.544+0000 7fd9fea00640 1 --2- 172.21.15.123:0/66154425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e44cad3320 0x55e44cae24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:28.581 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.544+0000 7fd9fea00640 1 -- 172.21.15.123:0/66154425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e44cad8e80 msgr2=0x55e44cae39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:28.581 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.544+0000 7fd9fea00640 1 --2- 172.21.15.123:0/66154425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e44cad8e80 0x55e44cae39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.544+0000 7fd9fea00640 1 -- 172.21.15.123:0/66154425 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e44cae3fe0 con 0x55e44cad3d10 2024-05-08T06:12:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.544+0000 7fd9ffe00640 1 --2- 172.21.15.123:0/66154425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e44cad3320 0x55e44cae24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:12:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.544+0000 7fd9fea00640 1 --2- 172.21.15.123:0/66154425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e44cad3d10 0x55e44cae29e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd9e801c960 tx=0x7fd9e8009ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74ba9d953c96cdc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.544+0000 7fd9ed600640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-05-08T06:12:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.545+0000 7fd9ed600640 1 -- 172.21.15.123:0/66154425 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e803d060 con 0x55e44cad3d10 2024-05-08T06:12:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.545+0000 7fda0967f740 1 -- 172.21.15.123:0/66154425 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9e8005670 con 0x55e44cad3d10 2024-05-08T06:12:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.545+0000 7fd9ed600640 1 -- 172.21.15.123:0/66154425 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fd9e80051c0 con 0x55e44cad3d10 2024-05-08T06:12:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.545+0000 7fd9ed600640 1 -- 172.21.15.123:0/66154425 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd9e8005400 con 0x55e44cad3d10 2024-05-08T06:12:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.545+0000 7fda0967f740 10 client.25555 fetch_fsmap learned FSMap version 24 2024-05-08T06:12:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.545+0000 7fda0967f740 1 -- 172.21.15.123:0/66154425 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e44cb5c5a0 con 0x55e44cad3d10 2024-05-08T06:12:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.545+0000 7fd9ed600640 1 -- 172.21.15.123:0/66154425 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9e8005670 con 0x55e44cad3d10 2024-05-08T06:12:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.546+0000 7fd9ed600640 1 -- 172.21.15.123:0/66154425 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fd9e802e070 con 0x55e44cad3d10 2024-05-08T06:12:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.546+0000 7fda0967f740 10 client.25555 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:12:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.546+0000 7fda0967f740 10 client.25555 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T06:12:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.546+0000 7fda0967f740 20 client.25555 populate_metadata read hostname 'smithi123' 2024-05-08T06:12:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.546+0000 7fda0967f740 1 --2- 172.21.15.123:0/66154425 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55e44cb5cec0 0x55e44cb5f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.546+0000 7fda0967f740 4 client.25555 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T06:12:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.546+0000 7fda0967f740 1 -- 172.21.15.123:0/66154425 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e44ca3efd0 con 0x55e44cb5cec0 2024-05-08T06:12:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.546+0000 7fd9ffe00640 1 --2- 172.21.15.123:0/66154425 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55e44cb5cec0 0x55e44cb5f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.547+0000 7fd9ffe00640 1 --2- 172.21.15.123:0/66154425 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55e44cb5cec0 0x55e44cb5f2e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74bd40ff16f906b3 server_cookie=6cfa336b88a59e4 in_seq=0 out_seq=0 2024-05-08T06:12:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.547+0000 7fd9ed600640 10 client.25555 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.559+0000 7fd9ed600640 1 -- 172.21.15.123:0/66154425 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e44ca3efd0 con 0x55e44cb5cec0 2024-05-08T06:12:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.559+0000 7fd9ed600640 10 client.25555 handle_command_reply: tid=0 2024-05-08T06:12:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.559+0000 7fda0967f740 1 client.25555 shutdown 2024-05-08T06:12:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.559+0000 7fda0967f740 1 -- 172.21.15.123:0/66154425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e44cad3d10 msgr2=0x55e44cae29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.559+0000 7fda0967f740 1 --2- 172.21.15.123:0/66154425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e44cad3d10 0x55e44cae29e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd9e801c960 tx=0x7fd9e8009ec0 comp rx=0 tx=0).stop 2024-05-08T06:12:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.559+0000 7fda0967f740 1 -- 172.21.15.123:0/66154425 shutdown_connections 2024-05-08T06:12:28.584 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.559+0000 7fda0967f740 1 --2- 172.21.15.123:0/66154425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e44cad8e80 0x55e44cae39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:28.584 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.559+0000 7fda0967f740 1 --2- 172.21.15.123:0/66154425 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55e44cb5cec0 0x55e44cb5f2e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:28.584 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.559+0000 7fda0967f740 1 --2- 172.21.15.123:0/66154425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e44cad3d10 0x55e44cae29e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:28.584 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.559+0000 7fda0967f740 1 --2- 172.21.15.123:0/66154425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e44cad3320 0x55e44cae24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:28.584 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.559+0000 7fda0967f740 1 -- 172.21.15.123:0/66154425 >> 172.21.15.123:0/66154425 conn(0x55e44ca9b230 msgr2=0x55e44cace820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:28.584 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.559+0000 7fd9ed600640 0 client.25555 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:28.584 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.559+0000 7fda0967f740 1 -- 172.21.15.123:0/66154425 shutdown_connections 2024-05-08T06:12:28.584 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.559+0000 7fda0967f740 1 -- 172.21.15.123:0/66154425 wait complete. 2024-05-08T06:12:28.584 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.560+0000 7fda0967f740 20 client.25555 trim_cache size 0 max 16384 2024-05-08T06:12:28.584 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout:{ 2024-05-08T06:12:28.584 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout: "message": "", 2024-05-08T06:12:28.585 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout: "return_code": 0 2024-05-08T06:12:28.585 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout:} 2024-05-08T06:12:28.585 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.560+0000 7fda05200640 1 -- 172.21.15.123:0/4094049630 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd9e40b2860 msgr2=0x7fd9e40b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:28.585 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.560+0000 7fda05200640 1 --2- 172.21.15.123:0/4094049630 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd9e40b2860 0x7fd9e40b4d20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd9f80a9d40 tx=0x7fd9f80e8530 comp rx=0 tx=0).stop 2024-05-08T06:12:28.585 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.560+0000 7fda05200640 1 -- 172.21.15.123:0/4094049630 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda00102870 msgr2=0x7fda0019ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:28.585 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.560+0000 7fda05200640 1 --2- 172.21.15.123:0/4094049630 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda00102870 0x7fda0019ac70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009fd0 tx=0x7fd9f800c270 comp rx=0 tx=0).stop 2024-05-08T06:12:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.561+0000 7fda05200640 1 -- 172.21.15.123:0/4094049630 shutdown_connections 2024-05-08T06:12:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.561+0000 7fda05200640 1 --2- 172.21.15.123:0/4094049630 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd9e40b2860 0x7fd9e40b4d20 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.561+0000 7fda05200640 1 --2- 172.21.15.123:0/4094049630 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda00104420 0x7fda001a2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.561+0000 7fda05200640 1 --2- 172.21.15.123:0/4094049630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda00103a60 0x7fda0019b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.561+0000 7fda05200640 1 --2- 172.21.15.123:0/4094049630 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda00102870 0x7fda0019ac70 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.561+0000 7fda05200640 1 -- 172.21.15.123:0/4094049630 >> 172.21.15.123:0/4094049630 conn(0x7fda000fdde0 msgr2=0x7fda00074be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.561+0000 7fda05200640 1 -- 172.21.15.123:0/4094049630 shutdown_connections 2024-05-08T06:12:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:28.561+0000 7fda05200640 1 -- 172.21.15.123:0/4094049630 wait complete. 2024-05-08T06:12:28.586 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4197376 len 8192 2024-05-08T06:12:28.586 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4197376 len 8192 2024-05-08T06:12:28.586 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4197376 len 8192 2024-05-08T06:12:28.586 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4197376 len 8192 2024-05-08T06:12:28.587 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4197376 len 16384 2024-05-08T06:12:28.587 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4197376 len 16384 2024-05-08T06:12:28.587 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4197376 len 16384 2024-05-08T06:12:28.587 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4197376 len 16384 2024-05-08T06:12:28.587 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4197888 len 1024 2024-05-08T06:12:28.587 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4197888 len 1024 2024-05-08T06:12:28.587 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4197888 len 1024 2024-05-08T06:12:28.587 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4197888 len 1024 2024-05-08T06:12:28.587 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4197888 len 2048 2024-05-08T06:12:28.587 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4197888 len 2048 2024-05-08T06:12:28.587 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4197888 len 2048 2024-05-08T06:12:28.587 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4197888 len 2048 2024-05-08T06:12:28.587 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4197888 len 4096 2024-05-08T06:12:28.587 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4197888 len 4096 2024-05-08T06:12:28.587 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4197888 len 4096 2024-05-08T06:12:28.587 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4197888 len 4096 2024-05-08T06:12:28.588 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4197888 len 8192 2024-05-08T06:12:28.588 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4197888 len 8192 2024-05-08T06:12:28.588 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4197888 len 8192 2024-05-08T06:12:28.588 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4197888 len 8192 2024-05-08T06:12:28.588 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1024 offset 4197888 len 16384 2024-05-08T06:12:28.588 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4197888 len 16384 2024-05-08T06:12:28.588 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1024 offset 4197888 len 16384 2024-05-08T06:12:28.588 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1024 offset 4197888 len 16384 2024-05-08T06:12:28.588 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4190208 len 1024 2024-05-08T06:12:28.588 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4190208 len 1024 2024-05-08T06:12:28.588 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4190208 len 1024 2024-05-08T06:12:28.588 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4190208 len 1024 2024-05-08T06:12:28.588 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4190208 len 2048 2024-05-08T06:12:28.588 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4190208 len 2048 2024-05-08T06:12:28.588 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4190208 len 2048 2024-05-08T06:12:28.589 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4190208 len 2048 2024-05-08T06:12:28.589 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4190208 len 4096 2024-05-08T06:12:28.589 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4190208 len 4096 2024-05-08T06:12:28.589 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4190208 len 4096 2024-05-08T06:12:28.589 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4190208 len 4096 2024-05-08T06:12:28.589 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4190208 len 8192 2024-05-08T06:12:28.589 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4190208 len 8192 2024-05-08T06:12:28.589 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4190208 len 8192 2024-05-08T06:12:28.589 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4190208 len 8192 2024-05-08T06:12:28.589 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4190208 len 16384 2024-05-08T06:12:28.589 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4190208 len 16384 2024-05-08T06:12:28.589 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4190208 len 16384 2024-05-08T06:12:28.589 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4190208 len 16384 2024-05-08T06:12:28.589 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4190720 len 1024 2024-05-08T06:12:28.589 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4190720 len 1024 2024-05-08T06:12:28.589 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4190720 len 1024 2024-05-08T06:12:28.590 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4190720 len 1024 2024-05-08T06:12:28.590 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4190720 len 2048 2024-05-08T06:12:28.590 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4190720 len 2048 2024-05-08T06:12:28.590 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4190720 len 2048 2024-05-08T06:12:28.590 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4190720 len 2048 2024-05-08T06:12:28.590 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4190720 len 4096 2024-05-08T06:12:28.590 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4190720 len 4096 2024-05-08T06:12:28.590 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4190720 len 4096 2024-05-08T06:12:28.590 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4190720 len 4096 2024-05-08T06:12:28.590 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4190720 len 8192 2024-05-08T06:12:28.590 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4190720 len 8192 2024-05-08T06:12:28.590 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4190720 len 8192 2024-05-08T06:12:28.590 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4190720 len 8192 2024-05-08T06:12:28.590 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4190720 len 16384 2024-05-08T06:12:28.590 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4190720 len 16384 2024-05-08T06:12:28.590 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4190720 len 16384 2024-05-08T06:12:28.591 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4190720 len 16384 2024-05-08T06:12:28.591 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4191232 len 1024 2024-05-08T06:12:28.591 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4191232 len 1024 2024-05-08T06:12:28.591 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4191232 len 1024 2024-05-08T06:12:28.591 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4191232 len 1024 2024-05-08T06:12:28.591 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4191232 len 2048 2024-05-08T06:12:28.591 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4191232 len 2048 2024-05-08T06:12:28.591 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4191232 len 2048 2024-05-08T06:12:28.591 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4191232 len 2048 2024-05-08T06:12:28.591 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4191232 len 4096 2024-05-08T06:12:28.591 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4191232 len 4096 2024-05-08T06:12:28.591 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4191232 len 4096 2024-05-08T06:12:28.591 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4191232 len 4096 2024-05-08T06:12:28.591 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4191232 len 8192 2024-05-08T06:12:28.591 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4191232 len 8192 2024-05-08T06:12:28.621 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4191232 len 8192 2024-05-08T06:12:28.622 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4191232 len 8192 2024-05-08T06:12:28.622 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4191232 len 16384 2024-05-08T06:12:28.622 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4191232 len 16384 2024-05-08T06:12:28.622 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4191232 len 16384 2024-05-08T06:12:28.622 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4191232 len 16384 2024-05-08T06:12:28.622 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4191744 len 1024 2024-05-08T06:12:28.622 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4191744 len 1024 2024-05-08T06:12:28.622 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4191744 len 1024 2024-05-08T06:12:28.622 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4191744 len 1024 2024-05-08T06:12:28.622 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4191744 len 2048 2024-05-08T06:12:28.622 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4191744 len 2048 2024-05-08T06:12:28.622 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4191744 len 2048 2024-05-08T06:12:28.622 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4191744 len 2048 2024-05-08T06:12:28.622 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4191744 len 4096 2024-05-08T06:12:28.623 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4191744 len 4096 2024-05-08T06:12:28.623 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4191744 len 4096 2024-05-08T06:12:28.623 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4191744 len 4096 2024-05-08T06:12:28.623 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4191744 len 8192 2024-05-08T06:12:28.623 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4191744 len 8192 2024-05-08T06:12:28.623 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4191744 len 8192 2024-05-08T06:12:28.623 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4191744 len 8192 2024-05-08T06:12:28.623 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4191744 len 16384 2024-05-08T06:12:28.623 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4191744 len 16384 2024-05-08T06:12:28.623 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4191744 len 16384 2024-05-08T06:12:28.623 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4191744 len 16384 2024-05-08T06:12:28.623 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4192256 len 1024 2024-05-08T06:12:28.623 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4192256 len 1024 2024-05-08T06:12:28.623 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4192256 len 1024 2024-05-08T06:12:28.623 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4192256 len 1024 2024-05-08T06:12:28.624 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4192256 len 2048 2024-05-08T06:12:28.624 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4192256 len 2048 2024-05-08T06:12:28.624 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4192256 len 2048 2024-05-08T06:12:28.624 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4192256 len 2048 2024-05-08T06:12:28.624 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4192256 len 4096 2024-05-08T06:12:28.624 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4192256 len 4096 2024-05-08T06:12:28.624 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4192256 len 4096 2024-05-08T06:12:28.624 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4192256 len 4096 2024-05-08T06:12:28.624 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4192256 len 8192 2024-05-08T06:12:28.624 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4192256 len 8192 2024-05-08T06:12:28.624 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4192256 len 8192 2024-05-08T06:12:28.624 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4192256 len 8192 2024-05-08T06:12:28.624 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4192256 len 16384 2024-05-08T06:12:28.624 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4192256 len 16384 2024-05-08T06:12:28.624 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4192256 len 16384 2024-05-08T06:12:28.624 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4192256 len 16384 2024-05-08T06:12:28.625 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4192768 len 1024 2024-05-08T06:12:28.625 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4192768 len 1024 2024-05-08T06:12:28.625 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4192768 len 1024 2024-05-08T06:12:28.625 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4192768 len 1024 2024-05-08T06:12:28.625 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4192768 len 2048 2024-05-08T06:12:28.625 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4192768 len 2048 2024-05-08T06:12:28.625 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4192768 len 2048 2024-05-08T06:12:28.625 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4192768 len 2048 2024-05-08T06:12:28.625 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4192768 len 4096 2024-05-08T06:12:28.625 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4192768 len 4096 2024-05-08T06:12:28.625 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4192768 len 4096 2024-05-08T06:12:28.625 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4192768 len 4096 2024-05-08T06:12:28.625 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4192768 len 8192 2024-05-08T06:12:28.625 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4192768 len 8192 2024-05-08T06:12:28.625 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4192768 len 8192 2024-05-08T06:12:28.625 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4192768 len 8192 2024-05-08T06:12:28.626 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4192768 len 16384 2024-05-08T06:12:28.626 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4192768 len 16384 2024-05-08T06:12:28.626 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4192768 len 16384 2024-05-08T06:12:28.626 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4192768 len 16384 2024-05-08T06:12:28.626 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4193280 len 1024 2024-05-08T06:12:28.626 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4193280 len 1024 2024-05-08T06:12:28.626 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4193280 len 1024 2024-05-08T06:12:28.626 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4193280 len 1024 2024-05-08T06:12:28.626 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4193280 len 2048 2024-05-08T06:12:28.626 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4193280 len 2048 2024-05-08T06:12:28.626 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4193280 len 2048 2024-05-08T06:12:28.626 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4193280 len 2048 2024-05-08T06:12:28.626 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4193280 len 4096 2024-05-08T06:12:28.626 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4193280 len 4096 2024-05-08T06:12:28.626 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4193280 len 4096 2024-05-08T06:12:28.627 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4193280 len 4096 2024-05-08T06:12:28.627 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4193280 len 8192 2024-05-08T06:12:28.627 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4193280 len 8192 2024-05-08T06:12:28.627 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4193280 len 8192 2024-05-08T06:12:28.627 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4193280 len 8192 2024-05-08T06:12:28.663 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4193280 len 16384 2024-05-08T06:12:28.663 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4193280 len 16384 2024-05-08T06:12:28.664 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4193280 len 16384 2024-05-08T06:12:28.664 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4193280 len 16384 2024-05-08T06:12:28.664 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4193792 len 1024 2024-05-08T06:12:28.664 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4193792 len 1024 2024-05-08T06:12:28.664 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4193792 len 1024 2024-05-08T06:12:28.664 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4193792 len 1024 2024-05-08T06:12:28.664 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4193792 len 2048 2024-05-08T06:12:28.664 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4193792 len 2048 2024-05-08T06:12:28.664 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4193792 len 2048 2024-05-08T06:12:28.664 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4193792 len 2048 2024-05-08T06:12:28.664 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4193792 len 4096 2024-05-08T06:12:28.664 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4193792 len 4096 2024-05-08T06:12:28.664 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4193792 len 4096 2024-05-08T06:12:28.664 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4193792 len 4096 2024-05-08T06:12:28.664 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4193792 len 8192 2024-05-08T06:12:28.665 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4193792 len 8192 2024-05-08T06:12:28.665 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4193792 len 8192 2024-05-08T06:12:28.665 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4193792 len 8192 2024-05-08T06:12:28.665 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4193792 len 16384 2024-05-08T06:12:28.665 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4193792 len 16384 2024-05-08T06:12:28.665 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4193792 len 16384 2024-05-08T06:12:28.665 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4193792 len 16384 2024-05-08T06:12:28.665 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4194304 len 1024 2024-05-08T06:12:28.665 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4194304 len 1024 2024-05-08T06:12:28.665 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4194304 len 1024 2024-05-08T06:12:28.665 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4194304 len 1024 2024-05-08T06:12:28.665 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4194304 len 2048 2024-05-08T06:12:28.665 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4194304 len 2048 2024-05-08T06:12:28.665 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4194304 len 2048 2024-05-08T06:12:28.665 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4194304 len 2048 2024-05-08T06:12:28.665 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4194304 len 4096 2024-05-08T06:12:28.666 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4194304 len 4096 2024-05-08T06:12:28.666 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4194304 len 4096 2024-05-08T06:12:28.666 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4194304 len 4096 2024-05-08T06:12:28.666 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4194304 len 8192 2024-05-08T06:12:28.666 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4194304 len 8192 2024-05-08T06:12:28.666 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4194304 len 8192 2024-05-08T06:12:28.666 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4194304 len 8192 2024-05-08T06:12:28.666 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4194304 len 16384 2024-05-08T06:12:28.666 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4194304 len 16384 2024-05-08T06:12:28.666 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4194304 len 16384 2024-05-08T06:12:28.666 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4194304 len 16384 2024-05-08T06:12:28.666 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4194816 len 1024 2024-05-08T06:12:28.666 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4194816 len 1024 2024-05-08T06:12:28.666 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4194816 len 1024 2024-05-08T06:12:28.666 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4194816 len 1024 2024-05-08T06:12:28.667 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4194816 len 2048 2024-05-08T06:12:28.667 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4194816 len 2048 2024-05-08T06:12:28.667 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4194816 len 2048 2024-05-08T06:12:28.667 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4194816 len 2048 2024-05-08T06:12:28.667 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4194816 len 4096 2024-05-08T06:12:28.667 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4194816 len 4096 2024-05-08T06:12:28.667 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4194816 len 4096 2024-05-08T06:12:28.667 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4194816 len 4096 2024-05-08T06:12:28.667 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4194816 len 8192 2024-05-08T06:12:28.667 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4194816 len 8192 2024-05-08T06:12:28.667 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4194816 len 8192 2024-05-08T06:12:28.667 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4194816 len 8192 2024-05-08T06:12:28.667 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4194816 len 16384 2024-05-08T06:12:28.667 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4194816 len 16384 2024-05-08T06:12:28.667 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4194816 len 16384 2024-05-08T06:12:28.667 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4194816 len 16384 2024-05-08T06:12:28.668 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4195328 len 1024 2024-05-08T06:12:28.668 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4195328 len 1024 2024-05-08T06:12:28.668 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4195328 len 1024 2024-05-08T06:12:28.668 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4195328 len 1024 2024-05-08T06:12:28.668 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4195328 len 2048 2024-05-08T06:12:28.668 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4195328 len 2048 2024-05-08T06:12:28.668 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4195328 len 2048 2024-05-08T06:12:28.668 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4195328 len 2048 2024-05-08T06:12:28.668 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4195328 len 4096 2024-05-08T06:12:28.668 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4195328 len 4096 2024-05-08T06:12:28.668 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4195328 len 4096 2024-05-08T06:12:28.668 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4195328 len 4096 2024-05-08T06:12:28.668 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4195328 len 8192 2024-05-08T06:12:28.668 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4195328 len 8192 2024-05-08T06:12:28.668 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4195328 len 8192 2024-05-08T06:12:28.668 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4195328 len 8192 2024-05-08T06:12:28.669 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4195328 len 16384 2024-05-08T06:12:28.669 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4195328 len 16384 2024-05-08T06:12:28.669 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4195328 len 16384 2024-05-08T06:12:28.704 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-08T06:12:28.712 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4195328 len 16384 2024-05-08T06:12:28.712 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4195840 len 1024 2024-05-08T06:12:28.712 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4195840 len 1024 2024-05-08T06:12:28.712 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4195840 len 1024 2024-05-08T06:12:28.712 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4195840 len 1024 2024-05-08T06:12:28.712 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4195840 len 2048 2024-05-08T06:12:28.712 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4195840 len 2048 2024-05-08T06:12:28.713 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4195840 len 2048 2024-05-08T06:12:28.713 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4195840 len 2048 2024-05-08T06:12:28.713 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4195840 len 4096 2024-05-08T06:12:28.713 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4195840 len 4096 2024-05-08T06:12:28.713 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4195840 len 4096 2024-05-08T06:12:28.713 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4195840 len 4096 2024-05-08T06:12:28.713 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4195840 len 8192 2024-05-08T06:12:28.713 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4195840 len 8192 2024-05-08T06:12:28.713 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4195840 len 8192 2024-05-08T06:12:28.713 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4195840 len 8192 2024-05-08T06:12:28.713 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4195840 len 16384 2024-05-08T06:12:28.713 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4195840 len 16384 2024-05-08T06:12:28.714 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4195840 len 16384 2024-05-08T06:12:28.714 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4195840 len 16384 2024-05-08T06:12:28.714 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4196352 len 1024 2024-05-08T06:12:28.714 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4196352 len 1024 2024-05-08T06:12:28.714 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4196352 len 1024 2024-05-08T06:12:28.714 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4196352 len 1024 2024-05-08T06:12:28.714 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4196352 len 2048 2024-05-08T06:12:28.714 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4196352 len 2048 2024-05-08T06:12:28.714 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4196352 len 2048 2024-05-08T06:12:28.714 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4196352 len 2048 2024-05-08T06:12:28.714 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4196352 len 4096 2024-05-08T06:12:28.714 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4196352 len 4096 2024-05-08T06:12:28.715 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4196352 len 4096 2024-05-08T06:12:28.715 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4196352 len 4096 2024-05-08T06:12:28.715 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4196352 len 8192 2024-05-08T06:12:28.715 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4196352 len 8192 2024-05-08T06:12:28.715 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4196352 len 8192 2024-05-08T06:12:28.715 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4196352 len 8192 2024-05-08T06:12:28.715 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4196352 len 16384 2024-05-08T06:12:28.715 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4196352 len 16384 2024-05-08T06:12:28.715 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4196352 len 16384 2024-05-08T06:12:28.715 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4196352 len 16384 2024-05-08T06:12:28.715 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4196864 len 1024 2024-05-08T06:12:28.715 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4196864 len 1024 2024-05-08T06:12:28.715 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4196864 len 1024 2024-05-08T06:12:28.716 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4196864 len 1024 2024-05-08T06:12:28.716 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4196864 len 2048 2024-05-08T06:12:28.716 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4196864 len 2048 2024-05-08T06:12:28.716 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4196864 len 2048 2024-05-08T06:12:28.716 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4196864 len 2048 2024-05-08T06:12:28.716 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4196864 len 4096 2024-05-08T06:12:28.716 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4196864 len 4096 2024-05-08T06:12:28.716 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4196864 len 4096 2024-05-08T06:12:28.716 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4196864 len 4096 2024-05-08T06:12:28.716 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4196864 len 8192 2024-05-08T06:12:28.716 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4196864 len 8192 2024-05-08T06:12:28.716 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4196864 len 8192 2024-05-08T06:12:28.716 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4196864 len 8192 2024-05-08T06:12:28.717 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4196864 len 16384 2024-05-08T06:12:28.717 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4196864 len 16384 2024-05-08T06:12:28.717 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4196864 len 16384 2024-05-08T06:12:28.717 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4196864 len 16384 2024-05-08T06:12:28.717 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4197376 len 1024 2024-05-08T06:12:28.717 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4197376 len 1024 2024-05-08T06:12:28.717 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4197376 len 1024 2024-05-08T06:12:28.717 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4197376 len 1024 2024-05-08T06:12:28.717 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4197376 len 2048 2024-05-08T06:12:28.717 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4197376 len 2048 2024-05-08T06:12:28.717 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4197376 len 2048 2024-05-08T06:12:28.718 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4197376 len 2048 2024-05-08T06:12:28.718 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4197376 len 4096 2024-05-08T06:12:28.718 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4197376 len 4096 2024-05-08T06:12:28.718 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4197376 len 4096 2024-05-08T06:12:28.718 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4197376 len 4096 2024-05-08T06:12:28.718 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4197376 len 8192 2024-05-08T06:12:28.718 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4197376 len 8192 2024-05-08T06:12:28.718 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4197376 len 8192 2024-05-08T06:12:28.718 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4197376 len 8192 2024-05-08T06:12:28.718 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4197376 len 16384 2024-05-08T06:12:28.718 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4197376 len 16384 2024-05-08T06:12:28.718 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4197376 len 16384 2024-05-08T06:12:28.718 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4197376 len 16384 2024-05-08T06:12:28.719 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4197888 len 1024 2024-05-08T06:12:28.749 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4197888 len 1024 2024-05-08T06:12:28.749 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4197888 len 1024 2024-05-08T06:12:28.749 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4197888 len 1024 2024-05-08T06:12:28.749 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4197888 len 2048 2024-05-08T06:12:28.750 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4197888 len 2048 2024-05-08T06:12:28.750 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4197888 len 2048 2024-05-08T06:12:28.750 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4197888 len 2048 2024-05-08T06:12:28.750 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4197888 len 4096 2024-05-08T06:12:28.750 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4197888 len 4096 2024-05-08T06:12:28.750 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4197888 len 4096 2024-05-08T06:12:28.750 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4197888 len 4096 2024-05-08T06:12:28.750 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4197888 len 8192 2024-05-08T06:12:28.750 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4197888 len 8192 2024-05-08T06:12:28.750 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4197888 len 8192 2024-05-08T06:12:28.750 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4197888 len 8192 2024-05-08T06:12:28.751 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 1536 offset 4197888 len 16384 2024-05-08T06:12:28.751 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4197888 len 16384 2024-05-08T06:12:28.751 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 1536 offset 4197888 len 16384 2024-05-08T06:12:28.751 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 1536 offset 4197888 len 16384 2024-05-08T06:12:28.751 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4190208 len 1024 2024-05-08T06:12:28.751 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4190208 len 1024 2024-05-08T06:12:28.751 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4190208 len 1024 2024-05-08T06:12:28.751 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4190208 len 1024 2024-05-08T06:12:28.751 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4190208 len 2048 2024-05-08T06:12:28.751 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4190208 len 2048 2024-05-08T06:12:28.751 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4190208 len 2048 2024-05-08T06:12:28.751 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4190208 len 2048 2024-05-08T06:12:28.752 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4190208 len 4096 2024-05-08T06:12:28.752 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4190208 len 4096 2024-05-08T06:12:28.752 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4190208 len 4096 2024-05-08T06:12:28.752 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4190208 len 4096 2024-05-08T06:12:28.752 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4190208 len 8192 2024-05-08T06:12:28.752 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4190208 len 8192 2024-05-08T06:12:28.752 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4190208 len 8192 2024-05-08T06:12:28.752 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4190208 len 8192 2024-05-08T06:12:28.752 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4190208 len 16384 2024-05-08T06:12:28.752 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4190208 len 16384 2024-05-08T06:12:28.752 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4190208 len 16384 2024-05-08T06:12:28.752 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4190208 len 16384 2024-05-08T06:12:28.753 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4190720 len 1024 2024-05-08T06:12:28.753 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4190720 len 1024 2024-05-08T06:12:28.753 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4190720 len 1024 2024-05-08T06:12:28.753 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4190720 len 1024 2024-05-08T06:12:28.753 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4190720 len 2048 2024-05-08T06:12:28.753 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4190720 len 2048 2024-05-08T06:12:28.753 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4190720 len 2048 2024-05-08T06:12:28.753 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4190720 len 2048 2024-05-08T06:12:28.753 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4190720 len 4096 2024-05-08T06:12:28.753 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4190720 len 4096 2024-05-08T06:12:28.753 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4190720 len 4096 2024-05-08T06:12:28.753 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4190720 len 4096 2024-05-08T06:12:28.754 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4190720 len 8192 2024-05-08T06:12:28.754 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4190720 len 8192 2024-05-08T06:12:28.754 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4190720 len 8192 2024-05-08T06:12:28.754 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4190720 len 8192 2024-05-08T06:12:28.754 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4190720 len 16384 2024-05-08T06:12:28.754 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4190720 len 16384 2024-05-08T06:12:28.754 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4190720 len 16384 2024-05-08T06:12:28.754 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4190720 len 16384 2024-05-08T06:12:28.754 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4191232 len 1024 2024-05-08T06:12:28.754 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4191232 len 1024 2024-05-08T06:12:28.754 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4191232 len 1024 2024-05-08T06:12:28.754 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4191232 len 1024 2024-05-08T06:12:28.755 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4191232 len 2048 2024-05-08T06:12:28.755 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4191232 len 2048 2024-05-08T06:12:28.755 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4191232 len 2048 2024-05-08T06:12:28.755 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4191232 len 2048 2024-05-08T06:12:28.755 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4191232 len 4096 2024-05-08T06:12:28.755 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4191232 len 4096 2024-05-08T06:12:28.755 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4191232 len 4096 2024-05-08T06:12:28.755 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4191232 len 4096 2024-05-08T06:12:28.755 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4191232 len 8192 2024-05-08T06:12:28.755 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4191232 len 8192 2024-05-08T06:12:28.755 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4191232 len 8192 2024-05-08T06:12:28.755 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4191232 len 8192 2024-05-08T06:12:28.756 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4191232 len 16384 2024-05-08T06:12:28.756 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4191232 len 16384 2024-05-08T06:12:28.756 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4191232 len 16384 2024-05-08T06:12:28.756 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4191232 len 16384 2024-05-08T06:12:28.756 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4191744 len 1024 2024-05-08T06:12:28.756 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4191744 len 1024 2024-05-08T06:12:28.756 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4191744 len 1024 2024-05-08T06:12:28.756 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4191744 len 1024 2024-05-08T06:12:28.797 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4191744 len 2048 2024-05-08T06:12:28.797 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4191744 len 2048 2024-05-08T06:12:28.797 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4191744 len 2048 2024-05-08T06:12:28.797 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4191744 len 2048 2024-05-08T06:12:28.797 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4191744 len 4096 2024-05-08T06:12:28.797 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4191744 len 4096 2024-05-08T06:12:28.797 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4191744 len 4096 2024-05-08T06:12:28.798 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4191744 len 4096 2024-05-08T06:12:28.798 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4191744 len 8192 2024-05-08T06:12:28.798 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4191744 len 8192 2024-05-08T06:12:28.798 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4191744 len 8192 2024-05-08T06:12:28.798 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4191744 len 8192 2024-05-08T06:12:28.798 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4191744 len 16384 2024-05-08T06:12:28.798 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4191744 len 16384 2024-05-08T06:12:28.798 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4191744 len 16384 2024-05-08T06:12:28.798 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4191744 len 16384 2024-05-08T06:12:28.798 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4192256 len 1024 2024-05-08T06:12:28.798 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4192256 len 1024 2024-05-08T06:12:28.798 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4192256 len 1024 2024-05-08T06:12:28.798 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4192256 len 1024 2024-05-08T06:12:28.798 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4192256 len 2048 2024-05-08T06:12:28.799 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4192256 len 2048 2024-05-08T06:12:28.799 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4192256 len 2048 2024-05-08T06:12:28.799 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4192256 len 2048 2024-05-08T06:12:28.799 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4192256 len 4096 2024-05-08T06:12:28.799 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4192256 len 4096 2024-05-08T06:12:28.799 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4192256 len 4096 2024-05-08T06:12:28.799 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4192256 len 4096 2024-05-08T06:12:28.799 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4192256 len 8192 2024-05-08T06:12:28.799 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4192256 len 8192 2024-05-08T06:12:28.799 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4192256 len 8192 2024-05-08T06:12:28.799 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4192256 len 8192 2024-05-08T06:12:28.799 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4192256 len 16384 2024-05-08T06:12:28.799 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4192256 len 16384 2024-05-08T06:12:28.799 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4192256 len 16384 2024-05-08T06:12:28.800 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4192256 len 16384 2024-05-08T06:12:28.800 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4192768 len 1024 2024-05-08T06:12:28.800 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4192768 len 1024 2024-05-08T06:12:28.800 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4192768 len 1024 2024-05-08T06:12:28.800 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4192768 len 1024 2024-05-08T06:12:28.800 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4192768 len 2048 2024-05-08T06:12:28.800 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4192768 len 2048 2024-05-08T06:12:28.800 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4192768 len 2048 2024-05-08T06:12:28.800 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4192768 len 2048 2024-05-08T06:12:28.800 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4192768 len 4096 2024-05-08T06:12:28.800 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4192768 len 4096 2024-05-08T06:12:28.800 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4192768 len 4096 2024-05-08T06:12:28.800 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4192768 len 4096 2024-05-08T06:12:28.800 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4192768 len 8192 2024-05-08T06:12:28.801 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4192768 len 8192 2024-05-08T06:12:28.801 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4192768 len 8192 2024-05-08T06:12:28.801 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4192768 len 8192 2024-05-08T06:12:28.801 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4192768 len 16384 2024-05-08T06:12:28.801 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4192768 len 16384 2024-05-08T06:12:28.801 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4192768 len 16384 2024-05-08T06:12:28.801 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4192768 len 16384 2024-05-08T06:12:28.801 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4193280 len 1024 2024-05-08T06:12:28.801 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4193280 len 1024 2024-05-08T06:12:28.801 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4193280 len 1024 2024-05-08T06:12:28.801 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4193280 len 1024 2024-05-08T06:12:28.801 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4193280 len 2048 2024-05-08T06:12:28.801 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4193280 len 2048 2024-05-08T06:12:28.801 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4193280 len 2048 2024-05-08T06:12:28.802 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4193280 len 2048 2024-05-08T06:12:28.802 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4193280 len 4096 2024-05-08T06:12:28.802 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4193280 len 4096 2024-05-08T06:12:28.802 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4193280 len 4096 2024-05-08T06:12:28.802 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4193280 len 4096 2024-05-08T06:12:28.802 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4193280 len 8192 2024-05-08T06:12:28.802 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4193280 len 8192 2024-05-08T06:12:28.802 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4193280 len 8192 2024-05-08T06:12:28.802 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4193280 len 8192 2024-05-08T06:12:28.802 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4193280 len 16384 2024-05-08T06:12:28.802 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4193280 len 16384 2024-05-08T06:12:28.802 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4193280 len 16384 2024-05-08T06:12:28.802 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4193280 len 16384 2024-05-08T06:12:28.803 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4193792 len 1024 2024-05-08T06:12:28.803 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4193792 len 1024 2024-05-08T06:12:28.803 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4193792 len 1024 2024-05-08T06:12:28.803 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4193792 len 1024 2024-05-08T06:12:28.803 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4193792 len 2048 2024-05-08T06:12:28.803 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4193792 len 2048 2024-05-08T06:12:28.845 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4193792 len 2048 2024-05-08T06:12:28.846 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4193792 len 2048 2024-05-08T06:12:28.846 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4193792 len 4096 2024-05-08T06:12:28.846 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4193792 len 4096 2024-05-08T06:12:28.846 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4193792 len 4096 2024-05-08T06:12:28.846 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4193792 len 4096 2024-05-08T06:12:28.846 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4193792 len 8192 2024-05-08T06:12:28.846 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4193792 len 8192 2024-05-08T06:12:28.846 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4193792 len 8192 2024-05-08T06:12:28.846 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4193792 len 8192 2024-05-08T06:12:28.846 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4193792 len 16384 2024-05-08T06:12:28.846 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4193792 len 16384 2024-05-08T06:12:28.847 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4193792 len 16384 2024-05-08T06:12:28.847 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4193792 len 16384 2024-05-08T06:12:28.847 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4194304 len 1024 2024-05-08T06:12:28.847 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4194304 len 1024 2024-05-08T06:12:28.847 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4194304 len 1024 2024-05-08T06:12:28.847 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4194304 len 1024 2024-05-08T06:12:28.847 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4194304 len 2048 2024-05-08T06:12:28.847 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4194304 len 2048 2024-05-08T06:12:28.847 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4194304 len 2048 2024-05-08T06:12:28.847 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4194304 len 2048 2024-05-08T06:12:28.847 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4194304 len 4096 2024-05-08T06:12:28.847 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4194304 len 4096 2024-05-08T06:12:28.847 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4194304 len 4096 2024-05-08T06:12:28.848 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4194304 len 4096 2024-05-08T06:12:28.848 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4194304 len 8192 2024-05-08T06:12:28.848 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4194304 len 8192 2024-05-08T06:12:28.848 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4194304 len 8192 2024-05-08T06:12:28.848 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4194304 len 8192 2024-05-08T06:12:28.848 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4194304 len 16384 2024-05-08T06:12:28.848 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4194304 len 16384 2024-05-08T06:12:28.848 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4194304 len 16384 2024-05-08T06:12:28.848 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4194304 len 16384 2024-05-08T06:12:28.848 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4194816 len 1024 2024-05-08T06:12:28.848 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4194816 len 1024 2024-05-08T06:12:28.848 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4194816 len 1024 2024-05-08T06:12:28.848 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4194816 len 1024 2024-05-08T06:12:28.848 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4194816 len 2048 2024-05-08T06:12:28.848 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4194816 len 2048 2024-05-08T06:12:28.849 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4194816 len 2048 2024-05-08T06:12:28.849 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4194816 len 2048 2024-05-08T06:12:28.849 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4194816 len 4096 2024-05-08T06:12:28.849 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4194816 len 4096 2024-05-08T06:12:28.849 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4194816 len 4096 2024-05-08T06:12:28.849 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4194816 len 4096 2024-05-08T06:12:28.849 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4194816 len 8192 2024-05-08T06:12:28.849 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4194816 len 8192 2024-05-08T06:12:28.849 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4194816 len 8192 2024-05-08T06:12:28.849 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4194816 len 8192 2024-05-08T06:12:28.849 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4194816 len 16384 2024-05-08T06:12:28.849 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4194816 len 16384 2024-05-08T06:12:28.849 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4194816 len 16384 2024-05-08T06:12:28.849 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4194816 len 16384 2024-05-08T06:12:28.850 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4195328 len 1024 2024-05-08T06:12:28.850 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4195328 len 1024 2024-05-08T06:12:28.850 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4195328 len 1024 2024-05-08T06:12:28.850 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4195328 len 1024 2024-05-08T06:12:28.850 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4195328 len 2048 2024-05-08T06:12:28.850 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4195328 len 2048 2024-05-08T06:12:28.850 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4195328 len 2048 2024-05-08T06:12:28.850 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4195328 len 2048 2024-05-08T06:12:28.850 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4195328 len 4096 2024-05-08T06:12:28.850 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4195328 len 4096 2024-05-08T06:12:28.850 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4195328 len 4096 2024-05-08T06:12:28.850 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4195328 len 4096 2024-05-08T06:12:28.850 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4195328 len 8192 2024-05-08T06:12:28.850 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4195328 len 8192 2024-05-08T06:12:28.850 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4195328 len 8192 2024-05-08T06:12:28.851 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4195328 len 8192 2024-05-08T06:12:28.851 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4195328 len 16384 2024-05-08T06:12:28.851 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4195328 len 16384 2024-05-08T06:12:28.851 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4195328 len 16384 2024-05-08T06:12:28.851 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4195328 len 16384 2024-05-08T06:12:28.851 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4195840 len 1024 2024-05-08T06:12:28.851 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4195840 len 1024 2024-05-08T06:12:28.851 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4195840 len 1024 2024-05-08T06:12:28.851 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4195840 len 1024 2024-05-08T06:12:28.851 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4195840 len 2048 2024-05-08T06:12:28.851 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4195840 len 2048 2024-05-08T06:12:28.851 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4195840 len 2048 2024-05-08T06:12:28.851 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4195840 len 2048 2024-05-08T06:12:28.886 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4195840 len 4096 2024-05-08T06:12:28.887 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4195840 len 4096 2024-05-08T06:12:28.887 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4195840 len 4096 2024-05-08T06:12:28.887 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4195840 len 4096 2024-05-08T06:12:28.887 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4195840 len 8192 2024-05-08T06:12:28.887 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4195840 len 8192 2024-05-08T06:12:28.887 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4195840 len 8192 2024-05-08T06:12:28.887 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4195840 len 8192 2024-05-08T06:12:28.887 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4195840 len 16384 2024-05-08T06:12:28.887 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4195840 len 16384 2024-05-08T06:12:28.888 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4195840 len 16384 2024-05-08T06:12:28.888 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4195840 len 16384 2024-05-08T06:12:28.888 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4196352 len 1024 2024-05-08T06:12:28.888 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4196352 len 1024 2024-05-08T06:12:28.888 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4196352 len 1024 2024-05-08T06:12:28.888 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4196352 len 1024 2024-05-08T06:12:28.888 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4196352 len 2048 2024-05-08T06:12:28.888 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4196352 len 2048 2024-05-08T06:12:28.888 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4196352 len 2048 2024-05-08T06:12:28.888 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4196352 len 2048 2024-05-08T06:12:28.888 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4196352 len 4096 2024-05-08T06:12:28.888 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4196352 len 4096 2024-05-08T06:12:28.888 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4196352 len 4096 2024-05-08T06:12:28.889 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4196352 len 4096 2024-05-08T06:12:28.889 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4196352 len 8192 2024-05-08T06:12:28.889 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4196352 len 8192 2024-05-08T06:12:28.889 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4196352 len 8192 2024-05-08T06:12:28.889 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4196352 len 8192 2024-05-08T06:12:28.889 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4196352 len 16384 2024-05-08T06:12:28.889 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4196352 len 16384 2024-05-08T06:12:28.889 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4196352 len 16384 2024-05-08T06:12:28.889 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4196352 len 16384 2024-05-08T06:12:28.889 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4196864 len 1024 2024-05-08T06:12:28.889 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4196864 len 1024 2024-05-08T06:12:28.890 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4196864 len 1024 2024-05-08T06:12:28.890 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4196864 len 1024 2024-05-08T06:12:28.890 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4196864 len 2048 2024-05-08T06:12:28.890 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4196864 len 2048 2024-05-08T06:12:28.890 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4196864 len 2048 2024-05-08T06:12:28.890 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4196864 len 2048 2024-05-08T06:12:28.890 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4196864 len 4096 2024-05-08T06:12:28.890 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4196864 len 4096 2024-05-08T06:12:28.890 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4196864 len 4096 2024-05-08T06:12:28.890 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4196864 len 4096 2024-05-08T06:12:28.890 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4196864 len 8192 2024-05-08T06:12:28.890 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4196864 len 8192 2024-05-08T06:12:28.891 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4196864 len 8192 2024-05-08T06:12:28.891 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4196864 len 8192 2024-05-08T06:12:28.891 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4196864 len 16384 2024-05-08T06:12:28.891 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4196864 len 16384 2024-05-08T06:12:28.891 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4196864 len 16384 2024-05-08T06:12:28.891 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4196864 len 16384 2024-05-08T06:12:28.891 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4197376 len 1024 2024-05-08T06:12:28.891 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4197376 len 1024 2024-05-08T06:12:28.891 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4197376 len 1024 2024-05-08T06:12:28.891 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4197376 len 1024 2024-05-08T06:12:28.891 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4197376 len 2048 2024-05-08T06:12:28.892 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4197376 len 2048 2024-05-08T06:12:28.892 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4197376 len 2048 2024-05-08T06:12:28.892 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4197376 len 2048 2024-05-08T06:12:28.892 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4197376 len 4096 2024-05-08T06:12:28.892 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4197376 len 4096 2024-05-08T06:12:28.892 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4197376 len 4096 2024-05-08T06:12:28.892 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4197376 len 4096 2024-05-08T06:12:28.892 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4197376 len 8192 2024-05-08T06:12:28.892 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4197376 len 8192 2024-05-08T06:12:28.892 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4197376 len 8192 2024-05-08T06:12:28.892 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4197376 len 8192 2024-05-08T06:12:28.893 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4197376 len 16384 2024-05-08T06:12:28.893 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4197376 len 16384 2024-05-08T06:12:28.893 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4197376 len 16384 2024-05-08T06:12:28.893 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4197376 len 16384 2024-05-08T06:12:28.893 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4197888 len 1024 2024-05-08T06:12:28.893 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4197888 len 1024 2024-05-08T06:12:28.893 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4197888 len 1024 2024-05-08T06:12:28.893 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4197888 len 1024 2024-05-08T06:12:28.893 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4197888 len 2048 2024-05-08T06:12:28.893 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4197888 len 2048 2024-05-08T06:12:28.893 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4197888 len 2048 2024-05-08T06:12:28.893 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4197888 len 2048 2024-05-08T06:12:28.893 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4197888 len 4096 2024-05-08T06:12:28.893 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4197888 len 4096 2024-05-08T06:12:28.894 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4197888 len 4096 2024-05-08T06:12:28.937 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4197888 len 4096 2024-05-08T06:12:28.938 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4197888 len 8192 2024-05-08T06:12:28.938 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4197888 len 8192 2024-05-08T06:12:28.938 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4197888 len 8192 2024-05-08T06:12:28.938 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4197888 len 8192 2024-05-08T06:12:28.938 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2048 offset 4197888 len 16384 2024-05-08T06:12:28.938 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4197888 len 16384 2024-05-08T06:12:28.938 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2048 offset 4197888 len 16384 2024-05-08T06:12:28.938 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2048 offset 4197888 len 16384 2024-05-08T06:12:28.938 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4190208 len 1024 2024-05-08T06:12:28.938 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4190208 len 1024 2024-05-08T06:12:28.938 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4190208 len 1024 2024-05-08T06:12:28.938 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4190208 len 1024 2024-05-08T06:12:28.939 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4190208 len 2048 2024-05-08T06:12:28.939 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4190208 len 2048 2024-05-08T06:12:28.939 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4190208 len 2048 2024-05-08T06:12:28.939 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4190208 len 2048 2024-05-08T06:12:28.939 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4190208 len 4096 2024-05-08T06:12:28.939 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4190208 len 4096 2024-05-08T06:12:28.939 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4190208 len 4096 2024-05-08T06:12:28.939 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4190208 len 4096 2024-05-08T06:12:28.939 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4190208 len 8192 2024-05-08T06:12:28.939 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4190208 len 8192 2024-05-08T06:12:28.939 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4190208 len 8192 2024-05-08T06:12:28.939 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4190208 len 8192 2024-05-08T06:12:28.939 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4190208 len 16384 2024-05-08T06:12:28.939 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4190208 len 16384 2024-05-08T06:12:28.940 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4190208 len 16384 2024-05-08T06:12:28.940 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4190208 len 16384 2024-05-08T06:12:28.940 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4190720 len 1024 2024-05-08T06:12:28.940 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4190720 len 1024 2024-05-08T06:12:28.940 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4190720 len 1024 2024-05-08T06:12:28.940 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4190720 len 1024 2024-05-08T06:12:28.940 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4190720 len 2048 2024-05-08T06:12:28.940 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4190720 len 2048 2024-05-08T06:12:28.940 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4190720 len 2048 2024-05-08T06:12:28.940 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4190720 len 2048 2024-05-08T06:12:28.940 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4190720 len 4096 2024-05-08T06:12:28.940 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4190720 len 4096 2024-05-08T06:12:28.940 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4190720 len 4096 2024-05-08T06:12:28.940 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4190720 len 4096 2024-05-08T06:12:28.940 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4190720 len 8192 2024-05-08T06:12:28.941 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4190720 len 8192 2024-05-08T06:12:28.941 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4190720 len 8192 2024-05-08T06:12:28.941 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4190720 len 8192 2024-05-08T06:12:28.941 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4190720 len 16384 2024-05-08T06:12:28.941 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4190720 len 16384 2024-05-08T06:12:28.941 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4190720 len 16384 2024-05-08T06:12:28.941 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4190720 len 16384 2024-05-08T06:12:28.941 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4191232 len 1024 2024-05-08T06:12:28.941 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4191232 len 1024 2024-05-08T06:12:28.942 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4191232 len 1024 2024-05-08T06:12:28.942 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4191232 len 1024 2024-05-08T06:12:28.942 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4191232 len 2048 2024-05-08T06:12:28.942 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4191232 len 2048 2024-05-08T06:12:28.942 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4191232 len 2048 2024-05-08T06:12:28.942 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4191232 len 2048 2024-05-08T06:12:28.942 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4191232 len 4096 2024-05-08T06:12:28.942 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4191232 len 4096 2024-05-08T06:12:28.942 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4191232 len 4096 2024-05-08T06:12:28.942 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4191232 len 4096 2024-05-08T06:12:28.942 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4191232 len 8192 2024-05-08T06:12:28.942 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4191232 len 8192 2024-05-08T06:12:28.942 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4191232 len 8192 2024-05-08T06:12:28.943 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4191232 len 8192 2024-05-08T06:12:28.943 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4191232 len 16384 2024-05-08T06:12:28.943 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4191232 len 16384 2024-05-08T06:12:28.943 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4191232 len 16384 2024-05-08T06:12:28.943 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4191232 len 16384 2024-05-08T06:12:28.943 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4191744 len 1024 2024-05-08T06:12:28.943 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4191744 len 1024 2024-05-08T06:12:28.943 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4191744 len 1024 2024-05-08T06:12:28.943 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4191744 len 1024 2024-05-08T06:12:28.943 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4191744 len 2048 2024-05-08T06:12:28.943 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4191744 len 2048 2024-05-08T06:12:28.944 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4191744 len 2048 2024-05-08T06:12:28.944 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4191744 len 2048 2024-05-08T06:12:28.944 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4191744 len 4096 2024-05-08T06:12:28.944 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4191744 len 4096 2024-05-08T06:12:28.944 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4191744 len 4096 2024-05-08T06:12:28.944 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4191744 len 4096 2024-05-08T06:12:28.944 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4191744 len 8192 2024-05-08T06:12:28.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:28 smithi123 ceph-mon[29787]: pgmap v520: 97 pgs: 97 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.4 KiB/s wr, 1.09k op/s 2024-05-08T06:12:28.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:28 smithi123 ceph-mon[29787]: scrub queued for path: / 2024-05-08T06:12:28.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:28 smithi123 ceph-mon[29787]: scrub summary: idle+waiting paths [/] 2024-05-08T06:12:28.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:28 smithi123 ceph-mon[29787]: scrub summary: active paths [/] 2024-05-08T06:12:28.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:28 smithi123 ceph-mon[29787]: scrub summary: idle+waiting paths [/] 2024-05-08T06:12:28.995 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4191744 len 8192 2024-05-08T06:12:28.995 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4191744 len 8192 2024-05-08T06:12:28.995 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4191744 len 8192 2024-05-08T06:12:28.995 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4191744 len 16384 2024-05-08T06:12:28.995 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4191744 len 16384 2024-05-08T06:12:28.995 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4191744 len 16384 2024-05-08T06:12:28.996 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4191744 len 16384 2024-05-08T06:12:28.996 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4192256 len 1024 2024-05-08T06:12:28.996 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4192256 len 1024 2024-05-08T06:12:28.996 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4192256 len 1024 2024-05-08T06:12:28.996 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4192256 len 1024 2024-05-08T06:12:28.996 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4192256 len 2048 2024-05-08T06:12:28.996 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4192256 len 2048 2024-05-08T06:12:28.996 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4192256 len 2048 2024-05-08T06:12:28.996 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4192256 len 2048 2024-05-08T06:12:28.996 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4192256 len 4096 2024-05-08T06:12:28.996 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4192256 len 4096 2024-05-08T06:12:28.996 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4192256 len 4096 2024-05-08T06:12:28.996 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4192256 len 4096 2024-05-08T06:12:28.996 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4192256 len 8192 2024-05-08T06:12:28.997 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4192256 len 8192 2024-05-08T06:12:28.997 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4192256 len 8192 2024-05-08T06:12:28.997 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4192256 len 8192 2024-05-08T06:12:28.997 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4192256 len 16384 2024-05-08T06:12:28.997 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4192256 len 16384 2024-05-08T06:12:28.997 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4192256 len 16384 2024-05-08T06:12:28.997 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4192256 len 16384 2024-05-08T06:12:28.997 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4192768 len 1024 2024-05-08T06:12:28.997 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4192768 len 1024 2024-05-08T06:12:28.997 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4192768 len 1024 2024-05-08T06:12:28.997 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4192768 len 1024 2024-05-08T06:12:28.997 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4192768 len 2048 2024-05-08T06:12:28.997 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4192768 len 2048 2024-05-08T06:12:28.997 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4192768 len 2048 2024-05-08T06:12:28.997 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4192768 len 2048 2024-05-08T06:12:28.998 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4192768 len 4096 2024-05-08T06:12:28.998 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4192768 len 4096 2024-05-08T06:12:28.998 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4192768 len 4096 2024-05-08T06:12:28.998 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4192768 len 4096 2024-05-08T06:12:28.998 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4192768 len 8192 2024-05-08T06:12:28.998 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4192768 len 8192 2024-05-08T06:12:28.998 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4192768 len 8192 2024-05-08T06:12:28.998 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4192768 len 8192 2024-05-08T06:12:28.998 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4192768 len 16384 2024-05-08T06:12:28.998 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4192768 len 16384 2024-05-08T06:12:28.998 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4192768 len 16384 2024-05-08T06:12:28.998 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4192768 len 16384 2024-05-08T06:12:28.998 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4193280 len 1024 2024-05-08T06:12:28.998 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4193280 len 1024 2024-05-08T06:12:28.999 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4193280 len 1024 2024-05-08T06:12:28.999 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4193280 len 1024 2024-05-08T06:12:28.999 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4193280 len 2048 2024-05-08T06:12:28.999 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4193280 len 2048 2024-05-08T06:12:28.999 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4193280 len 2048 2024-05-08T06:12:28.999 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4193280 len 2048 2024-05-08T06:12:28.999 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4193280 len 4096 2024-05-08T06:12:28.999 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4193280 len 4096 2024-05-08T06:12:28.999 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4193280 len 4096 2024-05-08T06:12:28.999 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4193280 len 4096 2024-05-08T06:12:28.999 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4193280 len 8192 2024-05-08T06:12:28.999 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4193280 len 8192 2024-05-08T06:12:28.999 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4193280 len 8192 2024-05-08T06:12:28.999 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4193280 len 8192 2024-05-08T06:12:28.999 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4193280 len 16384 2024-05-08T06:12:28.999 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4193280 len 16384 2024-05-08T06:12:29.000 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4193280 len 16384 2024-05-08T06:12:29.000 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4193280 len 16384 2024-05-08T06:12:29.000 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4193792 len 1024 2024-05-08T06:12:29.000 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4193792 len 1024 2024-05-08T06:12:29.000 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4193792 len 1024 2024-05-08T06:12:29.000 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4193792 len 1024 2024-05-08T06:12:29.000 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4193792 len 2048 2024-05-08T06:12:29.000 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4193792 len 2048 2024-05-08T06:12:29.000 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4193792 len 2048 2024-05-08T06:12:29.000 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4193792 len 2048 2024-05-08T06:12:29.000 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4193792 len 4096 2024-05-08T06:12:29.000 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4193792 len 4096 2024-05-08T06:12:29.000 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4193792 len 4096 2024-05-08T06:12:29.000 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4193792 len 4096 2024-05-08T06:12:29.001 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4193792 len 8192 2024-05-08T06:12:29.001 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4193792 len 8192 2024-05-08T06:12:29.001 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4193792 len 8192 2024-05-08T06:12:29.001 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4193792 len 8192 2024-05-08T06:12:29.013 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4193792 len 16384 2024-05-08T06:12:29.013 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4193792 len 16384 2024-05-08T06:12:29.014 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4193792 len 16384 2024-05-08T06:12:29.014 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4193792 len 16384 2024-05-08T06:12:29.014 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4194304 len 1024 2024-05-08T06:12:29.014 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4194304 len 1024 2024-05-08T06:12:29.014 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4194304 len 1024 2024-05-08T06:12:29.014 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4194304 len 1024 2024-05-08T06:12:29.014 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4194304 len 2048 2024-05-08T06:12:29.014 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4194304 len 2048 2024-05-08T06:12:29.014 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4194304 len 2048 2024-05-08T06:12:29.014 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4194304 len 2048 2024-05-08T06:12:29.014 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4194304 len 4096 2024-05-08T06:12:29.014 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4194304 len 4096 2024-05-08T06:12:29.014 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4194304 len 4096 2024-05-08T06:12:29.014 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4194304 len 4096 2024-05-08T06:12:29.014 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4194304 len 8192 2024-05-08T06:12:29.015 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4194304 len 8192 2024-05-08T06:12:29.015 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4194304 len 8192 2024-05-08T06:12:29.015 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4194304 len 8192 2024-05-08T06:12:29.015 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4194304 len 16384 2024-05-08T06:12:29.015 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4194304 len 16384 2024-05-08T06:12:29.015 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4194304 len 16384 2024-05-08T06:12:29.015 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4194304 len 16384 2024-05-08T06:12:29.015 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4194816 len 1024 2024-05-08T06:12:29.015 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4194816 len 1024 2024-05-08T06:12:29.015 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4194816 len 1024 2024-05-08T06:12:29.015 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4194816 len 1024 2024-05-08T06:12:29.015 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4194816 len 2048 2024-05-08T06:12:29.015 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4194816 len 2048 2024-05-08T06:12:29.015 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4194816 len 2048 2024-05-08T06:12:29.016 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4194816 len 2048 2024-05-08T06:12:29.016 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4194816 len 4096 2024-05-08T06:12:29.016 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4194816 len 4096 2024-05-08T06:12:29.016 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4194816 len 4096 2024-05-08T06:12:29.016 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4194816 len 4096 2024-05-08T06:12:29.016 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4194816 len 8192 2024-05-08T06:12:29.016 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4194816 len 8192 2024-05-08T06:12:29.016 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4194816 len 8192 2024-05-08T06:12:29.016 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4194816 len 8192 2024-05-08T06:12:29.016 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4194816 len 16384 2024-05-08T06:12:29.016 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4194816 len 16384 2024-05-08T06:12:29.016 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4194816 len 16384 2024-05-08T06:12:29.016 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4194816 len 16384 2024-05-08T06:12:29.016 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4195328 len 1024 2024-05-08T06:12:29.017 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4195328 len 1024 2024-05-08T06:12:29.017 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4195328 len 1024 2024-05-08T06:12:29.017 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4195328 len 1024 2024-05-08T06:12:29.017 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4195328 len 2048 2024-05-08T06:12:29.017 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4195328 len 2048 2024-05-08T06:12:29.017 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4195328 len 2048 2024-05-08T06:12:29.017 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4195328 len 2048 2024-05-08T06:12:29.017 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4195328 len 4096 2024-05-08T06:12:29.017 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4195328 len 4096 2024-05-08T06:12:29.017 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4195328 len 4096 2024-05-08T06:12:29.017 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4195328 len 4096 2024-05-08T06:12:29.017 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4195328 len 8192 2024-05-08T06:12:29.017 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4195328 len 8192 2024-05-08T06:12:29.017 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4195328 len 8192 2024-05-08T06:12:29.017 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4195328 len 8192 2024-05-08T06:12:29.018 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4195328 len 16384 2024-05-08T06:12:29.018 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4195328 len 16384 2024-05-08T06:12:29.018 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4195328 len 16384 2024-05-08T06:12:29.018 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4195328 len 16384 2024-05-08T06:12:29.018 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4195840 len 1024 2024-05-08T06:12:29.018 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4195840 len 1024 2024-05-08T06:12:29.018 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4195840 len 1024 2024-05-08T06:12:29.018 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4195840 len 1024 2024-05-08T06:12:29.018 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4195840 len 2048 2024-05-08T06:12:29.018 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4195840 len 2048 2024-05-08T06:12:29.018 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4195840 len 2048 2024-05-08T06:12:29.018 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4195840 len 2048 2024-05-08T06:12:29.018 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4195840 len 4096 2024-05-08T06:12:29.018 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4195840 len 4096 2024-05-08T06:12:29.019 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4195840 len 4096 2024-05-08T06:12:29.019 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4195840 len 4096 2024-05-08T06:12:29.019 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4195840 len 8192 2024-05-08T06:12:29.019 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4195840 len 8192 2024-05-08T06:12:29.019 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4195840 len 8192 2024-05-08T06:12:29.019 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4195840 len 8192 2024-05-08T06:12:29.019 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4195840 len 16384 2024-05-08T06:12:29.019 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4195840 len 16384 2024-05-08T06:12:29.031 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4195840 len 16384 2024-05-08T06:12:29.031 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4195840 len 16384 2024-05-08T06:12:29.031 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4196352 len 1024 2024-05-08T06:12:29.031 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4196352 len 1024 2024-05-08T06:12:29.031 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4196352 len 1024 2024-05-08T06:12:29.031 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4196352 len 1024 2024-05-08T06:12:29.031 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4196352 len 2048 2024-05-08T06:12:29.031 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4196352 len 2048 2024-05-08T06:12:29.031 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4196352 len 2048 2024-05-08T06:12:29.032 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4196352 len 2048 2024-05-08T06:12:29.032 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4196352 len 4096 2024-05-08T06:12:29.032 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4196352 len 4096 2024-05-08T06:12:29.032 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4196352 len 4096 2024-05-08T06:12:29.032 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4196352 len 4096 2024-05-08T06:12:29.032 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4196352 len 8192 2024-05-08T06:12:29.032 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4196352 len 8192 2024-05-08T06:12:29.032 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4196352 len 8192 2024-05-08T06:12:29.032 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4196352 len 8192 2024-05-08T06:12:29.032 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4196352 len 16384 2024-05-08T06:12:29.032 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4196352 len 16384 2024-05-08T06:12:29.032 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4196352 len 16384 2024-05-08T06:12:29.032 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4196352 len 16384 2024-05-08T06:12:29.033 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4196864 len 1024 2024-05-08T06:12:29.033 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4196864 len 1024 2024-05-08T06:12:29.033 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4196864 len 1024 2024-05-08T06:12:29.033 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4196864 len 1024 2024-05-08T06:12:29.033 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4196864 len 2048 2024-05-08T06:12:29.033 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4196864 len 2048 2024-05-08T06:12:29.033 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4196864 len 2048 2024-05-08T06:12:29.033 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4196864 len 2048 2024-05-08T06:12:29.033 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4196864 len 4096 2024-05-08T06:12:29.033 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4196864 len 4096 2024-05-08T06:12:29.033 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4196864 len 4096 2024-05-08T06:12:29.033 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4196864 len 4096 2024-05-08T06:12:29.033 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4196864 len 8192 2024-05-08T06:12:29.033 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4196864 len 8192 2024-05-08T06:12:29.033 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4196864 len 8192 2024-05-08T06:12:29.034 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4196864 len 8192 2024-05-08T06:12:29.034 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4196864 len 16384 2024-05-08T06:12:29.034 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4196864 len 16384 2024-05-08T06:12:29.034 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4196864 len 16384 2024-05-08T06:12:29.034 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4196864 len 16384 2024-05-08T06:12:29.034 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4197376 len 1024 2024-05-08T06:12:29.034 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4197376 len 1024 2024-05-08T06:12:29.034 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4197376 len 1024 2024-05-08T06:12:29.034 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4197376 len 1024 2024-05-08T06:12:29.034 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4197376 len 2048 2024-05-08T06:12:29.034 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4197376 len 2048 2024-05-08T06:12:29.034 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4197376 len 2048 2024-05-08T06:12:29.035 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4197376 len 2048 2024-05-08T06:12:29.035 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4197376 len 4096 2024-05-08T06:12:29.035 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4197376 len 4096 2024-05-08T06:12:29.035 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4197376 len 4096 2024-05-08T06:12:29.035 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4197376 len 4096 2024-05-08T06:12:29.035 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4197376 len 8192 2024-05-08T06:12:29.035 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4197376 len 8192 2024-05-08T06:12:29.035 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4197376 len 8192 2024-05-08T06:12:29.035 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4197376 len 8192 2024-05-08T06:12:29.035 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4197376 len 16384 2024-05-08T06:12:29.035 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4197376 len 16384 2024-05-08T06:12:29.035 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4197376 len 16384 2024-05-08T06:12:29.035 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4197376 len 16384 2024-05-08T06:12:29.035 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4197888 len 1024 2024-05-08T06:12:29.035 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4197888 len 1024 2024-05-08T06:12:29.036 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4197888 len 1024 2024-05-08T06:12:29.036 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4197888 len 1024 2024-05-08T06:12:29.036 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4197888 len 2048 2024-05-08T06:12:29.036 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4197888 len 2048 2024-05-08T06:12:29.036 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4197888 len 2048 2024-05-08T06:12:29.036 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4197888 len 2048 2024-05-08T06:12:29.036 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4197888 len 4096 2024-05-08T06:12:29.036 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4197888 len 4096 2024-05-08T06:12:29.036 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4197888 len 4096 2024-05-08T06:12:29.036 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4197888 len 4096 2024-05-08T06:12:29.036 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4197888 len 8192 2024-05-08T06:12:29.036 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4197888 len 8192 2024-05-08T06:12:29.036 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4197888 len 8192 2024-05-08T06:12:29.036 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4197888 len 8192 2024-05-08T06:12:29.037 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 2560 offset 4197888 len 16384 2024-05-08T06:12:29.037 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4197888 len 16384 2024-05-08T06:12:29.037 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 2560 offset 4197888 len 16384 2024-05-08T06:12:29.037 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 2560 offset 4197888 len 16384 2024-05-08T06:12:29.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:28 smithi163 ceph-mon[34520]: pgmap v520: 97 pgs: 97 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.4 KiB/s wr, 1.09k op/s 2024-05-08T06:12:29.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:28 smithi163 ceph-mon[34520]: scrub queued for path: / 2024-05-08T06:12:29.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:28 smithi163 ceph-mon[34520]: scrub summary: idle+waiting paths [/] 2024-05-08T06:12:29.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:28 smithi163 ceph-mon[34520]: scrub summary: active paths [/] 2024-05-08T06:12:29.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:28 smithi163 ceph-mon[34520]: scrub summary: idle+waiting paths [/] 2024-05-08T06:12:29.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:28 smithi192 ceph-mon[34664]: pgmap v520: 97 pgs: 97 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.4 KiB/s wr, 1.09k op/s 2024-05-08T06:12:29.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:28 smithi192 ceph-mon[34664]: scrub queued for path: / 2024-05-08T06:12:29.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:28 smithi192 ceph-mon[34664]: scrub summary: idle+waiting paths [/] 2024-05-08T06:12:29.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:28 smithi192 ceph-mon[34664]: scrub summary: active paths [/] 2024-05-08T06:12:29.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:28 smithi192 ceph-mon[34664]: scrub summary: idle+waiting paths [/] 2024-05-08T06:12:29.063 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4190208 len 1024 2024-05-08T06:12:29.064 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4190208 len 1024 2024-05-08T06:12:29.064 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4190208 len 1024 2024-05-08T06:12:29.064 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4190208 len 1024 2024-05-08T06:12:29.064 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4190208 len 2048 2024-05-08T06:12:29.064 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4190208 len 2048 2024-05-08T06:12:29.064 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4190208 len 2048 2024-05-08T06:12:29.064 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4190208 len 2048 2024-05-08T06:12:29.064 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4190208 len 4096 2024-05-08T06:12:29.064 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4190208 len 4096 2024-05-08T06:12:29.064 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4190208 len 4096 2024-05-08T06:12:29.064 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4190208 len 4096 2024-05-08T06:12:29.064 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4190208 len 8192 2024-05-08T06:12:29.064 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4190208 len 8192 2024-05-08T06:12:29.064 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4190208 len 8192 2024-05-08T06:12:29.065 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4190208 len 8192 2024-05-08T06:12:29.065 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4190208 len 16384 2024-05-08T06:12:29.065 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4190208 len 16384 2024-05-08T06:12:29.065 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4190208 len 16384 2024-05-08T06:12:29.065 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4190208 len 16384 2024-05-08T06:12:29.065 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4190720 len 1024 2024-05-08T06:12:29.065 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4190720 len 1024 2024-05-08T06:12:29.065 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4190720 len 1024 2024-05-08T06:12:29.065 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4190720 len 1024 2024-05-08T06:12:29.065 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4190720 len 2048 2024-05-08T06:12:29.065 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4190720 len 2048 2024-05-08T06:12:29.065 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4190720 len 2048 2024-05-08T06:12:29.065 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4190720 len 2048 2024-05-08T06:12:29.065 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4190720 len 4096 2024-05-08T06:12:29.066 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4190720 len 4096 2024-05-08T06:12:29.066 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4190720 len 4096 2024-05-08T06:12:29.066 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4190720 len 4096 2024-05-08T06:12:29.066 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4190720 len 8192 2024-05-08T06:12:29.066 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4190720 len 8192 2024-05-08T06:12:29.066 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4190720 len 8192 2024-05-08T06:12:29.066 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4190720 len 8192 2024-05-08T06:12:29.066 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4190720 len 16384 2024-05-08T06:12:29.066 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4190720 len 16384 2024-05-08T06:12:29.066 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4190720 len 16384 2024-05-08T06:12:29.066 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4190720 len 16384 2024-05-08T06:12:29.066 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4191232 len 1024 2024-05-08T06:12:29.066 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4191232 len 1024 2024-05-08T06:12:29.067 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4191232 len 1024 2024-05-08T06:12:29.067 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4191232 len 1024 2024-05-08T06:12:29.067 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4191232 len 2048 2024-05-08T06:12:29.067 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4191232 len 2048 2024-05-08T06:12:29.067 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4191232 len 2048 2024-05-08T06:12:29.067 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4191232 len 2048 2024-05-08T06:12:29.067 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4191232 len 4096 2024-05-08T06:12:29.067 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4191232 len 4096 2024-05-08T06:12:29.067 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4191232 len 4096 2024-05-08T06:12:29.067 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4191232 len 4096 2024-05-08T06:12:29.067 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4191232 len 8192 2024-05-08T06:12:29.067 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4191232 len 8192 2024-05-08T06:12:29.067 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4191232 len 8192 2024-05-08T06:12:29.067 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4191232 len 8192 2024-05-08T06:12:29.068 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4191232 len 16384 2024-05-08T06:12:29.068 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4191232 len 16384 2024-05-08T06:12:29.068 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4191232 len 16384 2024-05-08T06:12:29.068 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4191232 len 16384 2024-05-08T06:12:29.068 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4191744 len 1024 2024-05-08T06:12:29.068 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4191744 len 1024 2024-05-08T06:12:29.068 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4191744 len 1024 2024-05-08T06:12:29.068 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4191744 len 1024 2024-05-08T06:12:29.068 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4191744 len 2048 2024-05-08T06:12:29.068 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4191744 len 2048 2024-05-08T06:12:29.068 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4191744 len 2048 2024-05-08T06:12:29.068 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4191744 len 2048 2024-05-08T06:12:29.068 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4191744 len 4096 2024-05-08T06:12:29.068 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4191744 len 4096 2024-05-08T06:12:29.069 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4191744 len 4096 2024-05-08T06:12:29.069 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4191744 len 4096 2024-05-08T06:12:29.069 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4191744 len 8192 2024-05-08T06:12:29.069 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4191744 len 8192 2024-05-08T06:12:29.069 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4191744 len 8192 2024-05-08T06:12:29.069 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4191744 len 8192 2024-05-08T06:12:29.069 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4191744 len 16384 2024-05-08T06:12:29.069 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4191744 len 16384 2024-05-08T06:12:29.069 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4191744 len 16384 2024-05-08T06:12:29.069 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4191744 len 16384 2024-05-08T06:12:29.069 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4192256 len 1024 2024-05-08T06:12:29.069 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4192256 len 1024 2024-05-08T06:12:29.069 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4192256 len 1024 2024-05-08T06:12:29.096 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4192256 len 1024 2024-05-08T06:12:29.097 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4192256 len 2048 2024-05-08T06:12:29.097 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4192256 len 2048 2024-05-08T06:12:29.097 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4192256 len 2048 2024-05-08T06:12:29.097 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4192256 len 2048 2024-05-08T06:12:29.097 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4192256 len 4096 2024-05-08T06:12:29.097 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4192256 len 4096 2024-05-08T06:12:29.097 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4192256 len 4096 2024-05-08T06:12:29.097 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4192256 len 4096 2024-05-08T06:12:29.097 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4192256 len 8192 2024-05-08T06:12:29.097 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4192256 len 8192 2024-05-08T06:12:29.097 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4192256 len 8192 2024-05-08T06:12:29.097 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4192256 len 8192 2024-05-08T06:12:29.097 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4192256 len 16384 2024-05-08T06:12:29.098 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4192256 len 16384 2024-05-08T06:12:29.098 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4192256 len 16384 2024-05-08T06:12:29.098 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4192256 len 16384 2024-05-08T06:12:29.098 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4192768 len 1024 2024-05-08T06:12:29.098 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4192768 len 1024 2024-05-08T06:12:29.098 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4192768 len 1024 2024-05-08T06:12:29.098 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4192768 len 1024 2024-05-08T06:12:29.098 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4192768 len 2048 2024-05-08T06:12:29.098 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4192768 len 2048 2024-05-08T06:12:29.098 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4192768 len 2048 2024-05-08T06:12:29.098 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4192768 len 2048 2024-05-08T06:12:29.098 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4192768 len 4096 2024-05-08T06:12:29.099 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4192768 len 4096 2024-05-08T06:12:29.099 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4192768 len 4096 2024-05-08T06:12:29.099 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4192768 len 4096 2024-05-08T06:12:29.099 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4192768 len 8192 2024-05-08T06:12:29.099 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4192768 len 8192 2024-05-08T06:12:29.099 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4192768 len 8192 2024-05-08T06:12:29.099 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4192768 len 8192 2024-05-08T06:12:29.099 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4192768 len 16384 2024-05-08T06:12:29.099 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4192768 len 16384 2024-05-08T06:12:29.099 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4192768 len 16384 2024-05-08T06:12:29.099 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4192768 len 16384 2024-05-08T06:12:29.099 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4193280 len 1024 2024-05-08T06:12:29.099 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4193280 len 1024 2024-05-08T06:12:29.099 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4193280 len 1024 2024-05-08T06:12:29.099 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4193280 len 1024 2024-05-08T06:12:29.100 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4193280 len 2048 2024-05-08T06:12:29.100 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4193280 len 2048 2024-05-08T06:12:29.100 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4193280 len 2048 2024-05-08T06:12:29.100 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4193280 len 2048 2024-05-08T06:12:29.100 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4193280 len 4096 2024-05-08T06:12:29.100 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4193280 len 4096 2024-05-08T06:12:29.100 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4193280 len 4096 2024-05-08T06:12:29.100 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4193280 len 4096 2024-05-08T06:12:29.100 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4193280 len 8192 2024-05-08T06:12:29.100 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4193280 len 8192 2024-05-08T06:12:29.100 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4193280 len 8192 2024-05-08T06:12:29.100 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4193280 len 8192 2024-05-08T06:12:29.100 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4193280 len 16384 2024-05-08T06:12:29.100 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4193280 len 16384 2024-05-08T06:12:29.100 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4193280 len 16384 2024-05-08T06:12:29.101 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4193280 len 16384 2024-05-08T06:12:29.101 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4193792 len 1024 2024-05-08T06:12:29.101 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4193792 len 1024 2024-05-08T06:12:29.101 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4193792 len 1024 2024-05-08T06:12:29.101 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4193792 len 1024 2024-05-08T06:12:29.101 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4193792 len 2048 2024-05-08T06:12:29.101 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4193792 len 2048 2024-05-08T06:12:29.101 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4193792 len 2048 2024-05-08T06:12:29.101 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4193792 len 2048 2024-05-08T06:12:29.101 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4193792 len 4096 2024-05-08T06:12:29.101 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4193792 len 4096 2024-05-08T06:12:29.101 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4193792 len 4096 2024-05-08T06:12:29.101 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4193792 len 4096 2024-05-08T06:12:29.101 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4193792 len 8192 2024-05-08T06:12:29.102 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4193792 len 8192 2024-05-08T06:12:29.102 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4193792 len 8192 2024-05-08T06:12:29.102 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4193792 len 8192 2024-05-08T06:12:29.102 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4193792 len 16384 2024-05-08T06:12:29.102 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4193792 len 16384 2024-05-08T06:12:29.102 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4193792 len 16384 2024-05-08T06:12:29.102 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4193792 len 16384 2024-05-08T06:12:29.102 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4194304 len 1024 2024-05-08T06:12:29.102 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4194304 len 1024 2024-05-08T06:12:29.102 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4194304 len 1024 2024-05-08T06:12:29.102 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4194304 len 1024 2024-05-08T06:12:29.102 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4194304 len 2048 2024-05-08T06:12:29.113 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4194304 len 2048 2024-05-08T06:12:29.113 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4194304 len 2048 2024-05-08T06:12:29.113 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4194304 len 2048 2024-05-08T06:12:29.113 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4194304 len 4096 2024-05-08T06:12:29.114 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4194304 len 4096 2024-05-08T06:12:29.114 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4194304 len 4096 2024-05-08T06:12:29.114 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4194304 len 4096 2024-05-08T06:12:29.114 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4194304 len 8192 2024-05-08T06:12:29.114 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4194304 len 8192 2024-05-08T06:12:29.114 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4194304 len 8192 2024-05-08T06:12:29.114 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4194304 len 8192 2024-05-08T06:12:29.114 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4194304 len 16384 2024-05-08T06:12:29.114 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4194304 len 16384 2024-05-08T06:12:29.114 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4194304 len 16384 2024-05-08T06:12:29.114 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4194304 len 16384 2024-05-08T06:12:29.114 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4194816 len 1024 2024-05-08T06:12:29.114 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4194816 len 1024 2024-05-08T06:12:29.114 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4194816 len 1024 2024-05-08T06:12:29.114 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4194816 len 1024 2024-05-08T06:12:29.115 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4194816 len 2048 2024-05-08T06:12:29.115 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4194816 len 2048 2024-05-08T06:12:29.115 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4194816 len 2048 2024-05-08T06:12:29.115 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4194816 len 2048 2024-05-08T06:12:29.115 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4194816 len 4096 2024-05-08T06:12:29.115 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4194816 len 4096 2024-05-08T06:12:29.115 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4194816 len 4096 2024-05-08T06:12:29.115 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4194816 len 4096 2024-05-08T06:12:29.115 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4194816 len 8192 2024-05-08T06:12:29.115 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4194816 len 8192 2024-05-08T06:12:29.115 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4194816 len 8192 2024-05-08T06:12:29.115 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4194816 len 8192 2024-05-08T06:12:29.115 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4194816 len 16384 2024-05-08T06:12:29.115 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4194816 len 16384 2024-05-08T06:12:29.116 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4194816 len 16384 2024-05-08T06:12:29.116 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4194816 len 16384 2024-05-08T06:12:29.116 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4195328 len 1024 2024-05-08T06:12:29.116 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4195328 len 1024 2024-05-08T06:12:29.116 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4195328 len 1024 2024-05-08T06:12:29.116 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4195328 len 1024 2024-05-08T06:12:29.116 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4195328 len 2048 2024-05-08T06:12:29.116 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4195328 len 2048 2024-05-08T06:12:29.116 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4195328 len 2048 2024-05-08T06:12:29.116 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4195328 len 2048 2024-05-08T06:12:29.116 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4195328 len 4096 2024-05-08T06:12:29.116 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4195328 len 4096 2024-05-08T06:12:29.116 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4195328 len 4096 2024-05-08T06:12:29.116 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4195328 len 4096 2024-05-08T06:12:29.116 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4195328 len 8192 2024-05-08T06:12:29.117 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4195328 len 8192 2024-05-08T06:12:29.117 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4195328 len 8192 2024-05-08T06:12:29.117 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4195328 len 8192 2024-05-08T06:12:29.117 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4195328 len 16384 2024-05-08T06:12:29.117 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4195328 len 16384 2024-05-08T06:12:29.117 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4195328 len 16384 2024-05-08T06:12:29.117 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4195328 len 16384 2024-05-08T06:12:29.117 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4195840 len 1024 2024-05-08T06:12:29.117 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4195840 len 1024 2024-05-08T06:12:29.117 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4195840 len 1024 2024-05-08T06:12:29.117 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4195840 len 1024 2024-05-08T06:12:29.117 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4195840 len 2048 2024-05-08T06:12:29.117 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4195840 len 2048 2024-05-08T06:12:29.117 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4195840 len 2048 2024-05-08T06:12:29.117 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4195840 len 2048 2024-05-08T06:12:29.118 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4195840 len 4096 2024-05-08T06:12:29.118 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4195840 len 4096 2024-05-08T06:12:29.118 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4195840 len 4096 2024-05-08T06:12:29.118 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4195840 len 4096 2024-05-08T06:12:29.118 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4195840 len 8192 2024-05-08T06:12:29.118 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4195840 len 8192 2024-05-08T06:12:29.118 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4195840 len 8192 2024-05-08T06:12:29.118 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4195840 len 8192 2024-05-08T06:12:29.118 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4195840 len 16384 2024-05-08T06:12:29.118 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4195840 len 16384 2024-05-08T06:12:29.118 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4195840 len 16384 2024-05-08T06:12:29.118 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4195840 len 16384 2024-05-08T06:12:29.118 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4196352 len 1024 2024-05-08T06:12:29.118 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4196352 len 1024 2024-05-08T06:12:29.118 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4196352 len 1024 2024-05-08T06:12:29.119 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4196352 len 1024 2024-05-08T06:12:29.119 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4196352 len 2048 2024-05-08T06:12:29.119 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4196352 len 2048 2024-05-08T06:12:29.119 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4196352 len 2048 2024-05-08T06:12:29.119 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4196352 len 2048 2024-05-08T06:12:29.131 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4196352 len 4096 2024-05-08T06:12:29.131 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4196352 len 4096 2024-05-08T06:12:29.131 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4196352 len 4096 2024-05-08T06:12:29.131 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4196352 len 4096 2024-05-08T06:12:29.131 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4196352 len 8192 2024-05-08T06:12:29.131 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4196352 len 8192 2024-05-08T06:12:29.132 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4196352 len 8192 2024-05-08T06:12:29.132 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4196352 len 8192 2024-05-08T06:12:29.132 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4196352 len 16384 2024-05-08T06:12:29.132 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4196352 len 16384 2024-05-08T06:12:29.132 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4196352 len 16384 2024-05-08T06:12:29.132 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4196352 len 16384 2024-05-08T06:12:29.132 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4196864 len 1024 2024-05-08T06:12:29.132 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4196864 len 1024 2024-05-08T06:12:29.132 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4196864 len 1024 2024-05-08T06:12:29.132 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4196864 len 1024 2024-05-08T06:12:29.132 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4196864 len 2048 2024-05-08T06:12:29.132 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4196864 len 2048 2024-05-08T06:12:29.132 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4196864 len 2048 2024-05-08T06:12:29.132 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4196864 len 2048 2024-05-08T06:12:29.132 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4196864 len 4096 2024-05-08T06:12:29.133 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4196864 len 4096 2024-05-08T06:12:29.133 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4196864 len 4096 2024-05-08T06:12:29.133 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4196864 len 4096 2024-05-08T06:12:29.133 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4196864 len 8192 2024-05-08T06:12:29.133 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4196864 len 8192 2024-05-08T06:12:29.133 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4196864 len 8192 2024-05-08T06:12:29.133 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4196864 len 8192 2024-05-08T06:12:29.133 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4196864 len 16384 2024-05-08T06:12:29.133 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4196864 len 16384 2024-05-08T06:12:29.133 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4196864 len 16384 2024-05-08T06:12:29.133 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4196864 len 16384 2024-05-08T06:12:29.133 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4197376 len 1024 2024-05-08T06:12:29.133 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4197376 len 1024 2024-05-08T06:12:29.133 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4197376 len 1024 2024-05-08T06:12:29.134 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4197376 len 1024 2024-05-08T06:12:29.134 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4197376 len 2048 2024-05-08T06:12:29.134 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4197376 len 2048 2024-05-08T06:12:29.134 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4197376 len 2048 2024-05-08T06:12:29.134 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4197376 len 2048 2024-05-08T06:12:29.134 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4197376 len 4096 2024-05-08T06:12:29.134 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4197376 len 4096 2024-05-08T06:12:29.134 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4197376 len 4096 2024-05-08T06:12:29.134 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4197376 len 4096 2024-05-08T06:12:29.134 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4197376 len 8192 2024-05-08T06:12:29.134 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4197376 len 8192 2024-05-08T06:12:29.134 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4197376 len 8192 2024-05-08T06:12:29.134 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4197376 len 8192 2024-05-08T06:12:29.134 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4197376 len 16384 2024-05-08T06:12:29.134 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4197376 len 16384 2024-05-08T06:12:29.135 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4197376 len 16384 2024-05-08T06:12:29.135 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4197376 len 16384 2024-05-08T06:12:29.135 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4197888 len 1024 2024-05-08T06:12:29.135 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4197888 len 1024 2024-05-08T06:12:29.135 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4197888 len 1024 2024-05-08T06:12:29.135 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4197888 len 1024 2024-05-08T06:12:29.135 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4197888 len 2048 2024-05-08T06:12:29.135 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4197888 len 2048 2024-05-08T06:12:29.135 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4197888 len 2048 2024-05-08T06:12:29.135 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4197888 len 2048 2024-05-08T06:12:29.135 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4197888 len 4096 2024-05-08T06:12:29.135 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4197888 len 4096 2024-05-08T06:12:29.135 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4197888 len 4096 2024-05-08T06:12:29.135 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4197888 len 4096 2024-05-08T06:12:29.136 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4197888 len 8192 2024-05-08T06:12:29.136 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4197888 len 8192 2024-05-08T06:12:29.136 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4197888 len 8192 2024-05-08T06:12:29.136 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4197888 len 8192 2024-05-08T06:12:29.136 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3072 offset 4197888 len 16384 2024-05-08T06:12:29.136 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4197888 len 16384 2024-05-08T06:12:29.136 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3072 offset 4197888 len 16384 2024-05-08T06:12:29.136 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3072 offset 4197888 len 16384 2024-05-08T06:12:29.136 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4190208 len 1024 2024-05-08T06:12:29.136 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4190208 len 1024 2024-05-08T06:12:29.136 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4190208 len 1024 2024-05-08T06:12:29.136 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4190208 len 1024 2024-05-08T06:12:29.136 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4190208 len 2048 2024-05-08T06:12:29.136 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4190208 len 2048 2024-05-08T06:12:29.136 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4190208 len 2048 2024-05-08T06:12:29.137 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4190208 len 2048 2024-05-08T06:12:29.137 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4190208 len 4096 2024-05-08T06:12:29.137 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4190208 len 4096 2024-05-08T06:12:29.162 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:12:29.165 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4190208 len 4096 2024-05-08T06:12:29.165 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4190208 len 4096 2024-05-08T06:12:29.166 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4190208 len 8192 2024-05-08T06:12:29.166 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4190208 len 8192 2024-05-08T06:12:29.166 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4190208 len 8192 2024-05-08T06:12:29.166 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4190208 len 8192 2024-05-08T06:12:29.166 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4190208 len 16384 2024-05-08T06:12:29.166 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4190208 len 16384 2024-05-08T06:12:29.166 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4190208 len 16384 2024-05-08T06:12:29.166 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4190208 len 16384 2024-05-08T06:12:29.166 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4190720 len 1024 2024-05-08T06:12:29.166 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4190720 len 1024 2024-05-08T06:12:29.166 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4190720 len 1024 2024-05-08T06:12:29.167 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4190720 len 1024 2024-05-08T06:12:29.167 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4190720 len 2048 2024-05-08T06:12:29.167 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4190720 len 2048 2024-05-08T06:12:29.167 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4190720 len 2048 2024-05-08T06:12:29.167 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4190720 len 2048 2024-05-08T06:12:29.167 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4190720 len 4096 2024-05-08T06:12:29.167 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4190720 len 4096 2024-05-08T06:12:29.167 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4190720 len 4096 2024-05-08T06:12:29.167 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4190720 len 4096 2024-05-08T06:12:29.167 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4190720 len 8192 2024-05-08T06:12:29.167 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4190720 len 8192 2024-05-08T06:12:29.167 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4190720 len 8192 2024-05-08T06:12:29.167 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4190720 len 8192 2024-05-08T06:12:29.168 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4190720 len 16384 2024-05-08T06:12:29.168 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4190720 len 16384 2024-05-08T06:12:29.168 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4190720 len 16384 2024-05-08T06:12:29.168 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4190720 len 16384 2024-05-08T06:12:29.168 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4191232 len 1024 2024-05-08T06:12:29.168 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4191232 len 1024 2024-05-08T06:12:29.168 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4191232 len 1024 2024-05-08T06:12:29.168 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4191232 len 1024 2024-05-08T06:12:29.168 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4191232 len 2048 2024-05-08T06:12:29.168 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4191232 len 2048 2024-05-08T06:12:29.168 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4191232 len 2048 2024-05-08T06:12:29.169 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4191232 len 2048 2024-05-08T06:12:29.169 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4191232 len 4096 2024-05-08T06:12:29.169 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4191232 len 4096 2024-05-08T06:12:29.169 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4191232 len 4096 2024-05-08T06:12:29.169 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4191232 len 4096 2024-05-08T06:12:29.169 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4191232 len 8192 2024-05-08T06:12:29.169 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4191232 len 8192 2024-05-08T06:12:29.169 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4191232 len 8192 2024-05-08T06:12:29.169 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4191232 len 8192 2024-05-08T06:12:29.169 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4191232 len 16384 2024-05-08T06:12:29.169 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4191232 len 16384 2024-05-08T06:12:29.169 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4191232 len 16384 2024-05-08T06:12:29.170 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4191232 len 16384 2024-05-08T06:12:29.170 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4191744 len 1024 2024-05-08T06:12:29.170 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4191744 len 1024 2024-05-08T06:12:29.170 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4191744 len 1024 2024-05-08T06:12:29.170 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4191744 len 1024 2024-05-08T06:12:29.170 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4191744 len 2048 2024-05-08T06:12:29.170 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4191744 len 2048 2024-05-08T06:12:29.170 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4191744 len 2048 2024-05-08T06:12:29.170 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4191744 len 2048 2024-05-08T06:12:29.170 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4191744 len 4096 2024-05-08T06:12:29.170 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4191744 len 4096 2024-05-08T06:12:29.170 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4191744 len 4096 2024-05-08T06:12:29.171 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4191744 len 4096 2024-05-08T06:12:29.171 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4191744 len 8192 2024-05-08T06:12:29.171 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4191744 len 8192 2024-05-08T06:12:29.171 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4191744 len 8192 2024-05-08T06:12:29.171 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4191744 len 8192 2024-05-08T06:12:29.171 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4191744 len 16384 2024-05-08T06:12:29.171 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4191744 len 16384 2024-05-08T06:12:29.171 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4191744 len 16384 2024-05-08T06:12:29.171 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4191744 len 16384 2024-05-08T06:12:29.171 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4192256 len 1024 2024-05-08T06:12:29.171 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4192256 len 1024 2024-05-08T06:12:29.171 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4192256 len 1024 2024-05-08T06:12:29.171 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4192256 len 1024 2024-05-08T06:12:29.172 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4192256 len 2048 2024-05-08T06:12:29.172 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4192256 len 2048 2024-05-08T06:12:29.172 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4192256 len 2048 2024-05-08T06:12:29.172 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4192256 len 2048 2024-05-08T06:12:29.172 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4192256 len 4096 2024-05-08T06:12:29.172 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4192256 len 4096 2024-05-08T06:12:29.172 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4192256 len 4096 2024-05-08T06:12:29.172 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4192256 len 4096 2024-05-08T06:12:29.172 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4192256 len 8192 2024-05-08T06:12:29.206 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4192256 len 8192 2024-05-08T06:12:29.206 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4192256 len 8192 2024-05-08T06:12:29.206 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4192256 len 8192 2024-05-08T06:12:29.206 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4192256 len 16384 2024-05-08T06:12:29.206 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4192256 len 16384 2024-05-08T06:12:29.206 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4192256 len 16384 2024-05-08T06:12:29.206 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4192256 len 16384 2024-05-08T06:12:29.206 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4192768 len 1024 2024-05-08T06:12:29.207 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4192768 len 1024 2024-05-08T06:12:29.207 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4192768 len 1024 2024-05-08T06:12:29.207 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4192768 len 1024 2024-05-08T06:12:29.207 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4192768 len 2048 2024-05-08T06:12:29.207 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4192768 len 2048 2024-05-08T06:12:29.207 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4192768 len 2048 2024-05-08T06:12:29.207 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4192768 len 2048 2024-05-08T06:12:29.207 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4192768 len 4096 2024-05-08T06:12:29.207 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4192768 len 4096 2024-05-08T06:12:29.207 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4192768 len 4096 2024-05-08T06:12:29.207 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4192768 len 4096 2024-05-08T06:12:29.207 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4192768 len 8192 2024-05-08T06:12:29.208 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4192768 len 8192 2024-05-08T06:12:29.208 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4192768 len 8192 2024-05-08T06:12:29.208 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4192768 len 8192 2024-05-08T06:12:29.208 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4192768 len 16384 2024-05-08T06:12:29.208 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4192768 len 16384 2024-05-08T06:12:29.208 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4192768 len 16384 2024-05-08T06:12:29.208 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4192768 len 16384 2024-05-08T06:12:29.208 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4193280 len 1024 2024-05-08T06:12:29.208 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4193280 len 1024 2024-05-08T06:12:29.208 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4193280 len 1024 2024-05-08T06:12:29.208 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4193280 len 1024 2024-05-08T06:12:29.209 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4193280 len 2048 2024-05-08T06:12:29.209 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4193280 len 2048 2024-05-08T06:12:29.209 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4193280 len 2048 2024-05-08T06:12:29.209 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4193280 len 2048 2024-05-08T06:12:29.209 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4193280 len 4096 2024-05-08T06:12:29.209 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4193280 len 4096 2024-05-08T06:12:29.209 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4193280 len 4096 2024-05-08T06:12:29.209 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4193280 len 4096 2024-05-08T06:12:29.209 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4193280 len 8192 2024-05-08T06:12:29.209 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4193280 len 8192 2024-05-08T06:12:29.209 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4193280 len 8192 2024-05-08T06:12:29.209 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4193280 len 8192 2024-05-08T06:12:29.209 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4193280 len 16384 2024-05-08T06:12:29.210 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4193280 len 16384 2024-05-08T06:12:29.210 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4193280 len 16384 2024-05-08T06:12:29.210 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4193280 len 16384 2024-05-08T06:12:29.210 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4193792 len 1024 2024-05-08T06:12:29.210 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4193792 len 1024 2024-05-08T06:12:29.210 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4193792 len 1024 2024-05-08T06:12:29.210 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4193792 len 1024 2024-05-08T06:12:29.210 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4193792 len 2048 2024-05-08T06:12:29.210 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4193792 len 2048 2024-05-08T06:12:29.210 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4193792 len 2048 2024-05-08T06:12:29.210 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4193792 len 2048 2024-05-08T06:12:29.210 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4193792 len 4096 2024-05-08T06:12:29.211 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4193792 len 4096 2024-05-08T06:12:29.211 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4193792 len 4096 2024-05-08T06:12:29.211 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4193792 len 4096 2024-05-08T06:12:29.211 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4193792 len 8192 2024-05-08T06:12:29.211 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4193792 len 8192 2024-05-08T06:12:29.211 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4193792 len 8192 2024-05-08T06:12:29.211 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4193792 len 8192 2024-05-08T06:12:29.211 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4193792 len 16384 2024-05-08T06:12:29.211 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4193792 len 16384 2024-05-08T06:12:29.211 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4193792 len 16384 2024-05-08T06:12:29.211 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4193792 len 16384 2024-05-08T06:12:29.211 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4194304 len 1024 2024-05-08T06:12:29.211 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4194304 len 1024 2024-05-08T06:12:29.212 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4194304 len 1024 2024-05-08T06:12:29.212 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4194304 len 1024 2024-05-08T06:12:29.212 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4194304 len 2048 2024-05-08T06:12:29.212 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4194304 len 2048 2024-05-08T06:12:29.212 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4194304 len 2048 2024-05-08T06:12:29.212 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4194304 len 2048 2024-05-08T06:12:29.212 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4194304 len 4096 2024-05-08T06:12:29.212 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4194304 len 4096 2024-05-08T06:12:29.212 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4194304 len 4096 2024-05-08T06:12:29.212 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4194304 len 4096 2024-05-08T06:12:29.212 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4194304 len 8192 2024-05-08T06:12:29.212 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4194304 len 8192 2024-05-08T06:12:29.212 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4194304 len 8192 2024-05-08T06:12:29.224 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4194304 len 8192 2024-05-08T06:12:29.225 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4194304 len 16384 2024-05-08T06:12:29.225 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4194304 len 16384 2024-05-08T06:12:29.225 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4194304 len 16384 2024-05-08T06:12:29.225 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4194304 len 16384 2024-05-08T06:12:29.225 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4194816 len 1024 2024-05-08T06:12:29.225 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4194816 len 1024 2024-05-08T06:12:29.225 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4194816 len 1024 2024-05-08T06:12:29.225 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4194816 len 1024 2024-05-08T06:12:29.225 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4194816 len 2048 2024-05-08T06:12:29.225 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4194816 len 2048 2024-05-08T06:12:29.225 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4194816 len 2048 2024-05-08T06:12:29.225 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4194816 len 2048 2024-05-08T06:12:29.225 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4194816 len 4096 2024-05-08T06:12:29.226 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4194816 len 4096 2024-05-08T06:12:29.226 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4194816 len 4096 2024-05-08T06:12:29.226 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4194816 len 4096 2024-05-08T06:12:29.226 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4194816 len 8192 2024-05-08T06:12:29.226 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4194816 len 8192 2024-05-08T06:12:29.226 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4194816 len 8192 2024-05-08T06:12:29.226 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4194816 len 8192 2024-05-08T06:12:29.226 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4194816 len 16384 2024-05-08T06:12:29.226 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4194816 len 16384 2024-05-08T06:12:29.226 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4194816 len 16384 2024-05-08T06:12:29.226 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4194816 len 16384 2024-05-08T06:12:29.226 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4195328 len 1024 2024-05-08T06:12:29.227 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4195328 len 1024 2024-05-08T06:12:29.227 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4195328 len 1024 2024-05-08T06:12:29.227 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4195328 len 1024 2024-05-08T06:12:29.227 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4195328 len 2048 2024-05-08T06:12:29.227 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4195328 len 2048 2024-05-08T06:12:29.227 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4195328 len 2048 2024-05-08T06:12:29.227 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4195328 len 2048 2024-05-08T06:12:29.227 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4195328 len 4096 2024-05-08T06:12:29.227 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4195328 len 4096 2024-05-08T06:12:29.227 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4195328 len 4096 2024-05-08T06:12:29.227 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4195328 len 4096 2024-05-08T06:12:29.227 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4195328 len 8192 2024-05-08T06:12:29.227 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4195328 len 8192 2024-05-08T06:12:29.228 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4195328 len 8192 2024-05-08T06:12:29.228 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4195328 len 8192 2024-05-08T06:12:29.228 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4195328 len 16384 2024-05-08T06:12:29.228 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4195328 len 16384 2024-05-08T06:12:29.228 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4195328 len 16384 2024-05-08T06:12:29.228 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4195328 len 16384 2024-05-08T06:12:29.228 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4195840 len 1024 2024-05-08T06:12:29.228 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4195840 len 1024 2024-05-08T06:12:29.228 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4195840 len 1024 2024-05-08T06:12:29.228 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4195840 len 1024 2024-05-08T06:12:29.228 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4195840 len 2048 2024-05-08T06:12:29.228 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4195840 len 2048 2024-05-08T06:12:29.228 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4195840 len 2048 2024-05-08T06:12:29.229 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4195840 len 2048 2024-05-08T06:12:29.229 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4195840 len 4096 2024-05-08T06:12:29.229 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4195840 len 4096 2024-05-08T06:12:29.229 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4195840 len 4096 2024-05-08T06:12:29.229 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4195840 len 4096 2024-05-08T06:12:29.229 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4195840 len 8192 2024-05-08T06:12:29.229 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4195840 len 8192 2024-05-08T06:12:29.229 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4195840 len 8192 2024-05-08T06:12:29.229 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4195840 len 8192 2024-05-08T06:12:29.229 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4195840 len 16384 2024-05-08T06:12:29.229 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4195840 len 16384 2024-05-08T06:12:29.229 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4195840 len 16384 2024-05-08T06:12:29.229 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4195840 len 16384 2024-05-08T06:12:29.229 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4196352 len 1024 2024-05-08T06:12:29.230 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4196352 len 1024 2024-05-08T06:12:29.230 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4196352 len 1024 2024-05-08T06:12:29.230 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4196352 len 1024 2024-05-08T06:12:29.230 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4196352 len 2048 2024-05-08T06:12:29.230 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4196352 len 2048 2024-05-08T06:12:29.230 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4196352 len 2048 2024-05-08T06:12:29.230 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4196352 len 2048 2024-05-08T06:12:29.230 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4196352 len 4096 2024-05-08T06:12:29.230 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4196352 len 4096 2024-05-08T06:12:29.230 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4196352 len 4096 2024-05-08T06:12:29.230 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4196352 len 4096 2024-05-08T06:12:29.230 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4196352 len 8192 2024-05-08T06:12:29.230 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4196352 len 8192 2024-05-08T06:12:29.231 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4196352 len 8192 2024-05-08T06:12:29.231 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4196352 len 8192 2024-05-08T06:12:29.231 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4196352 len 16384 2024-05-08T06:12:29.715 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4196352 len 16384 2024-05-08T06:12:29.715 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4196352 len 16384 2024-05-08T06:12:29.715 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4196352 len 16384 2024-05-08T06:12:29.715 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4196864 len 1024 2024-05-08T06:12:29.715 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4196864 len 1024 2024-05-08T06:12:29.715 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4196864 len 1024 2024-05-08T06:12:29.715 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4196864 len 1024 2024-05-08T06:12:29.715 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4196864 len 2048 2024-05-08T06:12:29.715 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4196864 len 2048 2024-05-08T06:12:29.715 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4196864 len 2048 2024-05-08T06:12:29.716 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4196864 len 2048 2024-05-08T06:12:29.716 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4196864 len 4096 2024-05-08T06:12:29.716 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4196864 len 4096 2024-05-08T06:12:29.716 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4196864 len 4096 2024-05-08T06:12:29.716 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4196864 len 4096 2024-05-08T06:12:29.716 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4196864 len 8192 2024-05-08T06:12:29.716 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4196864 len 8192 2024-05-08T06:12:29.716 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4196864 len 8192 2024-05-08T06:12:29.716 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4196864 len 8192 2024-05-08T06:12:29.716 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4196864 len 16384 2024-05-08T06:12:29.716 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4196864 len 16384 2024-05-08T06:12:29.716 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4196864 len 16384 2024-05-08T06:12:29.716 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4196864 len 16384 2024-05-08T06:12:29.716 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4197376 len 1024 2024-05-08T06:12:29.716 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4197376 len 1024 2024-05-08T06:12:29.717 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4197376 len 1024 2024-05-08T06:12:29.717 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4197376 len 1024 2024-05-08T06:12:29.717 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4197376 len 2048 2024-05-08T06:12:29.717 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4197376 len 2048 2024-05-08T06:12:29.717 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4197376 len 2048 2024-05-08T06:12:29.717 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4197376 len 2048 2024-05-08T06:12:29.717 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4197376 len 4096 2024-05-08T06:12:29.717 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4197376 len 4096 2024-05-08T06:12:29.717 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4197376 len 4096 2024-05-08T06:12:29.717 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4197376 len 4096 2024-05-08T06:12:29.717 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4197376 len 8192 2024-05-08T06:12:29.717 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4197376 len 8192 2024-05-08T06:12:29.717 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4197376 len 8192 2024-05-08T06:12:29.717 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4197376 len 8192 2024-05-08T06:12:29.717 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4197376 len 16384 2024-05-08T06:12:29.717 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4197376 len 16384 2024-05-08T06:12:29.718 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4197376 len 16384 2024-05-08T06:12:29.718 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4197376 len 16384 2024-05-08T06:12:29.718 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4197888 len 1024 2024-05-08T06:12:29.718 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4197888 len 1024 2024-05-08T06:12:29.718 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4197888 len 1024 2024-05-08T06:12:29.718 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4197888 len 1024 2024-05-08T06:12:29.718 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4197888 len 2048 2024-05-08T06:12:29.718 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4197888 len 2048 2024-05-08T06:12:29.718 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4197888 len 2048 2024-05-08T06:12:29.718 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4197888 len 2048 2024-05-08T06:12:29.718 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4197888 len 4096 2024-05-08T06:12:29.718 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4197888 len 4096 2024-05-08T06:12:29.718 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4197888 len 4096 2024-05-08T06:12:29.718 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4197888 len 4096 2024-05-08T06:12:29.718 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4197888 len 8192 2024-05-08T06:12:29.719 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4197888 len 8192 2024-05-08T06:12:29.719 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4197888 len 8192 2024-05-08T06:12:29.719 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4197888 len 8192 2024-05-08T06:12:29.719 INFO:tasks.workunit.client.0.smithi123.stdout:read_direct buf_align 3584 offset 4197888 len 16384 2024-05-08T06:12:29.719 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4197888 len 16384 2024-05-08T06:12:29.719 INFO:tasks.workunit.client.0.smithi123.stdout:read_sync buf_align 3584 offset 4197888 len 16384 2024-05-08T06:12:29.719 INFO:tasks.workunit.client.0.smithi123.stdout:read_file buf_align 3584 offset 4197888 len 16384 2024-05-08T06:12:29.719 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4190720 len 1024 2024-05-08T06:12:29.719 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4190720 len 1024 2024-05-08T06:12:29.719 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4190720 len 1024 2024-05-08T06:12:29.719 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4190720 len 1024 2024-05-08T06:12:29.719 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4190720 len 2048 2024-05-08T06:12:29.719 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4190720 len 2048 2024-05-08T06:12:29.719 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4190720 len 2048 2024-05-08T06:12:29.719 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4190720 len 2048 2024-05-08T06:12:29.720 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4190720 len 4096 2024-05-08T06:12:29.720 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4190720 len 4096 2024-05-08T06:12:29.720 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4190720 len 4096 2024-05-08T06:12:29.720 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4190720 len 4096 2024-05-08T06:12:29.720 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4190720 len 8192 2024-05-08T06:12:29.720 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4190720 len 8192 2024-05-08T06:12:29.720 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4190720 len 8192 2024-05-08T06:12:29.720 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4190720 len 8192 2024-05-08T06:12:29.720 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4190720 len 16384 2024-05-08T06:12:29.720 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4190720 len 16384 2024-05-08T06:12:29.720 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4190720 len 16384 2024-05-08T06:12:29.720 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4190720 len 16384 2024-05-08T06:12:29.720 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4191232 len 1024 2024-05-08T06:12:30.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:30 smithi123 ceph-mon[29787]: pgmap v521: 97 pgs: 97 active+clean; 1.2 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 10 KiB/s wr, 1.09k op/s 2024-05-08T06:12:30.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:30 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:12:30.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:30 smithi123 ceph-mon[29787]: scrub summary: idle 2024-05-08T06:12:31.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:30 smithi163 ceph-mon[34520]: pgmap v521: 97 pgs: 97 active+clean; 1.2 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 10 KiB/s wr, 1.09k op/s 2024-05-08T06:12:31.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:30 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:12:31.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:30 smithi163 ceph-mon[34520]: scrub summary: idle 2024-05-08T06:12:31.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:30 smithi192 ceph-mon[34664]: pgmap v521: 97 pgs: 97 active+clean; 1.2 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 10 KiB/s wr, 1.09k op/s 2024-05-08T06:12:31.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:30 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:12:31.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:30 smithi192 ceph-mon[34664]: scrub summary: idle 2024-05-08T06:12:31.424 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4191232 len 1024 2024-05-08T06:12:31.424 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4191232 len 1024 2024-05-08T06:12:31.424 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4191232 len 1024 2024-05-08T06:12:31.424 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4191232 len 2048 2024-05-08T06:12:31.425 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4191232 len 2048 2024-05-08T06:12:31.425 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4191232 len 2048 2024-05-08T06:12:31.425 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4191232 len 2048 2024-05-08T06:12:31.425 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4191232 len 4096 2024-05-08T06:12:31.425 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4191232 len 4096 2024-05-08T06:12:31.425 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4191232 len 4096 2024-05-08T06:12:31.425 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4191232 len 4096 2024-05-08T06:12:31.425 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4191232 len 8192 2024-05-08T06:12:31.425 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4191232 len 8192 2024-05-08T06:12:31.425 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4191232 len 8192 2024-05-08T06:12:31.425 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4191232 len 8192 2024-05-08T06:12:31.425 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4191232 len 16384 2024-05-08T06:12:31.425 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4191232 len 16384 2024-05-08T06:12:31.426 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4191232 len 16384 2024-05-08T06:12:31.426 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4191232 len 16384 2024-05-08T06:12:31.426 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4191744 len 1024 2024-05-08T06:12:31.426 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4191744 len 1024 2024-05-08T06:12:31.426 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4191744 len 1024 2024-05-08T06:12:31.426 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4191744 len 1024 2024-05-08T06:12:31.426 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4191744 len 2048 2024-05-08T06:12:31.426 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4191744 len 2048 2024-05-08T06:12:31.426 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4191744 len 2048 2024-05-08T06:12:31.426 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4191744 len 2048 2024-05-08T06:12:31.426 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4191744 len 4096 2024-05-08T06:12:31.426 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4191744 len 4096 2024-05-08T06:12:31.426 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4191744 len 4096 2024-05-08T06:12:31.427 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4191744 len 4096 2024-05-08T06:12:31.427 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4191744 len 8192 2024-05-08T06:12:31.427 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4191744 len 8192 2024-05-08T06:12:31.427 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4191744 len 8192 2024-05-08T06:12:31.427 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4191744 len 8192 2024-05-08T06:12:31.427 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4191744 len 16384 2024-05-08T06:12:31.427 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4191744 len 16384 2024-05-08T06:12:31.427 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4191744 len 16384 2024-05-08T06:12:31.427 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4191744 len 16384 2024-05-08T06:12:31.427 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4192256 len 1024 2024-05-08T06:12:31.427 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4192256 len 1024 2024-05-08T06:12:31.427 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4192256 len 1024 2024-05-08T06:12:31.427 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4192256 len 1024 2024-05-08T06:12:31.427 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4192256 len 2048 2024-05-08T06:12:31.428 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4192256 len 2048 2024-05-08T06:12:31.428 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4192256 len 2048 2024-05-08T06:12:31.428 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4192256 len 2048 2024-05-08T06:12:31.428 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4192256 len 4096 2024-05-08T06:12:31.428 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4192256 len 4096 2024-05-08T06:12:31.428 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4192256 len 4096 2024-05-08T06:12:31.428 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4192256 len 4096 2024-05-08T06:12:31.428 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4192256 len 8192 2024-05-08T06:12:31.428 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4192256 len 8192 2024-05-08T06:12:31.428 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4192256 len 8192 2024-05-08T06:12:31.428 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4192256 len 8192 2024-05-08T06:12:31.428 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4192256 len 16384 2024-05-08T06:12:31.428 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4192256 len 16384 2024-05-08T06:12:31.428 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4192256 len 16384 2024-05-08T06:12:31.428 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4192256 len 16384 2024-05-08T06:12:31.429 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4192768 len 1024 2024-05-08T06:12:31.429 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4192768 len 1024 2024-05-08T06:12:31.429 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4192768 len 1024 2024-05-08T06:12:31.429 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4192768 len 1024 2024-05-08T06:12:31.429 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4192768 len 2048 2024-05-08T06:12:31.429 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4192768 len 2048 2024-05-08T06:12:31.429 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4192768 len 2048 2024-05-08T06:12:31.429 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4192768 len 2048 2024-05-08T06:12:31.429 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4192768 len 4096 2024-05-08T06:12:31.429 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4192768 len 4096 2024-05-08T06:12:31.429 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4192768 len 4096 2024-05-08T06:12:31.429 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4192768 len 4096 2024-05-08T06:12:31.429 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4192768 len 8192 2024-05-08T06:12:31.429 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4192768 len 8192 2024-05-08T06:12:31.429 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4192768 len 8192 2024-05-08T06:12:31.430 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4192768 len 8192 2024-05-08T06:12:31.430 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4192768 len 16384 2024-05-08T06:12:31.430 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4192768 len 16384 2024-05-08T06:12:31.430 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4192768 len 16384 2024-05-08T06:12:31.430 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4192768 len 16384 2024-05-08T06:12:31.430 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4193280 len 1024 2024-05-08T06:12:31.430 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4193280 len 1024 2024-05-08T06:12:31.430 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4193280 len 1024 2024-05-08T06:12:31.430 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4193280 len 1024 2024-05-08T06:12:31.430 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4193280 len 2048 2024-05-08T06:12:31.430 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4193280 len 2048 2024-05-08T06:12:31.430 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4193280 len 2048 2024-05-08T06:12:31.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.658+0000 7f47c1a00700 1 -- 172.21.15.123:0/2919608815 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47bc10b660 msgr2=0x7f47bc10ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:31.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.658+0000 7f47c1a00700 1 --2- 172.21.15.123:0/2919608815 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47bc10b660 0x7f47bc10ba40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f47ac009990 tx=0x7f47ac009ca0 comp rx=0 tx=0).stop 2024-05-08T06:12:31.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.659+0000 7f47c1a00700 1 -- 172.21.15.123:0/2919608815 shutdown_connections 2024-05-08T06:12:31.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.659+0000 7f47c1a00700 1 --2- 172.21.15.123:0/2919608815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47bc105f90 0x7f47bc1124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:31.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.659+0000 7f47c1a00700 1 --2- 172.21.15.123:0/2919608815 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47bc10b660 0x7f47bc10ba40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:31.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.659+0000 7f47c1a00700 1 --2- 172.21.15.123:0/2919608815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47bc1055d0 0x7f47bc105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:31.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.659+0000 7f47c1a00700 1 -- 172.21.15.123:0/2919608815 >> 172.21.15.123:0/2919608815 conn(0x7f47bc1010c0 msgr2=0x7f47bc1034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:31.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.659+0000 7f47c1a00700 1 -- 172.21.15.123:0/2919608815 shutdown_connections 2024-05-08T06:12:31.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.659+0000 7f47c1a00700 1 -- 172.21.15.123:0/2919608815 wait complete. 2024-05-08T06:12:31.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.661+0000 7f47c1a00700 1 Processor -- start 2024-05-08T06:12:31.662 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.661+0000 7f47c1a00700 1 -- start start 2024-05-08T06:12:31.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.662+0000 7f47c1a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47bc1055d0 0x7f47bc1a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:31.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.662+0000 7f47c1a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47bc105f90 0x7f47bc1a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:31.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.662+0000 7f47c1a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47bc10b660 0x7f47bc19bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:31.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.662+0000 7f47c1a00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc115820 con 0x7f47bc105f90 2024-05-08T06:12:31.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.662+0000 7f47c1a00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc1156a0 con 0x7f47bc10b660 2024-05-08T06:12:31.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.662+0000 7f47c1a00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc1159a0 con 0x7f47bc1055d0 2024-05-08T06:12:31.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.662+0000 7f47bbe00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47bc1055d0 0x7f47bc1a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:31.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.662+0000 7f47bbe00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47bc1055d0 0x7f47bc1a18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:46900/0 (socket says 172.21.15.123:46900) 2024-05-08T06:12:31.663 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.662+0000 7f47bbe00700 1 -- 172.21.15.123:0/540841620 learned_addr learned my addr 172.21.15.123:0/540841620 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:31.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.662+0000 7f47bb400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47bc105f90 0x7f47bc1a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:31.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.662+0000 7f47c1000700 1 --2- 172.21.15.123:0/540841620 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47bc10b660 0x7f47bc19bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:31.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.662+0000 7f47bbe00700 1 -- 172.21.15.123:0/540841620 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47bc10b660 msgr2=0x7f47bc19bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:31.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.662+0000 7f47bbe00700 1 --2- 172.21.15.123:0/540841620 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47bc10b660 0x7f47bc19bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:31.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.662+0000 7f47bbe00700 1 -- 172.21.15.123:0/540841620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47bc105f90 msgr2=0x7f47bc1a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:31.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.662+0000 7f47bbe00700 1 --2- 172.21.15.123:0/540841620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47bc105f90 0x7f47bc1a1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:31.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.663+0000 7f47bbe00700 1 -- 172.21.15.123:0/540841620 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47bc19c240 con 0x7f47bc1055d0 2024-05-08T06:12:31.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.663+0000 7f47c1000700 1 --2- 172.21.15.123:0/540841620 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47bc10b660 0x7f47bc19bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:12:31.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.663+0000 7f47bbe00700 1 --2- 172.21.15.123:0/540841620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47bc1055d0 0x7f47bc1a18d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f47ac000c00 tx=0x7f47ac0057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:31.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.663+0000 7f47b8c00700 1 -- 172.21.15.123:0/540841620 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac027070 con 0x7f47bc1055d0 2024-05-08T06:12:31.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.663+0000 7f47b8c00700 1 -- 172.21.15.123:0/540841620 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47ac022470 con 0x7f47bc1055d0 2024-05-08T06:12:31.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.663+0000 7f47b8c00700 1 -- 172.21.15.123:0/540841620 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac017650 con 0x7f47bc1055d0 2024-05-08T06:12:31.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.663+0000 7f47c1a00700 1 -- 172.21.15.123:0/540841620 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47bc19c4d0 con 0x7f47bc1055d0 2024-05-08T06:12:31.665 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.663+0000 7f47c1a00700 1 -- 172.21.15.123:0/540841620 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47bc1a87a0 con 0x7f47bc1055d0 2024-05-08T06:12:31.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.668+0000 7f47cd946b80 1 Processor -- start 2024-05-08T06:12:31.669 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.668+0000 7f47b8c00700 1 -- 172.21.15.123:0/540841620 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7f47ac02a070 con 0x7f47bc1055d0 2024-05-08T06:12:31.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.669+0000 7f47cd946b80 1 -- start start 2024-05-08T06:12:31.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.669+0000 7f47cd946b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf1a9641d0 0x55cf1a9645b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:31.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.669+0000 7f47cd946b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf1a964af0 0x55cf1a95e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:31.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.669+0000 7f47cd946b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf1a95ee10 0x55cf1a95f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:31.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.669+0000 7f47cd946b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cf1a8a8a20 con 0x55cf1a9641d0 2024-05-08T06:12:31.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.669+0000 7f47cd946b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55cf1a846760 con 0x55cf1a964af0 2024-05-08T06:12:31.670 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.669+0000 7f47cd946b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55cf1a8a49e0 con 0x55cf1a95ee10 2024-05-08T06:12:31.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.670+0000 7f47c1000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf1a9641d0 0x55cf1a9645b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:31.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.670+0000 7f47c1000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf1a9641d0 0x55cf1a9645b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41942/0 (socket says 172.21.15.123:41942) 2024-05-08T06:12:31.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.670+0000 7f47c1000700 1 -- 172.21.15.123:0/2600008947 learned_addr learned my addr 172.21.15.123:0/2600008947 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:31.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.670+0000 7f47bb400700 1 --2- 172.21.15.123:0/2600008947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf1a95ee10 0x55cf1a95f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:31.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.670+0000 7f47bbe00700 1 --2- 172.21.15.123:0/2600008947 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf1a964af0 0x55cf1a95e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:31.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.670+0000 7f47bb400700 1 -- 172.21.15.123:0/2600008947 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf1a964af0 msgr2=0x55cf1a95e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:31.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.670+0000 7f47bb400700 1 --2- 172.21.15.123:0/2600008947 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf1a964af0 0x55cf1a95e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:31.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.670+0000 7f47bb400700 1 -- 172.21.15.123:0/2600008947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf1a9641d0 msgr2=0x55cf1a9645b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:31.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.670+0000 7f47bb400700 1 --2- 172.21.15.123:0/2600008947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf1a9641d0 0x55cf1a9645b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:31.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.670+0000 7f47bb400700 1 -- 172.21.15.123:0/2600008947 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf1a7f58d0 con 0x55cf1a95ee10 2024-05-08T06:12:31.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.670+0000 7f47bbe00700 1 --2- 172.21.15.123:0/2600008947 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf1a964af0 0x55cf1a95e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:12:31.672 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.671+0000 7f47bb400700 1 --2- 172.21.15.123:0/2600008947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf1a95ee10 0x55cf1a95f270 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f47a8009c60 tx=0x7f47a8009f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daf72309c261affc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:31.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.671+0000 7f47b8c00700 1 --2- 172.21.15.123:0/540841620 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f47a00b2900 0x7f47a00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:31.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.671+0000 7f47b1600700 1 -- 172.21.15.123:0/2600008947 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a8011d10 con 0x55cf1a95ee10 2024-05-08T06:12:31.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.671+0000 7f47b1600700 1 -- 172.21.15.123:0/2600008947 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47a8004500 con 0x55cf1a95ee10 2024-05-08T06:12:31.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.671+0000 7f47b1600700 1 -- 172.21.15.123:0/2600008947 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a8005280 con 0x55cf1a95ee10 2024-05-08T06:12:31.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.671+0000 7f47c1000700 1 --2- 172.21.15.123:0/540841620 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f47a00b2900 0x7f47a00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:31.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.671+0000 7f47b8c00700 1 -- 172.21.15.123:0/540841620 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f47ac0e2e50 con 0x7f47bc1055d0 2024-05-08T06:12:31.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.672+0000 7f47cd946b80 1 -- 172.21.15.123:0/2600008947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf1a95ee10 msgr2=0x55cf1a95f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:31.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.672+0000 7f47cd946b80 1 --2- 172.21.15.123:0/2600008947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf1a95ee10 0x55cf1a95f270 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f47a8009c60 tx=0x7f47a8009f70 comp rx=0 tx=0).stop 2024-05-08T06:12:31.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.672+0000 7f47cd946b80 1 -- 172.21.15.123:0/2600008947 shutdown_connections 2024-05-08T06:12:31.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.672+0000 7f47cd946b80 1 --2- 172.21.15.123:0/2600008947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf1a9641d0 0x55cf1a9645b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:31.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.672+0000 7f47cd946b80 1 --2- 172.21.15.123:0/2600008947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf1a95ee10 0x55cf1a95f270 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:31.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.672+0000 7f47cd946b80 1 --2- 172.21.15.123:0/2600008947 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf1a964af0 0x55cf1a95e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:31.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.672+0000 7f47cd946b80 1 -- 172.21.15.123:0/2600008947 >> 172.21.15.123:0/2600008947 conn(0x55cf1a884b60 msgr2=0x55cf1a9509b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:31.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.672+0000 7f47c1000700 1 --2- 172.21.15.123:0/540841620 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f47a00b2900 0x7f47a00b4dc0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f47b4007fd0 tx=0x7f47b4007aa0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:31.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.672+0000 7f47cd946b80 1 -- 172.21.15.123:0/2600008947 shutdown_connections 2024-05-08T06:12:31.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.673+0000 7f47cd946b80 1 -- 172.21.15.123:0/2600008947 wait complete. 2024-05-08T06:12:31.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.673+0000 7f47cd946b80 1 -- start start 2024-05-08T06:12:31.675 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.673+0000 7f47cd946b80 1 Processor -- start 2024-05-08T06:12:31.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.674+0000 7f47cd946b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf1a95ee10 0x55cf1a96a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:31.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.674+0000 7f47cd946b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf1a9641d0 0x55cf1a967a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:31.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.674+0000 7f47cd946b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf1a964af0 0x55cf1a967fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:31.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.674+0000 7f47cd946b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cf1a96b5f0 con 0x55cf1a9641d0 2024-05-08T06:12:31.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.674+0000 7f47cd946b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55cf1a80eb20 con 0x55cf1a95ee10 2024-05-08T06:12:31.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.674+0000 7f47cd946b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55cf1a96b770 con 0x55cf1a964af0 2024-05-08T06:12:31.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.674+0000 7f47cd946b80 10 client.0 _ll_register_callbacks cb 0x55cf1a84fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:12:31.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.674+0000 7f47bbe00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf1a95ee10 0x55cf1a96a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:31.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.674+0000 7f47bbe00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf1a95ee10 0x55cf1a96a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:32812/0 (socket says 172.21.15.123:32812) 2024-05-08T06:12:31.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.674+0000 7f47bbe00700 1 -- 172.21.15.123:0/1709924259 learned_addr learned my addr 172.21.15.123:0/1709924259 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:31.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.674+0000 7f47c1000700 1 --2- 172.21.15.123:0/1709924259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf1a964af0 0x55cf1a967fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:31.678 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.674+0000 7f47bb400700 1 --2- 172.21.15.123:0/1709924259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf1a9641d0 0x55cf1a967a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:31.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.675+0000 7f47bbe00700 1 -- 172.21.15.123:0/1709924259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf1a964af0 msgr2=0x55cf1a967fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:31.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.675+0000 7f47bbe00700 1 --2- 172.21.15.123:0/1709924259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf1a964af0 0x55cf1a967fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:31.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.675+0000 7f47bbe00700 1 -- 172.21.15.123:0/1709924259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf1a9641d0 msgr2=0x55cf1a967a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:31.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.675+0000 7f47bbe00700 1 --2- 172.21.15.123:0/1709924259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf1a9641d0 0x55cf1a967a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:31.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.675+0000 7f47bbe00700 1 -- 172.21.15.123:0/1709924259 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf1a86e420 con 0x55cf1a95ee10 2024-05-08T06:12:31.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.675+0000 7f47bb400700 1 --2- 172.21.15.123:0/1709924259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf1a9641d0 0x55cf1a967a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:12:31.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.675+0000 7f47bbe00700 1 --2- 172.21.15.123:0/1709924259 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf1a95ee10 0x55cf1a96a6e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f47ac0a6590 tx=0x7f47ac022c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2f44257f8919e50 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:31.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.675+0000 7f47a6000700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-05-08T06:12:31.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.675+0000 7f47a6000700 1 -- 172.21.15.123:0/1709924259 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac0aa2e0 con 0x55cf1a95ee10 2024-05-08T06:12:31.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.675+0000 7f47cd946b80 1 -- 172.21.15.123:0/1709924259 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47a8004500 con 0x55cf1a95ee10 2024-05-08T06:12:31.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.675+0000 7f47a6000700 1 -- 172.21.15.123:0/1709924259 <== mon.2 v2:172.21.15.163:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f47ac0920d0 con 0x55cf1a95ee10 2024-05-08T06:12:31.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.675+0000 7f47a6000700 1 -- 172.21.15.123:0/1709924259 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f47ac0a9c80 con 0x55cf1a95ee10 2024-05-08T06:12:31.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.675+0000 7f47cd946b80 10 client.25577 fetch_fsmap learned FSMap version 24 2024-05-08T06:12:31.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.675+0000 7f47cd946b80 1 -- 172.21.15.123:0/1709924259 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55cf1a975d10 con 0x55cf1a95ee10 2024-05-08T06:12:31.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.676+0000 7f47a6000700 1 -- 172.21.15.123:0/1709924259 <== mon.2 v2:172.21.15.163:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47a8004500 con 0x55cf1a95ee10 2024-05-08T06:12:31.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.676+0000 7f47a6000700 1 -- 172.21.15.123:0/1709924259 <== mon.2 v2:172.21.15.163:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f47ac0b1bb0 con 0x55cf1a95ee10 2024-05-08T06:12:31.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.676+0000 7f47cd946b80 10 client.25577 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:12:31.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.676+0000 7f47cd946b80 10 client.25577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T06:12:31.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.676+0000 7f47cd946b80 20 client.25577 populate_metadata read hostname 'smithi123' 2024-05-08T06:12:31.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.676+0000 7f47cd946b80 1 --2- 172.21.15.123:0/1709924259 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55cf1a978320 0x55cf1a97a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:31.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.676+0000 7f47cd946b80 4 client.25577 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:12:31.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.676+0000 7f47cd946b80 1 -- 172.21.15.123:0/1709924259 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf1a9230a0 con 0x55cf1a978320 2024-05-08T06:12:31.680 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.676+0000 7f47bb400700 1 --2- 172.21.15.123:0/1709924259 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55cf1a978320 0x55cf1a97a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:31.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.677+0000 7f47bb400700 1 --2- 172.21.15.123:0/1709924259 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55cf1a978320 0x55cf1a97a7c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4216ddc3ce186f66 server_cookie=5920ce138cf0ee28 in_seq=0 out_seq=0 2024-05-08T06:12:31.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.677+0000 7f47a6000700 10 client.25577 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:31.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.678+0000 7f47a6000700 1 -- 172.21.15.123:0/1709924259 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cf1a9230a0 con 0x55cf1a978320 2024-05-08T06:12:31.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.678+0000 7f47a6000700 10 client.25577 handle_command_reply: tid=0 2024-05-08T06:12:31.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.678+0000 7f47cd946b80 1 client.25577 shutdown 2024-05-08T06:12:31.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.678+0000 7f47cd946b80 1 -- 172.21.15.123:0/1709924259 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf1a95ee10 msgr2=0x55cf1a96a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:31.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.678+0000 7f47cd946b80 1 --2- 172.21.15.123:0/1709924259 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf1a95ee10 0x55cf1a96a6e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f47ac0a6590 tx=0x7f47ac022c40 comp rx=0 tx=0).stop 2024-05-08T06:12:31.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.678+0000 7f47cd946b80 1 -- 172.21.15.123:0/1709924259 shutdown_connections 2024-05-08T06:12:31.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.678+0000 7f47cd946b80 1 --2- 172.21.15.123:0/1709924259 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55cf1a978320 0x55cf1a97a7c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:31.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.679+0000 7f47cd946b80 1 --2- 172.21.15.123:0/1709924259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf1a964af0 0x55cf1a967fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:31.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.679+0000 7f47cd946b80 1 --2- 172.21.15.123:0/1709924259 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf1a95ee10 0x55cf1a96a6e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:31.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.679+0000 7f47cd946b80 1 --2- 172.21.15.123:0/1709924259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf1a9641d0 0x55cf1a967a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:31.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.679+0000 7f47cd946b80 1 -- 172.21.15.123:0/1709924259 >> 172.21.15.123:0/1709924259 conn(0x55cf1a884b60 msgr2=0x55cf1a94b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:31.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.679+0000 7f47a6000700 0 client.25577 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:31.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.679+0000 7f47cd946b80 1 -- 172.21.15.123:0/1709924259 shutdown_connections 2024-05-08T06:12:31.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.679+0000 7f47cd946b80 1 -- 172.21.15.123:0/1709924259 wait complete. 2024-05-08T06:12:31.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.679+0000 7f47cd946b80 20 client.25577 trim_cache size 0 max 16384 2024-05-08T06:12:31.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.708+0000 7f47cd946b80 1 Processor -- start 2024-05-08T06:12:31.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.708+0000 7f47cd946b80 1 -- start start 2024-05-08T06:12:31.709 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.708+0000 7f47cd946b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf1a95ee10 0x55cf1a974680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:31.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.708+0000 7f47cd946b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf1a9641d0 0x55cf1a9656a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:31.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.708+0000 7f47cd946b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf1a964af0 0x55cf1a9800d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:31.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.708+0000 7f47cd946b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc1aded0 con 0x55cf1a9641d0 2024-05-08T06:12:31.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.708+0000 7f47cd946b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc1ab760 con 0x55cf1a964af0 2024-05-08T06:12:31.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.708+0000 7f47cd946b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc07b720 con 0x55cf1a95ee10 2024-05-08T06:12:31.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.709+0000 7f47bbe00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf1a95ee10 0x55cf1a974680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:31.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.709+0000 7f47bbe00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf1a95ee10 0x55cf1a974680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:46916/0 (socket says 172.21.15.123:46916) 2024-05-08T06:12:31.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.709+0000 7f47bbe00700 1 -- 172.21.15.123:0/155314478 learned_addr learned my addr 172.21.15.123:0/155314478 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:31.710 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.709+0000 7f47bb400700 1 --2- 172.21.15.123:0/155314478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf1a9641d0 0x55cf1a9656a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:31.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.709+0000 7f47c1000700 1 --2- 172.21.15.123:0/155314478 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf1a964af0 0x55cf1a9800d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:31.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.709+0000 7f47bbe00700 1 -- 172.21.15.123:0/155314478 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf1a964af0 msgr2=0x55cf1a9800d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:31.711 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.709+0000 7f47bbe00700 1 --2- 172.21.15.123:0/155314478 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf1a964af0 0x55cf1a9800d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:31.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.709+0000 7f47bbe00700 1 -- 172.21.15.123:0/155314478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf1a9641d0 msgr2=0x55cf1a9656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:31.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.709+0000 7f47bbe00700 1 --2- 172.21.15.123:0/155314478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf1a9641d0 0x55cf1a9656a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:31.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.709+0000 7f47bbe00700 1 -- 172.21.15.123:0/155314478 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf1a963c10 con 0x55cf1a95ee10 2024-05-08T06:12:31.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.709+0000 7f47bb400700 1 --2- 172.21.15.123:0/155314478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf1a9641d0 0x55cf1a9656a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:12:31.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.710+0000 7f47bbe00700 1 --2- 172.21.15.123:0/155314478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf1a95ee10 0x55cf1a974680 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f47ac0aa4a0 tx=0x7f47ac005c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77b5c52e1e734c55 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:31.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.710+0000 7f47b0c00700 1 -- 172.21.15.123:0/155314478 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac0a9e60 con 0x55cf1a95ee10 2024-05-08T06:12:31.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.710+0000 7f47b0c00700 1 -- 172.21.15.123:0/155314478 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47ac0b30b0 con 0x55cf1a95ee10 2024-05-08T06:12:31.712 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.710+0000 7f47b0c00700 1 -- 172.21.15.123:0/155314478 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac0b1bb0 con 0x55cf1a95ee10 2024-05-08T06:12:31.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.711+0000 7f47cd946b80 1 -- 172.21.15.123:0/155314478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf1a95ee10 msgr2=0x55cf1a974680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:31.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.711+0000 7f47cd946b80 1 --2- 172.21.15.123:0/155314478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf1a95ee10 0x55cf1a974680 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f47ac0aa4a0 tx=0x7f47ac005c40 comp rx=0 tx=0).stop 2024-05-08T06:12:31.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.711+0000 7f47cd946b80 1 -- 172.21.15.123:0/155314478 shutdown_connections 2024-05-08T06:12:31.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.711+0000 7f47cd946b80 1 --2- 172.21.15.123:0/155314478 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf1a964af0 0x55cf1a9800d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:31.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.711+0000 7f47cd946b80 1 --2- 172.21.15.123:0/155314478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf1a95ee10 0x55cf1a974680 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:31.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.711+0000 7f47cd946b80 1 --2- 172.21.15.123:0/155314478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf1a9641d0 0x55cf1a9656a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:31.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.711+0000 7f47cd946b80 1 -- 172.21.15.123:0/155314478 >> 172.21.15.123:0/155314478 conn(0x55cf1a884b60 msgr2=0x55cf1a964f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:31.713 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.711+0000 7f47cd946b80 1 -- 172.21.15.123:0/155314478 shutdown_connections 2024-05-08T06:12:31.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.711+0000 7f47cd946b80 1 -- 172.21.15.123:0/155314478 wait complete. 2024-05-08T06:12:31.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.712+0000 7f47cd946b80 1 -- start start 2024-05-08T06:12:31.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.712+0000 7f47cd946b80 1 Processor -- start 2024-05-08T06:12:31.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.713+0000 7f47cd946b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf1a95ee10 0x55cf1a970a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:31.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.713+0000 7f47cd946b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf1a9641d0 0x55cf1a970f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:31.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.713+0000 7f47cd946b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf1a964af0 0x55cf1a96d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:31.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.713+0000 7f47cd946b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc1a9c20 con 0x55cf1a95ee10 2024-05-08T06:12:31.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.713+0000 7f47cd946b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc1ab090 con 0x55cf1a9641d0 2024-05-08T06:12:31.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.713+0000 7f47cd946b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc1aa2f0 con 0x55cf1a964af0 2024-05-08T06:12:31.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.713+0000 7f47cd946b80 10 client.0 _ll_register_callbacks cb 0x55cf1a84fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:12:31.715 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.713+0000 7f47bb400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf1a9641d0 0x55cf1a970f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:31.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.713+0000 7f47bbe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf1a95ee10 0x55cf1a970a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:31.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.713+0000 7f47bb400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf1a9641d0 0x55cf1a970f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:32838/0 (socket says 172.21.15.123:32838) 2024-05-08T06:12:31.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.713+0000 7f47c1000700 1 --2- 172.21.15.123:0/790003855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf1a964af0 0x55cf1a96d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:31.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.713+0000 7f47bb400700 1 -- 172.21.15.123:0/790003855 learned_addr learned my addr 172.21.15.123:0/790003855 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:31.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.713+0000 7f47bbe00700 1 -- 172.21.15.123:0/790003855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf1a964af0 msgr2=0x55cf1a96d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:31.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.713+0000 7f47bbe00700 1 --2- 172.21.15.123:0/790003855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf1a964af0 0x55cf1a96d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:31.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.713+0000 7f47bbe00700 1 -- 172.21.15.123:0/790003855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf1a9641d0 msgr2=0x55cf1a970f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:31.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.713+0000 7f47bbe00700 1 --2- 172.21.15.123:0/790003855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf1a9641d0 0x55cf1a970f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:31.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.713+0000 7f47bbe00700 1 -- 172.21.15.123:0/790003855 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf1a96e230 con 0x55cf1a95ee10 2024-05-08T06:12:31.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.713+0000 7f47bb400700 1 --2- 172.21.15.123:0/790003855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf1a9641d0 0x55cf1a970f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:12:31.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.714+0000 7f47bbe00700 1 --2- 172.21.15.123:0/790003855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf1a95ee10 0x55cf1a970a00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f47ac0a7780 tx=0x7f47ac0a6490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97468723d18ce31f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:31.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.714+0000 7f47a6000700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-08T06:12:31.716 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.714+0000 7f47a6000700 1 -- 172.21.15.123:0/790003855 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac0c4070 con 0x55cf1a95ee10 2024-05-08T06:12:31.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.714+0000 7f47a6000700 1 -- 172.21.15.123:0/790003855 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f47ac0920d0 con 0x55cf1a95ee10 2024-05-08T06:12:31.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.714+0000 7f47cd946b80 1 -- 172.21.15.123:0/790003855 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47ac0b30b0 con 0x55cf1a95ee10 2024-05-08T06:12:31.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.714+0000 7f47a6000700 1 -- 172.21.15.123:0/790003855 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f47ac0aa510 con 0x55cf1a95ee10 2024-05-08T06:12:31.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.714+0000 7f47cd946b80 10 client.15468 fetch_fsmap learned FSMap version 24 2024-05-08T06:12:31.717 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.714+0000 7f47cd946b80 1 -- 172.21.15.123:0/790003855 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55cf1a96e230 con 0x55cf1a95ee10 2024-05-08T06:12:31.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.715+0000 7f47a6000700 1 -- 172.21.15.123:0/790003855 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47ac0b30b0 con 0x55cf1a95ee10 2024-05-08T06:12:31.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.715+0000 7f47a6000700 1 -- 172.21.15.123:0/790003855 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f47ac00fad0 con 0x55cf1a95ee10 2024-05-08T06:12:31.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.715+0000 7f47cd946b80 10 client.15468 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:12:31.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.715+0000 7f47cd946b80 10 client.15468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T06:12:31.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.715+0000 7f47cd946b80 20 client.15468 populate_metadata read hostname 'smithi123' 2024-05-08T06:12:31.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.715+0000 7f47cd946b80 1 --2- 172.21.15.123:0/790003855 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55cf1a9e3a40 0x55cf1a9e5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:31.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.715+0000 7f47cd946b80 4 client.15468 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T06:12:31.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.715+0000 7f47cd946b80 1 -- 172.21.15.123:0/790003855 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cf1a8b6400 con 0x55cf1a9e3a40 2024-05-08T06:12:31.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.715+0000 7f47bb400700 1 --2- 172.21.15.123:0/790003855 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55cf1a9e3a40 0x55cf1a9e5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:31.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.716+0000 7f47bb400700 1 --2- 172.21.15.123:0/790003855 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55cf1a9e3a40 0x55cf1a9e5e30 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8051a80ee0d09f3 server_cookie=3bfc220a5c4df75d in_seq=0 out_seq=0 2024-05-08T06:12:31.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.716+0000 7f47a6000700 10 client.15468 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:31.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.716+0000 7f47a6000700 1 -- 172.21.15.123:0/790003855 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cf1a8b6400 con 0x55cf1a9e3a40 2024-05-08T06:12:31.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.716+0000 7f47a6000700 10 client.15468 handle_command_reply: tid=0 2024-05-08T06:12:31.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.716+0000 7f47cd946b80 1 client.15468 shutdown 2024-05-08T06:12:31.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.716+0000 7f47cd946b80 1 -- 172.21.15.123:0/790003855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf1a95ee10 msgr2=0x55cf1a970a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:31.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.716+0000 7f47cd946b80 1 --2- 172.21.15.123:0/790003855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf1a95ee10 0x55cf1a970a00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f47ac0a7780 tx=0x7f47ac0a6490 comp rx=0 tx=0).stop 2024-05-08T06:12:31.719 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.716+0000 7f47cd946b80 1 -- 172.21.15.123:0/790003855 shutdown_connections 2024-05-08T06:12:31.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.716+0000 7f47cd946b80 1 --2- 172.21.15.123:0/790003855 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55cf1a9e3a40 0x55cf1a9e5e30 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:31.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.716+0000 7f47cd946b80 1 --2- 172.21.15.123:0/790003855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cf1a95ee10 0x55cf1a970a00 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:31.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.716+0000 7f47cd946b80 1 --2- 172.21.15.123:0/790003855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf1a964af0 0x55cf1a96d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:31.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.716+0000 7f47cd946b80 1 --2- 172.21.15.123:0/790003855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cf1a9641d0 0x55cf1a970f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:31.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.716+0000 7f47a6000700 0 client.15468 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:31.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.716+0000 7f47cd946b80 1 -- 172.21.15.123:0/790003855 >> 172.21.15.123:0/790003855 conn(0x55cf1a884b60 msgr2=0x55cf1a94abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:31.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.717+0000 7f47cd946b80 1 -- 172.21.15.123:0/790003855 shutdown_connections 2024-05-08T06:12:31.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.717+0000 7f47cd946b80 1 -- 172.21.15.123:0/790003855 wait complete. 2024-05-08T06:12:31.720 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.717+0000 7f47cd946b80 20 client.15468 trim_cache size 0 max 16384 2024-05-08T06:12:31.720 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-05-08T06:12:31.720 INFO:teuthology.orchestra.run.smithi123.stdout: "status": "no active scrubs running", 2024-05-08T06:12:31.721 INFO:teuthology.orchestra.run.smithi123.stdout: "scrubs": {} 2024-05-08T06:12:31.721 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-05-08T06:12:31.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.717+0000 7f47c1a00700 1 -- 172.21.15.123:0/540841620 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f47a00b2900 msgr2=0x7f47a00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:31.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.717+0000 7f47c1a00700 1 --2- 172.21.15.123:0/540841620 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f47a00b2900 0x7f47a00b4dc0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f47b4007fd0 tx=0x7f47b4007aa0 comp rx=0 tx=0).stop 2024-05-08T06:12:31.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.717+0000 7f47c1a00700 1 -- 172.21.15.123:0/540841620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47bc1055d0 msgr2=0x7f47bc1a18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:31.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.717+0000 7f47c1a00700 1 --2- 172.21.15.123:0/540841620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47bc1055d0 0x7f47bc1a18d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f47ac000c00 tx=0x7f47ac0057c0 comp rx=0 tx=0).stop 2024-05-08T06:12:31.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.718+0000 7f47c1a00700 1 -- 172.21.15.123:0/540841620 shutdown_connections 2024-05-08T06:12:31.721 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.718+0000 7f47c1a00700 1 --2- 172.21.15.123:0/540841620 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f47a00b2900 0x7f47a00b4dc0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:31.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.718+0000 7f47c1a00700 1 --2- 172.21.15.123:0/540841620 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47bc10b660 0x7f47bc19bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:31.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.718+0000 7f47c1a00700 1 --2- 172.21.15.123:0/540841620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47bc1055d0 0x7f47bc1a18d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:31.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.718+0000 7f47c1a00700 1 --2- 172.21.15.123:0/540841620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47bc105f90 0x7f47bc1a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:31.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.718+0000 7f47c1a00700 1 -- 172.21.15.123:0/540841620 >> 172.21.15.123:0/540841620 conn(0x7f47bc1010c0 msgr2=0x7f47bc1027f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:31.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.718+0000 7f47c1a00700 1 -- 172.21.15.123:0/540841620 shutdown_connections 2024-05-08T06:12:31.722 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:31.718+0000 7f47c1a00700 1 -- 172.21.15.123:0/540841620 wait complete. 2024-05-08T06:12:31.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:31 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:12:31.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:31 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:12:31.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:31 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:12:31.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:31 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:12:31.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:31 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:12:31.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:31 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:12:32.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:31 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:12:32.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:31 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:12:32.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:31 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:12:32.413 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4193280 len 2048 2024-05-08T06:12:32.413 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4193280 len 4096 2024-05-08T06:12:32.413 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4193280 len 4096 2024-05-08T06:12:32.413 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4193280 len 4096 2024-05-08T06:12:32.413 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4193280 len 4096 2024-05-08T06:12:32.413 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4193280 len 8192 2024-05-08T06:12:32.413 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4193280 len 8192 2024-05-08T06:12:32.413 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4193280 len 8192 2024-05-08T06:12:32.413 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4193280 len 8192 2024-05-08T06:12:32.413 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4193280 len 16384 2024-05-08T06:12:32.413 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4193280 len 16384 2024-05-08T06:12:32.413 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4193280 len 16384 2024-05-08T06:12:32.413 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4193280 len 16384 2024-05-08T06:12:32.413 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4193792 len 1024 2024-05-08T06:12:32.414 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4193792 len 1024 2024-05-08T06:12:32.414 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4193792 len 1024 2024-05-08T06:12:32.414 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4193792 len 1024 2024-05-08T06:12:32.414 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4193792 len 2048 2024-05-08T06:12:32.414 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4193792 len 2048 2024-05-08T06:12:32.414 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4193792 len 2048 2024-05-08T06:12:32.415 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4193792 len 2048 2024-05-08T06:12:32.415 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4193792 len 4096 2024-05-08T06:12:32.415 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4193792 len 4096 2024-05-08T06:12:32.415 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4193792 len 4096 2024-05-08T06:12:32.415 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4193792 len 4096 2024-05-08T06:12:32.415 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4193792 len 8192 2024-05-08T06:12:32.415 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4193792 len 8192 2024-05-08T06:12:32.415 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4193792 len 8192 2024-05-08T06:12:32.415 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4193792 len 8192 2024-05-08T06:12:32.415 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4193792 len 16384 2024-05-08T06:12:32.415 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4193792 len 16384 2024-05-08T06:12:32.415 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4193792 len 16384 2024-05-08T06:12:32.415 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4193792 len 16384 2024-05-08T06:12:32.415 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4194304 len 1024 2024-05-08T06:12:32.416 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4194304 len 1024 2024-05-08T06:12:32.416 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4194304 len 1024 2024-05-08T06:12:32.416 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4194304 len 1024 2024-05-08T06:12:32.416 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4194304 len 2048 2024-05-08T06:12:32.416 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4194304 len 2048 2024-05-08T06:12:32.416 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4194304 len 2048 2024-05-08T06:12:32.416 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4194304 len 2048 2024-05-08T06:12:32.416 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4194304 len 4096 2024-05-08T06:12:32.416 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4194304 len 4096 2024-05-08T06:12:32.416 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4194304 len 4096 2024-05-08T06:12:32.416 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4194304 len 4096 2024-05-08T06:12:32.416 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4194304 len 8192 2024-05-08T06:12:32.416 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4194304 len 8192 2024-05-08T06:12:32.416 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4194304 len 8192 2024-05-08T06:12:32.416 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4194304 len 8192 2024-05-08T06:12:32.416 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4194304 len 16384 2024-05-08T06:12:32.417 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4194304 len 16384 2024-05-08T06:12:32.417 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4194304 len 16384 2024-05-08T06:12:32.417 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4194304 len 16384 2024-05-08T06:12:32.417 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4194816 len 1024 2024-05-08T06:12:32.417 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4194816 len 1024 2024-05-08T06:12:32.417 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4194816 len 1024 2024-05-08T06:12:32.417 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4194816 len 1024 2024-05-08T06:12:32.417 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4194816 len 2048 2024-05-08T06:12:32.417 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4194816 len 2048 2024-05-08T06:12:32.417 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4194816 len 2048 2024-05-08T06:12:32.417 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4194816 len 2048 2024-05-08T06:12:32.417 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4194816 len 4096 2024-05-08T06:12:32.417 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4194816 len 4096 2024-05-08T06:12:32.417 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4194816 len 4096 2024-05-08T06:12:32.417 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4194816 len 4096 2024-05-08T06:12:32.418 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4194816 len 8192 2024-05-08T06:12:32.418 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4194816 len 8192 2024-05-08T06:12:32.418 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4194816 len 8192 2024-05-08T06:12:32.418 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4194816 len 8192 2024-05-08T06:12:32.418 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4194816 len 16384 2024-05-08T06:12:32.418 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4194816 len 16384 2024-05-08T06:12:32.418 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4194816 len 16384 2024-05-08T06:12:32.418 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4194816 len 16384 2024-05-08T06:12:32.418 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4195328 len 1024 2024-05-08T06:12:32.418 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4195328 len 1024 2024-05-08T06:12:32.418 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4195328 len 1024 2024-05-08T06:12:32.418 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4195328 len 1024 2024-05-08T06:12:32.418 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4195328 len 2048 2024-05-08T06:12:32.418 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4195328 len 2048 2024-05-08T06:12:32.418 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4195328 len 2048 2024-05-08T06:12:32.419 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4195328 len 2048 2024-05-08T06:12:32.419 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4195328 len 4096 2024-05-08T06:12:32.419 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4195328 len 4096 2024-05-08T06:12:32.419 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4195328 len 4096 2024-05-08T06:12:32.419 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4195328 len 4096 2024-05-08T06:12:32.887 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:32 smithi123 ceph-mon[29787]: pgmap v522: 97 pgs: 97 active+clean; 49 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 355 KiB/s wr, 1.15k op/s 2024-05-08T06:12:32.890 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T06:12:32.890 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T06:12:32.959 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4195328 len 8192 2024-05-08T06:12:32.959 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4195328 len 8192 2024-05-08T06:12:32.959 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4195328 len 8192 2024-05-08T06:12:32.959 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4195328 len 8192 2024-05-08T06:12:32.959 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4195328 len 16384 2024-05-08T06:12:32.959 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4195328 len 16384 2024-05-08T06:12:32.959 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4195328 len 16384 2024-05-08T06:12:32.960 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4195328 len 16384 2024-05-08T06:12:32.960 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4195840 len 1024 2024-05-08T06:12:32.960 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4195840 len 1024 2024-05-08T06:12:32.960 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4195840 len 1024 2024-05-08T06:12:32.960 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4195840 len 1024 2024-05-08T06:12:32.960 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4195840 len 2048 2024-05-08T06:12:32.960 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4195840 len 2048 2024-05-08T06:12:32.960 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4195840 len 2048 2024-05-08T06:12:32.960 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4195840 len 2048 2024-05-08T06:12:32.960 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4195840 len 4096 2024-05-08T06:12:32.960 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4195840 len 4096 2024-05-08T06:12:32.960 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4195840 len 4096 2024-05-08T06:12:32.960 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4195840 len 4096 2024-05-08T06:12:32.960 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4195840 len 8192 2024-05-08T06:12:32.960 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4195840 len 8192 2024-05-08T06:12:32.961 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4195840 len 8192 2024-05-08T06:12:32.961 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4195840 len 8192 2024-05-08T06:12:32.961 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4195840 len 16384 2024-05-08T06:12:32.961 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4195840 len 16384 2024-05-08T06:12:32.961 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4195840 len 16384 2024-05-08T06:12:32.961 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4195840 len 16384 2024-05-08T06:12:32.961 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4196352 len 1024 2024-05-08T06:12:32.961 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4196352 len 1024 2024-05-08T06:12:32.961 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4196352 len 1024 2024-05-08T06:12:32.961 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4196352 len 1024 2024-05-08T06:12:32.961 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4196352 len 2048 2024-05-08T06:12:32.961 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4196352 len 2048 2024-05-08T06:12:32.961 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4196352 len 2048 2024-05-08T06:12:32.961 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4196352 len 2048 2024-05-08T06:12:32.961 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4196352 len 4096 2024-05-08T06:12:32.962 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4196352 len 4096 2024-05-08T06:12:32.962 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4196352 len 4096 2024-05-08T06:12:32.962 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4196352 len 4096 2024-05-08T06:12:32.962 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4196352 len 8192 2024-05-08T06:12:32.962 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4196352 len 8192 2024-05-08T06:12:32.962 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4196352 len 8192 2024-05-08T06:12:32.962 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4196352 len 8192 2024-05-08T06:12:32.962 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4196352 len 16384 2024-05-08T06:12:32.962 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4196352 len 16384 2024-05-08T06:12:32.962 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4196352 len 16384 2024-05-08T06:12:32.962 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4196352 len 16384 2024-05-08T06:12:32.962 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4196864 len 1024 2024-05-08T06:12:32.962 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4196864 len 1024 2024-05-08T06:12:32.962 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4196864 len 1024 2024-05-08T06:12:32.963 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4196864 len 1024 2024-05-08T06:12:32.963 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4196864 len 2048 2024-05-08T06:12:32.963 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4196864 len 2048 2024-05-08T06:12:32.963 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4196864 len 2048 2024-05-08T06:12:32.963 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4196864 len 2048 2024-05-08T06:12:32.963 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4196864 len 4096 2024-05-08T06:12:32.963 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4196864 len 4096 2024-05-08T06:12:32.963 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4196864 len 4096 2024-05-08T06:12:32.963 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4196864 len 4096 2024-05-08T06:12:32.963 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4196864 len 8192 2024-05-08T06:12:32.963 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4196864 len 8192 2024-05-08T06:12:32.963 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4196864 len 8192 2024-05-08T06:12:32.963 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4196864 len 8192 2024-05-08T06:12:32.963 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4196864 len 16384 2024-05-08T06:12:32.963 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4196864 len 16384 2024-05-08T06:12:32.963 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4196864 len 16384 2024-05-08T06:12:32.964 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4196864 len 16384 2024-05-08T06:12:32.964 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4197376 len 1024 2024-05-08T06:12:32.964 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4197376 len 1024 2024-05-08T06:12:32.964 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4197376 len 1024 2024-05-08T06:12:32.964 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4197376 len 1024 2024-05-08T06:12:32.964 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4197376 len 2048 2024-05-08T06:12:32.964 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4197376 len 2048 2024-05-08T06:12:32.964 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4197376 len 2048 2024-05-08T06:12:32.964 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4197376 len 2048 2024-05-08T06:12:32.964 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4197376 len 4096 2024-05-08T06:12:32.964 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4197376 len 4096 2024-05-08T06:12:32.964 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4197376 len 4096 2024-05-08T06:12:32.964 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4197376 len 4096 2024-05-08T06:12:32.964 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4197376 len 8192 2024-05-08T06:12:32.964 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4197376 len 8192 2024-05-08T06:12:32.964 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4197376 len 8192 2024-05-08T06:12:32.965 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4197376 len 8192 2024-05-08T06:12:32.965 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4197376 len 16384 2024-05-08T06:12:32.965 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4197376 len 16384 2024-05-08T06:12:33.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:32 smithi163 ceph-mon[34520]: pgmap v522: 97 pgs: 97 active+clean; 49 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 355 KiB/s wr, 1.15k op/s 2024-05-08T06:12:33.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:32 smithi192 ceph-mon[34664]: pgmap v522: 97 pgs: 97 active+clean; 49 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 355 KiB/s wr, 1.15k op/s 2024-05-08T06:12:33.398 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:12:34.317 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4197376 len 16384 2024-05-08T06:12:34.317 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4197376 len 16384 2024-05-08T06:12:34.317 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4197888 len 1024 2024-05-08T06:12:34.318 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4197888 len 1024 2024-05-08T06:12:34.318 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4197888 len 1024 2024-05-08T06:12:34.318 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4197888 len 1024 2024-05-08T06:12:34.318 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4197888 len 2048 2024-05-08T06:12:34.318 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4197888 len 2048 2024-05-08T06:12:34.318 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4197888 len 2048 2024-05-08T06:12:34.318 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4197888 len 2048 2024-05-08T06:12:34.318 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4197888 len 4096 2024-05-08T06:12:34.318 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4197888 len 4096 2024-05-08T06:12:34.318 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4197888 len 4096 2024-05-08T06:12:34.318 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4197888 len 4096 2024-05-08T06:12:34.318 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4197888 len 8192 2024-05-08T06:12:34.318 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4197888 len 8192 2024-05-08T06:12:34.319 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4197888 len 8192 2024-05-08T06:12:34.319 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4197888 len 8192 2024-05-08T06:12:34.319 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 0 offset 4197888 len 16384 2024-05-08T06:12:34.319 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4197888 len 16384 2024-05-08T06:12:34.319 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 0 offset 4197888 len 16384 2024-05-08T06:12:34.319 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 0 offset 4197888 len 16384 2024-05-08T06:12:34.319 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4190720 len 1024 2024-05-08T06:12:34.319 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4190720 len 1024 2024-05-08T06:12:34.319 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4190720 len 1024 2024-05-08T06:12:34.319 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4190720 len 1024 2024-05-08T06:12:34.319 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4190720 len 2048 2024-05-08T06:12:34.319 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4190720 len 2048 2024-05-08T06:12:34.319 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4190720 len 2048 2024-05-08T06:12:34.319 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4190720 len 2048 2024-05-08T06:12:34.319 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4190720 len 4096 2024-05-08T06:12:34.319 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4190720 len 4096 2024-05-08T06:12:34.319 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4190720 len 4096 2024-05-08T06:12:34.320 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4190720 len 4096 2024-05-08T06:12:34.320 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4190720 len 8192 2024-05-08T06:12:34.320 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4190720 len 8192 2024-05-08T06:12:34.320 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4190720 len 8192 2024-05-08T06:12:34.320 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4190720 len 8192 2024-05-08T06:12:34.320 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4190720 len 16384 2024-05-08T06:12:34.320 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4190720 len 16384 2024-05-08T06:12:34.320 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4190720 len 16384 2024-05-08T06:12:34.320 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4190720 len 16384 2024-05-08T06:12:34.320 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4191232 len 1024 2024-05-08T06:12:34.320 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4191232 len 1024 2024-05-08T06:12:34.320 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4191232 len 1024 2024-05-08T06:12:34.320 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4191232 len 1024 2024-05-08T06:12:34.320 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4191232 len 2048 2024-05-08T06:12:34.320 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4191232 len 2048 2024-05-08T06:12:34.320 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4191232 len 2048 2024-05-08T06:12:34.321 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4191232 len 2048 2024-05-08T06:12:34.321 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4191232 len 4096 2024-05-08T06:12:34.321 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4191232 len 4096 2024-05-08T06:12:34.321 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4191232 len 4096 2024-05-08T06:12:34.321 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4191232 len 4096 2024-05-08T06:12:34.321 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4191232 len 8192 2024-05-08T06:12:34.321 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4191232 len 8192 2024-05-08T06:12:34.321 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4191232 len 8192 2024-05-08T06:12:34.321 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4191232 len 8192 2024-05-08T06:12:34.321 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4191232 len 16384 2024-05-08T06:12:34.321 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4191232 len 16384 2024-05-08T06:12:34.321 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4191232 len 16384 2024-05-08T06:12:34.321 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4191232 len 16384 2024-05-08T06:12:34.321 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4191744 len 1024 2024-05-08T06:12:34.321 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4191744 len 1024 2024-05-08T06:12:34.321 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4191744 len 1024 2024-05-08T06:12:34.322 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4191744 len 1024 2024-05-08T06:12:34.322 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4191744 len 2048 2024-05-08T06:12:34.322 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4191744 len 2048 2024-05-08T06:12:34.322 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4191744 len 2048 2024-05-08T06:12:34.322 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4191744 len 2048 2024-05-08T06:12:34.322 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4191744 len 4096 2024-05-08T06:12:34.322 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4191744 len 4096 2024-05-08T06:12:34.322 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4191744 len 4096 2024-05-08T06:12:34.322 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4191744 len 4096 2024-05-08T06:12:34.322 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4191744 len 8192 2024-05-08T06:12:34.322 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4191744 len 8192 2024-05-08T06:12:34.322 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4191744 len 8192 2024-05-08T06:12:34.322 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4191744 len 8192 2024-05-08T06:12:34.322 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4191744 len 16384 2024-05-08T06:12:34.323 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4191744 len 16384 2024-05-08T06:12:34.323 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4191744 len 16384 2024-05-08T06:12:34.323 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4191744 len 16384 2024-05-08T06:12:34.323 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4192256 len 1024 2024-05-08T06:12:34.323 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4192256 len 1024 2024-05-08T06:12:34.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:34 smithi123 ceph-mon[29787]: pgmap v523: 97 pgs: 97 active+clean; 181 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 722 KiB/s wr, 541 op/s 2024-05-08T06:12:35.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:34 smithi163 ceph-mon[34520]: pgmap v523: 97 pgs: 97 active+clean; 181 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 722 KiB/s wr, 541 op/s 2024-05-08T06:12:35.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:34 smithi192 ceph-mon[34664]: pgmap v523: 97 pgs: 97 active+clean; 181 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 722 KiB/s wr, 541 op/s 2024-05-08T06:12:36.038 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.038+0000 7ff33cc00700 1 -- 172.21.15.123:0/2317034866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff338109510 msgr2=0x7ff3381098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:36.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.038+0000 7ff33cc00700 1 --2- 172.21.15.123:0/2317034866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff338109510 0x7ff3381098f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff328009990 tx=0x7ff328009ca0 comp rx=0 tx=0).stop 2024-05-08T06:12:36.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.038+0000 7ff33cc00700 1 -- 172.21.15.123:0/2317034866 shutdown_connections 2024-05-08T06:12:36.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.038+0000 7ff33cc00700 1 --2- 172.21.15.123:0/2317034866 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff33806bb40 0x7ff338110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:36.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.038+0000 7ff33cc00700 1 --2- 172.21.15.123:0/2317034866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff338109510 0x7ff3381098f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:36.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.038+0000 7ff33cc00700 1 --2- 172.21.15.123:0/2317034866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff33806b180 0x7ff33806b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:36.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.038+0000 7ff33cc00700 1 -- 172.21.15.123:0/2317034866 >> 172.21.15.123:0/2317034866 conn(0x7ff338079350 msgr2=0x7ff338079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:36.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.038+0000 7ff33cc00700 1 -- 172.21.15.123:0/2317034866 shutdown_connections 2024-05-08T06:12:36.039 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.038+0000 7ff33cc00700 1 -- 172.21.15.123:0/2317034866 wait complete. 2024-05-08T06:12:36.040 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4192256 len 1024 2024-05-08T06:12:36.040 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4192256 len 1024 2024-05-08T06:12:36.040 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4192256 len 2048 2024-05-08T06:12:36.040 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4192256 len 2048 2024-05-08T06:12:36.040 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4192256 len 2048 2024-05-08T06:12:36.040 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4192256 len 2048 2024-05-08T06:12:36.040 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4192256 len 4096 2024-05-08T06:12:36.040 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4192256 len 4096 2024-05-08T06:12:36.040 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4192256 len 4096 2024-05-08T06:12:36.040 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4192256 len 4096 2024-05-08T06:12:36.040 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4192256 len 8192 2024-05-08T06:12:36.040 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4192256 len 8192 2024-05-08T06:12:36.041 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4192256 len 8192 2024-05-08T06:12:36.041 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4192256 len 8192 2024-05-08T06:12:36.041 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4192256 len 16384 2024-05-08T06:12:36.041 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4192256 len 16384 2024-05-08T06:12:36.041 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4192256 len 16384 2024-05-08T06:12:36.041 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4192256 len 16384 2024-05-08T06:12:36.041 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4192768 len 1024 2024-05-08T06:12:36.041 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4192768 len 1024 2024-05-08T06:12:36.041 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4192768 len 1024 2024-05-08T06:12:36.041 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4192768 len 1024 2024-05-08T06:12:36.041 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4192768 len 2048 2024-05-08T06:12:36.041 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4192768 len 2048 2024-05-08T06:12:36.041 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4192768 len 2048 2024-05-08T06:12:36.041 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4192768 len 2048 2024-05-08T06:12:36.042 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4192768 len 4096 2024-05-08T06:12:36.042 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4192768 len 4096 2024-05-08T06:12:36.042 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4192768 len 4096 2024-05-08T06:12:36.042 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4192768 len 4096 2024-05-08T06:12:36.042 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4192768 len 8192 2024-05-08T06:12:36.042 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4192768 len 8192 2024-05-08T06:12:36.042 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4192768 len 8192 2024-05-08T06:12:36.042 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4192768 len 8192 2024-05-08T06:12:36.042 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4192768 len 16384 2024-05-08T06:12:36.042 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4192768 len 16384 2024-05-08T06:12:36.042 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4192768 len 16384 2024-05-08T06:12:36.042 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4192768 len 16384 2024-05-08T06:12:36.042 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4193280 len 1024 2024-05-08T06:12:36.042 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4193280 len 1024 2024-05-08T06:12:36.043 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4193280 len 1024 2024-05-08T06:12:36.043 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4193280 len 1024 2024-05-08T06:12:36.043 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4193280 len 2048 2024-05-08T06:12:36.043 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4193280 len 2048 2024-05-08T06:12:36.043 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4193280 len 2048 2024-05-08T06:12:36.043 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4193280 len 2048 2024-05-08T06:12:36.043 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4193280 len 4096 2024-05-08T06:12:36.043 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4193280 len 4096 2024-05-08T06:12:36.043 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4193280 len 4096 2024-05-08T06:12:36.043 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4193280 len 4096 2024-05-08T06:12:36.043 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4193280 len 8192 2024-05-08T06:12:36.043 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4193280 len 8192 2024-05-08T06:12:36.043 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4193280 len 8192 2024-05-08T06:12:36.043 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4193280 len 8192 2024-05-08T06:12:36.043 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4193280 len 16384 2024-05-08T06:12:36.044 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4193280 len 16384 2024-05-08T06:12:36.044 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4193280 len 16384 2024-05-08T06:12:36.044 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4193280 len 16384 2024-05-08T06:12:36.044 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4193792 len 1024 2024-05-08T06:12:36.044 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4193792 len 1024 2024-05-08T06:12:36.044 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4193792 len 1024 2024-05-08T06:12:36.044 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4193792 len 1024 2024-05-08T06:12:36.044 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4193792 len 2048 2024-05-08T06:12:36.044 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4193792 len 2048 2024-05-08T06:12:36.044 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4193792 len 2048 2024-05-08T06:12:36.044 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4193792 len 2048 2024-05-08T06:12:36.044 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4193792 len 4096 2024-05-08T06:12:36.044 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4193792 len 4096 2024-05-08T06:12:36.044 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4193792 len 4096 2024-05-08T06:12:36.045 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4193792 len 4096 2024-05-08T06:12:36.045 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4193792 len 8192 2024-05-08T06:12:36.045 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4193792 len 8192 2024-05-08T06:12:36.045 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4193792 len 8192 2024-05-08T06:12:36.045 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4193792 len 8192 2024-05-08T06:12:36.045 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4193792 len 16384 2024-05-08T06:12:36.045 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4193792 len 16384 2024-05-08T06:12:36.045 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4193792 len 16384 2024-05-08T06:12:36.045 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4193792 len 16384 2024-05-08T06:12:36.045 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4194304 len 1024 2024-05-08T06:12:36.045 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4194304 len 1024 2024-05-08T06:12:36.045 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4194304 len 1024 2024-05-08T06:12:36.045 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4194304 len 1024 2024-05-08T06:12:36.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.039+0000 7ff33cc00700 1 Processor -- start 2024-05-08T06:12:36.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.039+0000 7ff33cc00700 1 -- start start 2024-05-08T06:12:36.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.040+0000 7ff33cc00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff33806b180 0x7ff33819bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:36.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.040+0000 7ff33cc00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff33806bb40 0x7ff33819c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:36.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.040+0000 7ff33cc00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff338109510 0x7ff3381a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:36.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.040+0000 7ff33cc00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff338113690 con 0x7ff33806bb40 2024-05-08T06:12:36.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.040+0000 7ff33cc00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff338113510 con 0x7ff338109510 2024-05-08T06:12:36.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.040+0000 7ff33cc00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff338113810 con 0x7ff33806b180 2024-05-08T06:12:36.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.040+0000 7ff336800700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff338109510 0x7ff3381a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:36.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.040+0000 7ff335400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff33806bb40 0x7ff33819c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:36.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.040+0000 7ff336800700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff338109510 0x7ff3381a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:40062/0 (socket says 172.21.15.123:40062) 2024-05-08T06:12:36.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.040+0000 7ff336800700 1 -- 172.21.15.123:0/368977184 learned_addr learned my addr 172.21.15.123:0/368977184 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:36.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.040+0000 7ff335e00700 1 --2- 172.21.15.123:0/368977184 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff33806b180 0x7ff33819bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:36.048 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.041+0000 7ff336800700 1 -- 172.21.15.123:0/368977184 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff33806b180 msgr2=0x7ff33819bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:36.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.041+0000 7ff336800700 1 --2- 172.21.15.123:0/368977184 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff33806b180 0x7ff33819bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:36.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.041+0000 7ff336800700 1 -- 172.21.15.123:0/368977184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff33806bb40 msgr2=0x7ff33819c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:36.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.041+0000 7ff336800700 1 --2- 172.21.15.123:0/368977184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff33806bb40 0x7ff33819c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:36.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.041+0000 7ff336800700 1 -- 172.21.15.123:0/368977184 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3381a0d90 con 0x7ff338109510 2024-05-08T06:12:36.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.041+0000 7ff335400700 1 --2- 172.21.15.123:0/368977184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff33806bb40 0x7ff33819c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:12:36.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.041+0000 7ff335e00700 1 --2- 172.21.15.123:0/368977184 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff33806b180 0x7ff33819bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:12:36.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.041+0000 7ff336800700 1 --2- 172.21.15.123:0/368977184 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff338109510 0x7ff3381a0650 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff33000ef30 tx=0x7ff33000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:36.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.041+0000 7ff32ea00700 1 -- 172.21.15.123:0/368977184 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3300092e0 con 0x7ff338109510 2024-05-08T06:12:36.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.041+0000 7ff32ea00700 1 -- 172.21.15.123:0/368977184 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff330004bb0 con 0x7ff338109510 2024-05-08T06:12:36.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.041+0000 7ff32ea00700 1 -- 172.21.15.123:0/368977184 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff330004020 con 0x7ff338109510 2024-05-08T06:12:36.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.041+0000 7ff33cc00700 1 -- 172.21.15.123:0/368977184 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3381a1020 con 0x7ff338109510 2024-05-08T06:12:36.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.042+0000 7ff33cc00700 1 -- 172.21.15.123:0/368977184 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3381a8900 con 0x7ff338109510 2024-05-08T06:12:36.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.043+0000 7ff33cc00700 1 -- 172.21.15.123:0/368977184 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff33806a190 con 0x7ff338109510 2024-05-08T06:12:36.049 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.045+0000 7ff32ea00700 1 -- 172.21.15.123:0/368977184 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7ff33001b030 con 0x7ff338109510 2024-05-08T06:12:36.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.047+0000 7ff32ea00700 1 --2- 172.21.15.123:0/368977184 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7ff31c0b2900 0x7ff31c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:36.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.047+0000 7ff32ea00700 1 -- 172.21.15.123:0/368977184 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7ff3300e25a0 con 0x7ff338109510 2024-05-08T06:12:36.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.047+0000 7ff335e00700 1 --2- 172.21.15.123:0/368977184 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7ff31c0b2900 0x7ff31c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:36.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.048+0000 7ff335e00700 1 --2- 172.21.15.123:0/368977184 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7ff31c0b2900 0x7ff31c0b4dc0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff328006010 tx=0x7ff32800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:36.050 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.049+0000 7ff32ea00700 1 -- 172.21.15.123:0/368977184 <== mon.2 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff330022020 con 0x7ff338109510 2024-05-08T06:12:36.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.338+0000 7ff33cc00700 1 -- 172.21.15.123:0/368977184 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3381a8ad0 con 0x7ff338109510 2024-05-08T06:12:36.339 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.339+0000 7ff32ea00700 1 -- 172.21.15.123:0/368977184 <== mon.2 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11885 (secure 0 0 0) 0x7ff330010040 con 0x7ff338109510 2024-05-08T06:12:36.340 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:12:36.340 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3646476522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3646476522},{"type":"v1","addr":"172.21.15.123:6835","nonce":3646476522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2763706497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2763706497},{"type":"v1","addr":"172.21.15.123:6837","nonce":2763706497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/274635265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":274635265},{"type":"v1","addr":"172.21.15.123:6839","nonce":274635265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/175423115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":175423115},{"type":"v1","addr":"172.21.15.192:6833","nonce":175423115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/3263009472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3263009472},{"type":"v1","addr":"172.21.15.163:6833","nonce":3263009472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3308612826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3308612826},{"type":"v1","addr":"172.21.15.163:6835","nonce":3308612826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T06:05:54.755550+0000","modified":"2024-05-08T06:10:12.600417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24512,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/202513680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":202513680},{"type":"v1","addr":"172.21.15.192:6835","nonce":202513680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/2483545969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2483545969},{"type":"v1","addr":"172.21.15.192:6837","nonce":2483545969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/660511838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":660511838},{"type":"v1","addr":"172.21.15.163:6837","nonce":660511838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.192:6839/3598682360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":3598682360},{"type":"v1","addr":"172.21.15.192:6839","nonce":3598682360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6841/1571726231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":1571726231},{"type":"v1","addr":"172.21.15.123:6841","nonce":1571726231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.163:6839/1568161515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":1568161515},{"type":"v1","addr":"172.21.15.163:6839","nonce":1568161515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24512,24496,24524]},"id":1}]} 2024-05-08T06:12:36.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.344+0000 7ff33cc00700 1 -- 172.21.15.123:0/368977184 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7ff31c0b2900 msgr2=0x7ff31c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:36.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.344+0000 7ff33cc00700 1 --2- 172.21.15.123:0/368977184 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7ff31c0b2900 0x7ff31c0b4dc0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff328006010 tx=0x7ff32800b260 comp rx=0 tx=0).stop 2024-05-08T06:12:36.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.344+0000 7ff33cc00700 1 -- 172.21.15.123:0/368977184 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff338109510 msgr2=0x7ff3381a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:36.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.344+0000 7ff33cc00700 1 --2- 172.21.15.123:0/368977184 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff338109510 0x7ff3381a0650 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff33000ef30 tx=0x7ff33000c3a0 comp rx=0 tx=0).stop 2024-05-08T06:12:36.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.344+0000 7ff33cc00700 1 -- 172.21.15.123:0/368977184 shutdown_connections 2024-05-08T06:12:36.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.344+0000 7ff33cc00700 1 --2- 172.21.15.123:0/368977184 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7ff31c0b2900 0x7ff31c0b4dc0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:36.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.344+0000 7ff33cc00700 1 --2- 172.21.15.123:0/368977184 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff338109510 0x7ff3381a0650 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:36.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.344+0000 7ff33cc00700 1 --2- 172.21.15.123:0/368977184 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff33806b180 0x7ff33819bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:36.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.344+0000 7ff33cc00700 1 --2- 172.21.15.123:0/368977184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff33806bb40 0x7ff33819c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:36.344 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.344+0000 7ff33cc00700 1 -- 172.21.15.123:0/368977184 >> 172.21.15.123:0/368977184 conn(0x7ff338079350 msgr2=0x7ff338100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:36.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.344+0000 7ff33cc00700 1 -- 172.21.15.123:0/368977184 shutdown_connections 2024-05-08T06:12:36.345 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:36.344+0000 7ff33cc00700 1 -- 172.21.15.123:0/368977184 wait complete. 2024-05-08T06:12:36.346 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 24 2024-05-08T06:12:36.536 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4194304 len 2048 2024-05-08T06:12:36.537 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4194304 len 2048 2024-05-08T06:12:36.537 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4194304 len 2048 2024-05-08T06:12:36.537 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4194304 len 2048 2024-05-08T06:12:36.537 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4194304 len 4096 2024-05-08T06:12:36.537 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4194304 len 4096 2024-05-08T06:12:36.537 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4194304 len 4096 2024-05-08T06:12:36.537 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4194304 len 4096 2024-05-08T06:12:36.537 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4194304 len 8192 2024-05-08T06:12:36.537 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4194304 len 8192 2024-05-08T06:12:36.537 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4194304 len 8192 2024-05-08T06:12:36.537 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4194304 len 8192 2024-05-08T06:12:36.537 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4194304 len 16384 2024-05-08T06:12:36.537 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4194304 len 16384 2024-05-08T06:12:36.538 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4194304 len 16384 2024-05-08T06:12:36.538 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4194304 len 16384 2024-05-08T06:12:36.538 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4194816 len 1024 2024-05-08T06:12:36.538 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4194816 len 1024 2024-05-08T06:12:36.538 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4194816 len 1024 2024-05-08T06:12:36.538 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4194816 len 1024 2024-05-08T06:12:36.538 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4194816 len 2048 2024-05-08T06:12:36.538 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4194816 len 2048 2024-05-08T06:12:36.538 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4194816 len 2048 2024-05-08T06:12:36.538 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4194816 len 2048 2024-05-08T06:12:36.538 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4194816 len 4096 2024-05-08T06:12:36.538 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4194816 len 4096 2024-05-08T06:12:36.538 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4194816 len 4096 2024-05-08T06:12:36.538 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4194816 len 4096 2024-05-08T06:12:36.539 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4194816 len 8192 2024-05-08T06:12:36.539 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4194816 len 8192 2024-05-08T06:12:36.539 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4194816 len 8192 2024-05-08T06:12:36.539 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4194816 len 8192 2024-05-08T06:12:36.539 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4194816 len 16384 2024-05-08T06:12:36.539 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4194816 len 16384 2024-05-08T06:12:36.539 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4194816 len 16384 2024-05-08T06:12:36.539 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4194816 len 16384 2024-05-08T06:12:36.539 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4195328 len 1024 2024-05-08T06:12:36.539 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4195328 len 1024 2024-05-08T06:12:36.539 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4195328 len 1024 2024-05-08T06:12:36.539 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4195328 len 1024 2024-05-08T06:12:36.539 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4195328 len 2048 2024-05-08T06:12:36.539 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4195328 len 2048 2024-05-08T06:12:36.539 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4195328 len 2048 2024-05-08T06:12:36.540 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4195328 len 2048 2024-05-08T06:12:36.540 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4195328 len 4096 2024-05-08T06:12:36.540 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4195328 len 4096 2024-05-08T06:12:36.540 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4195328 len 4096 2024-05-08T06:12:36.540 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4195328 len 4096 2024-05-08T06:12:36.540 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4195328 len 8192 2024-05-08T06:12:36.540 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4195328 len 8192 2024-05-08T06:12:36.540 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4195328 len 8192 2024-05-08T06:12:36.540 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4195328 len 8192 2024-05-08T06:12:36.540 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4195328 len 16384 2024-05-08T06:12:36.540 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4195328 len 16384 2024-05-08T06:12:36.540 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4195328 len 16384 2024-05-08T06:12:36.540 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4195328 len 16384 2024-05-08T06:12:36.540 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4195840 len 1024 2024-05-08T06:12:36.540 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4195840 len 1024 2024-05-08T06:12:36.541 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4195840 len 1024 2024-05-08T06:12:36.541 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4195840 len 1024 2024-05-08T06:12:36.541 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4195840 len 2048 2024-05-08T06:12:36.541 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4195840 len 2048 2024-05-08T06:12:36.541 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4195840 len 2048 2024-05-08T06:12:36.541 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4195840 len 2048 2024-05-08T06:12:36.541 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4195840 len 4096 2024-05-08T06:12:36.541 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4195840 len 4096 2024-05-08T06:12:36.541 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4195840 len 4096 2024-05-08T06:12:36.541 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4195840 len 4096 2024-05-08T06:12:36.541 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4195840 len 8192 2024-05-08T06:12:36.541 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4195840 len 8192 2024-05-08T06:12:36.541 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4195840 len 8192 2024-05-08T06:12:36.541 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4195840 len 8192 2024-05-08T06:12:36.541 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4195840 len 16384 2024-05-08T06:12:36.542 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4195840 len 16384 2024-05-08T06:12:36.542 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4195840 len 16384 2024-05-08T06:12:36.542 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4195840 len 16384 2024-05-08T06:12:36.542 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4196352 len 1024 2024-05-08T06:12:36.542 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4196352 len 1024 2024-05-08T06:12:36.542 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4196352 len 1024 2024-05-08T06:12:36.542 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4196352 len 1024 2024-05-08T06:12:36.542 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4196352 len 2048 2024-05-08T06:12:36.542 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4196352 len 2048 2024-05-08T06:12:36.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:36 smithi123 ceph-mon[29787]: pgmap v524: 97 pgs: 97 active+clean; 231 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 889 KiB/s wr, 582 op/s 2024-05-08T06:12:36.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:36 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/368977184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:12:37.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:36 smithi163 ceph-mon[34520]: pgmap v524: 97 pgs: 97 active+clean; 231 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 889 KiB/s wr, 582 op/s 2024-05-08T06:12:37.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:36 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/368977184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:12:37.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:36 smithi192 ceph-mon[34664]: pgmap v524: 97 pgs: 97 active+clean; 231 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 889 KiB/s wr, 582 op/s 2024-05-08T06:12:37.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:36 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/368977184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:12:37.180 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4196352 len 2048 2024-05-08T06:12:37.181 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4196352 len 2048 2024-05-08T06:12:37.181 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4196352 len 4096 2024-05-08T06:12:37.181 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4196352 len 4096 2024-05-08T06:12:37.181 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4196352 len 4096 2024-05-08T06:12:37.181 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4196352 len 4096 2024-05-08T06:12:37.181 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4196352 len 8192 2024-05-08T06:12:37.181 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4196352 len 8192 2024-05-08T06:12:37.181 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4196352 len 8192 2024-05-08T06:12:37.181 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4196352 len 8192 2024-05-08T06:12:37.181 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4196352 len 16384 2024-05-08T06:12:37.182 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4196352 len 16384 2024-05-08T06:12:37.182 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4196352 len 16384 2024-05-08T06:12:37.182 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4196352 len 16384 2024-05-08T06:12:37.182 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4196864 len 1024 2024-05-08T06:12:37.182 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4196864 len 1024 2024-05-08T06:12:37.182 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4196864 len 1024 2024-05-08T06:12:37.182 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4196864 len 1024 2024-05-08T06:12:37.182 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4196864 len 2048 2024-05-08T06:12:37.182 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4196864 len 2048 2024-05-08T06:12:37.182 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4196864 len 2048 2024-05-08T06:12:37.182 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4196864 len 2048 2024-05-08T06:12:37.182 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4196864 len 4096 2024-05-08T06:12:37.183 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4196864 len 4096 2024-05-08T06:12:37.183 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4196864 len 4096 2024-05-08T06:12:37.183 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4196864 len 4096 2024-05-08T06:12:37.183 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4196864 len 8192 2024-05-08T06:12:37.183 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4196864 len 8192 2024-05-08T06:12:37.183 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4196864 len 8192 2024-05-08T06:12:37.184 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4196864 len 8192 2024-05-08T06:12:37.184 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4196864 len 16384 2024-05-08T06:12:37.184 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4196864 len 16384 2024-05-08T06:12:37.184 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4196864 len 16384 2024-05-08T06:12:37.184 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4196864 len 16384 2024-05-08T06:12:37.184 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4197376 len 1024 2024-05-08T06:12:37.184 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4197376 len 1024 2024-05-08T06:12:37.184 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4197376 len 1024 2024-05-08T06:12:37.184 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4197376 len 1024 2024-05-08T06:12:37.184 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4197376 len 2048 2024-05-08T06:12:37.184 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4197376 len 2048 2024-05-08T06:12:37.184 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4197376 len 2048 2024-05-08T06:12:37.184 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4197376 len 2048 2024-05-08T06:12:37.184 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4197376 len 4096 2024-05-08T06:12:37.185 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4197376 len 4096 2024-05-08T06:12:37.185 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4197376 len 4096 2024-05-08T06:12:37.185 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4197376 len 4096 2024-05-08T06:12:37.185 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4197376 len 8192 2024-05-08T06:12:37.185 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4197376 len 8192 2024-05-08T06:12:37.185 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4197376 len 8192 2024-05-08T06:12:37.185 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4197376 len 8192 2024-05-08T06:12:37.185 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4197376 len 16384 2024-05-08T06:12:37.185 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4197376 len 16384 2024-05-08T06:12:37.185 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4197376 len 16384 2024-05-08T06:12:37.185 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4197376 len 16384 2024-05-08T06:12:37.186 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4197888 len 1024 2024-05-08T06:12:37.186 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4197888 len 1024 2024-05-08T06:12:37.186 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4197888 len 1024 2024-05-08T06:12:37.186 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4197888 len 1024 2024-05-08T06:12:37.186 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4197888 len 2048 2024-05-08T06:12:37.186 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4197888 len 2048 2024-05-08T06:12:37.186 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4197888 len 2048 2024-05-08T06:12:37.186 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4197888 len 2048 2024-05-08T06:12:37.186 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4197888 len 4096 2024-05-08T06:12:37.186 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4197888 len 4096 2024-05-08T06:12:37.186 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4197888 len 4096 2024-05-08T06:12:37.187 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4197888 len 4096 2024-05-08T06:12:37.187 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4197888 len 8192 2024-05-08T06:12:37.187 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4197888 len 8192 2024-05-08T06:12:37.187 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4197888 len 8192 2024-05-08T06:12:37.187 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4197888 len 8192 2024-05-08T06:12:37.187 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 512 offset 4197888 len 16384 2024-05-08T06:12:37.187 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4197888 len 16384 2024-05-08T06:12:37.187 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 512 offset 4197888 len 16384 2024-05-08T06:12:37.187 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 512 offset 4197888 len 16384 2024-05-08T06:12:37.187 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4190720 len 1024 2024-05-08T06:12:37.187 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4190720 len 1024 2024-05-08T06:12:37.187 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4190720 len 1024 2024-05-08T06:12:37.187 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4190720 len 1024 2024-05-08T06:12:37.187 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4190720 len 2048 2024-05-08T06:12:37.187 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4190720 len 2048 2024-05-08T06:12:37.187 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4190720 len 2048 2024-05-08T06:12:37.187 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4190720 len 2048 2024-05-08T06:12:37.188 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4190720 len 4096 2024-05-08T06:12:37.743 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-08T06:12:38.240 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:12:38.843 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4190720 len 4096 2024-05-08T06:12:38.843 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4190720 len 4096 2024-05-08T06:12:38.843 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4190720 len 4096 2024-05-08T06:12:38.844 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4190720 len 8192 2024-05-08T06:12:38.844 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4190720 len 8192 2024-05-08T06:12:38.844 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4190720 len 8192 2024-05-08T06:12:38.844 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4190720 len 8192 2024-05-08T06:12:38.844 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4190720 len 16384 2024-05-08T06:12:38.844 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4190720 len 16384 2024-05-08T06:12:38.844 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4190720 len 16384 2024-05-08T06:12:38.844 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4190720 len 16384 2024-05-08T06:12:38.844 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4191232 len 1024 2024-05-08T06:12:38.844 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4191232 len 1024 2024-05-08T06:12:38.844 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4191232 len 1024 2024-05-08T06:12:38.844 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4191232 len 1024 2024-05-08T06:12:38.844 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4191232 len 2048 2024-05-08T06:12:38.845 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4191232 len 2048 2024-05-08T06:12:38.845 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4191232 len 2048 2024-05-08T06:12:38.845 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4191232 len 2048 2024-05-08T06:12:38.845 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4191232 len 4096 2024-05-08T06:12:38.845 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4191232 len 4096 2024-05-08T06:12:38.845 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4191232 len 4096 2024-05-08T06:12:38.845 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4191232 len 4096 2024-05-08T06:12:38.845 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4191232 len 8192 2024-05-08T06:12:38.845 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4191232 len 8192 2024-05-08T06:12:38.845 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4191232 len 8192 2024-05-08T06:12:38.845 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4191232 len 8192 2024-05-08T06:12:38.845 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4191232 len 16384 2024-05-08T06:12:38.845 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4191232 len 16384 2024-05-08T06:12:38.846 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4191232 len 16384 2024-05-08T06:12:38.846 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4191232 len 16384 2024-05-08T06:12:38.846 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4191744 len 1024 2024-05-08T06:12:38.846 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4191744 len 1024 2024-05-08T06:12:38.846 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4191744 len 1024 2024-05-08T06:12:38.846 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4191744 len 1024 2024-05-08T06:12:38.846 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4191744 len 2048 2024-05-08T06:12:38.846 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4191744 len 2048 2024-05-08T06:12:38.846 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4191744 len 2048 2024-05-08T06:12:38.846 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4191744 len 2048 2024-05-08T06:12:38.847 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4191744 len 4096 2024-05-08T06:12:38.847 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4191744 len 4096 2024-05-08T06:12:38.847 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4191744 len 4096 2024-05-08T06:12:38.847 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4191744 len 4096 2024-05-08T06:12:38.847 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4191744 len 8192 2024-05-08T06:12:38.847 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4191744 len 8192 2024-05-08T06:12:38.847 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4191744 len 8192 2024-05-08T06:12:38.847 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4191744 len 8192 2024-05-08T06:12:38.847 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4191744 len 16384 2024-05-08T06:12:38.847 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4191744 len 16384 2024-05-08T06:12:38.847 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4191744 len 16384 2024-05-08T06:12:38.847 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4191744 len 16384 2024-05-08T06:12:38.847 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4192256 len 1024 2024-05-08T06:12:38.847 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4192256 len 1024 2024-05-08T06:12:38.848 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4192256 len 1024 2024-05-08T06:12:38.848 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4192256 len 1024 2024-05-08T06:12:38.848 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4192256 len 2048 2024-05-08T06:12:38.848 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4192256 len 2048 2024-05-08T06:12:38.848 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4192256 len 2048 2024-05-08T06:12:38.848 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4192256 len 2048 2024-05-08T06:12:38.848 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4192256 len 4096 2024-05-08T06:12:38.848 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4192256 len 4096 2024-05-08T06:12:38.848 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4192256 len 4096 2024-05-08T06:12:38.848 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4192256 len 4096 2024-05-08T06:12:38.848 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4192256 len 8192 2024-05-08T06:12:38.849 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4192256 len 8192 2024-05-08T06:12:38.849 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4192256 len 8192 2024-05-08T06:12:38.849 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4192256 len 8192 2024-05-08T06:12:38.849 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4192256 len 16384 2024-05-08T06:12:38.849 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4192256 len 16384 2024-05-08T06:12:38.849 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4192256 len 16384 2024-05-08T06:12:38.849 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4192256 len 16384 2024-05-08T06:12:38.849 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4192768 len 1024 2024-05-08T06:12:38.849 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4192768 len 1024 2024-05-08T06:12:38.849 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4192768 len 1024 2024-05-08T06:12:38.849 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4192768 len 1024 2024-05-08T06:12:38.849 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4192768 len 2048 2024-05-08T06:12:38.850 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4192768 len 2048 2024-05-08T06:12:38.850 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4192768 len 2048 2024-05-08T06:12:38.850 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4192768 len 2048 2024-05-08T06:12:38.850 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4192768 len 4096 2024-05-08T06:12:38.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:38 smithi123 ceph-mon[29787]: pgmap v525: 97 pgs: 97 active+clean; 187 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 35 MiB/s rd, 920 KiB/s wr, 232 op/s 2024-05-08T06:12:39.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:38 smithi163 ceph-mon[34520]: pgmap v525: 97 pgs: 97 active+clean; 187 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 35 MiB/s rd, 920 KiB/s wr, 232 op/s 2024-05-08T06:12:39.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:38 smithi192 ceph-mon[34664]: pgmap v525: 97 pgs: 97 active+clean; 187 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 35 MiB/s rd, 920 KiB/s wr, 232 op/s 2024-05-08T06:12:40.240 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4192768 len 4096 2024-05-08T06:12:40.241 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4192768 len 4096 2024-05-08T06:12:40.241 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4192768 len 4096 2024-05-08T06:12:40.241 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4192768 len 8192 2024-05-08T06:12:40.241 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4192768 len 8192 2024-05-08T06:12:40.241 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4192768 len 8192 2024-05-08T06:12:40.242 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4192768 len 8192 2024-05-08T06:12:40.242 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4192768 len 16384 2024-05-08T06:12:40.242 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4192768 len 16384 2024-05-08T06:12:40.242 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4192768 len 16384 2024-05-08T06:12:40.242 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4192768 len 16384 2024-05-08T06:12:40.242 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4193280 len 1024 2024-05-08T06:12:40.243 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4193280 len 1024 2024-05-08T06:12:40.243 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4193280 len 1024 2024-05-08T06:12:40.243 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4193280 len 1024 2024-05-08T06:12:40.243 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4193280 len 2048 2024-05-08T06:12:40.243 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4193280 len 2048 2024-05-08T06:12:40.243 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4193280 len 2048 2024-05-08T06:12:40.244 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4193280 len 2048 2024-05-08T06:12:40.244 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4193280 len 4096 2024-05-08T06:12:40.244 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4193280 len 4096 2024-05-08T06:12:40.244 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4193280 len 4096 2024-05-08T06:12:40.244 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4193280 len 4096 2024-05-08T06:12:40.244 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4193280 len 8192 2024-05-08T06:12:40.245 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4193280 len 8192 2024-05-08T06:12:40.245 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4193280 len 8192 2024-05-08T06:12:40.245 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4193280 len 8192 2024-05-08T06:12:40.245 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4193280 len 16384 2024-05-08T06:12:40.245 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4193280 len 16384 2024-05-08T06:12:40.245 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4193280 len 16384 2024-05-08T06:12:40.246 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4193280 len 16384 2024-05-08T06:12:40.246 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4193792 len 1024 2024-05-08T06:12:40.246 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4193792 len 1024 2024-05-08T06:12:40.246 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4193792 len 1024 2024-05-08T06:12:40.246 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4193792 len 1024 2024-05-08T06:12:40.246 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4193792 len 2048 2024-05-08T06:12:40.247 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4193792 len 2048 2024-05-08T06:12:40.247 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4193792 len 2048 2024-05-08T06:12:40.247 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4193792 len 2048 2024-05-08T06:12:40.247 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4193792 len 4096 2024-05-08T06:12:40.247 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4193792 len 4096 2024-05-08T06:12:40.247 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4193792 len 4096 2024-05-08T06:12:40.248 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4193792 len 4096 2024-05-08T06:12:40.248 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4193792 len 8192 2024-05-08T06:12:40.248 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4193792 len 8192 2024-05-08T06:12:40.248 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4193792 len 8192 2024-05-08T06:12:40.248 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4193792 len 8192 2024-05-08T06:12:40.248 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4193792 len 16384 2024-05-08T06:12:40.249 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4193792 len 16384 2024-05-08T06:12:40.249 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4193792 len 16384 2024-05-08T06:12:40.249 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4193792 len 16384 2024-05-08T06:12:40.249 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4194304 len 1024 2024-05-08T06:12:40.249 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4194304 len 1024 2024-05-08T06:12:40.249 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4194304 len 1024 2024-05-08T06:12:40.249 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4194304 len 1024 2024-05-08T06:12:40.250 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4194304 len 2048 2024-05-08T06:12:40.250 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4194304 len 2048 2024-05-08T06:12:40.250 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4194304 len 2048 2024-05-08T06:12:40.250 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4194304 len 2048 2024-05-08T06:12:40.250 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4194304 len 4096 2024-05-08T06:12:40.251 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4194304 len 4096 2024-05-08T06:12:40.251 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4194304 len 4096 2024-05-08T06:12:40.251 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4194304 len 4096 2024-05-08T06:12:40.251 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4194304 len 8192 2024-05-08T06:12:40.251 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4194304 len 8192 2024-05-08T06:12:40.251 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4194304 len 8192 2024-05-08T06:12:40.251 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4194304 len 8192 2024-05-08T06:12:40.252 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4194304 len 16384 2024-05-08T06:12:40.252 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4194304 len 16384 2024-05-08T06:12:40.252 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4194304 len 16384 2024-05-08T06:12:40.252 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4194304 len 16384 2024-05-08T06:12:40.252 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4194816 len 1024 2024-05-08T06:12:40.252 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4194816 len 1024 2024-05-08T06:12:40.253 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4194816 len 1024 2024-05-08T06:12:40.253 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4194816 len 1024 2024-05-08T06:12:40.253 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4194816 len 2048 2024-05-08T06:12:40.253 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4194816 len 2048 2024-05-08T06:12:40.253 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4194816 len 2048 2024-05-08T06:12:40.253 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4194816 len 2048 2024-05-08T06:12:40.254 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4194816 len 4096 2024-05-08T06:12:40.254 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4194816 len 4096 2024-05-08T06:12:40.778 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4194816 len 4096 2024-05-08T06:12:40.778 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4194816 len 4096 2024-05-08T06:12:40.778 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4194816 len 8192 2024-05-08T06:12:40.778 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4194816 len 8192 2024-05-08T06:12:40.779 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4194816 len 8192 2024-05-08T06:12:40.779 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4194816 len 8192 2024-05-08T06:12:40.779 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4194816 len 16384 2024-05-08T06:12:40.779 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4194816 len 16384 2024-05-08T06:12:40.779 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4194816 len 16384 2024-05-08T06:12:40.779 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4194816 len 16384 2024-05-08T06:12:40.779 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4195328 len 1024 2024-05-08T06:12:40.779 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4195328 len 1024 2024-05-08T06:12:40.779 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4195328 len 1024 2024-05-08T06:12:40.779 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4195328 len 1024 2024-05-08T06:12:40.779 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4195328 len 2048 2024-05-08T06:12:40.779 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4195328 len 2048 2024-05-08T06:12:40.780 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4195328 len 2048 2024-05-08T06:12:40.780 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4195328 len 2048 2024-05-08T06:12:40.780 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4195328 len 4096 2024-05-08T06:12:40.780 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4195328 len 4096 2024-05-08T06:12:40.780 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4195328 len 4096 2024-05-08T06:12:40.780 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4195328 len 4096 2024-05-08T06:12:40.780 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4195328 len 8192 2024-05-08T06:12:40.780 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4195328 len 8192 2024-05-08T06:12:40.780 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4195328 len 8192 2024-05-08T06:12:40.780 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4195328 len 8192 2024-05-08T06:12:40.780 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4195328 len 16384 2024-05-08T06:12:40.780 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4195328 len 16384 2024-05-08T06:12:40.780 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4195328 len 16384 2024-05-08T06:12:40.780 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4195328 len 16384 2024-05-08T06:12:40.781 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4195840 len 1024 2024-05-08T06:12:40.781 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4195840 len 1024 2024-05-08T06:12:40.781 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4195840 len 1024 2024-05-08T06:12:40.781 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4195840 len 1024 2024-05-08T06:12:40.781 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4195840 len 2048 2024-05-08T06:12:40.781 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4195840 len 2048 2024-05-08T06:12:40.781 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4195840 len 2048 2024-05-08T06:12:40.781 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4195840 len 2048 2024-05-08T06:12:40.781 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4195840 len 4096 2024-05-08T06:12:40.781 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4195840 len 4096 2024-05-08T06:12:40.781 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4195840 len 4096 2024-05-08T06:12:40.781 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4195840 len 4096 2024-05-08T06:12:40.781 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4195840 len 8192 2024-05-08T06:12:40.782 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4195840 len 8192 2024-05-08T06:12:40.782 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4195840 len 8192 2024-05-08T06:12:40.782 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4195840 len 8192 2024-05-08T06:12:40.782 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4195840 len 16384 2024-05-08T06:12:40.782 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4195840 len 16384 2024-05-08T06:12:40.782 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4195840 len 16384 2024-05-08T06:12:40.782 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4195840 len 16384 2024-05-08T06:12:40.782 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4196352 len 1024 2024-05-08T06:12:40.782 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4196352 len 1024 2024-05-08T06:12:40.782 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4196352 len 1024 2024-05-08T06:12:40.782 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4196352 len 1024 2024-05-08T06:12:40.782 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4196352 len 2048 2024-05-08T06:12:40.783 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4196352 len 2048 2024-05-08T06:12:40.783 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4196352 len 2048 2024-05-08T06:12:40.783 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4196352 len 2048 2024-05-08T06:12:40.783 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4196352 len 4096 2024-05-08T06:12:40.783 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4196352 len 4096 2024-05-08T06:12:40.783 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4196352 len 4096 2024-05-08T06:12:40.783 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4196352 len 4096 2024-05-08T06:12:40.783 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4196352 len 8192 2024-05-08T06:12:40.783 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4196352 len 8192 2024-05-08T06:12:40.783 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4196352 len 8192 2024-05-08T06:12:40.783 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4196352 len 8192 2024-05-08T06:12:40.783 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4196352 len 16384 2024-05-08T06:12:40.783 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4196352 len 16384 2024-05-08T06:12:40.783 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4196352 len 16384 2024-05-08T06:12:40.784 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4196352 len 16384 2024-05-08T06:12:40.784 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4196864 len 1024 2024-05-08T06:12:40.784 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4196864 len 1024 2024-05-08T06:12:40.784 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4196864 len 1024 2024-05-08T06:12:40.784 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4196864 len 1024 2024-05-08T06:12:40.784 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4196864 len 2048 2024-05-08T06:12:40.784 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4196864 len 2048 2024-05-08T06:12:40.784 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4196864 len 2048 2024-05-08T06:12:40.784 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4196864 len 2048 2024-05-08T06:12:40.784 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4196864 len 4096 2024-05-08T06:12:40.784 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4196864 len 4096 2024-05-08T06:12:40.784 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4196864 len 4096 2024-05-08T06:12:40.850 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:40 smithi123 ceph-mon[29787]: pgmap v526: 97 pgs: 97 active+clean; 140 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s rd, 1.1 MiB/s wr, 290 op/s 2024-05-08T06:12:40.918 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.918+0000 7f61c2a00700 1 -- 172.21.15.123:0/460109362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61bc10b660 msgr2=0x7f61bc10ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:40.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.918+0000 7f61c2a00700 1 --2- 172.21.15.123:0/460109362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61bc10b660 0x7f61bc10ba40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f61b0009990 tx=0x7f61b0009ca0 comp rx=0 tx=0).stop 2024-05-08T06:12:40.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.918+0000 7f61c2a00700 1 -- 172.21.15.123:0/460109362 shutdown_connections 2024-05-08T06:12:40.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.918+0000 7f61c2a00700 1 --2- 172.21.15.123:0/460109362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61bc105f90 0x7f61bc1124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:40.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.918+0000 7f61c2a00700 1 --2- 172.21.15.123:0/460109362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61bc10b660 0x7f61bc10ba40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:40.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.918+0000 7f61c2a00700 1 --2- 172.21.15.123:0/460109362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61bc1055d0 0x7f61bc105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:40.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.918+0000 7f61c2a00700 1 -- 172.21.15.123:0/460109362 >> 172.21.15.123:0/460109362 conn(0x7f61bc1010c0 msgr2=0x7f61bc1034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:40.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.918+0000 7f61c2a00700 1 -- 172.21.15.123:0/460109362 shutdown_connections 2024-05-08T06:12:40.920 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.918+0000 7f61c2a00700 1 -- 172.21.15.123:0/460109362 wait complete. 2024-05-08T06:12:40.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.919+0000 7f61c2a00700 1 Processor -- start 2024-05-08T06:12:40.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.919+0000 7f61c2a00700 1 -- start start 2024-05-08T06:12:40.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.920+0000 7f61c2a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61bc1055d0 0x7f61bc1a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:40.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.920+0000 7f61c2a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61bc105f90 0x7f61bc1a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:40.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.920+0000 7f61c2a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61bc10b660 0x7f61bc19ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:40.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.920+0000 7f61c2a00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc115800 con 0x7f61bc1055d0 2024-05-08T06:12:40.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.920+0000 7f61c2a00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc115680 con 0x7f61bc10b660 2024-05-08T06:12:40.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.920+0000 7f61c2a00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc115980 con 0x7f61bc105f90 2024-05-08T06:12:40.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.920+0000 7f61c0c00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61bc105f90 0x7f61bc1a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:40.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.920+0000 7f61c1600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61bc1055d0 0x7f61bc1a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:40.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.920+0000 7f61c0c00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61bc105f90 0x7f61bc1a1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:47710/0 (socket says 172.21.15.123:47710) 2024-05-08T06:12:40.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.920+0000 7f61c0c00700 1 -- 172.21.15.123:0/970974449 learned_addr learned my addr 172.21.15.123:0/970974449 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:40.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.920+0000 7f61c2000700 1 --2- 172.21.15.123:0/970974449 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61bc10b660 0x7f61bc19ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:40.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.920+0000 7f61c0c00700 1 -- 172.21.15.123:0/970974449 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61bc10b660 msgr2=0x7f61bc19ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:40.925 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.920+0000 7f61c0c00700 1 --2- 172.21.15.123:0/970974449 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61bc10b660 0x7f61bc19ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:40.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.920+0000 7f61c0c00700 1 -- 172.21.15.123:0/970974449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61bc1055d0 msgr2=0x7f61bc1a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:40.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.920+0000 7f61c0c00700 1 --2- 172.21.15.123:0/970974449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61bc1055d0 0x7f61bc1a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:40.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.920+0000 7f61c0c00700 1 -- 172.21.15.123:0/970974449 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61bc19c1c0 con 0x7f61bc105f90 2024-05-08T06:12:40.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.921+0000 7f61c0c00700 1 --2- 172.21.15.123:0/970974449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61bc105f90 0x7f61bc1a1de0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f61a800e9a0 tx=0x7f61a800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:40.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.921+0000 7f61ba000700 1 -- 172.21.15.123:0/970974449 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a800ca10 con 0x7f61bc105f90 2024-05-08T06:12:40.926 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.921+0000 7f61ba000700 1 -- 172.21.15.123:0/970974449 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61a800cbb0 con 0x7f61bc105f90 2024-05-08T06:12:40.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.921+0000 7f61c2a00700 1 -- 172.21.15.123:0/970974449 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61bc19c4b0 con 0x7f61bc105f90 2024-05-08T06:12:40.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.921+0000 7f61ba000700 1 -- 172.21.15.123:0/970974449 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a8018680 con 0x7f61bc105f90 2024-05-08T06:12:40.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.921+0000 7f61c2a00700 1 -- 172.21.15.123:0/970974449 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61bc1a8770 con 0x7f61bc105f90 2024-05-08T06:12:40.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.925+0000 7f61ba000700 1 -- 172.21.15.123:0/970974449 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7f61a8010430 con 0x7f61bc105f90 2024-05-08T06:12:40.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.926+0000 7f61ce9c5b80 1 Processor -- start 2024-05-08T06:12:40.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.926+0000 7f61ce9c5b80 1 -- start start 2024-05-08T06:12:40.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.927+0000 7f61ce9c5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615ddb4d9e0 0x5615ddb4ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:40.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.927+0000 7f61ce9c5b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5615ddb4e300 0x5615ddb47fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:40.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.927+0000 7f61ce9c5b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615ddb48600 0x5615ddb48a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:40.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.927+0000 7f61ce9c5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5615dda9aa20 con 0x5615ddb4d9e0 2024-05-08T06:12:40.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.927+0000 7f61ce9c5b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5615dda38760 con 0x5615ddb4e300 2024-05-08T06:12:40.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.927+0000 7f61ce9c5b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5615dda969e0 con 0x5615ddb48600 2024-05-08T06:12:40.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.927+0000 7f61c1600700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5615ddb4e300 0x5615ddb47fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:40.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.927+0000 7f61c1600700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5615ddb4e300 0x5615ddb47fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:40084/0 (socket says 172.21.15.123:40084) 2024-05-08T06:12:40.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.927+0000 7f61c1600700 1 -- 172.21.15.123:0/2160105159 learned_addr learned my addr 172.21.15.123:0/2160105159 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:40.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.927+0000 7f61c0c00700 1 --2- 172.21.15.123:0/2160105159 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615ddb48600 0x5615ddb48a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:40.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.927+0000 7f61c2000700 1 --2- 172.21.15.123:0/2160105159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615ddb4d9e0 0x5615ddb4ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:40.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.927+0000 7f61ba000700 1 --2- 172.21.15.123:0/970974449 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f61a40b2880 0x7f61a40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:40.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.927+0000 7f61c1600700 1 -- 172.21.15.123:0/2160105159 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615ddb48600 msgr2=0x5615ddb48a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:40.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.927+0000 7f61c1600700 1 --2- 172.21.15.123:0/2160105159 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615ddb48600 0x5615ddb48a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:40.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.927+0000 7f61c1600700 1 -- 172.21.15.123:0/2160105159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615ddb4d9e0 msgr2=0x5615ddb4ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:40.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.927+0000 7f61c1600700 1 --2- 172.21.15.123:0/2160105159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615ddb4d9e0 0x5615ddb4ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:40.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.927+0000 7f61c1600700 1 -- 172.21.15.123:0/2160105159 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615dd9e78d0 con 0x5615ddb4e300 2024-05-08T06:12:40.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.928+0000 7f61ba000700 1 -- 172.21.15.123:0/970974449 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f61a80e23b0 con 0x7f61bc105f90 2024-05-08T06:12:40.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.928+0000 7f61c1600700 1 --2- 172.21.15.123:0/2160105159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5615ddb4e300 0x5615ddb47fe0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f61bc106710 tx=0x7f61b0005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e79f77938e723b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:40.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.928+0000 7f61c2000700 1 --2- 172.21.15.123:0/970974449 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f61a40b2880 0x7f61a40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:40.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.928+0000 7f61aea00700 1 -- 172.21.15.123:0/2160105159 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b000b720 con 0x5615ddb4e300 2024-05-08T06:12:40.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.928+0000 7f61aea00700 1 -- 172.21.15.123:0/2160105159 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61b0027070 con 0x5615ddb4e300 2024-05-08T06:12:40.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.928+0000 7f61aea00700 1 -- 172.21.15.123:0/2160105159 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b0003a20 con 0x5615ddb4e300 2024-05-08T06:12:40.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.929+0000 7f61c2000700 1 --2- 172.21.15.123:0/970974449 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f61a40b2880 0x7f61a40b4d40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f61b4009ec0 tx=0x7f61b4009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:40.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.929+0000 7f61ce9c5b80 1 -- 172.21.15.123:0/2160105159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5615ddb4e300 msgr2=0x5615ddb47fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:40.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.929+0000 7f61ce9c5b80 1 --2- 172.21.15.123:0/2160105159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5615ddb4e300 0x5615ddb47fe0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f61bc106710 tx=0x7f61b0005fd0 comp rx=0 tx=0).stop 2024-05-08T06:12:40.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.929+0000 7f61ce9c5b80 1 -- 172.21.15.123:0/2160105159 shutdown_connections 2024-05-08T06:12:40.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.929+0000 7f61ce9c5b80 1 --2- 172.21.15.123:0/2160105159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615ddb4d9e0 0x5615ddb4ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:40.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.929+0000 7f61ce9c5b80 1 --2- 172.21.15.123:0/2160105159 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615ddb48600 0x5615ddb48a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:40.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.929+0000 7f61ce9c5b80 1 --2- 172.21.15.123:0/2160105159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5615ddb4e300 0x5615ddb47fe0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:40.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.929+0000 7f61ce9c5b80 1 -- 172.21.15.123:0/2160105159 >> 172.21.15.123:0/2160105159 conn(0x5615dda76b60 msgr2=0x5615ddb429b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:40.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.930+0000 7f61ce9c5b80 1 -- 172.21.15.123:0/2160105159 shutdown_connections 2024-05-08T06:12:40.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.930+0000 7f61ce9c5b80 1 -- 172.21.15.123:0/2160105159 wait complete. 2024-05-08T06:12:40.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.930+0000 7f61ce9c5b80 1 -- start start 2024-05-08T06:12:40.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.931+0000 7f61ce9c5b80 1 Processor -- start 2024-05-08T06:12:40.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.931+0000 7f61ce9c5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615ddb48600 0x5615ddb53ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:40.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.931+0000 7f61ce9c5b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615ddb4d9e0 0x5615ddb51270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:40.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.931+0000 7f61ce9c5b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5615ddb4e300 0x5615ddb517b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:40.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.931+0000 7f61ce9c5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5615ddb54d50 con 0x5615ddb48600 2024-05-08T06:12:40.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.931+0000 7f61ce9c5b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5615dda00b20 con 0x5615ddb4e300 2024-05-08T06:12:40.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.931+0000 7f61ce9c5b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5615ddb54ed0 con 0x5615ddb4d9e0 2024-05-08T06:12:40.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.931+0000 7f61ce9c5b80 10 client.0 _ll_register_callbacks cb 0x5615dda41c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:12:40.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.932+0000 7f61c0c00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615ddb4d9e0 0x5615ddb51270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:40.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.932+0000 7f61c2000700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5615ddb4e300 0x5615ddb517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:40.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.932+0000 7f61c0c00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615ddb4d9e0 0x5615ddb51270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:47718/0 (socket says 172.21.15.123:47718) 2024-05-08T06:12:40.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.932+0000 7f61c0c00700 1 -- 172.21.15.123:0/2330653830 learned_addr learned my addr 172.21.15.123:0/2330653830 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:40.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.932+0000 7f61c1600700 1 --2- 172.21.15.123:0/2330653830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615ddb48600 0x5615ddb53ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:40.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.932+0000 7f61c0c00700 1 -- 172.21.15.123:0/2330653830 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5615ddb4e300 msgr2=0x5615ddb517b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:40.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.932+0000 7f61c0c00700 1 --2- 172.21.15.123:0/2330653830 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5615ddb4e300 0x5615ddb517b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:40.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.932+0000 7f61c0c00700 1 -- 172.21.15.123:0/2330653830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615ddb48600 msgr2=0x5615ddb53ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:40.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.932+0000 7f61c0c00700 1 --2- 172.21.15.123:0/2330653830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615ddb48600 0x5615ddb53ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:40.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.932+0000 7f61c0c00700 1 -- 172.21.15.123:0/2330653830 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615dda60420 con 0x5615ddb4d9e0 2024-05-08T06:12:40.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.932+0000 7f61c0c00700 1 --2- 172.21.15.123:0/2330653830 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615ddb4d9e0 0x5615ddb51270 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f61a800e970 tx=0x7f61a80e5b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b8cbfa5bcc31ef6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:40.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.932+0000 7f61a3400700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-05-08T06:12:40.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.933+0000 7f61a3400700 1 -- 172.21.15.123:0/2330653830 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a80a92d0 con 0x5615ddb4d9e0 2024-05-08T06:12:40.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.933+0000 7f61a3400700 1 -- 172.21.15.123:0/2330653830 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f61a80b2550 con 0x5615ddb4d9e0 2024-05-08T06:12:40.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.933+0000 7f61ce9c5b80 1 -- 172.21.15.123:0/2330653830 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61b0027070 con 0x5615ddb4d9e0 2024-05-08T06:12:40.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.933+0000 7f61a3400700 1 -- 172.21.15.123:0/2330653830 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f61a80a5110 con 0x5615ddb4d9e0 2024-05-08T06:12:40.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.933+0000 7f61ce9c5b80 10 client.25591 fetch_fsmap learned FSMap version 24 2024-05-08T06:12:40.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.933+0000 7f61ce9c5b80 1 -- 172.21.15.123:0/2330653830 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5615ddb5f450 con 0x5615ddb4d9e0 2024-05-08T06:12:40.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.933+0000 7f61a3400700 1 -- 172.21.15.123:0/2330653830 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61b0027070 con 0x5615ddb4d9e0 2024-05-08T06:12:40.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.933+0000 7f61a3400700 1 -- 172.21.15.123:0/2330653830 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f61a80a8be0 con 0x5615ddb4d9e0 2024-05-08T06:12:40.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.933+0000 7f61ce9c5b80 10 client.25591 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:12:40.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.933+0000 7f61ce9c5b80 10 client.25591 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T06:12:40.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.933+0000 7f61ce9c5b80 20 client.25591 populate_metadata read hostname 'smithi123' 2024-05-08T06:12:40.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.933+0000 7f61ce9c5b80 1 --2- 172.21.15.123:0/2330653830 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x5615ddb61a90 0x5615ddb63e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:40.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.933+0000 7f61ce9c5b80 4 client.25591 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:12:40.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.933+0000 7f61ce9c5b80 1 -- 172.21.15.123:0/2330653830 --> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615ddb150a0 con 0x5615ddb61a90 2024-05-08T06:12:40.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.934+0000 7f61c1600700 1 --2- 172.21.15.123:0/2330653830 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x5615ddb61a90 0x5615ddb63e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:40.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.934+0000 7f61c1600700 1 --2- 172.21.15.123:0/2330653830 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x5615ddb61a90 0x5615ddb63e80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=89c4a6ef74472897 server_cookie=ae38f082ff8b4ec4 in_seq=0 out_seq=0 2024-05-08T06:12:40.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.934+0000 7f61a3400700 10 client.25591 ms_handle_connect on v2:172.21.15.192:6838/3598682360 2024-05-08T06:12:40.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.935+0000 7f61a3400700 1 -- 172.21.15.123:0/2330653830 <== mds.2 v2:172.21.15.192:6838/3598682360 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5615ddb150a0 con 0x5615ddb61a90 2024-05-08T06:12:40.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.935+0000 7f61a3400700 10 client.25591 handle_command_reply: tid=0 2024-05-08T06:12:40.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.936+0000 7f61ce9c5b80 1 client.25591 shutdown 2024-05-08T06:12:40.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.936+0000 7f61ce9c5b80 1 -- 172.21.15.123:0/2330653830 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615ddb4d9e0 msgr2=0x5615ddb51270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:40.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.936+0000 7f61ce9c5b80 1 --2- 172.21.15.123:0/2330653830 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615ddb4d9e0 0x5615ddb51270 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f61a800e970 tx=0x7f61a80e5b50 comp rx=0 tx=0).stop 2024-05-08T06:12:40.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.936+0000 7f61ce9c5b80 1 -- 172.21.15.123:0/2330653830 shutdown_connections 2024-05-08T06:12:40.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.936+0000 7f61ce9c5b80 1 --2- 172.21.15.123:0/2330653830 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x5615ddb61a90 0x5615ddb63e80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:40.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.936+0000 7f61ce9c5b80 1 --2- 172.21.15.123:0/2330653830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615ddb48600 0x5615ddb53ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:40.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.936+0000 7f61ce9c5b80 1 --2- 172.21.15.123:0/2330653830 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5615ddb4e300 0x5615ddb517b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:40.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.936+0000 7f61ce9c5b80 1 --2- 172.21.15.123:0/2330653830 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615ddb4d9e0 0x5615ddb51270 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:40.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.936+0000 7f61a3400700 0 client.25591 ms_handle_reset on v2:172.21.15.192:6838/3598682360 2024-05-08T06:12:40.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.936+0000 7f61ce9c5b80 1 -- 172.21.15.123:0/2330653830 >> 172.21.15.123:0/2330653830 conn(0x5615dda76b60 msgr2=0x5615ddb3d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:40.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.936+0000 7f61ce9c5b80 1 -- 172.21.15.123:0/2330653830 shutdown_connections 2024-05-08T06:12:40.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.936+0000 7f61ce9c5b80 1 -- 172.21.15.123:0/2330653830 wait complete. 2024-05-08T06:12:40.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.936+0000 7f61ce9c5b80 20 client.25591 trim_cache size 0 max 16384 2024-05-08T06:12:40.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.955+0000 7f61ce9c5b80 1 Processor -- start 2024-05-08T06:12:40.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.955+0000 7f61ce9c5b80 1 -- start start 2024-05-08T06:12:40.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.955+0000 7f61ce9c5b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615ddb48600 0x5615ddb47fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:40.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.955+0000 7f61ce9c5b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5615ddb4d9e0 0x5615ddb4fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:40.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.955+0000 7f61ce9c5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615ddb4e300 0x5615ddb500a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:40.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.955+0000 7f61ce9c5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc1ae0b0 con 0x5615ddb4e300 2024-05-08T06:12:40.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.955+0000 7f61ce9c5b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc1ab940 con 0x5615ddb4d9e0 2024-05-08T06:12:40.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.955+0000 7f61ce9c5b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc07b800 con 0x5615ddb48600 2024-05-08T06:12:40.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.955+0000 7f61c1600700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615ddb48600 0x5615ddb47fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:40.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.955+0000 7f61c0c00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5615ddb4d9e0 0x5615ddb4fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:40.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.956+0000 7f61c2000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615ddb4e300 0x5615ddb500a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:40.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.956+0000 7f61c1600700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615ddb48600 0x5615ddb47fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:47724/0 (socket says 172.21.15.123:47724) 2024-05-08T06:12:40.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.956+0000 7f61c1600700 1 -- 172.21.15.123:0/3882990527 learned_addr learned my addr 172.21.15.123:0/3882990527 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:40.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.956+0000 7f61c1600700 1 -- 172.21.15.123:0/3882990527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5615ddb4d9e0 msgr2=0x5615ddb4fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:40.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.956+0000 7f61c1600700 1 --2- 172.21.15.123:0/3882990527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5615ddb4d9e0 0x5615ddb4fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:40.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.956+0000 7f61c1600700 1 -- 172.21.15.123:0/3882990527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615ddb4e300 msgr2=0x5615ddb500a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:40.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.956+0000 7f61c1600700 1 --2- 172.21.15.123:0/3882990527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615ddb4e300 0x5615ddb500a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:40.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.956+0000 7f61c1600700 1 -- 172.21.15.123:0/3882990527 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615ddb644e0 con 0x5615ddb48600 2024-05-08T06:12:40.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.956+0000 7f61c0c00700 1 --2- 172.21.15.123:0/3882990527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5615ddb4d9e0 0x5615ddb4fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:12:40.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.956+0000 7f61c1600700 1 --2- 172.21.15.123:0/3882990527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615ddb48600 0x5615ddb47fe0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f61b0009ce0 tx=0x7f61b00058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73a9c10bccdcb05b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:40.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.956+0000 7f61ae000700 1 -- 172.21.15.123:0/3882990527 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b000bbb0 con 0x5615ddb48600 2024-05-08T06:12:40.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.957+0000 7f61ae000700 1 -- 172.21.15.123:0/3882990527 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61b0027040 con 0x5615ddb48600 2024-05-08T06:12:40.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.957+0000 7f61ae000700 1 -- 172.21.15.123:0/3882990527 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b00345d0 con 0x5615ddb48600 2024-05-08T06:12:40.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.957+0000 7f61ce9c5b80 1 -- 172.21.15.123:0/3882990527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615ddb48600 msgr2=0x5615ddb47fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:40.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.957+0000 7f61ce9c5b80 1 --2- 172.21.15.123:0/3882990527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615ddb48600 0x5615ddb47fe0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f61b0009ce0 tx=0x7f61b00058e0 comp rx=0 tx=0).stop 2024-05-08T06:12:40.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.957+0000 7f61ce9c5b80 1 -- 172.21.15.123:0/3882990527 shutdown_connections 2024-05-08T06:12:40.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.957+0000 7f61ce9c5b80 1 --2- 172.21.15.123:0/3882990527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615ddb4e300 0x5615ddb500a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:40.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.957+0000 7f61ce9c5b80 1 --2- 172.21.15.123:0/3882990527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615ddb48600 0x5615ddb47fe0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:40.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.957+0000 7f61ce9c5b80 1 --2- 172.21.15.123:0/3882990527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5615ddb4d9e0 0x5615ddb4fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:40.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.957+0000 7f61ce9c5b80 1 -- 172.21.15.123:0/3882990527 >> 172.21.15.123:0/3882990527 conn(0x5615dda76b60 msgr2=0x5615ddb51230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:40.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.958+0000 7f61ce9c5b80 1 -- 172.21.15.123:0/3882990527 shutdown_connections 2024-05-08T06:12:40.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.958+0000 7f61ce9c5b80 1 -- 172.21.15.123:0/3882990527 wait complete. 2024-05-08T06:12:40.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.958+0000 7f61ce9c5b80 1 -- start start 2024-05-08T06:12:40.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.958+0000 7f61ce9c5b80 1 Processor -- start 2024-05-08T06:12:40.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.959+0000 7f61ce9c5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615ddb48600 0x5615ddb68fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:40.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.959+0000 7f61ce9c5b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615ddb4d9e0 0x5615ddb694f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:40.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.959+0000 7f61ce9c5b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5615ddb4e300 0x5615ddb69a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:40.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.959+0000 7f61ce9c5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc1a9e00 con 0x5615ddb48600 2024-05-08T06:12:40.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.959+0000 7f61ce9c5b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc1ab270 con 0x5615ddb4e300 2024-05-08T06:12:40.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.959+0000 7f61ce9c5b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc1aa4d0 con 0x5615ddb4d9e0 2024-05-08T06:12:40.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.959+0000 7f61ce9c5b80 10 client.0 _ll_register_callbacks cb 0x5615dda41c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:12:40.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.959+0000 7f61c0c00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615ddb4d9e0 0x5615ddb694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:40.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.959+0000 7f61c1600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615ddb48600 0x5615ddb68fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:40.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.959+0000 7f61c0c00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615ddb4d9e0 0x5615ddb694f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:47728/0 (socket says 172.21.15.123:47728) 2024-05-08T06:12:40.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.959+0000 7f61c2000700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5615ddb4e300 0x5615ddb69a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:40.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.959+0000 7f61c0c00700 1 -- 172.21.15.123:0/4176554217 learned_addr learned my addr 172.21.15.123:0/4176554217 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:40.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.960+0000 7f61c0c00700 1 -- 172.21.15.123:0/4176554217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5615ddb4e300 msgr2=0x5615ddb69a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:40.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.960+0000 7f61c0c00700 1 --2- 172.21.15.123:0/4176554217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5615ddb4e300 0x5615ddb69a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:40.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.960+0000 7f61c0c00700 1 -- 172.21.15.123:0/4176554217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615ddb48600 msgr2=0x5615ddb68fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:40.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.960+0000 7f61c0c00700 1 --2- 172.21.15.123:0/4176554217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615ddb48600 0x5615ddb68fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:40.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.960+0000 7f61c0c00700 1 -- 172.21.15.123:0/4176554217 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615ddbcb370 con 0x5615ddb4d9e0 2024-05-08T06:12:40.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.960+0000 7f61c0c00700 1 --2- 172.21.15.123:0/4176554217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615ddb4d9e0 0x5615ddb694f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f61a800c2d0 tx=0x7f61a80b2600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8e4e42b32ef53fc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:40.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.960+0000 7f61a3400700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-05-08T06:12:40.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.960+0000 7f61a3400700 1 -- 172.21.15.123:0/4176554217 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a80a90f0 con 0x5615ddb4d9e0 2024-05-08T06:12:40.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.960+0000 7f61a3400700 1 -- 172.21.15.123:0/4176554217 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f61a8010720 con 0x5615ddb4d9e0 2024-05-08T06:12:40.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.960+0000 7f61ce9c5b80 1 -- 172.21.15.123:0/4176554217 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61b0027040 con 0x5615ddb4d9e0 2024-05-08T06:12:40.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.960+0000 7f61a3400700 1 -- 172.21.15.123:0/4176554217 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f61a80a92d0 con 0x5615ddb4d9e0 2024-05-08T06:12:40.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.960+0000 7f61ce9c5b80 10 client.25597 fetch_fsmap learned FSMap version 24 2024-05-08T06:12:40.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.961+0000 7f61ce9c5b80 1 -- 172.21.15.123:0/4176554217 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5615ddbcc650 con 0x5615ddb4d9e0 2024-05-08T06:12:40.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.961+0000 7f61a3400700 1 -- 172.21.15.123:0/4176554217 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61b0027040 con 0x5615ddb4d9e0 2024-05-08T06:12:40.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.961+0000 7f61a3400700 1 -- 172.21.15.123:0/4176554217 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f61a80aeec0 con 0x5615ddb4d9e0 2024-05-08T06:12:40.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.961+0000 7f61ce9c5b80 10 client.25597 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:12:40.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.961+0000 7f61ce9c5b80 10 client.25597 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T06:12:40.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.961+0000 7f61ce9c5b80 20 client.25597 populate_metadata read hostname 'smithi123' 2024-05-08T06:12:40.964 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.961+0000 7f61ce9c5b80 1 --2- 172.21.15.123:0/4176554217 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x5615ddbccd00 0x5615ddbcf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:40.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.961+0000 7f61ce9c5b80 4 client.25597 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T06:12:40.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.961+0000 7f61ce9c5b80 1 -- 172.21.15.123:0/4176554217 --> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5615ddaa8400 con 0x5615ddbccd00 2024-05-08T06:12:40.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.961+0000 7f61c1600700 1 --2- 172.21.15.123:0/4176554217 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x5615ddbccd00 0x5615ddbcf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:40.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.962+0000 7f61c1600700 1 --2- 172.21.15.123:0/4176554217 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x5615ddbccd00 0x5615ddbcf120 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a0af80314156d32d server_cookie=4044b80dbe0da6a in_seq=0 out_seq=0 2024-05-08T06:12:40.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.962+0000 7f61a3400700 10 client.25597 ms_handle_connect on v2:172.21.15.192:6838/3598682360 2024-05-08T06:12:40.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.962+0000 7f61a3400700 1 -- 172.21.15.123:0/4176554217 <== mds.2 v2:172.21.15.192:6838/3598682360 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5615ddaa8400 con 0x5615ddbccd00 2024-05-08T06:12:40.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.962+0000 7f61a3400700 10 client.25597 handle_command_reply: tid=0 2024-05-08T06:12:40.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.962+0000 7f61ce9c5b80 1 client.25597 shutdown 2024-05-08T06:12:40.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.962+0000 7f61ce9c5b80 1 -- 172.21.15.123:0/4176554217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615ddb4d9e0 msgr2=0x5615ddb694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:40.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.962+0000 7f61ce9c5b80 1 --2- 172.21.15.123:0/4176554217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615ddb4d9e0 0x5615ddb694f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f61a800c2d0 tx=0x7f61a80b2600 comp rx=0 tx=0).stop 2024-05-08T06:12:40.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.962+0000 7f61ce9c5b80 1 -- 172.21.15.123:0/4176554217 shutdown_connections 2024-05-08T06:12:40.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.962+0000 7f61ce9c5b80 1 --2- 172.21.15.123:0/4176554217 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x5615ddbccd00 0x5615ddbcf120 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:40.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.962+0000 7f61ce9c5b80 1 --2- 172.21.15.123:0/4176554217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615ddb48600 0x5615ddb68fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:40.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.962+0000 7f61a3400700 0 client.25597 ms_handle_reset on v2:172.21.15.192:6838/3598682360 2024-05-08T06:12:40.965 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.962+0000 7f61ce9c5b80 1 --2- 172.21.15.123:0/4176554217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5615ddb4e300 0x5615ddb69a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:40.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.962+0000 7f61ce9c5b80 1 --2- 172.21.15.123:0/4176554217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615ddb4d9e0 0x5615ddb694f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:40.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.962+0000 7f61ce9c5b80 1 -- 172.21.15.123:0/4176554217 >> 172.21.15.123:0/4176554217 conn(0x5615dda76b60 msgr2=0x5615ddb4e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:40.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.963+0000 7f61ce9c5b80 1 -- 172.21.15.123:0/4176554217 shutdown_connections 2024-05-08T06:12:40.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.963+0000 7f61ce9c5b80 1 -- 172.21.15.123:0/4176554217 wait complete. 2024-05-08T06:12:40.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.963+0000 7f61ce9c5b80 20 client.25597 trim_cache size 0 max 16384 2024-05-08T06:12:40.966 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-05-08T06:12:40.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.963+0000 7f61c2a00700 1 -- 172.21.15.123:0/970974449 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f61a40b2880 msgr2=0x7f61a40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:40.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.963+0000 7f61c2a00700 1 --2- 172.21.15.123:0/970974449 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f61a40b2880 0x7f61a40b4d40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f61b4009ec0 tx=0x7f61b4009040 comp rx=0 tx=0).stop 2024-05-08T06:12:40.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.963+0000 7f61c2a00700 1 -- 172.21.15.123:0/970974449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61bc105f90 msgr2=0x7f61bc1a1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:40.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.963+0000 7f61c2a00700 1 --2- 172.21.15.123:0/970974449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61bc105f90 0x7f61bc1a1de0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f61a800e9a0 tx=0x7f61a800ecb0 comp rx=0 tx=0).stop 2024-05-08T06:12:40.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.964+0000 7f61c2a00700 1 -- 172.21.15.123:0/970974449 shutdown_connections 2024-05-08T06:12:40.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.964+0000 7f61c2a00700 1 --2- 172.21.15.123:0/970974449 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f61a40b2880 0x7f61a40b4d40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:40.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.964+0000 7f61c2a00700 1 --2- 172.21.15.123:0/970974449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61bc1055d0 0x7f61bc1a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:40.967 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.964+0000 7f61c2a00700 1 --2- 172.21.15.123:0/970974449 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61bc10b660 0x7f61bc19ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:40.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.964+0000 7f61c2a00700 1 --2- 172.21.15.123:0/970974449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61bc105f90 0x7f61bc1a1de0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:40.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.964+0000 7f61c2a00700 1 -- 172.21.15.123:0/970974449 >> 172.21.15.123:0/970974449 conn(0x7f61bc1010c0 msgr2=0x7f61bc102820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:40.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.964+0000 7f61c2a00700 1 -- 172.21.15.123:0/970974449 shutdown_connections 2024-05-08T06:12:40.968 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:40.964+0000 7f61c2a00700 1 -- 172.21.15.123:0/970974449 wait complete. 2024-05-08T06:12:41.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:40 smithi163 ceph-mon[34520]: pgmap v526: 97 pgs: 97 active+clean; 140 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s rd, 1.1 MiB/s wr, 290 op/s 2024-05-08T06:12:41.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:40 smithi192 ceph-mon[34664]: pgmap v526: 97 pgs: 97 active+clean; 140 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s rd, 1.1 MiB/s wr, 290 op/s 2024-05-08T06:12:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.730+0000 7fa363c00640 1 -- 172.21.15.123:0/2104095368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa35c10a8e0 msgr2=0x7fa35c10cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.730+0000 7fa363c00640 1 --2- 172.21.15.123:0/2104095368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa35c10a8e0 0x7fa35c10cd70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa354009ca0 tx=0x7fa35402a8c0 comp rx=0 tx=0).stop 2024-05-08T06:12:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.731+0000 7fa363c00640 1 -- 172.21.15.123:0/2104095368 shutdown_connections 2024-05-08T06:12:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.731+0000 7fa363c00640 1 --2- 172.21.15.123:0/2104095368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa35c10a8e0 0x7fa35c10cd70 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.731+0000 7fa363c00640 1 --2- 172.21.15.123:0/2104095368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa35c075410 0x7fa35c075890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.731+0000 7fa363c00640 1 --2- 172.21.15.123:0/2104095368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa35c0769f0 0x7fa35c074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.731+0000 7fa363c00640 1 -- 172.21.15.123:0/2104095368 >> 172.21.15.123:0/2104095368 conn(0x7fa35c0fdf70 msgr2=0x7fa35c1003d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.731+0000 7fa363c00640 1 -- 172.21.15.123:0/2104095368 shutdown_connections 2024-05-08T06:12:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.731+0000 7fa363c00640 1 -- 172.21.15.123:0/2104095368 wait complete. 2024-05-08T06:12:41.734 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.732+0000 7fa363c00640 1 Processor -- start 2024-05-08T06:12:41.734 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.732+0000 7fa363c00640 1 -- start start 2024-05-08T06:12:41.734 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.732+0000 7fa363c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa35c075410 0x7fa35c19ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:41.734 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.732+0000 7fa363c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa35c0769f0 0x7fa35c19b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.732+0000 7fa363c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa35c10a8e0 0x7fa35c1a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.732+0000 7fa363c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c110760 con 0x7fa35c0769f0 2024-05-08T06:12:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.732+0000 7fa363c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c1105e0 con 0x7fa35c10a8e0 2024-05-08T06:12:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.732+0000 7fa363c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c1108e0 con 0x7fa35c075410 2024-05-08T06:12:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.732+0000 7fa35be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa35c0769f0 0x7fa35c19b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.732+0000 7fa361200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa35c075410 0x7fa35c19ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.733+0000 7fa361200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa35c075410 0x7fa35c19ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:47748/0 (socket says 172.21.15.123:47748) 2024-05-08T06:12:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.733+0000 7fa361200640 1 -- 172.21.15.123:0/568005443 learned_addr learned my addr 172.21.15.123:0/568005443 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.733+0000 7fa35be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa35c0769f0 0x7fa35c19b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36278/0 (socket says 172.21.15.123:36278) 2024-05-08T06:12:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.733+0000 7fa361c00640 1 --2- 172.21.15.123:0/568005443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa35c10a8e0 0x7fa35c1a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.733+0000 7fa35be00640 1 -- 172.21.15.123:0/568005443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa35c075410 msgr2=0x7fa35c19ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.733+0000 7fa35be00640 1 --2- 172.21.15.123:0/568005443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa35c075410 0x7fa35c19ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.733+0000 7fa35be00640 1 -- 172.21.15.123:0/568005443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa35c10a8e0 msgr2=0x7fa35c1a22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.733+0000 7fa35be00640 1 --2- 172.21.15.123:0/568005443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa35c10a8e0 0x7fa35c1a22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.733+0000 7fa35be00640 1 -- 172.21.15.123:0/568005443 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35c1a2940 con 0x7fa35c0769f0 2024-05-08T06:12:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.733+0000 7fa35be00640 1 --2- 172.21.15.123:0/568005443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa35c0769f0 0x7fa35c19b270 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa34800e9f0 tx=0x7fa34800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.733+0000 7fa359600640 1 -- 172.21.15.123:0/568005443 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34800ccc0 con 0x7fa35c0769f0 2024-05-08T06:12:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.734+0000 7fa359600640 1 -- 172.21.15.123:0/568005443 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3480108e0 con 0x7fa35c0769f0 2024-05-08T06:12:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.734+0000 7fa363c00640 1 -- 172.21.15.123:0/568005443 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa35c1a2c30 con 0x7fa35c0769f0 2024-05-08T06:12:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.734+0000 7fa359600640 1 -- 172.21.15.123:0/568005443 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa348018880 con 0x7fa35c0769f0 2024-05-08T06:12:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.734+0000 7fa363c00640 1 -- 172.21.15.123:0/568005443 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa35c1a31d0 con 0x7fa35c0769f0 2024-05-08T06:12:41.740 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.738+0000 7fa3669fb740 1 Processor -- start 2024-05-08T06:12:41.740 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.738+0000 7fa3669fb740 1 -- start start 2024-05-08T06:12:41.740 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.738+0000 7fa359600640 1 -- 172.21.15.123:0/568005443 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7fa348010430 con 0x7fa35c0769f0 2024-05-08T06:12:41.740 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.738+0000 7fa3669fb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563673194120 0x563673194540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:41.740 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.738+0000 7fa3669fb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563673194b10 0x563673199610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:41.740 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.739+0000 7fa3669fb740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563673199c80 0x56367319c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:41.740 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.739+0000 7fa3669fb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5636730f8550 con 0x563673194120 2024-05-08T06:12:41.741 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.739+0000 7fa3669fb740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x56367309dbc0 con 0x563673199c80 2024-05-08T06:12:41.741 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.739+0000 7fa3669fb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5636730f4510 con 0x563673194b10 2024-05-08T06:12:41.741 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.739+0000 7fa361200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563673194b10 0x563673199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:41.741 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.739+0000 7fa361c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563673194120 0x563673194540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:41.741 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.739+0000 7fa35be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563673199c80 0x56367319c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:41.741 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.739+0000 7fa361200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563673194b10 0x563673199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:47762/0 (socket says 172.21.15.123:47762) 2024-05-08T06:12:41.741 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.739+0000 7fa361200640 1 -- 172.21.15.123:0/4184268049 learned_addr learned my addr 172.21.15.123:0/4184268049 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:41.741 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.739+0000 7fa361200640 1 -- 172.21.15.123:0/4184268049 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563673199c80 msgr2=0x56367319c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:41.741 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.739+0000 7fa361200640 1 --2- 172.21.15.123:0/4184268049 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563673199c80 0x56367319c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:41.741 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.739+0000 7fa361200640 1 -- 172.21.15.123:0/4184268049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563673194120 msgr2=0x563673194540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:41.742 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.739+0000 7fa361200640 1 --2- 172.21.15.123:0/4184268049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563673194120 0x563673194540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:41.742 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.739+0000 7fa361200640 1 -- 172.21.15.123:0/4184268049 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563672e0a210 con 0x563673194b10 2024-05-08T06:12:41.742 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.740+0000 7fa361200640 1 --2- 172.21.15.123:0/4184268049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563673194b10 0x563673199610 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa350009da0 tx=0x7fa350011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af90e736e636bc2d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:41.742 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.740+0000 7fa34e000640 1 -- 172.21.15.123:0/4184268049 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35001e070 con 0x563673194b10 2024-05-08T06:12:41.742 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.740+0000 7fa34e000640 1 -- 172.21.15.123:0/4184268049 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa350019ae0 con 0x563673194b10 2024-05-08T06:12:41.742 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.740+0000 7fa34e000640 1 -- 172.21.15.123:0/4184268049 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35000fac0 con 0x563673194b10 2024-05-08T06:12:41.748 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.740+0000 7fa359600640 1 --2- 172.21.15.123:0/568005443 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa3440b2920 0x7fa3440b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:41.748 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.741+0000 7fa359600640 1 -- 172.21.15.123:0/568005443 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fa3480e2550 con 0x7fa35c0769f0 2024-05-08T06:12:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.741+0000 7fa361c00640 1 --2- 172.21.15.123:0/568005443 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa3440b2920 0x7fa3440b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.741+0000 7fa3669fb740 1 -- 172.21.15.123:0/4184268049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563673194b10 msgr2=0x563673199610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.741+0000 7fa3669fb740 1 --2- 172.21.15.123:0/4184268049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563673194b10 0x563673199610 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa350009da0 tx=0x7fa350011810 comp rx=0 tx=0).stop 2024-05-08T06:12:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.741+0000 7fa3669fb740 1 -- 172.21.15.123:0/4184268049 shutdown_connections 2024-05-08T06:12:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.741+0000 7fa3669fb740 1 --2- 172.21.15.123:0/4184268049 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563673199c80 0x56367319c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.741+0000 7fa3669fb740 1 --2- 172.21.15.123:0/4184268049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563673194b10 0x563673199610 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.741+0000 7fa3669fb740 1 --2- 172.21.15.123:0/4184268049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563673194120 0x563673194540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.741+0000 7fa3669fb740 1 -- 172.21.15.123:0/4184268049 >> 172.21.15.123:0/4184268049 conn(0x56367315c230 msgr2=0x56367318fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.741+0000 7fa3669fb740 1 -- 172.21.15.123:0/4184268049 shutdown_connections 2024-05-08T06:12:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.741+0000 7fa3669fb740 1 -- 172.21.15.123:0/4184268049 wait complete. 2024-05-08T06:12:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.742+0000 7fa361c00640 1 --2- 172.21.15.123:0/568005443 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa3440b2920 0x7fa3440b4de0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa354005e30 tx=0x7fa354031af0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.742+0000 7fa3669fb740 1 -- start start 2024-05-08T06:12:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.742+0000 7fa3669fb740 1 Processor -- start 2024-05-08T06:12:41.750 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.743+0000 7fa3669fb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563673194120 0x56367319e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:41.750 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.743+0000 7fa3669fb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563673194b10 0x56367319e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:41.750 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.743+0000 7fa3669fb740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563673199c80 0x5636731a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:41.750 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.743+0000 7fa3669fb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56367319cc30 con 0x563673194b10 2024-05-08T06:12:41.750 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.743+0000 7fa3669fb740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5636730b1660 con 0x563673199c80 2024-05-08T06:12:41.750 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.743+0000 7fa3669fb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56367319cdb0 con 0x563673194120 2024-05-08T06:12:41.750 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.743+0000 7fa3669fb740 10 client.0 _ll_register_callbacks cb 0x56367307b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:12:41.750 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.743+0000 7fa361200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563673194120 0x56367319e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:41.750 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.743+0000 7fa361200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563673194120 0x56367319e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:47774/0 (socket says 172.21.15.123:47774) 2024-05-08T06:12:41.750 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.743+0000 7fa361200640 1 -- 172.21.15.123:0/1990004739 learned_addr learned my addr 172.21.15.123:0/1990004739 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:41.750 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.743+0000 7fa361c00640 1 --2- 172.21.15.123:0/1990004739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563673199c80 0x5636731a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:41.750 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.743+0000 7fa35be00640 1 --2- 172.21.15.123:0/1990004739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563673194b10 0x56367319e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:41.750 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.743+0000 7fa361200640 1 -- 172.21.15.123:0/1990004739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563673199c80 msgr2=0x5636731a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:41.751 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.743+0000 7fa361200640 1 --2- 172.21.15.123:0/1990004739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563673199c80 0x5636731a56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:41.751 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.743+0000 7fa361200640 1 -- 172.21.15.123:0/1990004739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563673194b10 msgr2=0x56367319e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:41.751 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.743+0000 7fa361200640 1 --2- 172.21.15.123:0/1990004739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563673194b10 0x56367319e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:41.751 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.743+0000 7fa361200640 1 -- 172.21.15.123:0/1990004739 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563672ed6620 con 0x563673194120 2024-05-08T06:12:41.751 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.743+0000 7fa35be00640 1 --2- 172.21.15.123:0/1990004739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563673194b10 0x56367319e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:12:41.751 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.744+0000 7fa361200640 1 --2- 172.21.15.123:0/1990004739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563673194120 0x56367319e120 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa35000f400 tx=0x7fa35000f460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13effd8c921e8637 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:41.751 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.744+0000 7fa342a00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-05-08T06:12:41.751 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.744+0000 7fa342a00640 1 -- 172.21.15.123:0/1990004739 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa350029020 con 0x563673194120 2024-05-08T06:12:41.751 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.744+0000 7fa3669fb740 1 -- 172.21.15.123:0/1990004739 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa350019ae0 con 0x563673194120 2024-05-08T06:12:41.751 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.744+0000 7fa342a00640 1 -- 172.21.15.123:0/1990004739 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fa3500042c0 con 0x563673194120 2024-05-08T06:12:41.751 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.744+0000 7fa342a00640 1 -- 172.21.15.123:0/1990004739 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa35001e040 con 0x563673194120 2024-05-08T06:12:41.751 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.744+0000 7fa3669fb740 10 client.25609 fetch_fsmap learned FSMap version 24 2024-05-08T06:12:41.751 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.744+0000 7fa3669fb740 1 -- 172.21.15.123:0/1990004739 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5636731a86d0 con 0x563673194120 2024-05-08T06:12:41.752 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.745+0000 7fa342a00640 1 -- 172.21.15.123:0/1990004739 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa350019ae0 con 0x563673194120 2024-05-08T06:12:41.752 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.745+0000 7fa342a00640 1 -- 172.21.15.123:0/1990004739 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fa3500035f0 con 0x563673194120 2024-05-08T06:12:41.752 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.745+0000 7fa3669fb740 10 client.25609 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:12:41.752 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.745+0000 7fa3669fb740 10 client.25609 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T06:12:41.752 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.745+0000 7fa3669fb740 20 client.25609 populate_metadata read hostname 'smithi123' 2024-05-08T06:12:41.752 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.745+0000 7fa3669fb740 1 --2- 172.21.15.123:0/1990004739 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5636731a9ad0 0x5636731abec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:41.752 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.745+0000 7fa3669fb740 4 client.25609 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:12:41.752 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.745+0000 7fa3669fb740 1 -- 172.21.15.123:0/1990004739 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563673106160 con 0x5636731a9ad0 2024-05-08T06:12:41.752 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.745+0000 7fa35be00640 1 --2- 172.21.15.123:0/1990004739 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5636731a9ad0 0x5636731abec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:41.752 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.746+0000 7fa35be00640 1 --2- 172.21.15.123:0/1990004739 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5636731a9ad0 0x5636731abec0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efa890f908ed1af1 server_cookie=532442bf16a46417 in_seq=0 out_seq=0 2024-05-08T06:12:41.752 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.746+0000 7fa342a00640 10 client.25609 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:41.752 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.746+0000 7fa342a00640 1 -- 172.21.15.123:0/1990004739 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563673106160 con 0x5636731a9ad0 2024-05-08T06:12:41.752 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.746+0000 7fa342a00640 10 client.25609 handle_command_reply: tid=0 2024-05-08T06:12:41.753 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.746+0000 7fa3669fb740 1 client.25609 shutdown 2024-05-08T06:12:41.753 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.747+0000 7fa3669fb740 1 -- 172.21.15.123:0/1990004739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563673194120 msgr2=0x56367319e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:41.753 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.747+0000 7fa3669fb740 1 --2- 172.21.15.123:0/1990004739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563673194120 0x56367319e120 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa35000f400 tx=0x7fa35000f460 comp rx=0 tx=0).stop 2024-05-08T06:12:41.753 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.747+0000 7fa3669fb740 1 -- 172.21.15.123:0/1990004739 shutdown_connections 2024-05-08T06:12:41.753 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.747+0000 7fa3669fb740 1 --2- 172.21.15.123:0/1990004739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563673199c80 0x5636731a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:41.753 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.747+0000 7fa3669fb740 1 --2- 172.21.15.123:0/1990004739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563673194b10 0x56367319e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:41.753 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.747+0000 7fa3669fb740 1 --2- 172.21.15.123:0/1990004739 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5636731a9ad0 0x5636731abec0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:41.753 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.747+0000 7fa3669fb740 1 --2- 172.21.15.123:0/1990004739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563673194120 0x56367319e120 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:41.753 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.747+0000 7fa342a00640 0 client.25609 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:41.754 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.747+0000 7fa3669fb740 1 -- 172.21.15.123:0/1990004739 >> 172.21.15.123:0/1990004739 conn(0x56367315c230 msgr2=0x56367318ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:41.754 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.747+0000 7fa3669fb740 1 -- 172.21.15.123:0/1990004739 shutdown_connections 2024-05-08T06:12:41.754 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.747+0000 7fa3669fb740 1 -- 172.21.15.123:0/1990004739 wait complete. 2024-05-08T06:12:41.754 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.747+0000 7fa3669fb740 20 client.25609 trim_cache size 0 max 16384 2024-05-08T06:12:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.772+0000 7fa3669fb740 1 Processor -- start 2024-05-08T06:12:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.772+0000 7fa3669fb740 1 -- start start 2024-05-08T06:12:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.773+0000 7fa3669fb740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563673194120 0x56367318fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.773+0000 7fa3669fb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563673194b10 0x5636731a5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.773+0000 7fa3669fb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563673199c80 0x563673192860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.773+0000 7fa3669fb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c1ae820 con 0x563673199c80 2024-05-08T06:12:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.773+0000 7fa3669fb740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c1ac0f0 con 0x563673194120 2024-05-08T06:12:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.773+0000 7fa3669fb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c1a9f40 con 0x563673194b10 2024-05-08T06:12:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.773+0000 7fa35be00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563673194b10 0x5636731a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.773+0000 7fa361200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563673194120 0x56367318fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.773+0000 7fa35be00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563673194b10 0x5636731a5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:47790/0 (socket says 172.21.15.123:47790) 2024-05-08T06:12:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.773+0000 7fa35be00640 1 -- 172.21.15.123:0/802547367 learned_addr learned my addr 172.21.15.123:0/802547367 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.773+0000 7fa361c00640 1 --2- 172.21.15.123:0/802547367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563673199c80 0x563673192860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.773+0000 7fa35be00640 1 -- 172.21.15.123:0/802547367 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563673194120 msgr2=0x56367318fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.773+0000 7fa35be00640 1 --2- 172.21.15.123:0/802547367 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563673194120 0x56367318fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.773+0000 7fa35be00640 1 -- 172.21.15.123:0/802547367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563673199c80 msgr2=0x563673192860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.773+0000 7fa35be00640 1 --2- 172.21.15.123:0/802547367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563673199c80 0x563673192860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.773+0000 7fa35be00640 1 -- 172.21.15.123:0/802547367 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636731963e0 con 0x563673194b10 2024-05-08T06:12:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.773+0000 7fa361c00640 1 --2- 172.21.15.123:0/802547367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563673199c80 0x563673192860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:12:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.774+0000 7fa35be00640 1 --2- 172.21.15.123:0/802547367 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563673194b10 0x5636731a5660 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa34800e9c0 tx=0x7fa3480e80e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=779ab510bd61ccd8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.774+0000 7fa34d600640 1 -- 172.21.15.123:0/802547367 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3480a5330 con 0x563673194b10 2024-05-08T06:12:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.774+0000 7fa34d600640 1 -- 172.21.15.123:0/802547367 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa348010720 con 0x563673194b10 2024-05-08T06:12:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.774+0000 7fa34d600640 1 -- 172.21.15.123:0/802547367 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3480b29e0 con 0x563673194b10 2024-05-08T06:12:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.775+0000 7fa3669fb740 1 -- 172.21.15.123:0/802547367 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563673194b10 msgr2=0x5636731a5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.775+0000 7fa3669fb740 1 --2- 172.21.15.123:0/802547367 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563673194b10 0x5636731a5660 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa34800e9c0 tx=0x7fa3480e80e0 comp rx=0 tx=0).stop 2024-05-08T06:12:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.775+0000 7fa3669fb740 1 -- 172.21.15.123:0/802547367 shutdown_connections 2024-05-08T06:12:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.775+0000 7fa3669fb740 1 --2- 172.21.15.123:0/802547367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563673199c80 0x563673192860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.775+0000 7fa3669fb740 1 --2- 172.21.15.123:0/802547367 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563673194b10 0x5636731a5660 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.775+0000 7fa3669fb740 1 --2- 172.21.15.123:0/802547367 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563673194120 0x56367318fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.775+0000 7fa3669fb740 1 -- 172.21.15.123:0/802547367 >> 172.21.15.123:0/802547367 conn(0x56367315c230 msgr2=0x56367318f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.775+0000 7fa3669fb740 1 -- 172.21.15.123:0/802547367 shutdown_connections 2024-05-08T06:12:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.775+0000 7fa3669fb740 1 -- 172.21.15.123:0/802547367 wait complete. 2024-05-08T06:12:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.775+0000 7fa3669fb740 1 -- start start 2024-05-08T06:12:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.776+0000 7fa3669fb740 1 Processor -- start 2024-05-08T06:12:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.776+0000 7fa3669fb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563673194120 0x5636731a2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.776+0000 7fa3669fb740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563673194b10 0x5636731a28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.776+0000 7fa3669fb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563673199c80 0x563673212370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.776+0000 7fa3669fb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c1aa5b0 con 0x563673199c80 2024-05-08T06:12:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.776+0000 7fa3669fb740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c1aba20 con 0x563673194b10 2024-05-08T06:12:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.776+0000 7fa3669fb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c1aac80 con 0x563673194120 2024-05-08T06:12:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.776+0000 7fa3669fb740 10 client.0 _ll_register_callbacks cb 0x7fa35001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:12:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.776+0000 7fa361c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563673199c80 0x563673212370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.776+0000 7fa361c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563673199c80 0x563673212370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36324/0 (socket says 172.21.15.123:36324) 2024-05-08T06:12:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.776+0000 7fa361c00640 1 -- 172.21.15.123:0/516117777 learned_addr learned my addr 172.21.15.123:0/516117777 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.776+0000 7fa361200640 1 --2- 172.21.15.123:0/516117777 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563673194120 0x5636731a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.776+0000 7fa35be00640 1 --2- 172.21.15.123:0/516117777 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563673194b10 0x5636731a28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.777+0000 7fa361200640 1 -- 172.21.15.123:0/516117777 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563673194b10 msgr2=0x5636731a28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.777+0000 7fa361200640 1 --2- 172.21.15.123:0/516117777 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563673194b10 0x5636731a28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.777+0000 7fa361200640 1 -- 172.21.15.123:0/516117777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563673199c80 msgr2=0x563673212370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.777+0000 7fa361200640 1 --2- 172.21.15.123:0/516117777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563673199c80 0x563673212370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.777+0000 7fa361200640 1 -- 172.21.15.123:0/516117777 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636731a2ed0 con 0x563673194120 2024-05-08T06:12:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.777+0000 7fa361c00640 1 --2- 172.21.15.123:0/516117777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563673199c80 0x563673212370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:12:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.777+0000 7fa35be00640 1 --2- 172.21.15.123:0/516117777 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563673194b10 0x5636731a28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:12:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.777+0000 7fa361200640 1 --2- 172.21.15.123:0/516117777 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563673194120 0x5636731a2380 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa350032040 tx=0x7fa350025000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55dcd053d07092f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.777+0000 7fa342a00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-05-08T06:12:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.777+0000 7fa342a00640 1 -- 172.21.15.123:0/516117777 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa350029070 con 0x563673194120 2024-05-08T06:12:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.777+0000 7fa3669fb740 1 -- 172.21.15.123:0/516117777 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5636730fffd0 con 0x563673194120 2024-05-08T06:12:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.777+0000 7fa342a00640 1 -- 172.21.15.123:0/516117777 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fa35000f4d0 con 0x563673194120 2024-05-08T06:12:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.778+0000 7fa342a00640 1 -- 172.21.15.123:0/516117777 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa35000f6f0 con 0x563673194120 2024-05-08T06:12:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.778+0000 7fa3669fb740 10 client.25615 fetch_fsmap learned FSMap version 24 2024-05-08T06:12:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.778+0000 7fa3669fb740 1 -- 172.21.15.123:0/516117777 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563673213950 con 0x563673194120 2024-05-08T06:12:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.778+0000 7fa342a00640 1 -- 172.21.15.123:0/516117777 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5636730fffd0 con 0x563673194120 2024-05-08T06:12:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.778+0000 7fa342a00640 1 -- 172.21.15.123:0/516117777 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fa350024750 con 0x563673194120 2024-05-08T06:12:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.778+0000 7fa3669fb740 10 client.25615 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:12:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.778+0000 7fa3669fb740 10 client.25615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T06:12:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.778+0000 7fa3669fb740 20 client.25615 populate_metadata read hostname 'smithi123' 2024-05-08T06:12:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.778+0000 7fa3669fb740 1 --2- 172.21.15.123:0/516117777 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x563673214270 0x563673216690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.778+0000 7fa3669fb740 4 client.25615 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T06:12:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.778+0000 7fa3669fb740 1 -- 172.21.15.123:0/516117777 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56367307b4b0 con 0x563673214270 2024-05-08T06:12:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.779+0000 7fa35be00640 1 --2- 172.21.15.123:0/516117777 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x563673214270 0x563673216690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.779+0000 7fa35be00640 1 --2- 172.21.15.123:0/516117777 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x563673214270 0x563673216690 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9b551805721d0dc server_cookie=ad5fc1c21dd7071b in_seq=0 out_seq=0 2024-05-08T06:12:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.779+0000 7fa342a00640 10 client.25615 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.804+0000 7fa342a00640 1 -- 172.21.15.123:0/516117777 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56367307b4b0 con 0x563673214270 2024-05-08T06:12:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.804+0000 7fa342a00640 10 client.25615 handle_command_reply: tid=0 2024-05-08T06:12:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.804+0000 7fa3669fb740 1 client.25615 shutdown 2024-05-08T06:12:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.804+0000 7fa3669fb740 1 -- 172.21.15.123:0/516117777 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563673194120 msgr2=0x5636731a2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.804+0000 7fa3669fb740 1 --2- 172.21.15.123:0/516117777 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563673194120 0x5636731a2380 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa350032040 tx=0x7fa350025000 comp rx=0 tx=0).stop 2024-05-08T06:12:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.804+0000 7fa3669fb740 1 -- 172.21.15.123:0/516117777 shutdown_connections 2024-05-08T06:12:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.804+0000 7fa3669fb740 1 --2- 172.21.15.123:0/516117777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563673199c80 0x563673212370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.804+0000 7fa3669fb740 1 --2- 172.21.15.123:0/516117777 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563673194b10 0x5636731a28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.804+0000 7fa3669fb740 1 --2- 172.21.15.123:0/516117777 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x563673214270 0x563673216690 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.804+0000 7fa3669fb740 1 --2- 172.21.15.123:0/516117777 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563673194120 0x5636731a2380 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.804+0000 7fa342a00640 0 client.25615 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.804+0000 7fa3669fb740 1 -- 172.21.15.123:0/516117777 >> 172.21.15.123:0/516117777 conn(0x56367315c230 msgr2=0x563673192860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.804+0000 7fa3669fb740 1 -- 172.21.15.123:0/516117777 shutdown_connections 2024-05-08T06:12:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.804+0000 7fa3669fb740 1 -- 172.21.15.123:0/516117777 wait complete. 2024-05-08T06:12:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.804+0000 7fa3669fb740 20 client.25615 trim_cache size 0 max 16384 2024-05-08T06:12:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout:{ 2024-05-08T06:12:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout: "message": "", 2024-05-08T06:12:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout: "return_code": 0 2024-05-08T06:12:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout:} 2024-05-08T06:12:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.805+0000 7fa363c00640 1 -- 172.21.15.123:0/568005443 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa3440b2920 msgr2=0x7fa3440b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.805+0000 7fa363c00640 1 --2- 172.21.15.123:0/568005443 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa3440b2920 0x7fa3440b4de0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa354005e30 tx=0x7fa354031af0 comp rx=0 tx=0).stop 2024-05-08T06:12:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.805+0000 7fa363c00640 1 -- 172.21.15.123:0/568005443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa35c0769f0 msgr2=0x7fa35c19b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.805+0000 7fa363c00640 1 --2- 172.21.15.123:0/568005443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa35c0769f0 0x7fa35c19b270 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa34800e9f0 tx=0x7fa34800eeb0 comp rx=0 tx=0).stop 2024-05-08T06:12:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.806+0000 7fa363c00640 1 -- 172.21.15.123:0/568005443 shutdown_connections 2024-05-08T06:12:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.806+0000 7fa363c00640 1 --2- 172.21.15.123:0/568005443 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa3440b2920 0x7fa3440b4de0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.806+0000 7fa363c00640 1 --2- 172.21.15.123:0/568005443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa35c10a8e0 0x7fa35c1a22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.806+0000 7fa363c00640 1 --2- 172.21.15.123:0/568005443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa35c0769f0 0x7fa35c19b270 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.806+0000 7fa363c00640 1 --2- 172.21.15.123:0/568005443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa35c075410 0x7fa35c19ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.806+0000 7fa363c00640 1 -- 172.21.15.123:0/568005443 >> 172.21.15.123:0/568005443 conn(0x7fa35c0fdf70 msgr2=0x7fa35c1067e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.806+0000 7fa363c00640 1 -- 172.21.15.123:0/568005443 shutdown_connections 2024-05-08T06:12:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:41.806+0000 7fa363c00640 1 -- 172.21.15.123:0/568005443 wait complete. 2024-05-08T06:12:41.808 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4196864 len 4096 2024-05-08T06:12:41.808 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4196864 len 8192 2024-05-08T06:12:41.808 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4196864 len 8192 2024-05-08T06:12:41.809 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4196864 len 8192 2024-05-08T06:12:41.809 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4196864 len 8192 2024-05-08T06:12:41.809 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4196864 len 16384 2024-05-08T06:12:41.809 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4196864 len 16384 2024-05-08T06:12:41.809 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4196864 len 16384 2024-05-08T06:12:41.809 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4196864 len 16384 2024-05-08T06:12:41.809 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4197376 len 1024 2024-05-08T06:12:41.809 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4197376 len 1024 2024-05-08T06:12:41.809 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4197376 len 1024 2024-05-08T06:12:41.809 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4197376 len 1024 2024-05-08T06:12:41.809 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4197376 len 2048 2024-05-08T06:12:41.809 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4197376 len 2048 2024-05-08T06:12:41.809 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4197376 len 2048 2024-05-08T06:12:41.809 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4197376 len 2048 2024-05-08T06:12:41.809 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4197376 len 4096 2024-05-08T06:12:41.809 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4197376 len 4096 2024-05-08T06:12:41.810 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4197376 len 4096 2024-05-08T06:12:41.810 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4197376 len 4096 2024-05-08T06:12:41.810 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4197376 len 8192 2024-05-08T06:12:41.810 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4197376 len 8192 2024-05-08T06:12:41.810 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4197376 len 8192 2024-05-08T06:12:41.810 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4197376 len 8192 2024-05-08T06:12:41.810 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4197376 len 16384 2024-05-08T06:12:41.810 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4197376 len 16384 2024-05-08T06:12:41.810 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4197376 len 16384 2024-05-08T06:12:41.810 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4197376 len 16384 2024-05-08T06:12:41.810 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4197888 len 1024 2024-05-08T06:12:41.810 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4197888 len 1024 2024-05-08T06:12:41.810 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4197888 len 1024 2024-05-08T06:12:41.810 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4197888 len 1024 2024-05-08T06:12:41.810 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4197888 len 2048 2024-05-08T06:12:41.811 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4197888 len 2048 2024-05-08T06:12:41.811 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4197888 len 2048 2024-05-08T06:12:41.811 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4197888 len 2048 2024-05-08T06:12:41.811 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4197888 len 4096 2024-05-08T06:12:41.811 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4197888 len 4096 2024-05-08T06:12:41.811 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4197888 len 4096 2024-05-08T06:12:41.811 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4197888 len 4096 2024-05-08T06:12:41.811 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4197888 len 8192 2024-05-08T06:12:41.811 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4197888 len 8192 2024-05-08T06:12:41.811 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4197888 len 8192 2024-05-08T06:12:41.811 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4197888 len 8192 2024-05-08T06:12:41.811 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1024 offset 4197888 len 16384 2024-05-08T06:12:41.811 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4197888 len 16384 2024-05-08T06:12:41.811 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1024 offset 4197888 len 16384 2024-05-08T06:12:41.811 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1024 offset 4197888 len 16384 2024-05-08T06:12:41.811 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4190720 len 1024 2024-05-08T06:12:41.812 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4190720 len 1024 2024-05-08T06:12:41.812 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4190720 len 1024 2024-05-08T06:12:41.812 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4190720 len 1024 2024-05-08T06:12:41.812 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4190720 len 2048 2024-05-08T06:12:41.812 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4190720 len 2048 2024-05-08T06:12:41.812 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4190720 len 2048 2024-05-08T06:12:41.812 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4190720 len 2048 2024-05-08T06:12:41.812 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4190720 len 4096 2024-05-08T06:12:41.812 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4190720 len 4096 2024-05-08T06:12:41.812 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4190720 len 4096 2024-05-08T06:12:41.812 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4190720 len 4096 2024-05-08T06:12:41.812 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4190720 len 8192 2024-05-08T06:12:41.812 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4190720 len 8192 2024-05-08T06:12:41.812 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4190720 len 8192 2024-05-08T06:12:41.812 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4190720 len 8192 2024-05-08T06:12:41.812 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4190720 len 16384 2024-05-08T06:12:41.812 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4190720 len 16384 2024-05-08T06:12:41.813 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4190720 len 16384 2024-05-08T06:12:41.813 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4190720 len 16384 2024-05-08T06:12:41.813 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4191232 len 1024 2024-05-08T06:12:41.813 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4191232 len 1024 2024-05-08T06:12:41.813 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4191232 len 1024 2024-05-08T06:12:41.813 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4191232 len 1024 2024-05-08T06:12:41.813 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4191232 len 2048 2024-05-08T06:12:41.813 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4191232 len 2048 2024-05-08T06:12:41.813 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4191232 len 2048 2024-05-08T06:12:41.813 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4191232 len 2048 2024-05-08T06:12:41.813 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4191232 len 4096 2024-05-08T06:12:41.813 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4191232 len 4096 2024-05-08T06:12:41.813 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4191232 len 4096 2024-05-08T06:12:41.813 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4191232 len 4096 2024-05-08T06:12:41.840 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-08T06:12:42.310 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:12:42.754 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:42 smithi123 ceph-mon[29787]: pgmap v527: 97 pgs: 97 active+clean; 177 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 60 MiB/s rd, 1.2 MiB/s wr, 318 op/s 2024-05-08T06:12:43.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:42 smithi163 ceph-mon[34520]: pgmap v527: 97 pgs: 97 active+clean; 177 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 60 MiB/s rd, 1.2 MiB/s wr, 318 op/s 2024-05-08T06:12:43.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:42 smithi192 ceph-mon[34664]: pgmap v527: 97 pgs: 97 active+clean; 177 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 60 MiB/s rd, 1.2 MiB/s wr, 318 op/s 2024-05-08T06:12:43.191 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4191232 len 8192 2024-05-08T06:12:43.191 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4191232 len 8192 2024-05-08T06:12:43.191 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4191232 len 8192 2024-05-08T06:12:43.191 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4191232 len 8192 2024-05-08T06:12:43.191 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4191232 len 16384 2024-05-08T06:12:43.191 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4191232 len 16384 2024-05-08T06:12:43.192 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4191232 len 16384 2024-05-08T06:12:43.192 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4191232 len 16384 2024-05-08T06:12:43.192 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4191744 len 1024 2024-05-08T06:12:43.192 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4191744 len 1024 2024-05-08T06:12:43.192 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4191744 len 1024 2024-05-08T06:12:43.192 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4191744 len 1024 2024-05-08T06:12:43.192 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4191744 len 2048 2024-05-08T06:12:43.192 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4191744 len 2048 2024-05-08T06:12:43.192 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4191744 len 2048 2024-05-08T06:12:43.192 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4191744 len 2048 2024-05-08T06:12:43.192 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4191744 len 4096 2024-05-08T06:12:43.192 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4191744 len 4096 2024-05-08T06:12:43.192 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4191744 len 4096 2024-05-08T06:12:43.192 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4191744 len 4096 2024-05-08T06:12:43.192 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4191744 len 8192 2024-05-08T06:12:43.192 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4191744 len 8192 2024-05-08T06:12:43.193 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4191744 len 8192 2024-05-08T06:12:43.193 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4191744 len 8192 2024-05-08T06:12:43.193 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4191744 len 16384 2024-05-08T06:12:43.193 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4191744 len 16384 2024-05-08T06:12:43.193 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4191744 len 16384 2024-05-08T06:12:43.193 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4191744 len 16384 2024-05-08T06:12:43.193 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4192256 len 1024 2024-05-08T06:12:43.193 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4192256 len 1024 2024-05-08T06:12:43.193 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4192256 len 1024 2024-05-08T06:12:43.193 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4192256 len 1024 2024-05-08T06:12:43.193 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4192256 len 2048 2024-05-08T06:12:43.193 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4192256 len 2048 2024-05-08T06:12:43.193 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4192256 len 2048 2024-05-08T06:12:43.193 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4192256 len 2048 2024-05-08T06:12:43.193 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4192256 len 4096 2024-05-08T06:12:43.193 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4192256 len 4096 2024-05-08T06:12:43.194 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4192256 len 4096 2024-05-08T06:12:43.194 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4192256 len 4096 2024-05-08T06:12:43.194 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4192256 len 8192 2024-05-08T06:12:43.194 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4192256 len 8192 2024-05-08T06:12:43.194 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4192256 len 8192 2024-05-08T06:12:43.194 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4192256 len 8192 2024-05-08T06:12:43.194 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4192256 len 16384 2024-05-08T06:12:43.194 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4192256 len 16384 2024-05-08T06:12:43.194 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4192256 len 16384 2024-05-08T06:12:43.194 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4192256 len 16384 2024-05-08T06:12:43.194 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4192768 len 1024 2024-05-08T06:12:43.194 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4192768 len 1024 2024-05-08T06:12:43.194 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4192768 len 1024 2024-05-08T06:12:43.194 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4192768 len 1024 2024-05-08T06:12:43.194 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4192768 len 2048 2024-05-08T06:12:43.194 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4192768 len 2048 2024-05-08T06:12:43.195 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4192768 len 2048 2024-05-08T06:12:43.195 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4192768 len 2048 2024-05-08T06:12:43.195 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4192768 len 4096 2024-05-08T06:12:43.195 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4192768 len 4096 2024-05-08T06:12:43.195 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4192768 len 4096 2024-05-08T06:12:43.195 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4192768 len 4096 2024-05-08T06:12:43.195 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4192768 len 8192 2024-05-08T06:12:43.195 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4192768 len 8192 2024-05-08T06:12:43.195 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4192768 len 8192 2024-05-08T06:12:43.195 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4192768 len 8192 2024-05-08T06:12:43.195 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4192768 len 16384 2024-05-08T06:12:43.195 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4192768 len 16384 2024-05-08T06:12:43.195 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4192768 len 16384 2024-05-08T06:12:43.195 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4192768 len 16384 2024-05-08T06:12:43.195 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4193280 len 1024 2024-05-08T06:12:43.195 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4193280 len 1024 2024-05-08T06:12:43.196 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4193280 len 1024 2024-05-08T06:12:43.196 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4193280 len 1024 2024-05-08T06:12:43.196 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4193280 len 2048 2024-05-08T06:12:43.196 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4193280 len 2048 2024-05-08T06:12:43.196 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4193280 len 2048 2024-05-08T06:12:43.196 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4193280 len 2048 2024-05-08T06:12:43.196 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4193280 len 4096 2024-05-08T06:12:43.196 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4193280 len 4096 2024-05-08T06:12:43.196 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4193280 len 4096 2024-05-08T06:12:43.196 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4193280 len 4096 2024-05-08T06:12:43.196 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4193280 len 8192 2024-05-08T06:12:44.251 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4193280 len 8192 2024-05-08T06:12:44.251 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4193280 len 8192 2024-05-08T06:12:44.251 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4193280 len 8192 2024-05-08T06:12:44.251 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4193280 len 16384 2024-05-08T06:12:44.251 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4193280 len 16384 2024-05-08T06:12:44.251 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4193280 len 16384 2024-05-08T06:12:44.251 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4193280 len 16384 2024-05-08T06:12:44.251 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4193792 len 1024 2024-05-08T06:12:44.251 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4193792 len 1024 2024-05-08T06:12:44.251 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4193792 len 1024 2024-05-08T06:12:44.251 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4193792 len 1024 2024-05-08T06:12:44.251 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4193792 len 2048 2024-05-08T06:12:44.252 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4193792 len 2048 2024-05-08T06:12:44.252 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4193792 len 2048 2024-05-08T06:12:44.252 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4193792 len 2048 2024-05-08T06:12:44.252 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4193792 len 4096 2024-05-08T06:12:44.252 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4193792 len 4096 2024-05-08T06:12:44.252 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4193792 len 4096 2024-05-08T06:12:44.252 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4193792 len 4096 2024-05-08T06:12:44.252 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4193792 len 8192 2024-05-08T06:12:44.252 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4193792 len 8192 2024-05-08T06:12:44.252 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4193792 len 8192 2024-05-08T06:12:44.252 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4193792 len 8192 2024-05-08T06:12:44.252 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4193792 len 16384 2024-05-08T06:12:44.252 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4193792 len 16384 2024-05-08T06:12:44.253 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4193792 len 16384 2024-05-08T06:12:44.253 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4193792 len 16384 2024-05-08T06:12:44.253 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4194304 len 1024 2024-05-08T06:12:44.253 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4194304 len 1024 2024-05-08T06:12:44.253 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4194304 len 1024 2024-05-08T06:12:44.253 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4194304 len 1024 2024-05-08T06:12:44.253 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4194304 len 2048 2024-05-08T06:12:44.253 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4194304 len 2048 2024-05-08T06:12:44.253 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4194304 len 2048 2024-05-08T06:12:44.253 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4194304 len 2048 2024-05-08T06:12:44.253 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4194304 len 4096 2024-05-08T06:12:44.253 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4194304 len 4096 2024-05-08T06:12:44.253 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4194304 len 4096 2024-05-08T06:12:44.253 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4194304 len 4096 2024-05-08T06:12:44.253 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4194304 len 8192 2024-05-08T06:12:44.254 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4194304 len 8192 2024-05-08T06:12:44.254 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4194304 len 8192 2024-05-08T06:12:44.254 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4194304 len 8192 2024-05-08T06:12:44.254 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4194304 len 16384 2024-05-08T06:12:44.254 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4194304 len 16384 2024-05-08T06:12:44.254 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4194304 len 16384 2024-05-08T06:12:44.254 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4194304 len 16384 2024-05-08T06:12:44.254 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4194816 len 1024 2024-05-08T06:12:44.254 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4194816 len 1024 2024-05-08T06:12:44.254 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4194816 len 1024 2024-05-08T06:12:44.254 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4194816 len 1024 2024-05-08T06:12:44.254 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4194816 len 2048 2024-05-08T06:12:44.254 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4194816 len 2048 2024-05-08T06:12:44.254 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4194816 len 2048 2024-05-08T06:12:44.255 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4194816 len 2048 2024-05-08T06:12:44.255 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4194816 len 4096 2024-05-08T06:12:44.255 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4194816 len 4096 2024-05-08T06:12:44.255 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4194816 len 4096 2024-05-08T06:12:44.255 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4194816 len 4096 2024-05-08T06:12:44.255 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4194816 len 8192 2024-05-08T06:12:44.255 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4194816 len 8192 2024-05-08T06:12:44.255 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4194816 len 8192 2024-05-08T06:12:44.255 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4194816 len 8192 2024-05-08T06:12:44.255 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4194816 len 16384 2024-05-08T06:12:44.255 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4194816 len 16384 2024-05-08T06:12:44.255 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4194816 len 16384 2024-05-08T06:12:44.255 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4194816 len 16384 2024-05-08T06:12:44.255 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4195328 len 1024 2024-05-08T06:12:44.255 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4195328 len 1024 2024-05-08T06:12:44.256 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4195328 len 1024 2024-05-08T06:12:44.256 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4195328 len 1024 2024-05-08T06:12:44.256 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4195328 len 2048 2024-05-08T06:12:44.256 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4195328 len 2048 2024-05-08T06:12:44.256 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4195328 len 2048 2024-05-08T06:12:44.256 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4195328 len 2048 2024-05-08T06:12:44.256 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4195328 len 4096 2024-05-08T06:12:44.256 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4195328 len 4096 2024-05-08T06:12:44.256 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4195328 len 4096 2024-05-08T06:12:44.256 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4195328 len 4096 2024-05-08T06:12:44.256 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4195328 len 8192 2024-05-08T06:12:44.724 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4195328 len 8192 2024-05-08T06:12:44.725 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4195328 len 8192 2024-05-08T06:12:44.725 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4195328 len 8192 2024-05-08T06:12:44.725 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4195328 len 16384 2024-05-08T06:12:44.725 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4195328 len 16384 2024-05-08T06:12:44.725 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4195328 len 16384 2024-05-08T06:12:44.725 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4195328 len 16384 2024-05-08T06:12:44.725 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4195840 len 1024 2024-05-08T06:12:44.725 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4195840 len 1024 2024-05-08T06:12:44.725 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4195840 len 1024 2024-05-08T06:12:44.725 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4195840 len 1024 2024-05-08T06:12:44.725 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4195840 len 2048 2024-05-08T06:12:44.726 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4195840 len 2048 2024-05-08T06:12:44.726 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4195840 len 2048 2024-05-08T06:12:44.726 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4195840 len 2048 2024-05-08T06:12:44.726 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4195840 len 4096 2024-05-08T06:12:44.726 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4195840 len 4096 2024-05-08T06:12:44.726 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4195840 len 4096 2024-05-08T06:12:44.726 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4195840 len 4096 2024-05-08T06:12:44.726 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4195840 len 8192 2024-05-08T06:12:44.726 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4195840 len 8192 2024-05-08T06:12:44.726 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4195840 len 8192 2024-05-08T06:12:44.726 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4195840 len 8192 2024-05-08T06:12:44.726 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4195840 len 16384 2024-05-08T06:12:44.726 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4195840 len 16384 2024-05-08T06:12:44.726 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4195840 len 16384 2024-05-08T06:12:44.727 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4195840 len 16384 2024-05-08T06:12:44.727 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4196352 len 1024 2024-05-08T06:12:44.727 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4196352 len 1024 2024-05-08T06:12:44.727 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4196352 len 1024 2024-05-08T06:12:44.727 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4196352 len 1024 2024-05-08T06:12:44.727 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4196352 len 2048 2024-05-08T06:12:44.727 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4196352 len 2048 2024-05-08T06:12:44.727 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4196352 len 2048 2024-05-08T06:12:44.727 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4196352 len 2048 2024-05-08T06:12:44.727 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4196352 len 4096 2024-05-08T06:12:44.727 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4196352 len 4096 2024-05-08T06:12:44.727 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4196352 len 4096 2024-05-08T06:12:44.727 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4196352 len 4096 2024-05-08T06:12:44.727 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4196352 len 8192 2024-05-08T06:12:44.727 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4196352 len 8192 2024-05-08T06:12:44.727 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4196352 len 8192 2024-05-08T06:12:44.728 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4196352 len 8192 2024-05-08T06:12:44.728 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4196352 len 16384 2024-05-08T06:12:44.728 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4196352 len 16384 2024-05-08T06:12:44.728 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4196352 len 16384 2024-05-08T06:12:44.728 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4196352 len 16384 2024-05-08T06:12:44.728 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4196864 len 1024 2024-05-08T06:12:44.728 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4196864 len 1024 2024-05-08T06:12:44.728 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4196864 len 1024 2024-05-08T06:12:44.728 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4196864 len 1024 2024-05-08T06:12:44.728 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4196864 len 2048 2024-05-08T06:12:44.729 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4196864 len 2048 2024-05-08T06:12:44.729 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4196864 len 2048 2024-05-08T06:12:44.729 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4196864 len 2048 2024-05-08T06:12:44.729 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4196864 len 4096 2024-05-08T06:12:44.729 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4196864 len 4096 2024-05-08T06:12:44.729 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4196864 len 4096 2024-05-08T06:12:44.729 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4196864 len 4096 2024-05-08T06:12:44.729 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4196864 len 8192 2024-05-08T06:12:44.729 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4196864 len 8192 2024-05-08T06:12:44.729 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4196864 len 8192 2024-05-08T06:12:44.729 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4196864 len 8192 2024-05-08T06:12:44.729 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4196864 len 16384 2024-05-08T06:12:44.729 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4196864 len 16384 2024-05-08T06:12:44.729 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4196864 len 16384 2024-05-08T06:12:44.729 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4196864 len 16384 2024-05-08T06:12:44.730 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4197376 len 1024 2024-05-08T06:12:44.730 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4197376 len 1024 2024-05-08T06:12:44.730 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4197376 len 1024 2024-05-08T06:12:44.730 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4197376 len 1024 2024-05-08T06:12:44.730 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4197376 len 2048 2024-05-08T06:12:44.730 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4197376 len 2048 2024-05-08T06:12:44.730 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4197376 len 2048 2024-05-08T06:12:44.730 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4197376 len 2048 2024-05-08T06:12:44.730 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4197376 len 4096 2024-05-08T06:12:44.730 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4197376 len 4096 2024-05-08T06:12:44.730 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4197376 len 4096 2024-05-08T06:12:44.730 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4197376 len 4096 2024-05-08T06:12:44.731 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4197376 len 8192 2024-05-08T06:12:44.731 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4197376 len 8192 2024-05-08T06:12:44.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:44 smithi123 ceph-mon[29787]: pgmap v528: 97 pgs: 97 active+clean; 181 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s rd, 892 KiB/s wr, 293 op/s 2024-05-08T06:12:45.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:44 smithi163 ceph-mon[34520]: pgmap v528: 97 pgs: 97 active+clean; 181 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s rd, 892 KiB/s wr, 293 op/s 2024-05-08T06:12:45.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:44 smithi192 ceph-mon[34664]: pgmap v528: 97 pgs: 97 active+clean; 181 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s rd, 892 KiB/s wr, 293 op/s 2024-05-08T06:12:45.990 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4197376 len 8192 2024-05-08T06:12:45.990 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4197376 len 8192 2024-05-08T06:12:45.990 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4197376 len 16384 2024-05-08T06:12:45.990 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4197376 len 16384 2024-05-08T06:12:45.990 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4197376 len 16384 2024-05-08T06:12:45.990 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4197376 len 16384 2024-05-08T06:12:45.990 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4197888 len 1024 2024-05-08T06:12:45.990 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4197888 len 1024 2024-05-08T06:12:45.990 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4197888 len 1024 2024-05-08T06:12:45.990 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4197888 len 1024 2024-05-08T06:12:45.990 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4197888 len 2048 2024-05-08T06:12:45.991 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4197888 len 2048 2024-05-08T06:12:45.991 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4197888 len 2048 2024-05-08T06:12:45.991 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4197888 len 2048 2024-05-08T06:12:45.991 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4197888 len 4096 2024-05-08T06:12:45.991 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4197888 len 4096 2024-05-08T06:12:45.991 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4197888 len 4096 2024-05-08T06:12:45.991 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4197888 len 4096 2024-05-08T06:12:45.991 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4197888 len 8192 2024-05-08T06:12:45.991 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4197888 len 8192 2024-05-08T06:12:45.991 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4197888 len 8192 2024-05-08T06:12:45.991 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4197888 len 8192 2024-05-08T06:12:45.991 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 1536 offset 4197888 len 16384 2024-05-08T06:12:45.991 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4197888 len 16384 2024-05-08T06:12:45.991 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 1536 offset 4197888 len 16384 2024-05-08T06:12:45.991 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 1536 offset 4197888 len 16384 2024-05-08T06:12:45.992 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4190720 len 1024 2024-05-08T06:12:45.992 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4190720 len 1024 2024-05-08T06:12:45.992 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4190720 len 1024 2024-05-08T06:12:45.992 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4190720 len 1024 2024-05-08T06:12:45.992 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4190720 len 2048 2024-05-08T06:12:45.992 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4190720 len 2048 2024-05-08T06:12:45.992 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4190720 len 2048 2024-05-08T06:12:45.992 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4190720 len 2048 2024-05-08T06:12:45.992 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4190720 len 4096 2024-05-08T06:12:45.992 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4190720 len 4096 2024-05-08T06:12:45.992 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4190720 len 4096 2024-05-08T06:12:45.992 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4190720 len 4096 2024-05-08T06:12:45.992 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4190720 len 8192 2024-05-08T06:12:45.992 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4190720 len 8192 2024-05-08T06:12:45.993 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4190720 len 8192 2024-05-08T06:12:45.993 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4190720 len 8192 2024-05-08T06:12:45.993 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4190720 len 16384 2024-05-08T06:12:45.993 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4190720 len 16384 2024-05-08T06:12:45.993 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4190720 len 16384 2024-05-08T06:12:45.993 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4190720 len 16384 2024-05-08T06:12:45.993 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4191232 len 1024 2024-05-08T06:12:45.993 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4191232 len 1024 2024-05-08T06:12:45.993 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4191232 len 1024 2024-05-08T06:12:45.993 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4191232 len 1024 2024-05-08T06:12:45.993 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4191232 len 2048 2024-05-08T06:12:45.993 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4191232 len 2048 2024-05-08T06:12:45.993 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4191232 len 2048 2024-05-08T06:12:45.993 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4191232 len 2048 2024-05-08T06:12:45.993 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4191232 len 4096 2024-05-08T06:12:45.994 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4191232 len 4096 2024-05-08T06:12:45.994 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4191232 len 4096 2024-05-08T06:12:45.994 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4191232 len 4096 2024-05-08T06:12:45.994 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4191232 len 8192 2024-05-08T06:12:45.994 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4191232 len 8192 2024-05-08T06:12:45.994 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4191232 len 8192 2024-05-08T06:12:45.994 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4191232 len 8192 2024-05-08T06:12:45.994 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4191232 len 16384 2024-05-08T06:12:45.994 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4191232 len 16384 2024-05-08T06:12:45.994 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4191232 len 16384 2024-05-08T06:12:45.994 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4191232 len 16384 2024-05-08T06:12:45.994 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4191744 len 1024 2024-05-08T06:12:45.994 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4191744 len 1024 2024-05-08T06:12:45.994 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4191744 len 1024 2024-05-08T06:12:45.994 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4191744 len 1024 2024-05-08T06:12:45.995 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4191744 len 2048 2024-05-08T06:12:45.995 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4191744 len 2048 2024-05-08T06:12:45.995 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4191744 len 2048 2024-05-08T06:12:45.995 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4191744 len 2048 2024-05-08T06:12:45.995 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4191744 len 4096 2024-05-08T06:12:45.995 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4191744 len 4096 2024-05-08T06:12:45.995 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4191744 len 4096 2024-05-08T06:12:45.995 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4191744 len 4096 2024-05-08T06:12:45.995 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4191744 len 8192 2024-05-08T06:12:45.995 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4191744 len 8192 2024-05-08T06:12:45.995 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4191744 len 8192 2024-05-08T06:12:46.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.198+0000 7f5bf3c00700 1 -- 172.21.15.123:0/1870450624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bec109510 msgr2=0x7f5bec1098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:46.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.198+0000 7f5bf3c00700 1 --2- 172.21.15.123:0/1870450624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bec109510 0x7f5bec1098f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5be00098f0 tx=0x7f5be0009c00 comp rx=0 tx=0).stop 2024-05-08T06:12:46.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.198+0000 7f5bf3c00700 1 -- 172.21.15.123:0/1870450624 shutdown_connections 2024-05-08T06:12:46.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.198+0000 7f5bf3c00700 1 --2- 172.21.15.123:0/1870450624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bec06bb40 0x7f5bec110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:46.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.198+0000 7f5bf3c00700 1 --2- 172.21.15.123:0/1870450624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bec109510 0x7f5bec1098f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:46.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.198+0000 7f5bf3c00700 1 --2- 172.21.15.123:0/1870450624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bec06b180 0x7f5bec06b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:46.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.198+0000 7f5bf3c00700 1 -- 172.21.15.123:0/1870450624 >> 172.21.15.123:0/1870450624 conn(0x7f5bec079350 msgr2=0x7f5bec079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:46.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.198+0000 7f5bf3c00700 1 -- 172.21.15.123:0/1870450624 shutdown_connections 2024-05-08T06:12:46.200 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.199+0000 7f5bf3c00700 1 -- 172.21.15.123:0/1870450624 wait complete. 2024-05-08T06:12:46.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.200+0000 7f5bf3c00700 1 Processor -- start 2024-05-08T06:12:46.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.200+0000 7f5bf3c00700 1 -- start start 2024-05-08T06:12:46.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.200+0000 7f5bf3c00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bec06b180 0x7f5bec19f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:46.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.200+0000 7f5bf3c00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bec06bb40 0x7f5bec19fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:46.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.200+0000 7f5bf3c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bec109510 0x7f5bec199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:46.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.200+0000 7f5bf3c00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec113670 con 0x7f5bec109510 2024-05-08T06:12:46.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.200+0000 7f5bf3c00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec1134f0 con 0x7f5bec06b180 2024-05-08T06:12:46.201 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.200+0000 7f5bf3c00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec1137f0 con 0x7f5bec06bb40 2024-05-08T06:12:46.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.201+0000 7f5bf1200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bec06b180 0x7f5bec19f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:46.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.201+0000 7f5bf1200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bec06b180 0x7f5bec19f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:48898/0 (socket says 172.21.15.123:48898) 2024-05-08T06:12:46.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.201+0000 7f5bebe00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bec06bb40 0x7f5bec19fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:46.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.201+0000 7f5bf1200700 1 -- 172.21.15.123:0/232322720 learned_addr learned my addr 172.21.15.123:0/232322720 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:46.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.201+0000 7f5bf1c00700 1 --2- 172.21.15.123:0/232322720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bec109510 0x7f5bec199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:46.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.201+0000 7f5bf1200700 1 -- 172.21.15.123:0/232322720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bec06bb40 msgr2=0x7f5bec19fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:46.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.201+0000 7f5bf1200700 1 --2- 172.21.15.123:0/232322720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bec06bb40 0x7f5bec19fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:46.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.201+0000 7f5bf1200700 1 -- 172.21.15.123:0/232322720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bec109510 msgr2=0x7f5bec199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:46.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.201+0000 7f5bf1200700 1 --2- 172.21.15.123:0/232322720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bec109510 0x7f5bec199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:46.202 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.201+0000 7f5bf1200700 1 -- 172.21.15.123:0/232322720 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bec19a190 con 0x7f5bec06b180 2024-05-08T06:12:46.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.202+0000 7f5bf1200700 1 --2- 172.21.15.123:0/232322720 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bec06b180 0x7f5bec19f870 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5be0009230 tx=0x7f5be000fa70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:46.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.202+0000 7f5be9600700 1 -- 172.21.15.123:0/232322720 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be0005140 con 0x7f5bec06b180 2024-05-08T06:12:46.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.202+0000 7f5be9600700 1 -- 172.21.15.123:0/232322720 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5be000b560 con 0x7f5bec06b180 2024-05-08T06:12:46.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.202+0000 7f5be9600700 1 -- 172.21.15.123:0/232322720 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be0003970 con 0x7f5bec06b180 2024-05-08T06:12:46.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.202+0000 7f5bf3c00700 1 -- 172.21.15.123:0/232322720 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bec19a420 con 0x7f5bec06b180 2024-05-08T06:12:46.203 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.202+0000 7f5bf3c00700 1 -- 172.21.15.123:0/232322720 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bec1023a0 con 0x7f5bec06b180 2024-05-08T06:12:46.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.206+0000 7f5be9600700 1 -- 172.21.15.123:0/232322720 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7f5be000ba50 con 0x7f5bec06b180 2024-05-08T06:12:46.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.207+0000 7f5be9600700 1 --2- 172.21.15.123:0/232322720 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5bd40b2900 0x7f5bd40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:46.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.207+0000 7f5bfe621b80 1 Processor -- start 2024-05-08T06:12:46.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.207+0000 7f5be9600700 1 -- 172.21.15.123:0/232322720 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f5be00e3130 con 0x7f5bec06b180 2024-05-08T06:12:46.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.207+0000 7f5bebe00700 1 --2- 172.21.15.123:0/232322720 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5bd40b2900 0x7f5bd40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:46.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.208+0000 7f5bfe621b80 1 -- start start 2024-05-08T06:12:46.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.208+0000 7f5bfe621b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd64ba8cc0 0x55dd64ba90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:46.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.208+0000 7f5bfe621b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dd64ba95e0 0x55dd64bacfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:46.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.208+0000 7f5bfe621b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd64bad4f0 0x55dd64bad910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:46.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.208+0000 7f5bfe621b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55dd64af2a20 con 0x55dd64ba8cc0 2024-05-08T06:12:46.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.208+0000 7f5bfe621b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55dd64a90760 con 0x55dd64ba95e0 2024-05-08T06:12:46.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.208+0000 7f5bfe621b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55dd64aee9e0 con 0x55dd64bad4f0 2024-05-08T06:12:46.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.208+0000 7f5bf1c00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd64bad4f0 0x55dd64bad910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:46.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.208+0000 7f5bf1200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd64ba8cc0 0x55dd64ba90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:46.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.208+0000 7f5bebe00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dd64ba95e0 0x55dd64bacfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:46.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.208+0000 7f5bf1200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd64ba8cc0 0x55dd64ba90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34016/0 (socket says 172.21.15.123:34016) 2024-05-08T06:12:46.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.208+0000 7f5bf1200700 1 -- 172.21.15.123:0/2928943264 learned_addr learned my addr 172.21.15.123:0/2928943264 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:46.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.208+0000 7f5bf1c00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd64bad4f0 0x55dd64bad910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:41646/0 (socket says 172.21.15.123:41646) 2024-05-08T06:12:46.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.209+0000 7f5bebe00700 1 --2- 172.21.15.123:0/232322720 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5bd40b2900 0x7f5bd40b4dc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5bec1085a0 tx=0x7f5bd8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:46.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.209+0000 7f5bf1200700 1 -- 172.21.15.123:0/2928943264 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd64bad4f0 msgr2=0x55dd64bad910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:46.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.209+0000 7f5bf1200700 1 --2- 172.21.15.123:0/2928943264 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd64bad4f0 0x55dd64bad910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:46.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.209+0000 7f5bf1200700 1 -- 172.21.15.123:0/2928943264 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dd64ba95e0 msgr2=0x55dd64bacfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:46.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.209+0000 7f5bf1200700 1 --2- 172.21.15.123:0/2928943264 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dd64ba95e0 0x55dd64bacfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:46.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.209+0000 7f5bf1200700 1 -- 172.21.15.123:0/2928943264 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd64a3f8d0 con 0x55dd64ba8cc0 2024-05-08T06:12:46.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.209+0000 7f5bf1c00700 1 --2- 172.21.15.123:0/2928943264 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd64bad4f0 0x55dd64bad910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:12:46.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.209+0000 7f5bebe00700 1 --2- 172.21.15.123:0/2928943264 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dd64ba95e0 0x55dd64bacfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:12:46.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.209+0000 7f5bf1200700 1 --2- 172.21.15.123:0/2928943264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd64ba8cc0 0x55dd64ba90a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5be00aa490 tx=0x7f5be00aa690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd20b2a4a6e18602 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:46.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.209+0000 7f5bde000700 1 -- 172.21.15.123:0/2928943264 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be00aae90 con 0x55dd64ba8cc0 2024-05-08T06:12:46.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.210+0000 7f5bde000700 1 -- 172.21.15.123:0/2928943264 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5be00aa790 con 0x55dd64ba8cc0 2024-05-08T06:12:46.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.210+0000 7f5bde000700 1 -- 172.21.15.123:0/2928943264 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be000ec90 con 0x55dd64ba8cc0 2024-05-08T06:12:46.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.210+0000 7f5bfe621b80 1 -- 172.21.15.123:0/2928943264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd64ba8cc0 msgr2=0x55dd64ba90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:46.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.210+0000 7f5bfe621b80 1 --2- 172.21.15.123:0/2928943264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd64ba8cc0 0x55dd64ba90a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5be00aa490 tx=0x7f5be00aa690 comp rx=0 tx=0).stop 2024-05-08T06:12:46.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.211+0000 7f5bfe621b80 1 -- 172.21.15.123:0/2928943264 shutdown_connections 2024-05-08T06:12:46.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.211+0000 7f5bfe621b80 1 --2- 172.21.15.123:0/2928943264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd64ba8cc0 0x55dd64ba90a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:46.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.211+0000 7f5bfe621b80 1 --2- 172.21.15.123:0/2928943264 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd64bad4f0 0x55dd64bad910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:46.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.211+0000 7f5bfe621b80 1 --2- 172.21.15.123:0/2928943264 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dd64ba95e0 0x55dd64bacfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:46.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.211+0000 7f5bfe621b80 1 -- 172.21.15.123:0/2928943264 >> 172.21.15.123:0/2928943264 conn(0x55dd64b186d0 msgr2=0x55dd64b9ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:46.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.211+0000 7f5bfe621b80 1 -- 172.21.15.123:0/2928943264 shutdown_connections 2024-05-08T06:12:46.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.211+0000 7f5bfe621b80 1 -- 172.21.15.123:0/2928943264 wait complete. 2024-05-08T06:12:46.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.211+0000 7f5bfe621b80 1 -- start start 2024-05-08T06:12:46.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.212+0000 7f5bfe621b80 1 Processor -- start 2024-05-08T06:12:46.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.212+0000 7f5bfe621b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd64ba8cc0 0x55dd64bb4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:46.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.212+0000 7f5bfe621b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd64ba95e0 0x55dd64bb1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:46.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.212+0000 7f5bfe621b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dd64bad4f0 0x55dd64bb2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:46.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.212+0000 7f5bfe621b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55dd64bb54e0 con 0x55dd64ba8cc0 2024-05-08T06:12:46.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.212+0000 7f5bfe621b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55dd64a58b20 con 0x55dd64bad4f0 2024-05-08T06:12:46.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.212+0000 7f5bfe621b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55dd64bb5660 con 0x55dd64ba95e0 2024-05-08T06:12:46.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.212+0000 7f5bfe621b80 10 client.0 _ll_register_callbacks cb 0x55dd64a99c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:12:46.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.212+0000 7f5bebe00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd64ba95e0 0x55dd64bb1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:46.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.213+0000 7f5bebe00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd64ba95e0 0x55dd64bb1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:41652/0 (socket says 172.21.15.123:41652) 2024-05-08T06:12:46.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.213+0000 7f5bebe00700 1 -- 172.21.15.123:0/2950714961 learned_addr learned my addr 172.21.15.123:0/2950714961 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:46.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.213+0000 7f5bf1200700 1 --2- 172.21.15.123:0/2950714961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd64ba8cc0 0x55dd64bb4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:46.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.213+0000 7f5bf1c00700 1 --2- 172.21.15.123:0/2950714961 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dd64bad4f0 0x55dd64bb2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:46.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.213+0000 7f5bebe00700 1 -- 172.21.15.123:0/2950714961 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dd64bad4f0 msgr2=0x55dd64bb2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:46.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.213+0000 7f5bebe00700 1 --2- 172.21.15.123:0/2950714961 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dd64bad4f0 0x55dd64bb2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:46.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.213+0000 7f5bebe00700 1 -- 172.21.15.123:0/2950714961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd64ba8cc0 msgr2=0x55dd64bb4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:46.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.213+0000 7f5bebe00700 1 --2- 172.21.15.123:0/2950714961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd64ba8cc0 0x55dd64bb4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:46.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.213+0000 7f5bebe00700 1 -- 172.21.15.123:0/2950714961 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd64ab8420 con 0x55dd64ba95e0 2024-05-08T06:12:46.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.213+0000 7f5bebe00700 1 --2- 172.21.15.123:0/2950714961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd64ba95e0 0x55dd64bb1ce0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8003930 tx=0x7f5bd80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20c57c5287b4cc43 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:46.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.213+0000 7f5bd2a00700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-05-08T06:12:46.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.214+0000 7f5bd2a00700 1 -- 172.21.15.123:0/2950714961 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd801f840 con 0x55dd64ba95e0 2024-05-08T06:12:46.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.214+0000 7f5bfe621b80 1 -- 172.21.15.123:0/2950714961 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5be00aa790 con 0x55dd64ba95e0 2024-05-08T06:12:46.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.214+0000 7f5bd2a00700 1 -- 172.21.15.123:0/2950714961 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f5bd801f9e0 con 0x55dd64ba95e0 2024-05-08T06:12:46.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.214+0000 7f5bd2a00700 1 -- 172.21.15.123:0/2950714961 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5bd801fc20 con 0x55dd64ba95e0 2024-05-08T06:12:46.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.214+0000 7f5bfe621b80 10 client.25627 fetch_fsmap learned FSMap version 24 2024-05-08T06:12:46.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.214+0000 7f5bfe621b80 1 -- 172.21.15.123:0/2950714961 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55dd64bbfca0 con 0x55dd64ba95e0 2024-05-08T06:12:46.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.214+0000 7f5bd2a00700 1 -- 172.21.15.123:0/2950714961 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5be00aa790 con 0x55dd64ba95e0 2024-05-08T06:12:46.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.215+0000 7f5bd2a00700 1 -- 172.21.15.123:0/2950714961 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f5bd8005440 con 0x55dd64ba95e0 2024-05-08T06:12:46.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.215+0000 7f5bfe621b80 10 client.25627 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:12:46.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.215+0000 7f5bfe621b80 10 client.25627 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-08T06:12:46.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.215+0000 7f5bfe621b80 20 client.25627 populate_metadata read hostname 'smithi123' 2024-05-08T06:12:46.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.215+0000 7f5bfe621b80 1 --2- 172.21.15.123:0/2950714961 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x55dd64bc22e0 0x55dd64bc46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:46.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.215+0000 7f5bfe621b80 4 client.25627 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:12:46.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.215+0000 7f5bfe621b80 1 -- 172.21.15.123:0/2950714961 --> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd64b6d140 con 0x55dd64bc22e0 2024-05-08T06:12:46.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.215+0000 7f5bf1200700 1 --2- 172.21.15.123:0/2950714961 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x55dd64bc22e0 0x55dd64bc46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:46.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.216+0000 7f5bf1200700 1 --2- 172.21.15.123:0/2950714961 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x55dd64bc22e0 0x55dd64bc46d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cf38248a20f092ef server_cookie=e5469dfbe9451a83 in_seq=0 out_seq=0 2024-05-08T06:12:46.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.216+0000 7f5bd2a00700 10 client.25627 ms_handle_connect on v2:172.21.15.123:6840/1571726231 2024-05-08T06:12:46.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.216+0000 7f5bd2a00700 1 -- 172.21.15.123:0/2950714961 <== mds.1 v2:172.21.15.123:6840/1571726231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dd64b6d140 con 0x55dd64bc22e0 2024-05-08T06:12:46.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.216+0000 7f5bd2a00700 10 client.25627 handle_command_reply: tid=0 2024-05-08T06:12:46.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.217+0000 7f5bfe621b80 1 client.25627 shutdown 2024-05-08T06:12:46.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.217+0000 7f5bfe621b80 1 -- 172.21.15.123:0/2950714961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd64ba95e0 msgr2=0x55dd64bb1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:46.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.217+0000 7f5bfe621b80 1 --2- 172.21.15.123:0/2950714961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd64ba95e0 0x55dd64bb1ce0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8003930 tx=0x7f5bd80047c0 comp rx=0 tx=0).stop 2024-05-08T06:12:46.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.217+0000 7f5bfe621b80 1 -- 172.21.15.123:0/2950714961 shutdown_connections 2024-05-08T06:12:46.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.217+0000 7f5bfe621b80 1 --2- 172.21.15.123:0/2950714961 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x55dd64bc22e0 0x55dd64bc46d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:46.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.217+0000 7f5bfe621b80 1 --2- 172.21.15.123:0/2950714961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd64ba8cc0 0x55dd64bb4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:46.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.217+0000 7f5bfe621b80 1 --2- 172.21.15.123:0/2950714961 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dd64bad4f0 0x55dd64bb2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:46.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.217+0000 7f5bfe621b80 1 --2- 172.21.15.123:0/2950714961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd64ba95e0 0x55dd64bb1ce0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:46.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.217+0000 7f5bfe621b80 1 -- 172.21.15.123:0/2950714961 >> 172.21.15.123:0/2950714961 conn(0x55dd64b186d0 msgr2=0x55dd64b9a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:46.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.217+0000 7f5bd2a00700 0 client.25627 ms_handle_reset on v2:172.21.15.123:6840/1571726231 2024-05-08T06:12:46.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.217+0000 7f5bfe621b80 1 -- 172.21.15.123:0/2950714961 shutdown_connections 2024-05-08T06:12:46.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.217+0000 7f5bfe621b80 1 -- 172.21.15.123:0/2950714961 wait complete. 2024-05-08T06:12:46.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.217+0000 7f5bfe621b80 20 client.25627 trim_cache size 0 max 16384 2024-05-08T06:12:46.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.235+0000 7f5bfe621b80 1 Processor -- start 2024-05-08T06:12:46.235 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.235+0000 7f5bfe621b80 1 -- start start 2024-05-08T06:12:46.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.235+0000 7f5bfe621b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd64ba8cc0 0x55dd64bbe530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:46.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.235+0000 7f5bfe621b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dd64ba95e0 0x55dd64bc46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:46.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.235+0000 7f5bfe621b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd64bad4f0 0x55dd64baf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:46.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.235+0000 7f5bfe621b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec1ae210 con 0x55dd64bad4f0 2024-05-08T06:12:46.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.235+0000 7f5bfe621b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec1abaa0 con 0x55dd64ba95e0 2024-05-08T06:12:46.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.235+0000 7f5bfe621b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec1a98f0 con 0x55dd64ba8cc0 2024-05-08T06:12:46.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.236+0000 7f5bf1200700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd64ba8cc0 0x55dd64bbe530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:46.236 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.236+0000 7f5bf1c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd64bad4f0 0x55dd64baf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:46.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.236+0000 7f5bf1200700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd64ba8cc0 0x55dd64bbe530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:41656/0 (socket says 172.21.15.123:41656) 2024-05-08T06:12:46.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.236+0000 7f5bf1200700 1 -- 172.21.15.123:0/2515171680 learned_addr learned my addr 172.21.15.123:0/2515171680 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:46.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.236+0000 7f5bebe00700 1 --2- 172.21.15.123:0/2515171680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dd64ba95e0 0x55dd64bc46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:46.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.236+0000 7f5bf1200700 1 -- 172.21.15.123:0/2515171680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dd64ba95e0 msgr2=0x55dd64bc46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:46.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.236+0000 7f5bf1200700 1 --2- 172.21.15.123:0/2515171680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dd64ba95e0 0x55dd64bc46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:46.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.236+0000 7f5bf1200700 1 -- 172.21.15.123:0/2515171680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd64bad4f0 msgr2=0x55dd64baf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:46.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.236+0000 7f5bf1200700 1 --2- 172.21.15.123:0/2515171680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd64bad4f0 0x55dd64baf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:46.237 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.236+0000 7f5bf1200700 1 -- 172.21.15.123:0/2515171680 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd64bb0250 con 0x55dd64ba8cc0 2024-05-08T06:12:46.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.237+0000 7f5bf1200700 1 --2- 172.21.15.123:0/2515171680 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd64ba8cc0 0x55dd64bbe530 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5be0009810 tx=0x7f5be0019040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fb7b687ad82bd80 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:46.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.237+0000 7f5bdd600700 1 -- 172.21.15.123:0/2515171680 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be00a9f00 con 0x55dd64ba8cc0 2024-05-08T06:12:46.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.237+0000 7f5bdd600700 1 -- 172.21.15.123:0/2515171680 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5be00aa790 con 0x55dd64ba8cc0 2024-05-08T06:12:46.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.237+0000 7f5bdd600700 1 -- 172.21.15.123:0/2515171680 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be00aa9f0 con 0x55dd64ba8cc0 2024-05-08T06:12:46.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.238+0000 7f5bfe621b80 1 -- 172.21.15.123:0/2515171680 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd64ba8cc0 msgr2=0x55dd64bbe530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:46.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.238+0000 7f5bfe621b80 1 --2- 172.21.15.123:0/2515171680 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd64ba8cc0 0x55dd64bbe530 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5be0009810 tx=0x7f5be0019040 comp rx=0 tx=0).stop 2024-05-08T06:12:46.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.238+0000 7f5bfe621b80 1 -- 172.21.15.123:0/2515171680 shutdown_connections 2024-05-08T06:12:46.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.238+0000 7f5bfe621b80 1 --2- 172.21.15.123:0/2515171680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd64bad4f0 0x55dd64baf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:46.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.238+0000 7f5bfe621b80 1 --2- 172.21.15.123:0/2515171680 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd64ba8cc0 0x55dd64bbe530 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:46.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.238+0000 7f5bfe621b80 1 --2- 172.21.15.123:0/2515171680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dd64ba95e0 0x55dd64bc46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:46.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.238+0000 7f5bfe621b80 1 -- 172.21.15.123:0/2515171680 >> 172.21.15.123:0/2515171680 conn(0x55dd64b186d0 msgr2=0x55dd64baf1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:46.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.238+0000 7f5bfe621b80 1 -- 172.21.15.123:0/2515171680 shutdown_connections 2024-05-08T06:12:46.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.238+0000 7f5bfe621b80 1 -- 172.21.15.123:0/2515171680 wait complete. 2024-05-08T06:12:46.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.238+0000 7f5bfe621b80 1 -- start start 2024-05-08T06:12:46.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.239+0000 7f5bfe621b80 1 Processor -- start 2024-05-08T06:12:46.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.239+0000 7f5bfe621b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd64ba8cc0 0x55dd64bbacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:46.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.239+0000 7f5bfe621b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd64ba95e0 0x55dd64baaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:46.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.239+0000 7f5bfe621b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dd64bad4f0 0x55dd64bab4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:46.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.239+0000 7f5bfe621b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec1a9f60 con 0x55dd64ba95e0 2024-05-08T06:12:46.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.239+0000 7f5bfe621b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec1ab3d0 con 0x55dd64bad4f0 2024-05-08T06:12:46.241 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.239+0000 7f5bfe621b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec1aa630 con 0x55dd64ba8cc0 2024-05-08T06:12:46.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.239+0000 7f5bfe621b80 10 client.0 _ll_register_callbacks cb 0x55dd64a99c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:12:46.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.239+0000 7f5bf1200700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd64ba8cc0 0x55dd64bbacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:46.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.239+0000 7f5bf1200700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd64ba8cc0 0x55dd64bbacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:41664/0 (socket says 172.21.15.123:41664) 2024-05-08T06:12:46.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.239+0000 7f5bf1200700 1 -- 172.21.15.123:0/806180495 learned_addr learned my addr 172.21.15.123:0/806180495 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:46.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.240+0000 7f5bf1c00700 1 --2- 172.21.15.123:0/806180495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dd64bad4f0 0x55dd64bab4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:46.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.240+0000 7f5bf1200700 1 -- 172.21.15.123:0/806180495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dd64bad4f0 msgr2=0x55dd64bab4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:46.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.240+0000 7f5bebe00700 1 --2- 172.21.15.123:0/806180495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd64ba95e0 0x55dd64baaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:46.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.240+0000 7f5bf1200700 1 --2- 172.21.15.123:0/806180495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dd64bad4f0 0x55dd64bab4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:46.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.240+0000 7f5bf1200700 1 -- 172.21.15.123:0/806180495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd64ba95e0 msgr2=0x55dd64baaf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:46.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.240+0000 7f5bf1200700 1 --2- 172.21.15.123:0/806180495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd64ba95e0 0x55dd64baaf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:46.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.240+0000 7f5bf1200700 1 -- 172.21.15.123:0/806180495 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd64babc10 con 0x55dd64ba8cc0 2024-05-08T06:12:46.242 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.240+0000 7f5bebe00700 1 --2- 172.21.15.123:0/806180495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd64ba95e0 0x55dd64baaf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:12:46.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.240+0000 7f5bf1200700 1 --2- 172.21.15.123:0/806180495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd64ba8cc0 0x55dd64bbacf0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5be00a7a00 tx=0x7f5be00aa690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f51bba5d8a658515 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:46.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.240+0000 7f5bd2a00700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-05-08T06:12:46.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.241+0000 7f5bd2a00700 1 -- 172.21.15.123:0/806180495 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be00aae90 con 0x55dd64ba8cc0 2024-05-08T06:12:46.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.241+0000 7f5bd2a00700 1 -- 172.21.15.123:0/806180495 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f5be00a6a20 con 0x55dd64ba8cc0 2024-05-08T06:12:46.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.241+0000 7f5bfe621b80 1 -- 172.21.15.123:0/806180495 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5be00aa790 con 0x55dd64ba8cc0 2024-05-08T06:12:46.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.241+0000 7f5bd2a00700 1 -- 172.21.15.123:0/806180495 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5be000ec90 con 0x55dd64ba8cc0 2024-05-08T06:12:46.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.241+0000 7f5bfe621b80 10 client.25633 fetch_fsmap learned FSMap version 24 2024-05-08T06:12:46.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.241+0000 7f5bfe621b80 1 -- 172.21.15.123:0/806180495 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55dd64babc10 con 0x55dd64ba8cc0 2024-05-08T06:12:46.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.241+0000 7f5bd2a00700 1 -- 172.21.15.123:0/806180495 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5be00aa790 con 0x55dd64ba8cc0 2024-05-08T06:12:46.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.241+0000 7f5bd2a00700 1 -- 172.21.15.123:0/806180495 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f5be00b30d0 con 0x55dd64ba8cc0 2024-05-08T06:12:46.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.241+0000 7f5bfe621b80 10 client.25633 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:12:46.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.241+0000 7f5bfe621b80 10 client.25633 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-08T06:12:46.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.241+0000 7f5bfe621b80 20 client.25633 populate_metadata read hostname 'smithi123' 2024-05-08T06:12:46.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.241+0000 7f5bfe621b80 1 --2- 172.21.15.123:0/806180495 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x55dd64c2cc50 0x55dd64c2f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:46.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.241+0000 7f5bfe621b80 4 client.25633 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T06:12:46.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.241+0000 7f5bfe621b80 1 -- 172.21.15.123:0/806180495 --> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dd64b00400 con 0x55dd64c2cc50 2024-05-08T06:12:46.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.242+0000 7f5bebe00700 1 --2- 172.21.15.123:0/806180495 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x55dd64c2cc50 0x55dd64c2f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:46.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.242+0000 7f5bebe00700 1 --2- 172.21.15.123:0/806180495 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x55dd64c2cc50 0x55dd64c2f040 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=994a30aec65e085e server_cookie=8af991e7e1d03f29 in_seq=0 out_seq=0 2024-05-08T06:12:46.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.242+0000 7f5bd2a00700 10 client.25633 ms_handle_connect on v2:172.21.15.123:6840/1571726231 2024-05-08T06:12:46.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.243+0000 7f5bd2a00700 1 -- 172.21.15.123:0/806180495 <== mds.1 v2:172.21.15.123:6840/1571726231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dd64b00400 con 0x55dd64c2cc50 2024-05-08T06:12:46.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.243+0000 7f5bd2a00700 10 client.25633 handle_command_reply: tid=0 2024-05-08T06:12:46.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.243+0000 7f5bfe621b80 1 client.25633 shutdown 2024-05-08T06:12:46.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.243+0000 7f5bfe621b80 1 -- 172.21.15.123:0/806180495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd64ba8cc0 msgr2=0x55dd64bbacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:46.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.243+0000 7f5bfe621b80 1 --2- 172.21.15.123:0/806180495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd64ba8cc0 0x55dd64bbacf0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5be00a7a00 tx=0x7f5be00aa690 comp rx=0 tx=0).stop 2024-05-08T06:12:46.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.243+0000 7f5bfe621b80 1 -- 172.21.15.123:0/806180495 shutdown_connections 2024-05-08T06:12:46.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.243+0000 7f5bfe621b80 1 --2- 172.21.15.123:0/806180495 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x55dd64c2cc50 0x55dd64c2f040 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:46.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.243+0000 7f5bfe621b80 1 --2- 172.21.15.123:0/806180495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dd64bad4f0 0x55dd64bab4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:46.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.243+0000 7f5bfe621b80 1 --2- 172.21.15.123:0/806180495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55dd64ba8cc0 0x55dd64bbacf0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:46.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.243+0000 7f5bfe621b80 1 --2- 172.21.15.123:0/806180495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd64ba95e0 0x55dd64baaf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:46.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.243+0000 7f5bd2a00700 0 client.25633 ms_handle_reset on v2:172.21.15.123:6840/1571726231 2024-05-08T06:12:46.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.243+0000 7f5bfe621b80 1 -- 172.21.15.123:0/806180495 >> 172.21.15.123:0/806180495 conn(0x55dd64b186d0 msgr2=0x55dd64bb32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:46.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.243+0000 7f5bfe621b80 1 -- 172.21.15.123:0/806180495 shutdown_connections 2024-05-08T06:12:46.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.243+0000 7f5bfe621b80 1 -- 172.21.15.123:0/806180495 wait complete. 2024-05-08T06:12:46.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.244+0000 7f5bfe621b80 20 client.25633 trim_cache size 0 max 16384 2024-05-08T06:12:46.246 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-05-08T06:12:46.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.244+0000 7f5bf3c00700 1 -- 172.21.15.123:0/232322720 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5bd40b2900 msgr2=0x7f5bd40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:46.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.244+0000 7f5bf3c00700 1 --2- 172.21.15.123:0/232322720 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5bd40b2900 0x7f5bd40b4dc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5bec1085a0 tx=0x7f5bd8009310 comp rx=0 tx=0).stop 2024-05-08T06:12:46.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.244+0000 7f5bf3c00700 1 -- 172.21.15.123:0/232322720 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bec06b180 msgr2=0x7f5bec19f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:46.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.244+0000 7f5bf3c00700 1 --2- 172.21.15.123:0/232322720 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bec06b180 0x7f5bec19f870 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5be0009230 tx=0x7f5be000fa70 comp rx=0 tx=0).stop 2024-05-08T06:12:46.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.245+0000 7f5bf3c00700 1 -- 172.21.15.123:0/232322720 shutdown_connections 2024-05-08T06:12:46.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.245+0000 7f5bf3c00700 1 --2- 172.21.15.123:0/232322720 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5bd40b2900 0x7f5bd40b4dc0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:46.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.245+0000 7f5bf3c00700 1 --2- 172.21.15.123:0/232322720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bec109510 0x7f5bec199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:46.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.245+0000 7f5bf3c00700 1 --2- 172.21.15.123:0/232322720 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bec06b180 0x7f5bec19f870 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:46.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.245+0000 7f5bf3c00700 1 --2- 172.21.15.123:0/232322720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bec06bb40 0x7f5bec19fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:46.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.245+0000 7f5bf3c00700 1 -- 172.21.15.123:0/232322720 >> 172.21.15.123:0/232322720 conn(0x7f5bec079350 msgr2=0x7f5bec101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:46.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.245+0000 7f5bf3c00700 1 -- 172.21.15.123:0/232322720 shutdown_connections 2024-05-08T06:12:46.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:46.245+0000 7f5bf3c00700 1 -- 172.21.15.123:0/232322720 wait complete. 2024-05-08T06:12:46.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:46 smithi123 ceph-mon[29787]: pgmap v529: 97 pgs: 97 active+clean; 199 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 70 MiB/s rd, 798 KiB/s wr, 258 op/s 2024-05-08T06:12:47.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:46 smithi163 ceph-mon[34520]: pgmap v529: 97 pgs: 97 active+clean; 199 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 70 MiB/s rd, 798 KiB/s wr, 258 op/s 2024-05-08T06:12:47.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:46 smithi192 ceph-mon[34664]: pgmap v529: 97 pgs: 97 active+clean; 199 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 70 MiB/s rd, 798 KiB/s wr, 258 op/s 2024-05-08T06:12:47.303 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-08T06:12:47.754 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4191744 len 8192 2024-05-08T06:12:47.755 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4191744 len 16384 2024-05-08T06:12:47.755 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4191744 len 16384 2024-05-08T06:12:47.755 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4191744 len 16384 2024-05-08T06:12:47.755 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4191744 len 16384 2024-05-08T06:12:47.755 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4192256 len 1024 2024-05-08T06:12:47.755 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4192256 len 1024 2024-05-08T06:12:47.755 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4192256 len 1024 2024-05-08T06:12:47.755 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4192256 len 1024 2024-05-08T06:12:47.755 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4192256 len 2048 2024-05-08T06:12:47.755 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4192256 len 2048 2024-05-08T06:12:47.755 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4192256 len 2048 2024-05-08T06:12:47.755 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4192256 len 2048 2024-05-08T06:12:47.756 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4192256 len 4096 2024-05-08T06:12:47.756 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4192256 len 4096 2024-05-08T06:12:47.756 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4192256 len 4096 2024-05-08T06:12:47.756 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4192256 len 4096 2024-05-08T06:12:47.756 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4192256 len 8192 2024-05-08T06:12:47.756 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4192256 len 8192 2024-05-08T06:12:47.756 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4192256 len 8192 2024-05-08T06:12:47.756 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4192256 len 8192 2024-05-08T06:12:47.756 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4192256 len 16384 2024-05-08T06:12:47.756 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4192256 len 16384 2024-05-08T06:12:47.756 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4192256 len 16384 2024-05-08T06:12:47.756 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4192256 len 16384 2024-05-08T06:12:47.756 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4192768 len 1024 2024-05-08T06:12:47.756 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4192768 len 1024 2024-05-08T06:12:47.757 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4192768 len 1024 2024-05-08T06:12:47.757 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4192768 len 1024 2024-05-08T06:12:47.757 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4192768 len 2048 2024-05-08T06:12:47.757 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4192768 len 2048 2024-05-08T06:12:47.757 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4192768 len 2048 2024-05-08T06:12:47.757 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4192768 len 2048 2024-05-08T06:12:47.757 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4192768 len 4096 2024-05-08T06:12:47.757 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4192768 len 4096 2024-05-08T06:12:47.757 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4192768 len 4096 2024-05-08T06:12:47.757 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4192768 len 4096 2024-05-08T06:12:47.757 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4192768 len 8192 2024-05-08T06:12:47.757 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4192768 len 8192 2024-05-08T06:12:47.757 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4192768 len 8192 2024-05-08T06:12:47.757 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4192768 len 8192 2024-05-08T06:12:47.757 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4192768 len 16384 2024-05-08T06:12:47.758 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4192768 len 16384 2024-05-08T06:12:47.758 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4192768 len 16384 2024-05-08T06:12:47.758 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4192768 len 16384 2024-05-08T06:12:47.758 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4193280 len 1024 2024-05-08T06:12:47.758 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4193280 len 1024 2024-05-08T06:12:47.758 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4193280 len 1024 2024-05-08T06:12:47.758 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4193280 len 1024 2024-05-08T06:12:47.758 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4193280 len 2048 2024-05-08T06:12:47.758 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4193280 len 2048 2024-05-08T06:12:47.758 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4193280 len 2048 2024-05-08T06:12:47.758 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4193280 len 2048 2024-05-08T06:12:47.758 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4193280 len 4096 2024-05-08T06:12:47.758 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4193280 len 4096 2024-05-08T06:12:47.758 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4193280 len 4096 2024-05-08T06:12:47.759 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4193280 len 4096 2024-05-08T06:12:47.759 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4193280 len 8192 2024-05-08T06:12:47.759 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4193280 len 8192 2024-05-08T06:12:47.759 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4193280 len 8192 2024-05-08T06:12:47.759 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4193280 len 8192 2024-05-08T06:12:47.759 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4193280 len 16384 2024-05-08T06:12:47.759 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4193280 len 16384 2024-05-08T06:12:47.759 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4193280 len 16384 2024-05-08T06:12:47.759 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4193280 len 16384 2024-05-08T06:12:47.759 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4193792 len 1024 2024-05-08T06:12:47.759 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4193792 len 1024 2024-05-08T06:12:47.759 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4193792 len 1024 2024-05-08T06:12:47.759 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4193792 len 1024 2024-05-08T06:12:47.759 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4193792 len 2048 2024-05-08T06:12:47.759 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4193792 len 2048 2024-05-08T06:12:47.760 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4193792 len 2048 2024-05-08T06:12:47.760 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4193792 len 2048 2024-05-08T06:12:47.760 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4193792 len 4096 2024-05-08T06:12:47.760 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4193792 len 4096 2024-05-08T06:12:47.760 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4193792 len 4096 2024-05-08T06:12:47.760 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4193792 len 4096 2024-05-08T06:12:47.760 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4193792 len 8192 2024-05-08T06:12:47.760 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4193792 len 8192 2024-05-08T06:12:47.760 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4193792 len 8192 2024-05-08T06:12:47.760 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4193792 len 8192 2024-05-08T06:12:47.770 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:12:48.258 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4193792 len 16384 2024-05-08T06:12:48.258 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4193792 len 16384 2024-05-08T06:12:48.258 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4193792 len 16384 2024-05-08T06:12:48.258 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4193792 len 16384 2024-05-08T06:12:48.258 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4194304 len 1024 2024-05-08T06:12:48.258 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4194304 len 1024 2024-05-08T06:12:48.259 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4194304 len 1024 2024-05-08T06:12:48.259 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4194304 len 1024 2024-05-08T06:12:48.259 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4194304 len 2048 2024-05-08T06:12:48.259 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4194304 len 2048 2024-05-08T06:12:48.259 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4194304 len 2048 2024-05-08T06:12:48.259 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4194304 len 2048 2024-05-08T06:12:48.259 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4194304 len 4096 2024-05-08T06:12:48.259 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4194304 len 4096 2024-05-08T06:12:48.259 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4194304 len 4096 2024-05-08T06:12:48.259 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4194304 len 4096 2024-05-08T06:12:48.259 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4194304 len 8192 2024-05-08T06:12:48.259 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4194304 len 8192 2024-05-08T06:12:48.259 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4194304 len 8192 2024-05-08T06:12:48.259 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4194304 len 8192 2024-05-08T06:12:48.259 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4194304 len 16384 2024-05-08T06:12:48.260 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4194304 len 16384 2024-05-08T06:12:48.260 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4194304 len 16384 2024-05-08T06:12:48.260 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4194304 len 16384 2024-05-08T06:12:48.260 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4194816 len 1024 2024-05-08T06:12:48.260 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4194816 len 1024 2024-05-08T06:12:48.260 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4194816 len 1024 2024-05-08T06:12:48.260 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4194816 len 1024 2024-05-08T06:12:48.260 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4194816 len 2048 2024-05-08T06:12:48.260 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4194816 len 2048 2024-05-08T06:12:48.260 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4194816 len 2048 2024-05-08T06:12:48.260 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4194816 len 2048 2024-05-08T06:12:48.260 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4194816 len 4096 2024-05-08T06:12:48.260 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4194816 len 4096 2024-05-08T06:12:48.260 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4194816 len 4096 2024-05-08T06:12:48.260 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4194816 len 4096 2024-05-08T06:12:48.260 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4194816 len 8192 2024-05-08T06:12:48.261 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4194816 len 8192 2024-05-08T06:12:48.261 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4194816 len 8192 2024-05-08T06:12:48.261 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4194816 len 8192 2024-05-08T06:12:48.261 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4194816 len 16384 2024-05-08T06:12:48.261 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4194816 len 16384 2024-05-08T06:12:48.261 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4194816 len 16384 2024-05-08T06:12:48.261 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4194816 len 16384 2024-05-08T06:12:48.261 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4195328 len 1024 2024-05-08T06:12:48.261 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4195328 len 1024 2024-05-08T06:12:48.261 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4195328 len 1024 2024-05-08T06:12:48.261 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4195328 len 1024 2024-05-08T06:12:48.261 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4195328 len 2048 2024-05-08T06:12:48.261 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4195328 len 2048 2024-05-08T06:12:48.261 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4195328 len 2048 2024-05-08T06:12:48.261 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4195328 len 2048 2024-05-08T06:12:48.261 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4195328 len 4096 2024-05-08T06:12:48.262 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4195328 len 4096 2024-05-08T06:12:48.262 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4195328 len 4096 2024-05-08T06:12:48.262 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4195328 len 4096 2024-05-08T06:12:48.262 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4195328 len 8192 2024-05-08T06:12:48.262 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4195328 len 8192 2024-05-08T06:12:48.262 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4195328 len 8192 2024-05-08T06:12:48.262 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4195328 len 8192 2024-05-08T06:12:48.262 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4195328 len 16384 2024-05-08T06:12:48.262 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4195328 len 16384 2024-05-08T06:12:48.262 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4195328 len 16384 2024-05-08T06:12:48.262 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4195328 len 16384 2024-05-08T06:12:48.262 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4195840 len 1024 2024-05-08T06:12:48.263 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4195840 len 1024 2024-05-08T06:12:48.263 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4195840 len 1024 2024-05-08T06:12:48.263 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4195840 len 1024 2024-05-08T06:12:48.263 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4195840 len 2048 2024-05-08T06:12:48.263 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4195840 len 2048 2024-05-08T06:12:48.263 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4195840 len 2048 2024-05-08T06:12:48.263 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4195840 len 2048 2024-05-08T06:12:48.263 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4195840 len 4096 2024-05-08T06:12:48.263 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4195840 len 4096 2024-05-08T06:12:48.263 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4195840 len 4096 2024-05-08T06:12:48.263 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4195840 len 4096 2024-05-08T06:12:48.263 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4195840 len 8192 2024-05-08T06:12:48.263 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4195840 len 8192 2024-05-08T06:12:48.263 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4195840 len 8192 2024-05-08T06:12:48.263 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4195840 len 8192 2024-05-08T06:12:48.728 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4195840 len 16384 2024-05-08T06:12:48.728 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4195840 len 16384 2024-05-08T06:12:48.729 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4195840 len 16384 2024-05-08T06:12:48.729 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4195840 len 16384 2024-05-08T06:12:48.729 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4196352 len 1024 2024-05-08T06:12:48.729 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4196352 len 1024 2024-05-08T06:12:48.729 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4196352 len 1024 2024-05-08T06:12:48.729 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4196352 len 1024 2024-05-08T06:12:48.729 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4196352 len 2048 2024-05-08T06:12:48.729 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4196352 len 2048 2024-05-08T06:12:48.729 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4196352 len 2048 2024-05-08T06:12:48.729 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4196352 len 2048 2024-05-08T06:12:48.729 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4196352 len 4096 2024-05-08T06:12:48.729 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4196352 len 4096 2024-05-08T06:12:48.729 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4196352 len 4096 2024-05-08T06:12:48.729 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4196352 len 4096 2024-05-08T06:12:48.729 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4196352 len 8192 2024-05-08T06:12:48.730 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4196352 len 8192 2024-05-08T06:12:48.730 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4196352 len 8192 2024-05-08T06:12:48.730 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4196352 len 8192 2024-05-08T06:12:48.730 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4196352 len 16384 2024-05-08T06:12:48.730 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4196352 len 16384 2024-05-08T06:12:48.730 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4196352 len 16384 2024-05-08T06:12:48.730 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4196352 len 16384 2024-05-08T06:12:48.730 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4196864 len 1024 2024-05-08T06:12:48.730 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4196864 len 1024 2024-05-08T06:12:48.730 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4196864 len 1024 2024-05-08T06:12:48.730 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4196864 len 1024 2024-05-08T06:12:48.730 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4196864 len 2048 2024-05-08T06:12:48.730 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4196864 len 2048 2024-05-08T06:12:48.730 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4196864 len 2048 2024-05-08T06:12:48.731 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4196864 len 2048 2024-05-08T06:12:48.731 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4196864 len 4096 2024-05-08T06:12:48.731 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4196864 len 4096 2024-05-08T06:12:48.731 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4196864 len 4096 2024-05-08T06:12:48.731 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4196864 len 4096 2024-05-08T06:12:48.731 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4196864 len 8192 2024-05-08T06:12:48.731 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4196864 len 8192 2024-05-08T06:12:48.731 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4196864 len 8192 2024-05-08T06:12:48.731 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4196864 len 8192 2024-05-08T06:12:48.731 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4196864 len 16384 2024-05-08T06:12:48.731 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4196864 len 16384 2024-05-08T06:12:48.731 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4196864 len 16384 2024-05-08T06:12:48.731 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4196864 len 16384 2024-05-08T06:12:48.731 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4197376 len 1024 2024-05-08T06:12:48.731 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4197376 len 1024 2024-05-08T06:12:48.732 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4197376 len 1024 2024-05-08T06:12:48.732 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4197376 len 1024 2024-05-08T06:12:48.732 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4197376 len 2048 2024-05-08T06:12:48.732 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4197376 len 2048 2024-05-08T06:12:48.732 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4197376 len 2048 2024-05-08T06:12:48.732 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4197376 len 2048 2024-05-08T06:12:48.732 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4197376 len 4096 2024-05-08T06:12:48.732 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4197376 len 4096 2024-05-08T06:12:48.732 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4197376 len 4096 2024-05-08T06:12:48.732 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4197376 len 4096 2024-05-08T06:12:48.733 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4197376 len 8192 2024-05-08T06:12:48.733 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4197376 len 8192 2024-05-08T06:12:48.733 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4197376 len 8192 2024-05-08T06:12:48.733 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4197376 len 8192 2024-05-08T06:12:48.733 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4197376 len 16384 2024-05-08T06:12:48.733 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4197376 len 16384 2024-05-08T06:12:48.733 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4197376 len 16384 2024-05-08T06:12:48.733 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4197376 len 16384 2024-05-08T06:12:48.733 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4197888 len 1024 2024-05-08T06:12:48.733 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4197888 len 1024 2024-05-08T06:12:48.733 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4197888 len 1024 2024-05-08T06:12:48.733 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4197888 len 1024 2024-05-08T06:12:48.733 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4197888 len 2048 2024-05-08T06:12:48.734 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4197888 len 2048 2024-05-08T06:12:48.734 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4197888 len 2048 2024-05-08T06:12:48.734 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4197888 len 2048 2024-05-08T06:12:48.734 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4197888 len 4096 2024-05-08T06:12:48.734 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4197888 len 4096 2024-05-08T06:12:48.734 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4197888 len 4096 2024-05-08T06:12:48.734 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4197888 len 4096 2024-05-08T06:12:48.734 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4197888 len 8192 2024-05-08T06:12:48.734 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4197888 len 8192 2024-05-08T06:12:48.734 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4197888 len 8192 2024-05-08T06:12:48.734 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4197888 len 8192 2024-05-08T06:12:48.734 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2048 offset 4197888 len 16384 2024-05-08T06:12:48.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:48 smithi123 ceph-mon[29787]: pgmap v530: 97 pgs: 97 active+clean; 227 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 73 MiB/s rd, 664 KiB/s wr, 242 op/s 2024-05-08T06:12:49.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:48 smithi163 ceph-mon[34520]: pgmap v530: 97 pgs: 97 active+clean; 227 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 73 MiB/s rd, 664 KiB/s wr, 242 op/s 2024-05-08T06:12:49.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:48 smithi192 ceph-mon[34664]: pgmap v530: 97 pgs: 97 active+clean; 227 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 73 MiB/s rd, 664 KiB/s wr, 242 op/s 2024-05-08T06:12:50.414 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4197888 len 16384 2024-05-08T06:12:50.414 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2048 offset 4197888 len 16384 2024-05-08T06:12:50.414 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2048 offset 4197888 len 16384 2024-05-08T06:12:50.414 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4190720 len 1024 2024-05-08T06:12:50.414 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4190720 len 1024 2024-05-08T06:12:50.414 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4190720 len 1024 2024-05-08T06:12:50.414 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4190720 len 1024 2024-05-08T06:12:50.414 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4190720 len 2048 2024-05-08T06:12:50.414 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4190720 len 2048 2024-05-08T06:12:50.414 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4190720 len 2048 2024-05-08T06:12:50.415 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4190720 len 2048 2024-05-08T06:12:50.415 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4190720 len 4096 2024-05-08T06:12:50.415 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4190720 len 4096 2024-05-08T06:12:50.415 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4190720 len 4096 2024-05-08T06:12:50.415 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4190720 len 4096 2024-05-08T06:12:50.415 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4190720 len 8192 2024-05-08T06:12:50.415 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4190720 len 8192 2024-05-08T06:12:50.415 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4190720 len 8192 2024-05-08T06:12:50.415 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4190720 len 8192 2024-05-08T06:12:50.415 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4190720 len 16384 2024-05-08T06:12:50.415 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4190720 len 16384 2024-05-08T06:12:50.415 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4190720 len 16384 2024-05-08T06:12:50.415 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4190720 len 16384 2024-05-08T06:12:50.415 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4191232 len 1024 2024-05-08T06:12:50.416 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4191232 len 1024 2024-05-08T06:12:50.416 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4191232 len 1024 2024-05-08T06:12:50.416 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4191232 len 1024 2024-05-08T06:12:50.416 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4191232 len 2048 2024-05-08T06:12:50.416 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4191232 len 2048 2024-05-08T06:12:50.416 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4191232 len 2048 2024-05-08T06:12:50.416 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4191232 len 2048 2024-05-08T06:12:50.416 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4191232 len 4096 2024-05-08T06:12:50.416 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4191232 len 4096 2024-05-08T06:12:50.416 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4191232 len 4096 2024-05-08T06:12:50.416 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4191232 len 4096 2024-05-08T06:12:50.416 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4191232 len 8192 2024-05-08T06:12:50.416 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4191232 len 8192 2024-05-08T06:12:50.416 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4191232 len 8192 2024-05-08T06:12:50.416 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4191232 len 8192 2024-05-08T06:12:50.417 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4191232 len 16384 2024-05-08T06:12:50.417 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4191232 len 16384 2024-05-08T06:12:50.417 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4191232 len 16384 2024-05-08T06:12:50.417 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4191232 len 16384 2024-05-08T06:12:50.417 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4191744 len 1024 2024-05-08T06:12:50.417 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4191744 len 1024 2024-05-08T06:12:50.417 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4191744 len 1024 2024-05-08T06:12:50.417 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4191744 len 1024 2024-05-08T06:12:50.417 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4191744 len 2048 2024-05-08T06:12:50.417 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4191744 len 2048 2024-05-08T06:12:50.417 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4191744 len 2048 2024-05-08T06:12:50.417 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4191744 len 2048 2024-05-08T06:12:50.417 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4191744 len 4096 2024-05-08T06:12:50.417 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4191744 len 4096 2024-05-08T06:12:50.417 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4191744 len 4096 2024-05-08T06:12:50.418 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4191744 len 4096 2024-05-08T06:12:50.418 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4191744 len 8192 2024-05-08T06:12:50.418 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4191744 len 8192 2024-05-08T06:12:50.418 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4191744 len 8192 2024-05-08T06:12:50.418 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4191744 len 8192 2024-05-08T06:12:50.418 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4191744 len 16384 2024-05-08T06:12:50.418 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4191744 len 16384 2024-05-08T06:12:50.418 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4191744 len 16384 2024-05-08T06:12:50.418 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4191744 len 16384 2024-05-08T06:12:50.418 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4192256 len 1024 2024-05-08T06:12:50.418 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4192256 len 1024 2024-05-08T06:12:50.418 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4192256 len 1024 2024-05-08T06:12:50.418 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4192256 len 1024 2024-05-08T06:12:50.418 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4192256 len 2048 2024-05-08T06:12:50.419 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4192256 len 2048 2024-05-08T06:12:50.419 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4192256 len 2048 2024-05-08T06:12:50.419 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4192256 len 2048 2024-05-08T06:12:50.419 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4192256 len 4096 2024-05-08T06:12:50.419 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4192256 len 4096 2024-05-08T06:12:50.419 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4192256 len 4096 2024-05-08T06:12:50.419 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4192256 len 4096 2024-05-08T06:12:50.419 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4192256 len 8192 2024-05-08T06:12:50.419 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4192256 len 8192 2024-05-08T06:12:50.419 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4192256 len 8192 2024-05-08T06:12:50.419 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4192256 len 8192 2024-05-08T06:12:50.419 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4192256 len 16384 2024-05-08T06:12:50.419 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4192256 len 16384 2024-05-08T06:12:50.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.440+0000 7f879f200700 1 -- 172.21.15.123:0/759164227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87a0104640 msgr2=0x7f87a0104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:50.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.440+0000 7f879f200700 1 --2- 172.21.15.123:0/759164227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87a0104640 0x7f87a0104a20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f87900098f0 tx=0x7f8790009c00 comp rx=0 tx=0).stop 2024-05-08T06:12:50.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.441+0000 7f879f200700 1 -- 172.21.15.123:0/759164227 shutdown_connections 2024-05-08T06:12:50.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.441+0000 7f879f200700 1 --2- 172.21.15.123:0/759164227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87a0104640 0x7f87a0104a20 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:50.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.441+0000 7f879f200700 1 --2- 172.21.15.123:0/759164227 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87a010da20 0x7f87a010fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:50.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.441+0000 7f879f200700 1 --2- 172.21.15.123:0/759164227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87a0104f60 0x7f87a010d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:50.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.441+0000 7f879f200700 1 -- 172.21.15.123:0/759164227 >> 172.21.15.123:0/759164227 conn(0x7f87a00fde30 msgr2=0x7f87a0100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:50.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.441+0000 7f879f200700 1 -- 172.21.15.123:0/759164227 shutdown_connections 2024-05-08T06:12:50.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.441+0000 7f879f200700 1 -- 172.21.15.123:0/759164227 wait complete. 2024-05-08T06:12:50.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.442+0000 7f879f200700 1 Processor -- start 2024-05-08T06:12:50.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.443+0000 7f879f200700 1 -- start start 2024-05-08T06:12:50.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.443+0000 7f879f200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87a0104640 0x7f87a01022a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:50.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.443+0000 7f879f200700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87a0104f60 0x7f87a01027e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:50.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.443+0000 7f879f200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87a010da20 0x7f87a0102d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:50.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.444+0000 7f879f200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f87a01135f0 con 0x7f87a010da20 2024-05-08T06:12:50.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.444+0000 7f879f200700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f87a0113470 con 0x7f87a0104640 2024-05-08T06:12:50.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.444+0000 7f879f200700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f87a0113770 con 0x7f87a0104f60 2024-05-08T06:12:50.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.444+0000 7f879d400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87a0104f60 0x7f87a01027e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:50.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.444+0000 7f879d400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87a0104f60 0x7f87a01027e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:41674/0 (socket says 172.21.15.123:41674) 2024-05-08T06:12:50.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.444+0000 7f879d400700 1 -- 172.21.15.123:0/1113275185 learned_addr learned my addr 172.21.15.123:0/1113275185 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:50.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.444+0000 7f879de00700 1 --2- 172.21.15.123:0/1113275185 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87a0104640 0x7f87a01022a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:50.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.444+0000 7f879e800700 1 --2- 172.21.15.123:0/1113275185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87a010da20 0x7f87a0102d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:50.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.444+0000 7f879d400700 1 -- 172.21.15.123:0/1113275185 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87a0104640 msgr2=0x7f87a01022a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:50.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.444+0000 7f879d400700 1 --2- 172.21.15.123:0/1113275185 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87a0104640 0x7f87a01022a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:50.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.444+0000 7f879d400700 1 -- 172.21.15.123:0/1113275185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87a010da20 msgr2=0x7f87a0102d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:50.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.444+0000 7f879d400700 1 --2- 172.21.15.123:0/1113275185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87a010da20 0x7f87a0102d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:50.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.444+0000 7f879d400700 1 -- 172.21.15.123:0/1113275185 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a01a8090 con 0x7f87a0104f60 2024-05-08T06:12:50.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.445+0000 7f879d400700 1 --2- 172.21.15.123:0/1113275185 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87a0104f60 0x7f87a01027e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f878c00e9f0 tx=0x7f878c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:50.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.445+0000 7f8796a00700 1 -- 172.21.15.123:0/1113275185 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c00c9c0 con 0x7f87a0104f60 2024-05-08T06:12:50.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.445+0000 7f8796a00700 1 -- 172.21.15.123:0/1113275185 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f878c00cb60 con 0x7f87a0104f60 2024-05-08T06:12:50.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.445+0000 7f879f200700 1 -- 172.21.15.123:0/1113275185 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87a01a8380 con 0x7f87a0104f60 2024-05-08T06:12:50.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.445+0000 7f8796a00700 1 -- 172.21.15.123:0/1113275185 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c018680 con 0x7f87a0104f60 2024-05-08T06:12:50.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.445+0000 7f879f200700 1 -- 172.21.15.123:0/1113275185 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87a01a88c0 con 0x7f87a0104f60 2024-05-08T06:12:50.449 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.449+0000 7f8796a00700 1 -- 172.21.15.123:0/1113275185 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7f878c018a30 con 0x7f87a0104f60 2024-05-08T06:12:50.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.450+0000 7f87af658b80 1 Processor -- start 2024-05-08T06:12:50.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.450+0000 7f87af658b80 1 -- start start 2024-05-08T06:12:50.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.451+0000 7f87af658b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564287f63da0 0x564287f64180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:50.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.451+0000 7f87af658b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564287f646c0 0x564287f5e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:50.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.451+0000 7f87af658b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564287f5ea50 0x564287f5eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:50.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.451+0000 7f87af658b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564287eb0a20 con 0x564287f63da0 2024-05-08T06:12:50.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.451+0000 7f87af658b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x564287e4e760 con 0x564287f646c0 2024-05-08T06:12:50.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.451+0000 7f87af658b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564287eac9e0 con 0x564287f5ea50 2024-05-08T06:12:50.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.451+0000 7f879de00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564287f646c0 0x564287f5e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:50.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.451+0000 7f879de00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564287f646c0 0x564287f5e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:48954/0 (socket says 172.21.15.123:48954) 2024-05-08T06:12:50.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.451+0000 7f879de00700 1 -- 172.21.15.123:0/3691768994 learned_addr learned my addr 172.21.15.123:0/3691768994 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:50.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.451+0000 7f879e800700 1 --2- 172.21.15.123:0/3691768994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564287f63da0 0x564287f64180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:50.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.451+0000 7f879d400700 1 --2- 172.21.15.123:0/3691768994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564287f5ea50 0x564287f5eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:50.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.452+0000 7f8796a00700 1 --2- 172.21.15.123:0/1113275185 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f87840b2880 0x7f87840b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:50.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.452+0000 7f879de00700 1 -- 172.21.15.123:0/3691768994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564287f5ea50 msgr2=0x564287f5eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:50.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.452+0000 7f879de00700 1 --2- 172.21.15.123:0/3691768994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564287f5ea50 0x564287f5eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:50.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.452+0000 7f879de00700 1 -- 172.21.15.123:0/3691768994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564287f63da0 msgr2=0x564287f64180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:50.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.452+0000 7f879de00700 1 --2- 172.21.15.123:0/3691768994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564287f63da0 0x564287f64180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:50.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.452+0000 7f879de00700 1 -- 172.21.15.123:0/3691768994 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564287dfd8d0 con 0x564287f646c0 2024-05-08T06:12:50.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.452+0000 7f879d400700 1 --2- 172.21.15.123:0/3691768994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564287f5ea50 0x564287f5eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:12:50.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.452+0000 7f8796a00700 1 -- 172.21.15.123:0/1113275185 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f878c0e26d0 con 0x7f87a0104f60 2024-05-08T06:12:50.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.452+0000 7f879e800700 1 --2- 172.21.15.123:0/1113275185 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f87840b2880 0x7f87840b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:50.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.452+0000 7f879de00700 1 --2- 172.21.15.123:0/3691768994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564287f646c0 0x564287f5e430 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f87900098f0 tx=0x7f8790005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc39275ba7ad4973 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:50.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.452+0000 7f878b400700 1 -- 172.21.15.123:0/3691768994 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879000f460 con 0x564287f646c0 2024-05-08T06:12:50.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.452+0000 7f878b400700 1 -- 172.21.15.123:0/3691768994 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8790005040 con 0x564287f646c0 2024-05-08T06:12:50.453 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.452+0000 7f878b400700 1 -- 172.21.15.123:0/3691768994 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87900039b0 con 0x564287f646c0 2024-05-08T06:12:50.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.453+0000 7f87af658b80 1 -- 172.21.15.123:0/3691768994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564287f646c0 msgr2=0x564287f5e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:50.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.453+0000 7f87af658b80 1 --2- 172.21.15.123:0/3691768994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564287f646c0 0x564287f5e430 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f87900098f0 tx=0x7f8790005dc0 comp rx=0 tx=0).stop 2024-05-08T06:12:50.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.453+0000 7f879e800700 1 --2- 172.21.15.123:0/1113275185 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f87840b2880 0x7f87840b4d40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8798009fd0 tx=0x7f8798009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:50.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.453+0000 7f87af658b80 1 -- 172.21.15.123:0/3691768994 shutdown_connections 2024-05-08T06:12:50.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.453+0000 7f87af658b80 1 --2- 172.21.15.123:0/3691768994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564287f63da0 0x564287f64180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:50.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.453+0000 7f87af658b80 1 --2- 172.21.15.123:0/3691768994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564287f5ea50 0x564287f5eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:50.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.453+0000 7f87af658b80 1 --2- 172.21.15.123:0/3691768994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564287f646c0 0x564287f5e430 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:50.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.453+0000 7f87af658b80 1 -- 172.21.15.123:0/3691768994 >> 172.21.15.123:0/3691768994 conn(0x564287ed66d0 msgr2=0x564287f58ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:50.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.454+0000 7f87af658b80 1 -- 172.21.15.123:0/3691768994 shutdown_connections 2024-05-08T06:12:50.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.454+0000 7f87af658b80 1 -- 172.21.15.123:0/3691768994 wait complete. 2024-05-08T06:12:50.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.454+0000 7f87af658b80 1 -- start start 2024-05-08T06:12:50.455 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.454+0000 7f87af658b80 1 Processor -- start 2024-05-08T06:12:50.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.455+0000 7f87af658b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564287f5ea50 0x564287f6a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:50.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.455+0000 7f87af658b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564287f63da0 0x564287f67790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:50.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.455+0000 7f87af658b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564287f646c0 0x564287f67cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:50.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.455+0000 7f87af658b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564287f6b0f0 con 0x564287f646c0 2024-05-08T06:12:50.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.455+0000 7f87af658b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x564287e16b20 con 0x564287f63da0 2024-05-08T06:12:50.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.455+0000 7f87af658b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564287f6b270 con 0x564287f5ea50 2024-05-08T06:12:50.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.455+0000 7f87af658b80 10 client.0 _ll_register_callbacks cb 0x564287e57c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:12:50.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.455+0000 7f879de00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564287f5ea50 0x564287f6a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:50.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.455+0000 7f879d400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564287f63da0 0x564287f67790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:50.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.455+0000 7f879de00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564287f5ea50 0x564287f6a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:41694/0 (socket says 172.21.15.123:41694) 2024-05-08T06:12:50.457 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.455+0000 7f879de00700 1 -- 172.21.15.123:0/3659475944 learned_addr learned my addr 172.21.15.123:0/3659475944 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:50.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.455+0000 7f879e800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564287f646c0 0x564287f67cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:50.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.456+0000 7f879d400700 1 -- 172.21.15.123:0/3659475944 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564287f5ea50 msgr2=0x564287f6a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:50.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.456+0000 7f879d400700 1 --2- 172.21.15.123:0/3659475944 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564287f5ea50 0x564287f6a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:50.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.456+0000 7f879d400700 1 -- 172.21.15.123:0/3659475944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564287f646c0 msgr2=0x564287f67cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:50.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.456+0000 7f879d400700 1 --2- 172.21.15.123:0/3659475944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564287f646c0 0x564287f67cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:50.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.456+0000 7f879d400700 1 -- 172.21.15.123:0/3659475944 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564287e76420 con 0x564287f63da0 2024-05-08T06:12:50.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.456+0000 7f879d400700 1 --2- 172.21.15.123:0/3659475944 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564287f63da0 0x564287f67790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x564287f5f410 tx=0x7f878c0aa1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a567eb29fd6f7e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:50.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.456+0000 7f877fe00700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-05-08T06:12:50.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.456+0000 7f877fe00700 1 -- 172.21.15.123:0/3659475944 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c0a9410 con 0x564287f63da0 2024-05-08T06:12:50.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.457+0000 7f877fe00700 1 -- 172.21.15.123:0/3659475944 <== mon.2 v2:172.21.15.163:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f878c0a8f00 con 0x564287f63da0 2024-05-08T06:12:50.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.457+0000 7f87af658b80 1 -- 172.21.15.123:0/3659475944 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8790005040 con 0x564287f63da0 2024-05-08T06:12:50.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.457+0000 7f877fe00700 1 -- 172.21.15.123:0/3659475944 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f878c0a5430 con 0x564287f63da0 2024-05-08T06:12:50.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.457+0000 7f87af658b80 10 client.25655 fetch_fsmap learned FSMap version 24 2024-05-08T06:12:50.458 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.457+0000 7f87af658b80 1 -- 172.21.15.123:0/3659475944 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564287f75950 con 0x564287f63da0 2024-05-08T06:12:50.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.457+0000 7f877fe00700 1 -- 172.21.15.123:0/3659475944 <== mon.2 v2:172.21.15.163:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8790005040 con 0x564287f63da0 2024-05-08T06:12:50.459 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.457+0000 7f877fe00700 1 -- 172.21.15.123:0/3659475944 <== mon.2 v2:172.21.15.163:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f878c0e4bb0 con 0x564287f63da0 2024-05-08T06:12:50.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.457+0000 7f87af658b80 10 client.25655 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:12:50.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.457+0000 7f87af658b80 10 client.25655 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T06:12:50.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.457+0000 7f87af658b80 20 client.25655 populate_metadata read hostname 'smithi123' 2024-05-08T06:12:50.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.457+0000 7f87af658b80 1 --2- 172.21.15.123:0/3659475944 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x564287f77f90 0x564287f7a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:50.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.457+0000 7f87af658b80 4 client.25655 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:12:50.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.457+0000 7f87af658b80 1 -- 172.21.15.123:0/3659475944 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564287f2b140 con 0x564287f77f90 2024-05-08T06:12:50.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.458+0000 7f879de00700 1 --2- 172.21.15.123:0/3659475944 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x564287f77f90 0x564287f7a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:50.460 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.458+0000 7f879de00700 1 --2- 172.21.15.123:0/3659475944 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x564287f77f90 0x564287f7a380 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85b2cd8ea4e1ab60 server_cookie=93e6e32c66ffd39e in_seq=0 out_seq=0 2024-05-08T06:12:50.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.458+0000 7f877fe00700 10 client.25655 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:50.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.460+0000 7f877fe00700 1 -- 172.21.15.123:0/3659475944 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564287f2b140 con 0x564287f77f90 2024-05-08T06:12:50.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.460+0000 7f877fe00700 10 client.25655 handle_command_reply: tid=0 2024-05-08T06:12:50.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.460+0000 7f87af658b80 1 client.25655 shutdown 2024-05-08T06:12:50.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.460+0000 7f87af658b80 1 -- 172.21.15.123:0/3659475944 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564287f63da0 msgr2=0x564287f67790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:50.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.460+0000 7f87af658b80 1 --2- 172.21.15.123:0/3659475944 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564287f63da0 0x564287f67790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x564287f5f410 tx=0x7f878c0aa1e0 comp rx=0 tx=0).stop 2024-05-08T06:12:50.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.460+0000 7f87af658b80 1 -- 172.21.15.123:0/3659475944 shutdown_connections 2024-05-08T06:12:50.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.460+0000 7f87af658b80 1 --2- 172.21.15.123:0/3659475944 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x564287f77f90 0x564287f7a380 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:50.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.460+0000 7f87af658b80 1 --2- 172.21.15.123:0/3659475944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564287f646c0 0x564287f67cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:50.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.460+0000 7f87af658b80 1 --2- 172.21.15.123:0/3659475944 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564287f5ea50 0x564287f6a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:50.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.460+0000 7f87af658b80 1 --2- 172.21.15.123:0/3659475944 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564287f63da0 0x564287f67790 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:50.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.460+0000 7f877fe00700 0 client.25655 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:50.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.460+0000 7f87af658b80 1 -- 172.21.15.123:0/3659475944 >> 172.21.15.123:0/3659475944 conn(0x564287ed66d0 msgr2=0x564287f53750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:50.461 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.460+0000 7f87af658b80 1 -- 172.21.15.123:0/3659475944 shutdown_connections 2024-05-08T06:12:50.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.461+0000 7f87af658b80 1 -- 172.21.15.123:0/3659475944 wait complete. 2024-05-08T06:12:50.462 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.461+0000 7f87af658b80 20 client.25655 trim_cache size 0 max 16384 2024-05-08T06:12:50.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.479+0000 7f87af658b80 1 Processor -- start 2024-05-08T06:12:50.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.480+0000 7f87af658b80 1 -- start start 2024-05-08T06:12:50.480 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.480+0000 7f87af658b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564287f5ea50 0x564287f7a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:50.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.480+0000 7f87af658b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564287f63da0 0x564287f666f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:50.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.480+0000 7f87af658b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564287f646c0 0x564287f74180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:50.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.480+0000 7f87af658b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f87a01b24a0 con 0x564287f646c0 2024-05-08T06:12:50.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.480+0000 7f87af658b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f87a01afd30 con 0x564287f5ea50 2024-05-08T06:12:50.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.480+0000 7f87af658b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f87a01adba0 con 0x564287f63da0 2024-05-08T06:12:50.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.480+0000 7f879d400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564287f63da0 0x564287f666f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:50.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.481+0000 7f879de00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564287f5ea50 0x564287f7a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:50.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.481+0000 7f879d400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564287f63da0 0x564287f666f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:41710/0 (socket says 172.21.15.123:41710) 2024-05-08T06:12:50.481 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.481+0000 7f879d400700 1 -- 172.21.15.123:0/3669345362 learned_addr learned my addr 172.21.15.123:0/3669345362 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:50.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.481+0000 7f879e800700 1 --2- 172.21.15.123:0/3669345362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564287f646c0 0x564287f74180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:50.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.481+0000 7f879d400700 1 -- 172.21.15.123:0/3669345362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564287f5ea50 msgr2=0x564287f7a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:50.482 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.481+0000 7f879d400700 1 --2- 172.21.15.123:0/3669345362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564287f5ea50 0x564287f7a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:50.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.481+0000 7f879d400700 1 -- 172.21.15.123:0/3669345362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564287f646c0 msgr2=0x564287f74180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:50.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.481+0000 7f879d400700 1 --2- 172.21.15.123:0/3669345362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564287f646c0 0x564287f74180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:50.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.481+0000 7f879d400700 1 -- 172.21.15.123:0/3669345362 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564287f558a0 con 0x564287f63da0 2024-05-08T06:12:50.483 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.481+0000 7f879d400700 1 --2- 172.21.15.123:0/3669345362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564287f63da0 0x564287f666f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f878c0a9c30 tx=0x7f878c0aa250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2e48afe893423f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:50.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.481+0000 7f878aa00700 1 -- 172.21.15.123:0/3669345362 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c0c1020 con 0x564287f63da0 2024-05-08T06:12:50.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.481+0000 7f878aa00700 1 -- 172.21.15.123:0/3669345362 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f878c0aa3a0 con 0x564287f63da0 2024-05-08T06:12:50.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.481+0000 7f878aa00700 1 -- 172.21.15.123:0/3669345362 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c0a8970 con 0x564287f63da0 2024-05-08T06:12:50.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.482+0000 7f87af658b80 1 -- 172.21.15.123:0/3669345362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564287f63da0 msgr2=0x564287f666f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:50.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.482+0000 7f87af658b80 1 --2- 172.21.15.123:0/3669345362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564287f63da0 0x564287f666f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f878c0a9c30 tx=0x7f878c0aa250 comp rx=0 tx=0).stop 2024-05-08T06:12:50.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.482+0000 7f87af658b80 1 -- 172.21.15.123:0/3669345362 shutdown_connections 2024-05-08T06:12:50.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.482+0000 7f87af658b80 1 --2- 172.21.15.123:0/3669345362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564287f646c0 0x564287f74180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:50.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.482+0000 7f87af658b80 1 --2- 172.21.15.123:0/3669345362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564287f5ea50 0x564287f7a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:50.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.482+0000 7f87af658b80 1 --2- 172.21.15.123:0/3669345362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564287f63da0 0x564287f666f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:50.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.482+0000 7f87af658b80 1 -- 172.21.15.123:0/3669345362 >> 172.21.15.123:0/3669345362 conn(0x564287ed66d0 msgr2=0x564287f68d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:50.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.482+0000 7f87af658b80 1 -- 172.21.15.123:0/3669345362 shutdown_connections 2024-05-08T06:12:50.484 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.482+0000 7f87af658b80 1 -- 172.21.15.123:0/3669345362 wait complete. 2024-05-08T06:12:50.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.483+0000 7f87af658b80 1 -- start start 2024-05-08T06:12:50.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.483+0000 7f87af658b80 1 Processor -- start 2024-05-08T06:12:50.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.484+0000 7f87af658b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564287f5ea50 0x564287f70500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:50.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.484+0000 7f87af658b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564287f63da0 0x564287f70a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:50.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.484+0000 7f87af658b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564287f646c0 0x564287f7ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:50.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.484+0000 7f87af658b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f87a01ae1f0 con 0x564287f63da0 2024-05-08T06:12:50.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.484+0000 7f87af658b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f87a01af660 con 0x564287f646c0 2024-05-08T06:12:50.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.484+0000 7f87af658b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f87a01ae8c0 con 0x564287f5ea50 2024-05-08T06:12:50.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.484+0000 7f87af658b80 10 client.0 _ll_register_callbacks cb 0x564287e57c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:12:50.488 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.484+0000 7f879de00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564287f5ea50 0x564287f70500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:50.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.484+0000 7f879de00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564287f5ea50 0x564287f70500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:41712/0 (socket says 172.21.15.123:41712) 2024-05-08T06:12:50.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.484+0000 7f879de00700 1 -- 172.21.15.123:0/450163213 learned_addr learned my addr 172.21.15.123:0/450163213 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:50.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.484+0000 7f879d400700 1 --2- 172.21.15.123:0/450163213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564287f63da0 0x564287f70a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:50.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.484+0000 7f879e800700 1 --2- 172.21.15.123:0/450163213 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564287f646c0 0x564287f7ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:50.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.484+0000 7f879de00700 1 -- 172.21.15.123:0/450163213 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564287f646c0 msgr2=0x564287f7ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:50.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.484+0000 7f879de00700 1 --2- 172.21.15.123:0/450163213 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564287f646c0 0x564287f7ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:50.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.484+0000 7f879de00700 1 -- 172.21.15.123:0/450163213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564287f63da0 msgr2=0x564287f70a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:50.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.484+0000 7f879de00700 1 --2- 172.21.15.123:0/450163213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564287f63da0 0x564287f70a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:50.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.484+0000 7f879de00700 1 -- 172.21.15.123:0/450163213 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564287f7f440 con 0x564287f5ea50 2024-05-08T06:12:50.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.485+0000 7f879de00700 1 --2- 172.21.15.123:0/450163213 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564287f5ea50 0x564287f70500 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f87900098f0 tx=0x7f8790005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c0acb7da141b16f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:50.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.485+0000 7f877fe00700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-05-08T06:12:50.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.485+0000 7f877fe00700 1 -- 172.21.15.123:0/450163213 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8790037020 con 0x564287f5ea50 2024-05-08T06:12:50.489 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.485+0000 7f877fe00700 1 -- 172.21.15.123:0/450163213 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f879000f870 con 0x564287f5ea50 2024-05-08T06:12:50.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.485+0000 7f87af658b80 1 -- 172.21.15.123:0/450163213 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f878c0aa3a0 con 0x564287f5ea50 2024-05-08T06:12:50.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.485+0000 7f877fe00700 1 -- 172.21.15.123:0/450163213 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8790003680 con 0x564287f5ea50 2024-05-08T06:12:50.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.485+0000 7f87af658b80 10 client.25651 fetch_fsmap learned FSMap version 24 2024-05-08T06:12:50.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.485+0000 7f87af658b80 1 -- 172.21.15.123:0/450163213 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564287fe2250 con 0x564287f5ea50 2024-05-08T06:12:50.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.486+0000 7f877fe00700 1 -- 172.21.15.123:0/450163213 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f878c0aa3a0 con 0x564287f5ea50 2024-05-08T06:12:50.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.486+0000 7f877fe00700 1 -- 172.21.15.123:0/450163213 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f8790033ce0 con 0x564287f5ea50 2024-05-08T06:12:50.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.486+0000 7f87af658b80 10 client.25651 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:12:50.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.486+0000 7f87af658b80 10 client.25651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T06:12:50.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.486+0000 7f87af658b80 20 client.25651 populate_metadata read hostname 'smithi123' 2024-05-08T06:12:50.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.486+0000 7f87af658b80 1 --2- 172.21.15.123:0/450163213 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x564287fe2540 0x564287fe4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:50.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.486+0000 7f87af658b80 4 client.25651 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T06:12:50.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.486+0000 7f87af658b80 1 -- 172.21.15.123:0/450163213 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564287ebe400 con 0x564287fe2540 2024-05-08T06:12:50.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.486+0000 7f879d400700 1 --2- 172.21.15.123:0/450163213 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x564287fe2540 0x564287fe4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:50.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.487+0000 7f879d400700 1 --2- 172.21.15.123:0/450163213 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x564287fe2540 0x564287fe4930 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74409df3ab830260 server_cookie=166c39e5dce9c723 in_seq=0 out_seq=0 2024-05-08T06:12:50.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.487+0000 7f877fe00700 10 client.25651 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:50.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.487+0000 7f877fe00700 1 -- 172.21.15.123:0/450163213 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564287ebe400 con 0x564287fe2540 2024-05-08T06:12:50.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.487+0000 7f877fe00700 10 client.25651 handle_command_reply: tid=0 2024-05-08T06:12:50.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.487+0000 7f87af658b80 1 client.25651 shutdown 2024-05-08T06:12:50.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.487+0000 7f87af658b80 1 -- 172.21.15.123:0/450163213 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564287f5ea50 msgr2=0x564287f70500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:50.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.487+0000 7f87af658b80 1 --2- 172.21.15.123:0/450163213 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564287f5ea50 0x564287f70500 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f87900098f0 tx=0x7f8790005270 comp rx=0 tx=0).stop 2024-05-08T06:12:50.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.487+0000 7f87af658b80 1 -- 172.21.15.123:0/450163213 shutdown_connections 2024-05-08T06:12:50.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.487+0000 7f87af658b80 1 --2- 172.21.15.123:0/450163213 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x564287fe2540 0x564287fe4930 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:50.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.487+0000 7f87af658b80 1 --2- 172.21.15.123:0/450163213 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564287f646c0 0x564287f7ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:50.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.487+0000 7f87af658b80 1 --2- 172.21.15.123:0/450163213 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564287f5ea50 0x564287f70500 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:50.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.487+0000 7f87af658b80 1 --2- 172.21.15.123:0/450163213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564287f63da0 0x564287f70a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:50.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.487+0000 7f87af658b80 1 -- 172.21.15.123:0/450163213 >> 172.21.15.123:0/450163213 conn(0x564287ed66d0 msgr2=0x564287f651b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:50.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.487+0000 7f877fe00700 0 client.25651 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:50.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.488+0000 7f87af658b80 1 -- 172.21.15.123:0/450163213 shutdown_connections 2024-05-08T06:12:50.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.488+0000 7f87af658b80 1 -- 172.21.15.123:0/450163213 wait complete. 2024-05-08T06:12:50.492 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.488+0000 7f87af658b80 20 client.25651 trim_cache size 0 max 16384 2024-05-08T06:12:50.492 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-05-08T06:12:50.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.489+0000 7f879f200700 1 -- 172.21.15.123:0/1113275185 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f87840b2880 msgr2=0x7f87840b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:50.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.489+0000 7f879f200700 1 --2- 172.21.15.123:0/1113275185 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f87840b2880 0x7f87840b4d40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8798009fd0 tx=0x7f8798009040 comp rx=0 tx=0).stop 2024-05-08T06:12:50.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.489+0000 7f879f200700 1 -- 172.21.15.123:0/1113275185 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87a0104f60 msgr2=0x7f87a01027e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:50.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.489+0000 7f879f200700 1 --2- 172.21.15.123:0/1113275185 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87a0104f60 0x7f87a01027e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f878c00e9f0 tx=0x7f878c00ed00 comp rx=0 tx=0).stop 2024-05-08T06:12:50.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.489+0000 7f879f200700 1 -- 172.21.15.123:0/1113275185 shutdown_connections 2024-05-08T06:12:50.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.489+0000 7f879f200700 1 --2- 172.21.15.123:0/1113275185 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f87840b2880 0x7f87840b4d40 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:50.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.489+0000 7f879f200700 1 --2- 172.21.15.123:0/1113275185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87a010da20 0x7f87a0102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:50.493 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.489+0000 7f879f200700 1 --2- 172.21.15.123:0/1113275185 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87a0104640 0x7f87a01022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:50.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.489+0000 7f879f200700 1 --2- 172.21.15.123:0/1113275185 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87a0104f60 0x7f87a01027e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:50.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.489+0000 7f879f200700 1 -- 172.21.15.123:0/1113275185 >> 172.21.15.123:0/1113275185 conn(0x7f87a00fde30 msgr2=0x7f87a00febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:50.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.489+0000 7f879f200700 1 -- 172.21.15.123:0/1113275185 shutdown_connections 2024-05-08T06:12:50.494 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:50.489+0000 7f879f200700 1 -- 172.21.15.123:0/1113275185 wait complete. 2024-05-08T06:12:50.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:50 smithi123 ceph-mon[29787]: pgmap v531: 97 pgs: 97 active+clean; 199 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s rd, 666 KiB/s wr, 244 op/s 2024-05-08T06:12:51.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:50 smithi163 ceph-mon[34520]: pgmap v531: 97 pgs: 97 active+clean; 199 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s rd, 666 KiB/s wr, 244 op/s 2024-05-08T06:12:51.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:50 smithi192 ceph-mon[34664]: pgmap v531: 97 pgs: 97 active+clean; 199 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s rd, 666 KiB/s wr, 244 op/s 2024-05-08T06:12:51.924 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4192256 len 16384 2024-05-08T06:12:51.925 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4192256 len 16384 2024-05-08T06:12:51.925 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4192768 len 1024 2024-05-08T06:12:51.925 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4192768 len 1024 2024-05-08T06:12:51.925 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4192768 len 1024 2024-05-08T06:12:51.925 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4192768 len 1024 2024-05-08T06:12:51.925 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4192768 len 2048 2024-05-08T06:12:51.925 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4192768 len 2048 2024-05-08T06:12:51.925 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4192768 len 2048 2024-05-08T06:12:51.925 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4192768 len 2048 2024-05-08T06:12:51.925 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4192768 len 4096 2024-05-08T06:12:51.925 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4192768 len 4096 2024-05-08T06:12:51.925 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4192768 len 4096 2024-05-08T06:12:51.925 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4192768 len 4096 2024-05-08T06:12:51.925 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4192768 len 8192 2024-05-08T06:12:51.926 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4192768 len 8192 2024-05-08T06:12:51.926 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4192768 len 8192 2024-05-08T06:12:51.926 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4192768 len 8192 2024-05-08T06:12:51.926 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4192768 len 16384 2024-05-08T06:12:51.926 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4192768 len 16384 2024-05-08T06:12:51.926 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4192768 len 16384 2024-05-08T06:12:51.926 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4192768 len 16384 2024-05-08T06:12:51.926 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4193280 len 1024 2024-05-08T06:12:51.926 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4193280 len 1024 2024-05-08T06:12:51.926 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4193280 len 1024 2024-05-08T06:12:51.926 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4193280 len 1024 2024-05-08T06:12:51.926 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4193280 len 2048 2024-05-08T06:12:51.927 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4193280 len 2048 2024-05-08T06:12:51.927 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4193280 len 2048 2024-05-08T06:12:51.927 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4193280 len 2048 2024-05-08T06:12:51.927 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4193280 len 4096 2024-05-08T06:12:51.927 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4193280 len 4096 2024-05-08T06:12:51.927 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4193280 len 4096 2024-05-08T06:12:51.927 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4193280 len 4096 2024-05-08T06:12:51.927 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4193280 len 8192 2024-05-08T06:12:51.927 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4193280 len 8192 2024-05-08T06:12:51.927 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4193280 len 8192 2024-05-08T06:12:51.927 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4193280 len 8192 2024-05-08T06:12:51.927 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4193280 len 16384 2024-05-08T06:12:51.927 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4193280 len 16384 2024-05-08T06:12:51.927 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4193280 len 16384 2024-05-08T06:12:51.927 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4193280 len 16384 2024-05-08T06:12:51.927 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4193792 len 1024 2024-05-08T06:12:51.928 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4193792 len 1024 2024-05-08T06:12:51.928 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4193792 len 1024 2024-05-08T06:12:51.928 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4193792 len 1024 2024-05-08T06:12:51.928 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4193792 len 2048 2024-05-08T06:12:51.928 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4193792 len 2048 2024-05-08T06:12:51.928 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4193792 len 2048 2024-05-08T06:12:51.928 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4193792 len 2048 2024-05-08T06:12:51.928 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4193792 len 4096 2024-05-08T06:12:51.928 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4193792 len 4096 2024-05-08T06:12:51.928 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4193792 len 4096 2024-05-08T06:12:51.928 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4193792 len 4096 2024-05-08T06:12:51.928 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4193792 len 8192 2024-05-08T06:12:51.928 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4193792 len 8192 2024-05-08T06:12:51.928 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4193792 len 8192 2024-05-08T06:12:51.928 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4193792 len 8192 2024-05-08T06:12:51.929 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4193792 len 16384 2024-05-08T06:12:51.929 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4193792 len 16384 2024-05-08T06:12:51.929 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4193792 len 16384 2024-05-08T06:12:51.929 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4193792 len 16384 2024-05-08T06:12:51.929 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4194304 len 1024 2024-05-08T06:12:51.929 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4194304 len 1024 2024-05-08T06:12:51.929 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4194304 len 1024 2024-05-08T06:12:51.929 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4194304 len 1024 2024-05-08T06:12:51.929 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4194304 len 2048 2024-05-08T06:12:51.929 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4194304 len 2048 2024-05-08T06:12:51.929 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4194304 len 2048 2024-05-08T06:12:51.929 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4194304 len 2048 2024-05-08T06:12:51.929 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4194304 len 4096 2024-05-08T06:12:51.930 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4194304 len 4096 2024-05-08T06:12:51.930 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4194304 len 4096 2024-05-08T06:12:51.930 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4194304 len 4096 2024-05-08T06:12:51.930 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4194304 len 8192 2024-05-08T06:12:51.930 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4194304 len 8192 2024-05-08T06:12:51.930 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4194304 len 8192 2024-05-08T06:12:51.930 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4194304 len 8192 2024-05-08T06:12:51.930 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4194304 len 16384 2024-05-08T06:12:51.930 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4194304 len 16384 2024-05-08T06:12:51.930 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4194304 len 16384 2024-05-08T06:12:52.426 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4194304 len 16384 2024-05-08T06:12:52.426 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4194816 len 1024 2024-05-08T06:12:52.426 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4194816 len 1024 2024-05-08T06:12:52.427 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4194816 len 1024 2024-05-08T06:12:52.427 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4194816 len 1024 2024-05-08T06:12:52.427 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4194816 len 2048 2024-05-08T06:12:52.427 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4194816 len 2048 2024-05-08T06:12:52.427 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4194816 len 2048 2024-05-08T06:12:52.427 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4194816 len 2048 2024-05-08T06:12:52.427 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4194816 len 4096 2024-05-08T06:12:52.427 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4194816 len 4096 2024-05-08T06:12:52.427 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4194816 len 4096 2024-05-08T06:12:52.427 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4194816 len 4096 2024-05-08T06:12:52.427 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4194816 len 8192 2024-05-08T06:12:52.427 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4194816 len 8192 2024-05-08T06:12:52.427 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4194816 len 8192 2024-05-08T06:12:52.427 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4194816 len 8192 2024-05-08T06:12:52.427 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4194816 len 16384 2024-05-08T06:12:52.428 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4194816 len 16384 2024-05-08T06:12:52.428 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4194816 len 16384 2024-05-08T06:12:52.428 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4194816 len 16384 2024-05-08T06:12:52.428 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4195328 len 1024 2024-05-08T06:12:52.428 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4195328 len 1024 2024-05-08T06:12:52.428 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4195328 len 1024 2024-05-08T06:12:52.428 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4195328 len 1024 2024-05-08T06:12:52.428 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4195328 len 2048 2024-05-08T06:12:52.428 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4195328 len 2048 2024-05-08T06:12:52.428 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4195328 len 2048 2024-05-08T06:12:52.428 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4195328 len 2048 2024-05-08T06:12:52.428 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4195328 len 4096 2024-05-08T06:12:52.428 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4195328 len 4096 2024-05-08T06:12:52.428 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4195328 len 4096 2024-05-08T06:12:52.428 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4195328 len 4096 2024-05-08T06:12:52.428 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4195328 len 8192 2024-05-08T06:12:52.429 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4195328 len 8192 2024-05-08T06:12:52.429 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4195328 len 8192 2024-05-08T06:12:52.429 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4195328 len 8192 2024-05-08T06:12:52.429 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4195328 len 16384 2024-05-08T06:12:52.429 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4195328 len 16384 2024-05-08T06:12:52.429 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4195328 len 16384 2024-05-08T06:12:52.429 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4195328 len 16384 2024-05-08T06:12:52.429 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4195840 len 1024 2024-05-08T06:12:52.429 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4195840 len 1024 2024-05-08T06:12:52.429 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4195840 len 1024 2024-05-08T06:12:52.429 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4195840 len 1024 2024-05-08T06:12:52.429 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4195840 len 2048 2024-05-08T06:12:52.429 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4195840 len 2048 2024-05-08T06:12:52.429 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4195840 len 2048 2024-05-08T06:12:52.429 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4195840 len 2048 2024-05-08T06:12:52.429 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4195840 len 4096 2024-05-08T06:12:52.429 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4195840 len 4096 2024-05-08T06:12:52.430 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4195840 len 4096 2024-05-08T06:12:52.430 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4195840 len 4096 2024-05-08T06:12:52.430 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4195840 len 8192 2024-05-08T06:12:52.430 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4195840 len 8192 2024-05-08T06:12:52.430 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4195840 len 8192 2024-05-08T06:12:52.430 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4195840 len 8192 2024-05-08T06:12:52.430 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4195840 len 16384 2024-05-08T06:12:52.430 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4195840 len 16384 2024-05-08T06:12:52.430 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4195840 len 16384 2024-05-08T06:12:52.430 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4195840 len 16384 2024-05-08T06:12:52.430 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4196352 len 1024 2024-05-08T06:12:52.430 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4196352 len 1024 2024-05-08T06:12:52.430 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4196352 len 1024 2024-05-08T06:12:52.430 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4196352 len 1024 2024-05-08T06:12:52.430 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4196352 len 2048 2024-05-08T06:12:52.431 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4196352 len 2048 2024-05-08T06:12:52.431 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4196352 len 2048 2024-05-08T06:12:52.431 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4196352 len 2048 2024-05-08T06:12:52.431 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4196352 len 4096 2024-05-08T06:12:52.431 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4196352 len 4096 2024-05-08T06:12:52.431 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4196352 len 4096 2024-05-08T06:12:52.431 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4196352 len 4096 2024-05-08T06:12:52.431 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4196352 len 8192 2024-05-08T06:12:52.431 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4196352 len 8192 2024-05-08T06:12:52.431 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4196352 len 8192 2024-05-08T06:12:52.431 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4196352 len 8192 2024-05-08T06:12:52.431 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4196352 len 16384 2024-05-08T06:12:52.431 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4196352 len 16384 2024-05-08T06:12:52.431 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4196352 len 16384 2024-05-08T06:12:52.431 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4196352 len 16384 2024-05-08T06:12:52.781 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T06:12:52.781 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T06:12:52.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:52 smithi123 ceph-mon[29787]: pgmap v532: 97 pgs: 97 active+clean; 241 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 77 MiB/s rd, 682 KiB/s wr, 250 op/s 2024-05-08T06:12:53.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:52 smithi163 ceph-mon[34520]: pgmap v532: 97 pgs: 97 active+clean; 241 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 77 MiB/s rd, 682 KiB/s wr, 250 op/s 2024-05-08T06:12:53.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:52 smithi192 ceph-mon[34664]: pgmap v532: 97 pgs: 97 active+clean; 241 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 77 MiB/s rd, 682 KiB/s wr, 250 op/s 2024-05-08T06:12:53.175 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4196864 len 1024 2024-05-08T06:12:53.175 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4196864 len 1024 2024-05-08T06:12:53.176 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4196864 len 1024 2024-05-08T06:12:53.176 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4196864 len 1024 2024-05-08T06:12:53.176 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4196864 len 2048 2024-05-08T06:12:53.176 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4196864 len 2048 2024-05-08T06:12:53.176 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4196864 len 2048 2024-05-08T06:12:53.176 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4196864 len 2048 2024-05-08T06:12:53.176 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4196864 len 4096 2024-05-08T06:12:53.176 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4196864 len 4096 2024-05-08T06:12:53.176 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4196864 len 4096 2024-05-08T06:12:53.176 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4196864 len 4096 2024-05-08T06:12:53.176 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4196864 len 8192 2024-05-08T06:12:53.176 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4196864 len 8192 2024-05-08T06:12:53.176 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4196864 len 8192 2024-05-08T06:12:53.176 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4196864 len 8192 2024-05-08T06:12:53.176 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4196864 len 16384 2024-05-08T06:12:53.176 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4196864 len 16384 2024-05-08T06:12:53.177 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4196864 len 16384 2024-05-08T06:12:53.177 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4196864 len 16384 2024-05-08T06:12:53.177 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4197376 len 1024 2024-05-08T06:12:53.177 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4197376 len 1024 2024-05-08T06:12:53.177 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4197376 len 1024 2024-05-08T06:12:53.177 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4197376 len 1024 2024-05-08T06:12:53.177 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4197376 len 2048 2024-05-08T06:12:53.177 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4197376 len 2048 2024-05-08T06:12:53.177 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4197376 len 2048 2024-05-08T06:12:53.177 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4197376 len 2048 2024-05-08T06:12:53.177 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4197376 len 4096 2024-05-08T06:12:53.177 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4197376 len 4096 2024-05-08T06:12:53.177 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4197376 len 4096 2024-05-08T06:12:53.177 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4197376 len 4096 2024-05-08T06:12:53.177 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4197376 len 8192 2024-05-08T06:12:53.178 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4197376 len 8192 2024-05-08T06:12:53.178 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4197376 len 8192 2024-05-08T06:12:53.178 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4197376 len 8192 2024-05-08T06:12:53.178 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4197376 len 16384 2024-05-08T06:12:53.178 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4197376 len 16384 2024-05-08T06:12:53.178 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4197376 len 16384 2024-05-08T06:12:53.178 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4197376 len 16384 2024-05-08T06:12:53.178 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4197888 len 1024 2024-05-08T06:12:53.178 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4197888 len 1024 2024-05-08T06:12:53.178 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4197888 len 1024 2024-05-08T06:12:53.178 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4197888 len 1024 2024-05-08T06:12:53.178 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4197888 len 2048 2024-05-08T06:12:53.178 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4197888 len 2048 2024-05-08T06:12:53.178 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4197888 len 2048 2024-05-08T06:12:53.178 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4197888 len 2048 2024-05-08T06:12:53.179 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4197888 len 4096 2024-05-08T06:12:53.179 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4197888 len 4096 2024-05-08T06:12:53.179 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4197888 len 4096 2024-05-08T06:12:53.179 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4197888 len 4096 2024-05-08T06:12:53.179 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4197888 len 8192 2024-05-08T06:12:53.179 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4197888 len 8192 2024-05-08T06:12:53.179 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4197888 len 8192 2024-05-08T06:12:53.179 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4197888 len 8192 2024-05-08T06:12:53.179 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 2560 offset 4197888 len 16384 2024-05-08T06:12:53.179 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4197888 len 16384 2024-05-08T06:12:53.179 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 2560 offset 4197888 len 16384 2024-05-08T06:12:53.179 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 2560 offset 4197888 len 16384 2024-05-08T06:12:53.179 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4190720 len 1024 2024-05-08T06:12:53.179 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4190720 len 1024 2024-05-08T06:12:53.179 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4190720 len 1024 2024-05-08T06:12:53.179 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4190720 len 1024 2024-05-08T06:12:53.180 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4190720 len 2048 2024-05-08T06:12:53.180 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4190720 len 2048 2024-05-08T06:12:53.180 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4190720 len 2048 2024-05-08T06:12:53.180 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4190720 len 2048 2024-05-08T06:12:53.180 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4190720 len 4096 2024-05-08T06:12:53.180 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4190720 len 4096 2024-05-08T06:12:53.180 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4190720 len 4096 2024-05-08T06:12:53.180 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4190720 len 4096 2024-05-08T06:12:53.180 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4190720 len 8192 2024-05-08T06:12:53.180 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4190720 len 8192 2024-05-08T06:12:53.180 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4190720 len 8192 2024-05-08T06:12:53.180 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4190720 len 8192 2024-05-08T06:12:53.180 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4190720 len 16384 2024-05-08T06:12:53.180 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4190720 len 16384 2024-05-08T06:12:53.180 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4190720 len 16384 2024-05-08T06:12:53.181 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4190720 len 16384 2024-05-08T06:12:53.231 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:12:54.880 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4191232 len 1024 2024-05-08T06:12:54.880 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4191232 len 1024 2024-05-08T06:12:54.880 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4191232 len 1024 2024-05-08T06:12:54.880 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4191232 len 1024 2024-05-08T06:12:54.880 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4191232 len 2048 2024-05-08T06:12:54.880 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4191232 len 2048 2024-05-08T06:12:54.880 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4191232 len 2048 2024-05-08T06:12:54.881 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4191232 len 2048 2024-05-08T06:12:54.881 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4191232 len 4096 2024-05-08T06:12:54.881 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4191232 len 4096 2024-05-08T06:12:54.881 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4191232 len 4096 2024-05-08T06:12:54.881 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4191232 len 4096 2024-05-08T06:12:54.881 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4191232 len 8192 2024-05-08T06:12:54.881 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4191232 len 8192 2024-05-08T06:12:54.881 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4191232 len 8192 2024-05-08T06:12:54.881 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4191232 len 8192 2024-05-08T06:12:54.881 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4191232 len 16384 2024-05-08T06:12:54.881 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4191232 len 16384 2024-05-08T06:12:54.881 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4191232 len 16384 2024-05-08T06:12:54.881 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4191232 len 16384 2024-05-08T06:12:54.881 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4191744 len 1024 2024-05-08T06:12:54.882 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4191744 len 1024 2024-05-08T06:12:54.882 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4191744 len 1024 2024-05-08T06:12:54.882 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4191744 len 1024 2024-05-08T06:12:54.882 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4191744 len 2048 2024-05-08T06:12:54.882 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4191744 len 2048 2024-05-08T06:12:54.882 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4191744 len 2048 2024-05-08T06:12:54.882 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4191744 len 2048 2024-05-08T06:12:54.882 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4191744 len 4096 2024-05-08T06:12:54.882 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4191744 len 4096 2024-05-08T06:12:54.882 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4191744 len 4096 2024-05-08T06:12:54.882 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4191744 len 4096 2024-05-08T06:12:54.883 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4191744 len 8192 2024-05-08T06:12:54.883 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4191744 len 8192 2024-05-08T06:12:54.883 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4191744 len 8192 2024-05-08T06:12:54.883 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4191744 len 8192 2024-05-08T06:12:54.883 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4191744 len 16384 2024-05-08T06:12:54.883 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4191744 len 16384 2024-05-08T06:12:54.883 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4191744 len 16384 2024-05-08T06:12:54.883 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4191744 len 16384 2024-05-08T06:12:54.883 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4192256 len 1024 2024-05-08T06:12:54.883 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4192256 len 1024 2024-05-08T06:12:54.883 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4192256 len 1024 2024-05-08T06:12:54.883 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4192256 len 1024 2024-05-08T06:12:54.883 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4192256 len 2048 2024-05-08T06:12:54.883 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4192256 len 2048 2024-05-08T06:12:54.884 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4192256 len 2048 2024-05-08T06:12:54.884 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4192256 len 2048 2024-05-08T06:12:54.884 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4192256 len 4096 2024-05-08T06:12:54.884 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4192256 len 4096 2024-05-08T06:12:54.884 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4192256 len 4096 2024-05-08T06:12:54.884 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4192256 len 4096 2024-05-08T06:12:54.884 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4192256 len 8192 2024-05-08T06:12:54.884 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4192256 len 8192 2024-05-08T06:12:54.884 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4192256 len 8192 2024-05-08T06:12:54.884 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4192256 len 8192 2024-05-08T06:12:54.884 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4192256 len 16384 2024-05-08T06:12:54.884 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4192256 len 16384 2024-05-08T06:12:54.884 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4192256 len 16384 2024-05-08T06:12:54.885 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4192256 len 16384 2024-05-08T06:12:54.885 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4192768 len 1024 2024-05-08T06:12:54.885 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4192768 len 1024 2024-05-08T06:12:54.885 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4192768 len 1024 2024-05-08T06:12:54.885 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4192768 len 1024 2024-05-08T06:12:54.885 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4192768 len 2048 2024-05-08T06:12:54.885 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4192768 len 2048 2024-05-08T06:12:54.885 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4192768 len 2048 2024-05-08T06:12:54.885 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4192768 len 2048 2024-05-08T06:12:54.885 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4192768 len 4096 2024-05-08T06:12:54.885 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4192768 len 4096 2024-05-08T06:12:54.885 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4192768 len 4096 2024-05-08T06:12:54.886 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4192768 len 4096 2024-05-08T06:12:54.886 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4192768 len 8192 2024-05-08T06:12:54.886 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4192768 len 8192 2024-05-08T06:12:54.886 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4192768 len 8192 2024-05-08T06:12:54.886 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4192768 len 8192 2024-05-08T06:12:54.886 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4192768 len 16384 2024-05-08T06:12:54.886 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4192768 len 16384 2024-05-08T06:12:54.886 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4192768 len 16384 2024-05-08T06:12:54.886 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4192768 len 16384 2024-05-08T06:12:54.886 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4193280 len 1024 2024-05-08T06:12:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.950+0000 7f697fe00640 1 -- 172.21.15.123:0/2427411614 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6980108510 msgr2=0x7f698010a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.950+0000 7f697fe00640 1 --2- 172.21.15.123:0/2427411614 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6980108510 0x7f698010a960 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6978009ca0 tx=0x7f697802a8c0 comp rx=0 tx=0).stop 2024-05-08T06:12:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.950+0000 7f697fe00640 1 -- 172.21.15.123:0/2427411614 shutdown_connections 2024-05-08T06:12:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.950+0000 7f697fe00640 1 --2- 172.21.15.123:0/2427411614 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6980108510 0x7f698010a960 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.950+0000 7f697fe00640 1 --2- 172.21.15.123:0/2427411614 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6980105be0 0x7f6980107fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.950+0000 7f697fe00640 1 --2- 172.21.15.123:0/2427411614 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69801032b0 0x7f69801056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.950+0000 7f697fe00640 1 -- 172.21.15.123:0/2427411614 >> 172.21.15.123:0/2427411614 conn(0x7f69800fcca0 msgr2=0x7f69800ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.950+0000 7f697fe00640 1 -- 172.21.15.123:0/2427411614 shutdown_connections 2024-05-08T06:12:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.950+0000 7f697fe00640 1 -- 172.21.15.123:0/2427411614 wait complete. 2024-05-08T06:12:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.951+0000 7f697fe00640 1 Processor -- start 2024-05-08T06:12:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.952+0000 7f697fe00640 1 -- start start 2024-05-08T06:12:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.952+0000 7f697fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69801032b0 0x7f698019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.952+0000 7f697fe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6980105be0 0x7f698019b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.952+0000 7f697fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6980108510 0x7f69801a21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.952+0000 7f697fe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f698010e360 con 0x7f69801032b0 2024-05-08T06:12:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.952+0000 7f697fe00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f698010e1e0 con 0x7f6980108510 2024-05-08T06:12:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.952+0000 7f697fe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f698010e4e0 con 0x7f6980105be0 2024-05-08T06:12:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.952+0000 7f6977e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6980105be0 0x7f698019b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.952+0000 7f697f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6980108510 0x7f69801a21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.952+0000 7f6977e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6980105be0 0x7f698019b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:33474/0 (socket says 172.21.15.123:33474) 2024-05-08T06:12:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.952+0000 7f6977e00640 1 -- 172.21.15.123:0/349618287 learned_addr learned my addr 172.21.15.123:0/349618287 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.952+0000 7f697ea00640 1 --2- 172.21.15.123:0/349618287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69801032b0 0x7f698019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.953+0000 7f697f400640 1 -- 172.21.15.123:0/349618287 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6980105be0 msgr2=0x7f698019b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.953+0000 7f697f400640 1 --2- 172.21.15.123:0/349618287 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6980105be0 0x7f698019b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.953+0000 7f697f400640 1 -- 172.21.15.123:0/349618287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69801032b0 msgr2=0x7f698019ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.953+0000 7f697f400640 1 --2- 172.21.15.123:0/349618287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69801032b0 0x7f698019ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.953+0000 7f697f400640 1 -- 172.21.15.123:0/349618287 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69801a2840 con 0x7f6980108510 2024-05-08T06:12:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.953+0000 7f6977e00640 1 --2- 172.21.15.123:0/349618287 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6980105be0 0x7f698019b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:12:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.953+0000 7f697f400640 1 --2- 172.21.15.123:0/349618287 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6980108510 0x7f69801a21a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6978031a20 tx=0x7f69780317a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.953+0000 7f6977400640 1 -- 172.21.15.123:0/349618287 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6978003c30 con 0x7f6980108510 2024-05-08T06:12:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.953+0000 7f697fe00640 1 -- 172.21.15.123:0/349618287 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69801a2ad0 con 0x7f6980108510 2024-05-08T06:12:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.954+0000 7f6977400640 1 -- 172.21.15.123:0/349618287 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6978003dd0 con 0x7f6980108510 2024-05-08T06:12:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.954+0000 7f6977400640 1 -- 172.21.15.123:0/349618287 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697802fb60 con 0x7f6980108510 2024-05-08T06:12:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.954+0000 7f697fe00640 1 -- 172.21.15.123:0/349618287 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69801a3070 con 0x7f6980108510 2024-05-08T06:12:54.956 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:54 smithi123 ceph-mon[29787]: pgmap v533: 97 pgs: 97 active+clean; 209 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 76 MiB/s rd, 692 KiB/s wr, 255 op/s 2024-05-08T06:12:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.957+0000 7f6977400640 1 -- 172.21.15.123:0/349618287 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7f697802fd20 con 0x7f6980108510 2024-05-08T06:12:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.959+0000 7f6988be7740 1 Processor -- start 2024-05-08T06:12:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.959+0000 7f6988be7740 1 -- start start 2024-05-08T06:12:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.959+0000 7f6988be7740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d5b028320 0x564d5b028740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.959+0000 7f6988be7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d5b028d10 0x564d5b02d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.959+0000 7f6988be7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564d5b02de80 0x564d5b030310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.959+0000 7f6977400640 1 --2- 172.21.15.123:0/349618287 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f69640b27e0 0x7f69640b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.959+0000 7f6988be7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564d5af8c550 con 0x564d5b028d10 2024-05-08T06:12:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.959+0000 7f6988be7740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x564d5af31bc0 con 0x564d5b028320 2024-05-08T06:12:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.959+0000 7f6988be7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564d5af88510 con 0x564d5b02de80 2024-05-08T06:12:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.960+0000 7f6977400640 1 -- 172.21.15.123:0/349618287 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f69781037e0 con 0x7f6980108510 2024-05-08T06:12:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.960+0000 7f697ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d5b028d10 0x564d5b02d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.960+0000 7f697ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d5b028d10 0x564d5b02d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56320/0 (socket says 172.21.15.123:56320) 2024-05-08T06:12:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.960+0000 7f697ea00640 1 -- 172.21.15.123:0/2698938521 learned_addr learned my addr 172.21.15.123:0/2698938521 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.960+0000 7f697f400640 1 --2- 172.21.15.123:0/349618287 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f69640b27e0 0x7f69640b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.960+0000 7f6977e00640 1 --2- 172.21.15.123:0/2698938521 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d5b028320 0x564d5b028740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.960+0000 7f6977e00640 1 --2- 172.21.15.123:0/2698938521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564d5b02de80 0x564d5b030310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.960+0000 7f697ea00640 1 -- 172.21.15.123:0/2698938521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564d5b02de80 msgr2=0x564d5b030310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.960+0000 7f697ea00640 1 --2- 172.21.15.123:0/2698938521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564d5b02de80 0x564d5b030310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.960+0000 7f697ea00640 1 -- 172.21.15.123:0/2698938521 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d5b028320 msgr2=0x564d5b028740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.960+0000 7f697ea00640 1 --2- 172.21.15.123:0/2698938521 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d5b028320 0x564d5b028740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.960+0000 7f697ea00640 1 -- 172.21.15.123:0/2698938521 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d5ac9e210 con 0x564d5b028d10 2024-05-08T06:12:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.960+0000 7f6977e00640 1 --2- 172.21.15.123:0/2698938521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564d5b02de80 0x564d5b030310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:12:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.961+0000 7f697f400640 1 --2- 172.21.15.123:0/349618287 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f69640b27e0 0x7f69640b4ca0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f69780c8020 tx=0x7f697802a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.961+0000 7f697ea00640 1 --2- 172.21.15.123:0/2698938521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d5b028d10 0x564d5b02d810 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6970009c90 tx=0x7f6970011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96dd26f5c6df8b67 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.961+0000 7f696fe00640 1 -- 172.21.15.123:0/2698938521 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6970005220 con 0x564d5b028d10 2024-05-08T06:12:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.961+0000 7f696fe00640 1 -- 172.21.15.123:0/2698938521 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6970004c50 con 0x564d5b028d10 2024-05-08T06:12:54.962 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.961+0000 7f696fe00640 1 -- 172.21.15.123:0/2698938521 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6970019690 con 0x564d5b028d10 2024-05-08T06:12:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.962+0000 7f6988be7740 1 -- 172.21.15.123:0/2698938521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d5b028d10 msgr2=0x564d5b02d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.962+0000 7f6988be7740 1 --2- 172.21.15.123:0/2698938521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d5b028d10 0x564d5b02d810 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6970009c90 tx=0x7f6970011810 comp rx=0 tx=0).stop 2024-05-08T06:12:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.963+0000 7f6988be7740 1 -- 172.21.15.123:0/2698938521 shutdown_connections 2024-05-08T06:12:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.963+0000 7f6988be7740 1 --2- 172.21.15.123:0/2698938521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564d5b02de80 0x564d5b030310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.963+0000 7f6988be7740 1 --2- 172.21.15.123:0/2698938521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d5b028d10 0x564d5b02d810 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.963+0000 7f6988be7740 1 --2- 172.21.15.123:0/2698938521 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d5b028320 0x564d5b028740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.963+0000 7f6988be7740 1 -- 172.21.15.123:0/2698938521 >> 172.21.15.123:0/2698938521 conn(0x564d5aff0230 msgr2=0x564d5b023da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.963+0000 7f6988be7740 1 -- 172.21.15.123:0/2698938521 shutdown_connections 2024-05-08T06:12:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.963+0000 7f6988be7740 1 -- 172.21.15.123:0/2698938521 wait complete. 2024-05-08T06:12:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.963+0000 7f6988be7740 1 -- start start 2024-05-08T06:12:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.964+0000 7f6988be7740 1 Processor -- start 2024-05-08T06:12:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.964+0000 7f6988be7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564d5b028320 0x564d5b032300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.964+0000 7f6988be7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d5b028d10 0x564d5b032840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.964+0000 7f6988be7740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d5b02de80 0x564d5b039880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.964+0000 7f6988be7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564d5b030e30 con 0x564d5b028d10 2024-05-08T06:12:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.964+0000 7f6988be7740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x564d5af45660 con 0x564d5b02de80 2024-05-08T06:12:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.964+0000 7f6988be7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564d5b030fb0 con 0x564d5b028320 2024-05-08T06:12:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.965+0000 7f6988be7740 10 client.0 _ll_register_callbacks cb 0x564d5af0f400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:12:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.965+0000 7f6977e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564d5b028320 0x564d5b032300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.965+0000 7f6977e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564d5b028320 0x564d5b032300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:33494/0 (socket says 172.21.15.123:33494) 2024-05-08T06:12:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.965+0000 7f6977e00640 1 -- 172.21.15.123:0/2649094613 learned_addr learned my addr 172.21.15.123:0/2649094613 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.965+0000 7f697ea00640 1 --2- 172.21.15.123:0/2649094613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d5b028d10 0x564d5b032840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.965+0000 7f6977e00640 1 --2- 172.21.15.123:0/2649094613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d5b02de80 0x564d5b039880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.965+0000 7f6977e00640 1 -- 172.21.15.123:0/2649094613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d5b02de80 msgr2=0x564d5b039880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.965+0000 7f6977e00640 1 --2- 172.21.15.123:0/2649094613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d5b02de80 0x564d5b039880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.965+0000 7f6977e00640 1 -- 172.21.15.123:0/2649094613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d5b028d10 msgr2=0x564d5b032840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.965+0000 7f6977e00640 1 --2- 172.21.15.123:0/2649094613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d5b028d10 0x564d5b032840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.965+0000 7f6977e00640 1 -- 172.21.15.123:0/2649094613 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d5ad6a620 con 0x564d5b028320 2024-05-08T06:12:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.966+0000 7f6977e00640 1 --2- 172.21.15.123:0/2649094613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564d5b028320 0x564d5b032300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6968013e50 tx=0x7f6968015040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1683ecfe0468d42b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.966+0000 7f696cc00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-05-08T06:12:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.966+0000 7f696cc00640 1 -- 172.21.15.123:0/2649094613 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6968004030 con 0x564d5b028320 2024-05-08T06:12:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.966+0000 7f696cc00640 1 -- 172.21.15.123:0/2649094613 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f69680041d0 con 0x564d5b028320 2024-05-08T06:12:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.966+0000 7f6988be7740 1 -- 172.21.15.123:0/2649094613 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6970004c50 con 0x564d5b028320 2024-05-08T06:12:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.966+0000 7f696cc00640 1 -- 172.21.15.123:0/2649094613 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f69680043d0 con 0x564d5b028320 2024-05-08T06:12:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.966+0000 7f6988be7740 10 client.25660 fetch_fsmap learned FSMap version 24 2024-05-08T06:12:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.966+0000 7f6988be7740 1 -- 172.21.15.123:0/2649094613 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564d5b03c890 con 0x564d5b028320 2024-05-08T06:12:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.967+0000 7f696cc00640 1 -- 172.21.15.123:0/2649094613 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6970004c50 con 0x564d5b028320 2024-05-08T06:12:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.967+0000 7f696cc00640 1 -- 172.21.15.123:0/2649094613 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f696800f5d0 con 0x564d5b028320 2024-05-08T06:12:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.967+0000 7f6988be7740 10 client.25660 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:12:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.967+0000 7f6988be7740 10 client.25660 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T06:12:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.967+0000 7f6988be7740 20 client.25660 populate_metadata read hostname 'smithi123' 2024-05-08T06:12:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.967+0000 7f6988be7740 1 --2- 172.21.15.123:0/2649094613 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x564d5b03df10 0x564d5b040380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.967+0000 7f6988be7740 4 client.25660 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:12:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.967+0000 7f6988be7740 1 -- 172.21.15.123:0/2649094613 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d5af9a160 con 0x564d5b03df10 2024-05-08T06:12:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.967+0000 7f697ea00640 1 --2- 172.21.15.123:0/2649094613 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x564d5b03df10 0x564d5b040380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.968+0000 7f697ea00640 1 --2- 172.21.15.123:0/2649094613 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x564d5b03df10 0x564d5b040380 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a657aa96f1b8468 server_cookie=dd5acacaf207f339 in_seq=0 out_seq=0 2024-05-08T06:12:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.968+0000 7f696cc00640 10 client.25660 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.969+0000 7f696cc00640 1 -- 172.21.15.123:0/2649094613 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564d5af9a160 con 0x564d5b03df10 2024-05-08T06:12:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.969+0000 7f696cc00640 10 client.25660 handle_command_reply: tid=0 2024-05-08T06:12:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.969+0000 7f6988be7740 1 client.25660 shutdown 2024-05-08T06:12:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.969+0000 7f6988be7740 1 -- 172.21.15.123:0/2649094613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564d5b028320 msgr2=0x564d5b032300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.969+0000 7f6988be7740 1 --2- 172.21.15.123:0/2649094613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564d5b028320 0x564d5b032300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6968013e50 tx=0x7f6968015040 comp rx=0 tx=0).stop 2024-05-08T06:12:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.969+0000 7f6988be7740 1 -- 172.21.15.123:0/2649094613 shutdown_connections 2024-05-08T06:12:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.969+0000 7f6988be7740 1 --2- 172.21.15.123:0/2649094613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d5b02de80 0x564d5b039880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.969+0000 7f6988be7740 1 --2- 172.21.15.123:0/2649094613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d5b028d10 0x564d5b032840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.969+0000 7f6988be7740 1 --2- 172.21.15.123:0/2649094613 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x564d5b03df10 0x564d5b040380 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.969+0000 7f6988be7740 1 --2- 172.21.15.123:0/2649094613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564d5b028320 0x564d5b032300 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.969+0000 7f696cc00640 0 client.25660 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.969+0000 7f6988be7740 1 -- 172.21.15.123:0/2649094613 >> 172.21.15.123:0/2649094613 conn(0x564d5aff0230 msgr2=0x564d5b022e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.969+0000 7f6988be7740 1 -- 172.21.15.123:0/2649094613 shutdown_connections 2024-05-08T06:12:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.969+0000 7f6988be7740 1 -- 172.21.15.123:0/2649094613 wait complete. 2024-05-08T06:12:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.969+0000 7f6988be7740 20 client.25660 trim_cache size 0 max 16384 2024-05-08T06:12:54.994 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.994+0000 7f6988be7740 1 Processor -- start 2024-05-08T06:12:54.995 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.994+0000 7f6988be7740 1 -- start start 2024-05-08T06:12:54.995 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.994+0000 7f6988be7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564d5b028320 0x564d5b023f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:54.995 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.994+0000 7f6988be7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d5b028d10 0x564d5b039840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:54.995 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.994+0000 7f6988be7740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d5b02de80 0x564d5b031f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:54.995 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.994+0000 7f6988be7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f69801ae610 con 0x564d5b028d10 2024-05-08T06:12:54.995 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.994+0000 7f6988be7740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f69801abee0 con 0x564d5b02de80 2024-05-08T06:12:54.995 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.994+0000 7f6988be7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f69801a9d30 con 0x564d5b028320 2024-05-08T06:12:54.995 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.994+0000 7f6977e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564d5b028320 0x564d5b023f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:54.995 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.994+0000 7f697ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d5b028d10 0x564d5b039840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:54.995 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.994+0000 7f6977e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564d5b028320 0x564d5b023f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:33500/0 (socket says 172.21.15.123:33500) 2024-05-08T06:12:54.995 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.994+0000 7f6977e00640 1 -- 172.21.15.123:0/149854016 learned_addr learned my addr 172.21.15.123:0/149854016 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:54.995 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.994+0000 7f6977e00640 1 --2- 172.21.15.123:0/149854016 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d5b02de80 0x564d5b031f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:54.996 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.995+0000 7f6977e00640 1 -- 172.21.15.123:0/149854016 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d5b02de80 msgr2=0x564d5b031f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:54.996 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.995+0000 7f6977e00640 1 --2- 172.21.15.123:0/149854016 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d5b02de80 0x564d5b031f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:54.996 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.995+0000 7f6977e00640 1 -- 172.21.15.123:0/149854016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d5b028d10 msgr2=0x564d5b039840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:54.996 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.995+0000 7f6977e00640 1 --2- 172.21.15.123:0/149854016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d5b028d10 0x564d5b039840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:54.996 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.995+0000 7f6977e00640 1 -- 172.21.15.123:0/149854016 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d5b024a50 con 0x564d5b028320 2024-05-08T06:12:54.998 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.995+0000 7f6977e00640 1 --2- 172.21.15.123:0/149854016 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564d5b028320 0x564d5b023f90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6968001eb0 tx=0x7f6968009530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20e8aa19924cdced server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:54.998 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.995+0000 7f696f400640 1 -- 172.21.15.123:0/149854016 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696800f7d0 con 0x564d5b028320 2024-05-08T06:12:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.996+0000 7f696f400640 1 -- 172.21.15.123:0/149854016 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f696800fc30 con 0x564d5b028320 2024-05-08T06:12:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.996+0000 7f696f400640 1 -- 172.21.15.123:0/149854016 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6968021dd0 con 0x564d5b028320 2024-05-08T06:12:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.996+0000 7f6988be7740 1 -- 172.21.15.123:0/149854016 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564d5b028320 msgr2=0x564d5b023f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.996+0000 7f6988be7740 1 --2- 172.21.15.123:0/149854016 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564d5b028320 0x564d5b023f90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6968001eb0 tx=0x7f6968009530 comp rx=0 tx=0).stop 2024-05-08T06:12:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.996+0000 7f6988be7740 1 -- 172.21.15.123:0/149854016 shutdown_connections 2024-05-08T06:12:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.996+0000 7f6988be7740 1 --2- 172.21.15.123:0/149854016 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d5b02de80 0x564d5b031f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.996+0000 7f6988be7740 1 --2- 172.21.15.123:0/149854016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d5b028d10 0x564d5b039840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.996+0000 7f6988be7740 1 --2- 172.21.15.123:0/149854016 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564d5b028320 0x564d5b023f90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.996+0000 7f6988be7740 1 -- 172.21.15.123:0/149854016 >> 172.21.15.123:0/149854016 conn(0x564d5aff0230 msgr2=0x564d5b023880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.997+0000 7f6988be7740 1 -- 172.21.15.123:0/149854016 shutdown_connections 2024-05-08T06:12:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.997+0000 7f6988be7740 1 -- 172.21.15.123:0/149854016 wait complete. 2024-05-08T06:12:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.997+0000 7f6988be7740 1 -- start start 2024-05-08T06:12:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.997+0000 7f6988be7740 1 Processor -- start 2024-05-08T06:12:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.998+0000 7f6988be7740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d5b028320 0x564d5b037300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.998+0000 7f6988be7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d5b028d10 0x564d5b037840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.998+0000 7f6988be7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564d5b02de80 0x564d5b038800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.998+0000 7f6988be7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f69801aa3a0 con 0x564d5b028d10 2024-05-08T06:12:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.998+0000 7f6988be7740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f69801ab810 con 0x564d5b028320 2024-05-08T06:12:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.998+0000 7f6988be7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f69801aaa70 con 0x564d5b02de80 2024-05-08T06:12:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.998+0000 7f6988be7740 10 client.0 _ll_register_callbacks cb 0x7f6968013180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:12:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.998+0000 7f6977e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d5b028320 0x564d5b037300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.998+0000 7f697ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d5b028d10 0x564d5b037840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.998+0000 7f6977e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d5b028320 0x564d5b037300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:37954/0 (socket says 172.21.15.123:37954) 2024-05-08T06:12:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.998+0000 7f6977e00640 1 -- 172.21.15.123:0/3287129655 learned_addr learned my addr 172.21.15.123:0/3287129655 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.998+0000 7f6977e00640 1 --2- 172.21.15.123:0/3287129655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564d5b02de80 0x564d5b038800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.999+0000 7f697ea00640 1 -- 172.21.15.123:0/3287129655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564d5b02de80 msgr2=0x564d5b038800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.999+0000 7f697ea00640 1 --2- 172.21.15.123:0/3287129655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564d5b02de80 0x564d5b038800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.999+0000 7f697ea00640 1 -- 172.21.15.123:0/3287129655 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d5b028320 msgr2=0x564d5b037300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.999+0000 7f697ea00640 1 --2- 172.21.15.123:0/3287129655 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d5b028320 0x564d5b037300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.999+0000 7f697ea00640 1 -- 172.21.15.123:0/3287129655 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d5b038df0 con 0x564d5b028d10 2024-05-08T06:12:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.999+0000 7f6977e00640 1 --2- 172.21.15.123:0/3287129655 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d5b028320 0x564d5b037300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:12:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.999+0000 7f697ea00640 1 --2- 172.21.15.123:0/3287129655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d5b028d10 0x564d5b037840 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x564d5b024c50 tx=0x7f6970009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da689de62b0e08a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:54.999+0000 7f696cc00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-08T06:12:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.000+0000 7f696cc00640 1 -- 172.21.15.123:0/3287129655 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6970031070 con 0x564d5b028d10 2024-05-08T06:12:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.000+0000 7f696cc00640 1 -- 172.21.15.123:0/3287129655 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f697002d3a0 con 0x564d5b028d10 2024-05-08T06:12:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.000+0000 7f6988be7740 1 -- 172.21.15.123:0/3287129655 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f696800fc30 con 0x564d5b028d10 2024-05-08T06:12:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.000+0000 7f696cc00640 1 -- 172.21.15.123:0/3287129655 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6970011d10 con 0x564d5b028d10 2024-05-08T06:12:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.000+0000 7f6988be7740 10 client.15558 fetch_fsmap learned FSMap version 24 2024-05-08T06:12:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.000+0000 7f6988be7740 1 -- 172.21.15.123:0/3287129655 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564d5b0a8c00 con 0x564d5b028d10 2024-05-08T06:12:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.001+0000 7f696cc00640 1 -- 172.21.15.123:0/3287129655 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f696800fc30 con 0x564d5b028d10 2024-05-08T06:12:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.001+0000 7f696cc00640 1 -- 172.21.15.123:0/3287129655 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f6970034020 con 0x564d5b028d10 2024-05-08T06:12:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.001+0000 7f6988be7740 10 client.15558 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:12:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.001+0000 7f6988be7740 10 client.15558 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T06:12:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.001+0000 7f6988be7740 20 client.15558 populate_metadata read hostname 'smithi123' 2024-05-08T06:12:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.001+0000 7f6988be7740 1 --2- 172.21.15.123:0/3287129655 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x564d5b0a9460 0x564d5b0ab850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.001+0000 7f6988be7740 4 client.15558 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T06:12:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.001+0000 7f6988be7740 1 -- 172.21.15.123:0/3287129655 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564d5af93fd0 con 0x564d5b0a9460 2024-05-08T06:12:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.001+0000 7f6977e00640 1 --2- 172.21.15.123:0/3287129655 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x564d5b0a9460 0x564d5b0ab850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.002+0000 7f6977e00640 1 --2- 172.21.15.123:0/3287129655 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x564d5b0a9460 0x564d5b0ab850 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec8a9c0b4deec298 server_cookie=842b6520694cee5d in_seq=0 out_seq=0 2024-05-08T06:12:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.002+0000 7f696cc00640 10 client.15558 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:55.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:54 smithi163 ceph-mon[34520]: pgmap v533: 97 pgs: 97 active+clean; 209 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 76 MiB/s rd, 692 KiB/s wr, 255 op/s 2024-05-08T06:12:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.048+0000 7f696cc00640 1 -- 172.21.15.123:0/3287129655 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564d5af93fd0 con 0x564d5b0a9460 2024-05-08T06:12:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.048+0000 7f696cc00640 10 client.15558 handle_command_reply: tid=0 2024-05-08T06:12:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.048+0000 7f6988be7740 1 client.15558 shutdown 2024-05-08T06:12:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.048+0000 7f6988be7740 1 -- 172.21.15.123:0/3287129655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d5b028d10 msgr2=0x564d5b037840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.048+0000 7f6988be7740 1 --2- 172.21.15.123:0/3287129655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d5b028d10 0x564d5b037840 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x564d5b024c50 tx=0x7f6970009f90 comp rx=0 tx=0).stop 2024-05-08T06:12:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.048+0000 7f6988be7740 1 -- 172.21.15.123:0/3287129655 shutdown_connections 2024-05-08T06:12:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.048+0000 7f6988be7740 1 --2- 172.21.15.123:0/3287129655 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x564d5b0a9460 0x564d5b0ab850 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.048+0000 7f6988be7740 1 --2- 172.21.15.123:0/3287129655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564d5b02de80 0x564d5b038800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.048+0000 7f6988be7740 1 --2- 172.21.15.123:0/3287129655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d5b028d10 0x564d5b037840 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.048+0000 7f696cc00640 0 client.15558 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.048+0000 7f6988be7740 1 --2- 172.21.15.123:0/3287129655 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d5b028320 0x564d5b037300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.048+0000 7f6988be7740 1 -- 172.21.15.123:0/3287129655 >> 172.21.15.123:0/3287129655 conn(0x564d5aff0230 msgr2=0x564d5b031f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.048+0000 7f6988be7740 1 -- 172.21.15.123:0/3287129655 shutdown_connections 2024-05-08T06:12:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.048+0000 7f6988be7740 1 -- 172.21.15.123:0/3287129655 wait complete. 2024-05-08T06:12:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.048+0000 7f6988be7740 20 client.15558 trim_cache size 0 max 16384 2024-05-08T06:12:55.050 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout:{ 2024-05-08T06:12:55.050 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout: "message": "", 2024-05-08T06:12:55.050 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout: "return_code": 0 2024-05-08T06:12:55.050 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stdout:} 2024-05-08T06:12:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.049+0000 7f697fe00640 1 -- 172.21.15.123:0/349618287 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f69640b27e0 msgr2=0x7f69640b4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.049+0000 7f697fe00640 1 --2- 172.21.15.123:0/349618287 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f69640b27e0 0x7f69640b4ca0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f69780c8020 tx=0x7f697802a8c0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.049+0000 7f697fe00640 1 -- 172.21.15.123:0/349618287 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6980108510 msgr2=0x7f69801a21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.049+0000 7f697fe00640 1 --2- 172.21.15.123:0/349618287 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6980108510 0x7f69801a21a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6978031a20 tx=0x7f69780317a0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.049+0000 7f697fe00640 1 -- 172.21.15.123:0/349618287 shutdown_connections 2024-05-08T06:12:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.049+0000 7f697fe00640 1 --2- 172.21.15.123:0/349618287 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f69640b27e0 0x7f69640b4ca0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.049+0000 7f697fe00640 1 --2- 172.21.15.123:0/349618287 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6980108510 0x7f69801a21a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.049+0000 7f697fe00640 1 --2- 172.21.15.123:0/349618287 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6980105be0 0x7f698019b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.049+0000 7f697fe00640 1 --2- 172.21.15.123:0/349618287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69801032b0 0x7f698019ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.049+0000 7f697fe00640 1 -- 172.21.15.123:0/349618287 >> 172.21.15.123:0/349618287 conn(0x7f69800fcca0 msgr2=0x7f6980102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.050+0000 7f697fe00640 1 -- 172.21.15.123:0/349618287 shutdown_connections 2024-05-08T06:12:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi123.front.sepia.ceph.com.smithi123.stderr:2024-05-08T06:12:55.050+0000 7f697fe00640 1 -- 172.21.15.123:0/349618287 wait complete. 2024-05-08T06:12:55.052 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:54 smithi192 ceph-mon[34664]: pgmap v533: 97 pgs: 97 active+clean; 209 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 76 MiB/s rd, 692 KiB/s wr, 255 op/s 2024-05-08T06:12:55.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.941+0000 7fa256800700 1 -- 172.21.15.123:0/1016988479 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa25006b180 msgr2=0x7fa25006b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:55.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.941+0000 7fa256800700 1 --2- 172.21.15.123:0/1016988479 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa25006b180 0x7fa25006b600 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa23c009990 tx=0x7fa23c009ca0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.941+0000 7fa256800700 1 -- 172.21.15.123:0/1016988479 shutdown_connections 2024-05-08T06:12:55.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.941+0000 7fa256800700 1 --2- 172.21.15.123:0/1016988479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa25006bb40 0x7fa250110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.941+0000 7fa256800700 1 --2- 172.21.15.123:0/1016988479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa250109510 0x7fa2501098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.941+0000 7fa256800700 1 --2- 172.21.15.123:0/1016988479 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa25006b180 0x7fa25006b600 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.941+0000 7fa256800700 1 -- 172.21.15.123:0/1016988479 >> 172.21.15.123:0/1016988479 conn(0x7fa250079350 msgr2=0x7fa250079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:55.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.941+0000 7fa256800700 1 -- 172.21.15.123:0/1016988479 shutdown_connections 2024-05-08T06:12:55.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.942+0000 7fa256800700 1 -- 172.21.15.123:0/1016988479 wait complete. 2024-05-08T06:12:55.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.943+0000 7fa256800700 1 Processor -- start 2024-05-08T06:12:55.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.943+0000 7fa256800700 1 -- start start 2024-05-08T06:12:55.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.944+0000 7fa256800700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa25006b180 0x7fa25019f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:55.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.944+0000 7fa256800700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa25006bb40 0x7fa25019fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:55.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.944+0000 7fa256800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa250109510 0x7fa250199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:55.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.944+0000 7fa256800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa250113690 con 0x7fa250109510 2024-05-08T06:12:55.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.944+0000 7fa256800700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa250113510 con 0x7fa25006b180 2024-05-08T06:12:55.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.944+0000 7fa256800700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa250113810 con 0x7fa25006bb40 2024-05-08T06:12:55.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.944+0000 7fa24ea00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa25006bb40 0x7fa25019fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:55.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.944+0000 7fa24f400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa25006b180 0x7fa25019f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:55.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.944+0000 7fa24ea00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa25006bb40 0x7fa25019fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:33526/0 (socket says 172.21.15.123:33526) 2024-05-08T06:12:55.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.944+0000 7fa24ea00700 1 -- 172.21.15.123:0/116381868 learned_addr learned my addr 172.21.15.123:0/116381868 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:55.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.944+0000 7fa24fe00700 1 --2- 172.21.15.123:0/116381868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa250109510 0x7fa250199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:55.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.944+0000 7fa24f400700 1 -- 172.21.15.123:0/116381868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa25006bb40 msgr2=0x7fa25019fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:55.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.944+0000 7fa24f400700 1 --2- 172.21.15.123:0/116381868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa25006bb40 0x7fa25019fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.944+0000 7fa24f400700 1 -- 172.21.15.123:0/116381868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa250109510 msgr2=0x7fa250199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:55.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.945+0000 7fa24f400700 1 --2- 172.21.15.123:0/116381868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa250109510 0x7fa250199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.945+0000 7fa24f400700 1 -- 172.21.15.123:0/116381868 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa25019a1d0 con 0x7fa25006b180 2024-05-08T06:12:55.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.945+0000 7fa24fe00700 1 --2- 172.21.15.123:0/116381868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa250109510 0x7fa250199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:12:55.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.945+0000 7fa24f400700 1 --2- 172.21.15.123:0/116381868 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa25006b180 0x7fa25019f8b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa24000e9f0 tx=0x7fa24000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:55.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.945+0000 7fa247e00700 1 -- 172.21.15.123:0/116381868 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24000cbd0 con 0x7fa25006b180 2024-05-08T06:12:55.946 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.945+0000 7fa247e00700 1 -- 172.21.15.123:0/116381868 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2400044d0 con 0x7fa25006b180 2024-05-08T06:12:55.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.945+0000 7fa256800700 1 -- 172.21.15.123:0/116381868 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa25019a4c0 con 0x7fa25006b180 2024-05-08T06:12:55.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.945+0000 7fa247e00700 1 -- 172.21.15.123:0/116381868 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa240005020 con 0x7fa25006b180 2024-05-08T06:12:55.947 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.946+0000 7fa256800700 1 -- 172.21.15.123:0/116381868 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2501a6780 con 0x7fa25006b180 2024-05-08T06:12:55.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.949+0000 7fa247e00700 1 -- 172.21.15.123:0/116381868 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7fa240004750 con 0x7fa25006b180 2024-05-08T06:12:55.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.951+0000 7fa261100b80 1 Processor -- start 2024-05-08T06:12:55.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.951+0000 7fa261100b80 1 -- start start 2024-05-08T06:12:55.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.952+0000 7fa247e00700 1 --2- 172.21.15.123:0/116381868 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa2340b2900 0x7fa2340b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:55.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.952+0000 7fa261100b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562200c920e0 0x562200c924c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:55.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.952+0000 7fa261100b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562200c92a00 0x562200c96510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:55.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.952+0000 7fa261100b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562200c96a50 0x562200c96eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:55.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.952+0000 7fa261100b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562200be4aa0 con 0x562200c96a50 2024-05-08T06:12:55.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.952+0000 7fa261100b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x562200b827e0 con 0x562200c92a00 2024-05-08T06:12:55.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.952+0000 7fa261100b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x562200be0a60 con 0x562200c920e0 2024-05-08T06:12:55.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.952+0000 7fa247e00700 1 -- 172.21.15.123:0/116381868 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fa2400e20e0 con 0x7fa25006b180 2024-05-08T06:12:55.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.952+0000 7fa24fe00700 1 --2- 172.21.15.123:0/116381868 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa2340b2900 0x7fa2340b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:55.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.953+0000 7fa24ea00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562200c92a00 0x562200c96510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:55.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.953+0000 7fa24fe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562200c96a50 0x562200c96eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:55.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.953+0000 7fa24ea00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562200c92a00 0x562200c96510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:37990/0 (socket says 172.21.15.123:37990) 2024-05-08T06:12:55.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.953+0000 7fa24ea00700 1 -- 172.21.15.123:0/385563253 learned_addr learned my addr 172.21.15.123:0/385563253 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:55.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.953+0000 7fa24f400700 1 --2- 172.21.15.123:0/385563253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562200c920e0 0x562200c924c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:55.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.953+0000 7fa24ea00700 1 -- 172.21.15.123:0/385563253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562200c920e0 msgr2=0x562200c924c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:55.954 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.953+0000 7fa24ea00700 1 --2- 172.21.15.123:0/385563253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562200c920e0 0x562200c924c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.955 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.953+0000 7fa24ea00700 1 -- 172.21.15.123:0/385563253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562200c96a50 msgr2=0x562200c96eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:55.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.953+0000 7fa24ea00700 1 --2- 172.21.15.123:0/385563253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562200c96a50 0x562200c96eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.953+0000 7fa24ea00700 1 -- 172.21.15.123:0/385563253 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562200b31950 con 0x562200c92a00 2024-05-08T06:12:55.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.953+0000 7fa24fe00700 1 --2- 172.21.15.123:0/385563253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562200c96a50 0x562200c96eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:12:55.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.954+0000 7fa24ea00700 1 --2- 172.21.15.123:0/385563253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562200c92a00 0x562200c96510 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa23c005f20 tx=0x7fa23c005bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e561a993481522d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:55.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.954+0000 7fa244c00700 1 -- 172.21.15.123:0/385563253 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c027070 con 0x562200c92a00 2024-05-08T06:12:55.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.954+0000 7fa244c00700 1 -- 172.21.15.123:0/385563253 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa23c0056a0 con 0x562200c92a00 2024-05-08T06:12:55.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.954+0000 7fa244c00700 1 -- 172.21.15.123:0/385563253 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c003d00 con 0x562200c92a00 2024-05-08T06:12:55.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.955+0000 7fa261100b80 1 -- 172.21.15.123:0/385563253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562200c92a00 msgr2=0x562200c96510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:55.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.955+0000 7fa24fe00700 1 --2- 172.21.15.123:0/116381868 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa2340b2900 0x7fa2340b4dc0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa248009550 tx=0x7fa2480090f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:55.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.955+0000 7fa261100b80 1 --2- 172.21.15.123:0/385563253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562200c92a00 0x562200c96510 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa23c005f20 tx=0x7fa23c005bc0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.955+0000 7fa261100b80 1 -- 172.21.15.123:0/385563253 shutdown_connections 2024-05-08T06:12:55.956 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.955+0000 7fa261100b80 1 --2- 172.21.15.123:0/385563253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562200c96a50 0x562200c96eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.955+0000 7fa261100b80 1 --2- 172.21.15.123:0/385563253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562200c920e0 0x562200c924c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.955+0000 7fa261100b80 1 --2- 172.21.15.123:0/385563253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562200c92a00 0x562200c96510 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.955+0000 7fa261100b80 1 -- 172.21.15.123:0/385563253 >> 172.21.15.123:0/385563253 conn(0x562200bc0be0 msgr2=0x562200c8c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:55.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.955+0000 7fa261100b80 1 -- 172.21.15.123:0/385563253 shutdown_connections 2024-05-08T06:12:55.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.955+0000 7fa261100b80 1 -- 172.21.15.123:0/385563253 wait complete. 2024-05-08T06:12:55.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.956+0000 7fa261100b80 1 -- start start 2024-05-08T06:12:55.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.956+0000 7fa261100b80 1 Processor -- start 2024-05-08T06:12:55.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.957+0000 7fa261100b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562200c920e0 0x562200ca5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:55.958 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.957+0000 7fa261100b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562200c92a00 0x562200ca60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:55.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.957+0000 7fa261100b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562200c96a50 0x562200c9ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:55.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.957+0000 7fa261100b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562200c9ec00 con 0x562200c92a00 2024-05-08T06:12:55.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.957+0000 7fa261100b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x562200b4aba0 con 0x562200c920e0 2024-05-08T06:12:55.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.957+0000 7fa261100b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x562200c9ed80 con 0x562200c96a50 2024-05-08T06:12:55.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.957+0000 7fa261100b80 10 client.0 _ll_register_callbacks cb 0x562200b8bcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:12:55.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.957+0000 7fa24f400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562200c920e0 0x562200ca5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:55.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.957+0000 7fa24ea00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562200c92a00 0x562200ca60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:55.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.957+0000 7fa24f400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562200c920e0 0x562200ca5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:37996/0 (socket says 172.21.15.123:37996) 2024-05-08T06:12:55.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.957+0000 7fa24f400700 1 -- 172.21.15.123:0/2644848388 learned_addr learned my addr 172.21.15.123:0/2644848388 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:55.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.958+0000 7fa24f400700 1 -- 172.21.15.123:0/2644848388 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562200c96a50 msgr2=0x562200c9ff90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T06:12:55.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.958+0000 7fa24f400700 1 --2- 172.21.15.123:0/2644848388 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562200c96a50 0x562200c9ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.958+0000 7fa24f400700 1 -- 172.21.15.123:0/2644848388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562200c92a00 msgr2=0x562200ca60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:55.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.958+0000 7fa24f400700 1 --2- 172.21.15.123:0/2644848388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562200c92a00 0x562200ca60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.958+0000 7fa24f400700 1 -- 172.21.15.123:0/2644848388 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562200baa4a0 con 0x562200c920e0 2024-05-08T06:12:55.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.958+0000 7fa24ea00700 1 --2- 172.21.15.123:0/2644848388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562200c92a00 0x562200ca60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:12:55.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.958+0000 7fa24f400700 1 --2- 172.21.15.123:0/2644848388 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562200c920e0 0x562200ca5bb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa2400dfb90 tx=0x7fa2400a9bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97da87e3b8b75b37 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:55.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.958+0000 7fa239600700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-05-08T06:12:55.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.958+0000 7fa239600700 1 -- 172.21.15.123:0/2644848388 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24001c040 con 0x562200c920e0 2024-05-08T06:12:55.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.958+0000 7fa239600700 1 -- 172.21.15.123:0/2644848388 <== mon.2 v2:172.21.15.163:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fa2400a5ca0 con 0x562200c920e0 2024-05-08T06:12:55.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.958+0000 7fa261100b80 1 -- 172.21.15.123:0/2644848388 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa23c0056a0 con 0x562200c920e0 2024-05-08T06:12:55.960 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4193280 len 1024 2024-05-08T06:12:55.960 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4193280 len 1024 2024-05-08T06:12:55.960 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4193280 len 1024 2024-05-08T06:12:55.960 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4193280 len 2048 2024-05-08T06:12:55.960 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4193280 len 2048 2024-05-08T06:12:55.961 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4193280 len 2048 2024-05-08T06:12:55.961 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4193280 len 2048 2024-05-08T06:12:55.961 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4193280 len 4096 2024-05-08T06:12:55.961 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4193280 len 4096 2024-05-08T06:12:55.961 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4193280 len 4096 2024-05-08T06:12:55.961 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4193280 len 4096 2024-05-08T06:12:55.961 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4193280 len 8192 2024-05-08T06:12:55.961 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4193280 len 8192 2024-05-08T06:12:55.961 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4193280 len 8192 2024-05-08T06:12:55.961 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4193280 len 8192 2024-05-08T06:12:55.961 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4193280 len 16384 2024-05-08T06:12:55.961 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4193280 len 16384 2024-05-08T06:12:55.962 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4193280 len 16384 2024-05-08T06:12:55.962 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4193280 len 16384 2024-05-08T06:12:55.962 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4193792 len 1024 2024-05-08T06:12:55.962 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4193792 len 1024 2024-05-08T06:12:55.962 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4193792 len 1024 2024-05-08T06:12:55.962 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4193792 len 1024 2024-05-08T06:12:55.962 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4193792 len 2048 2024-05-08T06:12:55.962 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4193792 len 2048 2024-05-08T06:12:55.962 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4193792 len 2048 2024-05-08T06:12:55.962 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4193792 len 2048 2024-05-08T06:12:55.962 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4193792 len 4096 2024-05-08T06:12:55.963 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4193792 len 4096 2024-05-08T06:12:55.963 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4193792 len 4096 2024-05-08T06:12:55.963 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4193792 len 4096 2024-05-08T06:12:55.963 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4193792 len 8192 2024-05-08T06:12:55.963 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4193792 len 8192 2024-05-08T06:12:55.963 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4193792 len 8192 2024-05-08T06:12:55.963 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4193792 len 8192 2024-05-08T06:12:55.963 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4193792 len 16384 2024-05-08T06:12:55.963 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4193792 len 16384 2024-05-08T06:12:55.963 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4193792 len 16384 2024-05-08T06:12:55.963 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4193792 len 16384 2024-05-08T06:12:55.963 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4194304 len 1024 2024-05-08T06:12:55.963 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4194304 len 1024 2024-05-08T06:12:55.964 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4194304 len 1024 2024-05-08T06:12:55.964 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4194304 len 1024 2024-05-08T06:12:55.964 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4194304 len 2048 2024-05-08T06:12:55.964 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4194304 len 2048 2024-05-08T06:12:55.964 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4194304 len 2048 2024-05-08T06:12:55.964 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4194304 len 2048 2024-05-08T06:12:55.964 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4194304 len 4096 2024-05-08T06:12:55.964 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4194304 len 4096 2024-05-08T06:12:55.964 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4194304 len 4096 2024-05-08T06:12:55.964 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4194304 len 4096 2024-05-08T06:12:55.965 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4194304 len 8192 2024-05-08T06:12:55.965 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4194304 len 8192 2024-05-08T06:12:55.965 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4194304 len 8192 2024-05-08T06:12:55.965 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4194304 len 8192 2024-05-08T06:12:55.965 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4194304 len 16384 2024-05-08T06:12:55.965 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4194304 len 16384 2024-05-08T06:12:55.965 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4194304 len 16384 2024-05-08T06:12:55.965 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4194304 len 16384 2024-05-08T06:12:55.965 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4194816 len 1024 2024-05-08T06:12:55.965 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4194816 len 1024 2024-05-08T06:12:55.966 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4194816 len 1024 2024-05-08T06:12:55.966 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4194816 len 1024 2024-05-08T06:12:55.966 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4194816 len 2048 2024-05-08T06:12:55.966 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4194816 len 2048 2024-05-08T06:12:55.966 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4194816 len 2048 2024-05-08T06:12:55.966 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4194816 len 2048 2024-05-08T06:12:55.966 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4194816 len 4096 2024-05-08T06:12:55.966 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4194816 len 4096 2024-05-08T06:12:55.967 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4194816 len 4096 2024-05-08T06:12:55.967 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4194816 len 4096 2024-05-08T06:12:55.967 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4194816 len 8192 2024-05-08T06:12:55.967 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4194816 len 8192 2024-05-08T06:12:55.967 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4194816 len 8192 2024-05-08T06:12:55.967 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4194816 len 8192 2024-05-08T06:12:55.967 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4194816 len 16384 2024-05-08T06:12:55.967 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4194816 len 16384 2024-05-08T06:12:55.967 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4194816 len 16384 2024-05-08T06:12:55.967 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4194816 len 16384 2024-05-08T06:12:55.967 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4195328 len 1024 2024-05-08T06:12:55.968 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4195328 len 1024 2024-05-08T06:12:55.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.959+0000 7fa239600700 1 -- 172.21.15.123:0/2644848388 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa2400a8e20 con 0x562200c920e0 2024-05-08T06:12:55.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.959+0000 7fa261100b80 10 client.25691 fetch_fsmap learned FSMap version 24 2024-05-08T06:12:55.969 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.959+0000 7fa261100b80 1 -- 172.21.15.123:0/2644848388 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562200cad860 con 0x562200c920e0 2024-05-08T06:12:55.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.959+0000 7fa239600700 1 -- 172.21.15.123:0/2644848388 <== mon.2 v2:172.21.15.163:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa23c0056a0 con 0x562200c920e0 2024-05-08T06:12:55.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.959+0000 7fa239600700 1 -- 172.21.15.123:0/2644848388 <== mon.2 v2:172.21.15.163:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fa2400a5020 con 0x562200c920e0 2024-05-08T06:12:55.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.959+0000 7fa261100b80 10 client.25691 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:12:55.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.959+0000 7fa261100b80 10 client.25691 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T06:12:55.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.959+0000 7fa261100b80 20 client.25691 populate_metadata read hostname 'smithi123' 2024-05-08T06:12:55.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.959+0000 7fa261100b80 1 --2- 172.21.15.123:0/2644848388 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x562200cafdf0 0x562200cb2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:55.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.959+0000 7fa261100b80 4 client.25691 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:12:55.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.959+0000 7fa261100b80 1 -- 172.21.15.123:0/2644848388 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562200c5f120 con 0x562200cafdf0 2024-05-08T06:12:55.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.960+0000 7fa24ea00700 1 --2- 172.21.15.123:0/2644848388 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x562200cafdf0 0x562200cb2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:55.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.960+0000 7fa24ea00700 1 --2- 172.21.15.123:0/2644848388 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x562200cafdf0 0x562200cb2220 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49d4da4d433dd396 server_cookie=70fb49743f3b9550 in_seq=0 out_seq=0 2024-05-08T06:12:55.970 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.960+0000 7fa239600700 10 client.25691 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:55.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.961+0000 7fa239600700 1 -- 172.21.15.123:0/2644848388 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562200c5f120 con 0x562200cafdf0 2024-05-08T06:12:55.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.961+0000 7fa239600700 10 client.25691 handle_command_reply: tid=0 2024-05-08T06:12:55.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.961+0000 7fa261100b80 1 client.25691 shutdown 2024-05-08T06:12:55.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.961+0000 7fa261100b80 1 -- 172.21.15.123:0/2644848388 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562200c920e0 msgr2=0x562200ca5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:55.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.961+0000 7fa261100b80 1 --2- 172.21.15.123:0/2644848388 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562200c920e0 0x562200ca5bb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa2400dfb90 tx=0x7fa2400a9bf0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.961+0000 7fa261100b80 1 -- 172.21.15.123:0/2644848388 shutdown_connections 2024-05-08T06:12:55.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.961+0000 7fa261100b80 1 --2- 172.21.15.123:0/2644848388 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x562200cafdf0 0x562200cb2220 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.961+0000 7fa261100b80 1 --2- 172.21.15.123:0/2644848388 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562200c96a50 0x562200c9ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.961+0000 7fa261100b80 1 --2- 172.21.15.123:0/2644848388 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562200c920e0 0x562200ca5bb0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.961+0000 7fa239600700 0 client.25691 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:55.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.961+0000 7fa261100b80 1 --2- 172.21.15.123:0/2644848388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562200c92a00 0x562200ca60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.961+0000 7fa261100b80 1 -- 172.21.15.123:0/2644848388 >> 172.21.15.123:0/2644848388 conn(0x562200bc0be0 msgr2=0x562200c8c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:55.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.962+0000 7fa261100b80 1 -- 172.21.15.123:0/2644848388 shutdown_connections 2024-05-08T06:12:55.971 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.962+0000 7fa261100b80 1 -- 172.21.15.123:0/2644848388 wait complete. 2024-05-08T06:12:55.972 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.962+0000 7fa261100b80 20 client.25691 trim_cache size 0 max 16384 2024-05-08T06:12:55.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.985+0000 7fa261100b80 1 Processor -- start 2024-05-08T06:12:55.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.985+0000 7fa261100b80 1 -- start start 2024-05-08T06:12:55.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.986+0000 7fa261100b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562200c920e0 0x562200c9ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:55.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.986+0000 7fa261100b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562200c92a00 0x562200c94c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:55.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.986+0000 7fa261100b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562200c96a50 0x562200c95140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:55.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.986+0000 7fa261100b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa2501ae100 con 0x562200c920e0 2024-05-08T06:12:55.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.986+0000 7fa261100b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa2501ab990 con 0x562200c92a00 2024-05-08T06:12:55.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.986+0000 7fa261100b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa2501a97e0 con 0x562200c96a50 2024-05-08T06:12:55.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.986+0000 7fa24ea00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562200c92a00 0x562200c94c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:55.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.986+0000 7fa24f400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562200c920e0 0x562200c9ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:55.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.986+0000 7fa24ea00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562200c92a00 0x562200c94c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:38010/0 (socket says 172.21.15.123:38010) 2024-05-08T06:12:55.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.986+0000 7fa24fe00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562200c96a50 0x562200c95140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:55.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.986+0000 7fa24ea00700 1 -- 172.21.15.123:0/2276538206 learned_addr learned my addr 172.21.15.123:0/2276538206 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:55.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.987+0000 7fa24ea00700 1 -- 172.21.15.123:0/2276538206 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562200c96a50 msgr2=0x562200c95140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:55.987 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.987+0000 7fa24ea00700 1 --2- 172.21.15.123:0/2276538206 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562200c96a50 0x562200c95140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.987+0000 7fa24ea00700 1 -- 172.21.15.123:0/2276538206 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562200c920e0 msgr2=0x562200c9ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:55.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.987+0000 7fa24ea00700 1 --2- 172.21.15.123:0/2276538206 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562200c920e0 0x562200c9ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.987+0000 7fa24ea00700 1 -- 172.21.15.123:0/2276538206 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562200c93580 con 0x562200c92a00 2024-05-08T06:12:55.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.987+0000 7fa24ea00700 1 --2- 172.21.15.123:0/2276538206 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562200c92a00 0x562200c94c00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa23c005ef0 tx=0x7fa23c00b280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=423657a3728f08a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:55.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.987+0000 7fa23be00700 1 -- 172.21.15.123:0/2276538206 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c027070 con 0x562200c92a00 2024-05-08T06:12:55.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.987+0000 7fa23be00700 1 -- 172.21.15.123:0/2276538206 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa23c028040 con 0x562200c92a00 2024-05-08T06:12:55.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.987+0000 7fa23be00700 1 -- 172.21.15.123:0/2276538206 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c034520 con 0x562200c92a00 2024-05-08T06:12:55.988 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.988+0000 7fa261100b80 1 -- 172.21.15.123:0/2276538206 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562200c92a00 msgr2=0x562200c94c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:55.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.988+0000 7fa261100b80 1 --2- 172.21.15.123:0/2276538206 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562200c92a00 0x562200c94c00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa23c005ef0 tx=0x7fa23c00b280 comp rx=0 tx=0).stop 2024-05-08T06:12:55.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.989+0000 7fa261100b80 1 -- 172.21.15.123:0/2276538206 shutdown_connections 2024-05-08T06:12:55.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.989+0000 7fa261100b80 1 --2- 172.21.15.123:0/2276538206 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562200c920e0 0x562200c9ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.989+0000 7fa261100b80 1 --2- 172.21.15.123:0/2276538206 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562200c96a50 0x562200c95140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.989+0000 7fa261100b80 1 --2- 172.21.15.123:0/2276538206 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562200c92a00 0x562200c94c00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.989+0000 7fa261100b80 1 -- 172.21.15.123:0/2276538206 >> 172.21.15.123:0/2276538206 conn(0x562200bc0be0 msgr2=0x562200c9f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:55.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.989+0000 7fa261100b80 1 -- 172.21.15.123:0/2276538206 shutdown_connections 2024-05-08T06:12:55.989 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.989+0000 7fa261100b80 1 -- 172.21.15.123:0/2276538206 wait complete. 2024-05-08T06:12:55.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.989+0000 7fa261100b80 1 -- start start 2024-05-08T06:12:55.990 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.990+0000 7fa261100b80 1 Processor -- start 2024-05-08T06:12:55.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.990+0000 7fa261100b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562200c920e0 0x562200ca6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:55.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.990+0000 7fa261100b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562200c92a00 0x562200ca7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:55.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.990+0000 7fa261100b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562200c96a50 0x562200caaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:55.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.990+0000 7fa261100b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa2501a9e50 con 0x562200c96a50 2024-05-08T06:12:55.991 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.990+0000 7fa261100b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa2501ab2c0 con 0x562200c920e0 2024-05-08T06:12:55.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.990+0000 7fa261100b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa2501aa520 con 0x562200c92a00 2024-05-08T06:12:55.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.991+0000 7fa261100b80 10 client.0 _ll_register_callbacks cb 0x562200b8bcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:12:55.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.991+0000 7fa24ea00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562200c92a00 0x562200ca7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:55.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.991+0000 7fa24ea00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562200c92a00 0x562200ca7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:33566/0 (socket says 172.21.15.123:33566) 2024-05-08T06:12:55.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.991+0000 7fa24ea00700 1 -- 172.21.15.123:0/3412545538 learned_addr learned my addr 172.21.15.123:0/3412545538 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:55.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.991+0000 7fa24f400700 1 --2- 172.21.15.123:0/3412545538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562200c920e0 0x562200ca6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:55.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.991+0000 7fa24ea00700 1 -- 172.21.15.123:0/3412545538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562200c920e0 msgr2=0x562200ca6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:55.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.991+0000 7fa24ea00700 1 --2- 172.21.15.123:0/3412545538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562200c920e0 0x562200ca6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.991+0000 7fa24ea00700 1 -- 172.21.15.123:0/3412545538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562200c96a50 msgr2=0x562200caaa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:55.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.991+0000 7fa24fe00700 1 --2- 172.21.15.123:0/3412545538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562200c96a50 0x562200caaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:55.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.991+0000 7fa24ea00700 1 --2- 172.21.15.123:0/3412545538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562200c96a50 0x562200caaa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.991+0000 7fa24ea00700 1 -- 172.21.15.123:0/3412545538 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562200caaf40 con 0x562200c92a00 2024-05-08T06:12:55.992 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.991+0000 7fa24f400700 1 --2- 172.21.15.123:0/3412545538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562200c920e0 0x562200ca6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:12:55.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.991+0000 7fa24fe00700 1 --2- 172.21.15.123:0/3412545538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562200c96a50 0x562200caaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:12:55.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.992+0000 7fa24ea00700 1 --2- 172.21.15.123:0/3412545538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562200c92a00 0x562200ca7000 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa23c009750 tx=0x7fa23c004f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c8bbff843a41374 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:55.993 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.992+0000 7fa239600700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-05-08T06:12:55.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.992+0000 7fa239600700 1 -- 172.21.15.123:0/3412545538 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c041060 con 0x562200c92a00 2024-05-08T06:12:55.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.992+0000 7fa239600700 1 -- 172.21.15.123:0/3412545538 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fa23c00b720 con 0x562200c92a00 2024-05-08T06:12:55.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.992+0000 7fa261100b80 1 -- 172.21.15.123:0/3412545538 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa23c028040 con 0x562200c92a00 2024-05-08T06:12:55.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.992+0000 7fa239600700 1 -- 172.21.15.123:0/3412545538 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa23c027040 con 0x562200c92a00 2024-05-08T06:12:55.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.992+0000 7fa261100b80 10 client.25681 fetch_fsmap learned FSMap version 24 2024-05-08T06:12:55.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.992+0000 7fa261100b80 1 -- 172.21.15.123:0/3412545538 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562200d1f710 con 0x562200c92a00 2024-05-08T06:12:55.994 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.992+0000 7fa239600700 1 -- 172.21.15.123:0/3412545538 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa23c028040 con 0x562200c92a00 2024-05-08T06:12:55.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.993+0000 7fa239600700 1 -- 172.21.15.123:0/3412545538 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fa23c02d040 con 0x562200c92a00 2024-05-08T06:12:55.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.993+0000 7fa261100b80 10 client.25681 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:12:55.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.993+0000 7fa261100b80 10 client.25681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T06:12:55.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.993+0000 7fa261100b80 20 client.25681 populate_metadata read hostname 'smithi123' 2024-05-08T06:12:55.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.993+0000 7fa261100b80 1 --2- 172.21.15.123:0/3412545538 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x562200d1f940 0x562200d21d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:55.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.993+0000 7fa261100b80 4 client.25681 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T06:12:55.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.993+0000 7fa261100b80 1 -- 172.21.15.123:0/3412545538 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562200bf2480 con 0x562200d1f940 2024-05-08T06:12:55.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.993+0000 7fa24f400700 1 --2- 172.21.15.123:0/3412545538 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x562200d1f940 0x562200d21d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:55.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.994+0000 7fa24f400700 1 --2- 172.21.15.123:0/3412545538 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x562200d1f940 0x562200d21d30 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d223f5df255d7e5 server_cookie=e917c09a88707d64 in_seq=0 out_seq=0 2024-05-08T06:12:55.995 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.994+0000 7fa239600700 10 client.25681 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:55.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.995+0000 7fa239600700 1 -- 172.21.15.123:0/3412545538 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562200bf2480 con 0x562200d1f940 2024-05-08T06:12:55.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.995+0000 7fa239600700 10 client.25681 handle_command_reply: tid=0 2024-05-08T06:12:55.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.995+0000 7fa261100b80 1 client.25681 shutdown 2024-05-08T06:12:55.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.995+0000 7fa261100b80 1 -- 172.21.15.123:0/3412545538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562200c92a00 msgr2=0x562200ca7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:55.996 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.995+0000 7fa261100b80 1 --2- 172.21.15.123:0/3412545538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562200c92a00 0x562200ca7000 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa23c009750 tx=0x7fa23c004f60 comp rx=0 tx=0).stop 2024-05-08T06:12:55.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.995+0000 7fa261100b80 1 -- 172.21.15.123:0/3412545538 shutdown_connections 2024-05-08T06:12:55.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.995+0000 7fa261100b80 1 --2- 172.21.15.123:0/3412545538 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x562200d1f940 0x562200d21d30 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.995+0000 7fa261100b80 1 --2- 172.21.15.123:0/3412545538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562200c96a50 0x562200caaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.995+0000 7fa261100b80 1 --2- 172.21.15.123:0/3412545538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562200c920e0 0x562200ca6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.995+0000 7fa261100b80 1 --2- 172.21.15.123:0/3412545538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562200c92a00 0x562200ca7000 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.995+0000 7fa239600700 0 client.25681 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:55.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.995+0000 7fa261100b80 1 -- 172.21.15.123:0/3412545538 >> 172.21.15.123:0/3412545538 conn(0x562200bc0be0 msgr2=0x562200c86aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:55.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.996+0000 7fa261100b80 1 -- 172.21.15.123:0/3412545538 shutdown_connections 2024-05-08T06:12:55.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.996+0000 7fa261100b80 1 -- 172.21.15.123:0/3412545538 wait complete. 2024-05-08T06:12:55.997 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.996+0000 7fa261100b80 20 client.25681 trim_cache size 0 max 16384 2024-05-08T06:12:55.997 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-05-08T06:12:55.998 INFO:teuthology.orchestra.run.smithi123.stdout: "return_code": 0, 2024-05-08T06:12:55.998 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_tag": "a5574e44-adbc-4cfd-b305-22776ae5dc48", 2024-05-08T06:12:55.998 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": "asynchronous" 2024-05-08T06:12:55.998 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-05-08T06:12:55.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.997+0000 7fa256800700 1 -- 172.21.15.123:0/116381868 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa2340b2900 msgr2=0x7fa2340b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:55.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.997+0000 7fa256800700 1 --2- 172.21.15.123:0/116381868 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa2340b2900 0x7fa2340b4dc0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa248009550 tx=0x7fa2480090f0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.997+0000 7fa256800700 1 -- 172.21.15.123:0/116381868 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa25006b180 msgr2=0x7fa25019f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:55.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.997+0000 7fa256800700 1 --2- 172.21.15.123:0/116381868 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa25006b180 0x7fa25019f8b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa24000e9f0 tx=0x7fa24000ed00 comp rx=0 tx=0).stop 2024-05-08T06:12:55.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.997+0000 7fa256800700 1 -- 172.21.15.123:0/116381868 shutdown_connections 2024-05-08T06:12:55.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.997+0000 7fa256800700 1 --2- 172.21.15.123:0/116381868 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fa2340b2900 0x7fa2340b4dc0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.997+0000 7fa256800700 1 --2- 172.21.15.123:0/116381868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa250109510 0x7fa250199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.997+0000 7fa256800700 1 --2- 172.21.15.123:0/116381868 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa25006b180 0x7fa25019f8b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:55.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.997+0000 7fa256800700 1 --2- 172.21.15.123:0/116381868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa25006bb40 0x7fa25019fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:56.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.997+0000 7fa256800700 1 -- 172.21.15.123:0/116381868 >> 172.21.15.123:0/116381868 conn(0x7fa250079350 msgr2=0x7fa250101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:56.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.997+0000 7fa256800700 1 -- 172.21.15.123:0/116381868 shutdown_connections 2024-05-08T06:12:56.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:55.997+0000 7fa256800700 1 -- 172.21.15.123:0/116381868 wait complete. 2024-05-08T06:12:56.440 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4195328 len 1024 2024-05-08T06:12:56.440 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4195328 len 1024 2024-05-08T06:12:56.440 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4195328 len 2048 2024-05-08T06:12:56.440 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4195328 len 2048 2024-05-08T06:12:56.440 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4195328 len 2048 2024-05-08T06:12:56.440 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4195328 len 2048 2024-05-08T06:12:56.440 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4195328 len 4096 2024-05-08T06:12:56.440 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4195328 len 4096 2024-05-08T06:12:56.440 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4195328 len 4096 2024-05-08T06:12:56.440 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4195328 len 4096 2024-05-08T06:12:56.440 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4195328 len 8192 2024-05-08T06:12:56.440 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4195328 len 8192 2024-05-08T06:12:56.440 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4195328 len 8192 2024-05-08T06:12:56.440 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4195328 len 8192 2024-05-08T06:12:56.441 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4195328 len 16384 2024-05-08T06:12:56.441 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4195328 len 16384 2024-05-08T06:12:56.441 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4195328 len 16384 2024-05-08T06:12:56.441 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4195328 len 16384 2024-05-08T06:12:56.441 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4195840 len 1024 2024-05-08T06:12:56.441 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4195840 len 1024 2024-05-08T06:12:56.441 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4195840 len 1024 2024-05-08T06:12:56.441 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4195840 len 1024 2024-05-08T06:12:56.441 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4195840 len 2048 2024-05-08T06:12:56.441 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4195840 len 2048 2024-05-08T06:12:56.441 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4195840 len 2048 2024-05-08T06:12:56.441 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4195840 len 2048 2024-05-08T06:12:56.441 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4195840 len 4096 2024-05-08T06:12:56.441 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4195840 len 4096 2024-05-08T06:12:56.441 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4195840 len 4096 2024-05-08T06:12:56.441 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4195840 len 4096 2024-05-08T06:12:56.442 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4195840 len 8192 2024-05-08T06:12:56.442 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4195840 len 8192 2024-05-08T06:12:56.442 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4195840 len 8192 2024-05-08T06:12:56.442 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4195840 len 8192 2024-05-08T06:12:56.442 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4195840 len 16384 2024-05-08T06:12:56.442 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4195840 len 16384 2024-05-08T06:12:56.442 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4195840 len 16384 2024-05-08T06:12:56.442 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4195840 len 16384 2024-05-08T06:12:56.442 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4196352 len 1024 2024-05-08T06:12:56.442 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4196352 len 1024 2024-05-08T06:12:56.442 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4196352 len 1024 2024-05-08T06:12:56.442 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4196352 len 1024 2024-05-08T06:12:56.443 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4196352 len 2048 2024-05-08T06:12:56.443 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4196352 len 2048 2024-05-08T06:12:56.443 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4196352 len 2048 2024-05-08T06:12:56.443 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4196352 len 2048 2024-05-08T06:12:56.443 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4196352 len 4096 2024-05-08T06:12:56.443 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4196352 len 4096 2024-05-08T06:12:56.443 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4196352 len 4096 2024-05-08T06:12:56.443 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4196352 len 4096 2024-05-08T06:12:56.443 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4196352 len 8192 2024-05-08T06:12:56.443 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4196352 len 8192 2024-05-08T06:12:56.443 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4196352 len 8192 2024-05-08T06:12:56.443 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4196352 len 8192 2024-05-08T06:12:56.443 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4196352 len 16384 2024-05-08T06:12:56.443 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4196352 len 16384 2024-05-08T06:12:56.444 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4196352 len 16384 2024-05-08T06:12:56.444 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4196352 len 16384 2024-05-08T06:12:56.444 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4196864 len 1024 2024-05-08T06:12:56.444 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4196864 len 1024 2024-05-08T06:12:56.444 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4196864 len 1024 2024-05-08T06:12:56.444 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4196864 len 1024 2024-05-08T06:12:56.444 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4196864 len 2048 2024-05-08T06:12:56.444 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4196864 len 2048 2024-05-08T06:12:56.444 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4196864 len 2048 2024-05-08T06:12:56.444 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4196864 len 2048 2024-05-08T06:12:56.444 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4196864 len 4096 2024-05-08T06:12:56.444 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4196864 len 4096 2024-05-08T06:12:56.444 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4196864 len 4096 2024-05-08T06:12:56.445 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4196864 len 4096 2024-05-08T06:12:56.445 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4196864 len 8192 2024-05-08T06:12:56.445 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4196864 len 8192 2024-05-08T06:12:56.445 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4196864 len 8192 2024-05-08T06:12:56.445 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4196864 len 8192 2024-05-08T06:12:56.445 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4196864 len 16384 2024-05-08T06:12:56.445 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4196864 len 16384 2024-05-08T06:12:56.445 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4196864 len 16384 2024-05-08T06:12:56.445 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4196864 len 16384 2024-05-08T06:12:56.445 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4197376 len 1024 2024-05-08T06:12:56.445 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4197376 len 1024 2024-05-08T06:12:56.445 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4197376 len 1024 2024-05-08T06:12:56.840 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:56 smithi123 ceph-mon[29787]: pgmap v534: 97 pgs: 97 active+clean; 209 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 69 MiB/s rd, 698 KiB/s wr, 244 op/s 2024-05-08T06:12:56.885 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-08T06:12:57.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:56 smithi163 ceph-mon[34520]: pgmap v534: 97 pgs: 97 active+clean; 209 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 69 MiB/s rd, 698 KiB/s wr, 244 op/s 2024-05-08T06:12:57.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:56 smithi192 ceph-mon[34664]: pgmap v534: 97 pgs: 97 active+clean; 209 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 69 MiB/s rd, 698 KiB/s wr, 244 op/s 2024-05-08T06:12:57.330 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:12:57.435 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4197376 len 1024 2024-05-08T06:12:57.435 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4197376 len 2048 2024-05-08T06:12:57.436 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4197376 len 2048 2024-05-08T06:12:57.436 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4197376 len 2048 2024-05-08T06:12:57.436 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4197376 len 2048 2024-05-08T06:12:57.436 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4197376 len 4096 2024-05-08T06:12:57.436 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4197376 len 4096 2024-05-08T06:12:57.436 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4197376 len 4096 2024-05-08T06:12:57.436 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4197376 len 4096 2024-05-08T06:12:57.436 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4197376 len 8192 2024-05-08T06:12:57.436 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4197376 len 8192 2024-05-08T06:12:57.436 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4197376 len 8192 2024-05-08T06:12:57.436 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4197376 len 8192 2024-05-08T06:12:57.436 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4197376 len 16384 2024-05-08T06:12:57.436 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4197376 len 16384 2024-05-08T06:12:57.436 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4197376 len 16384 2024-05-08T06:12:57.437 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4197376 len 16384 2024-05-08T06:12:57.437 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4197888 len 1024 2024-05-08T06:12:57.437 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4197888 len 1024 2024-05-08T06:12:57.437 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4197888 len 1024 2024-05-08T06:12:57.437 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4197888 len 1024 2024-05-08T06:12:57.437 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4197888 len 2048 2024-05-08T06:12:57.437 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4197888 len 2048 2024-05-08T06:12:57.437 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4197888 len 2048 2024-05-08T06:12:57.437 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4197888 len 2048 2024-05-08T06:12:57.437 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4197888 len 4096 2024-05-08T06:12:57.437 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4197888 len 4096 2024-05-08T06:12:57.437 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4197888 len 4096 2024-05-08T06:12:57.437 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4197888 len 4096 2024-05-08T06:12:57.437 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4197888 len 8192 2024-05-08T06:12:57.438 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4197888 len 8192 2024-05-08T06:12:57.438 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4197888 len 8192 2024-05-08T06:12:57.438 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4197888 len 8192 2024-05-08T06:12:57.438 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3072 offset 4197888 len 16384 2024-05-08T06:12:57.438 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4197888 len 16384 2024-05-08T06:12:57.438 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3072 offset 4197888 len 16384 2024-05-08T06:12:57.438 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3072 offset 4197888 len 16384 2024-05-08T06:12:57.438 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4190720 len 1024 2024-05-08T06:12:57.438 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4190720 len 1024 2024-05-08T06:12:57.438 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4190720 len 1024 2024-05-08T06:12:57.438 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4190720 len 1024 2024-05-08T06:12:57.438 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4190720 len 2048 2024-05-08T06:12:57.438 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4190720 len 2048 2024-05-08T06:12:57.438 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4190720 len 2048 2024-05-08T06:12:57.438 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4190720 len 2048 2024-05-08T06:12:57.439 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4190720 len 4096 2024-05-08T06:12:57.439 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4190720 len 4096 2024-05-08T06:12:57.439 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4190720 len 4096 2024-05-08T06:12:57.439 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4190720 len 4096 2024-05-08T06:12:57.439 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4190720 len 8192 2024-05-08T06:12:57.439 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4190720 len 8192 2024-05-08T06:12:57.439 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4190720 len 8192 2024-05-08T06:12:57.439 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4190720 len 8192 2024-05-08T06:12:57.439 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4190720 len 16384 2024-05-08T06:12:57.439 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4190720 len 16384 2024-05-08T06:12:57.439 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4190720 len 16384 2024-05-08T06:12:57.439 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4190720 len 16384 2024-05-08T06:12:57.439 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4191232 len 1024 2024-05-08T06:12:57.439 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4191232 len 1024 2024-05-08T06:12:57.439 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4191232 len 1024 2024-05-08T06:12:57.440 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4191232 len 1024 2024-05-08T06:12:57.440 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4191232 len 2048 2024-05-08T06:12:57.440 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4191232 len 2048 2024-05-08T06:12:57.440 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4191232 len 2048 2024-05-08T06:12:57.440 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4191232 len 2048 2024-05-08T06:12:57.440 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4191232 len 4096 2024-05-08T06:12:57.440 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4191232 len 4096 2024-05-08T06:12:57.440 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4191232 len 4096 2024-05-08T06:12:57.440 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4191232 len 4096 2024-05-08T06:12:57.440 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4191232 len 8192 2024-05-08T06:12:57.440 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4191232 len 8192 2024-05-08T06:12:57.440 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4191232 len 8192 2024-05-08T06:12:57.440 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4191232 len 8192 2024-05-08T06:12:57.440 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4191232 len 16384 2024-05-08T06:12:57.440 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4191232 len 16384 2024-05-08T06:12:57.440 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4191232 len 16384 2024-05-08T06:12:57.441 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4191232 len 16384 2024-05-08T06:12:57.441 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4191744 len 1024 2024-05-08T06:12:57.441 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4191744 len 1024 2024-05-08T06:12:57.441 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4191744 len 1024 2024-05-08T06:12:57.441 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4191744 len 1024 2024-05-08T06:12:57.736 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:57 smithi123 ceph-mon[29787]: scrub queued for path: / 2024-05-08T06:12:57.736 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:57 smithi123 ceph-mon[29787]: scrub summary: idle+waiting paths [/] 2024-05-08T06:12:57.736 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:57 smithi123 ceph-mon[29787]: scrub summary: active paths [/] 2024-05-08T06:12:57.736 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:57 smithi123 ceph-mon[29787]: scrub summary: idle+waiting paths [/] 2024-05-08T06:12:58.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:57 smithi163 ceph-mon[34520]: scrub queued for path: / 2024-05-08T06:12:58.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:57 smithi163 ceph-mon[34520]: scrub summary: idle+waiting paths [/] 2024-05-08T06:12:58.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:57 smithi163 ceph-mon[34520]: scrub summary: active paths [/] 2024-05-08T06:12:58.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:57 smithi163 ceph-mon[34520]: scrub summary: idle+waiting paths [/] 2024-05-08T06:12:58.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:57 smithi192 ceph-mon[34664]: scrub queued for path: / 2024-05-08T06:12:58.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:57 smithi192 ceph-mon[34664]: scrub summary: idle+waiting paths [/] 2024-05-08T06:12:58.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:57 smithi192 ceph-mon[34664]: scrub summary: active paths [/] 2024-05-08T06:12:58.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:57 smithi192 ceph-mon[34664]: scrub summary: idle+waiting paths [/] 2024-05-08T06:12:58.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:12:58 smithi123 ceph-mon[29787]: pgmap v535: 97 pgs: 97 active+clean; 187 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 68 MiB/s rd, 634 KiB/s wr, 238 op/s 2024-05-08T06:12:59.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:12:58 smithi163 ceph-mon[34520]: pgmap v535: 97 pgs: 97 active+clean; 187 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 68 MiB/s rd, 634 KiB/s wr, 238 op/s 2024-05-08T06:12:59.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:12:58 smithi192 ceph-mon[34664]: pgmap v535: 97 pgs: 97 active+clean; 187 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 68 MiB/s rd, 634 KiB/s wr, 238 op/s 2024-05-08T06:12:59.131 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4191744 len 2048 2024-05-08T06:12:59.131 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4191744 len 2048 2024-05-08T06:12:59.131 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4191744 len 2048 2024-05-08T06:12:59.131 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4191744 len 2048 2024-05-08T06:12:59.131 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4191744 len 4096 2024-05-08T06:12:59.131 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4191744 len 4096 2024-05-08T06:12:59.131 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4191744 len 4096 2024-05-08T06:12:59.131 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4191744 len 4096 2024-05-08T06:12:59.131 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4191744 len 8192 2024-05-08T06:12:59.131 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4191744 len 8192 2024-05-08T06:12:59.132 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4191744 len 8192 2024-05-08T06:12:59.132 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4191744 len 8192 2024-05-08T06:12:59.132 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4191744 len 16384 2024-05-08T06:12:59.132 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4191744 len 16384 2024-05-08T06:12:59.132 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4191744 len 16384 2024-05-08T06:12:59.132 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4191744 len 16384 2024-05-08T06:12:59.132 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4192256 len 1024 2024-05-08T06:12:59.132 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4192256 len 1024 2024-05-08T06:12:59.132 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4192256 len 1024 2024-05-08T06:12:59.132 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4192256 len 1024 2024-05-08T06:12:59.132 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4192256 len 2048 2024-05-08T06:12:59.132 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4192256 len 2048 2024-05-08T06:12:59.132 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4192256 len 2048 2024-05-08T06:12:59.132 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4192256 len 2048 2024-05-08T06:12:59.132 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4192256 len 4096 2024-05-08T06:12:59.133 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4192256 len 4096 2024-05-08T06:12:59.133 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4192256 len 4096 2024-05-08T06:12:59.133 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4192256 len 4096 2024-05-08T06:12:59.133 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4192256 len 8192 2024-05-08T06:12:59.133 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4192256 len 8192 2024-05-08T06:12:59.133 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4192256 len 8192 2024-05-08T06:12:59.133 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4192256 len 8192 2024-05-08T06:12:59.133 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4192256 len 16384 2024-05-08T06:12:59.133 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4192256 len 16384 2024-05-08T06:12:59.133 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4192256 len 16384 2024-05-08T06:12:59.133 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4192256 len 16384 2024-05-08T06:12:59.133 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4192768 len 1024 2024-05-08T06:12:59.133 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4192768 len 1024 2024-05-08T06:12:59.133 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4192768 len 1024 2024-05-08T06:12:59.133 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4192768 len 1024 2024-05-08T06:12:59.133 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4192768 len 2048 2024-05-08T06:12:59.134 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4192768 len 2048 2024-05-08T06:12:59.134 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4192768 len 2048 2024-05-08T06:12:59.134 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4192768 len 2048 2024-05-08T06:12:59.134 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4192768 len 4096 2024-05-08T06:12:59.134 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4192768 len 4096 2024-05-08T06:12:59.134 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4192768 len 4096 2024-05-08T06:12:59.134 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4192768 len 4096 2024-05-08T06:12:59.134 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4192768 len 8192 2024-05-08T06:12:59.134 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4192768 len 8192 2024-05-08T06:12:59.135 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4192768 len 8192 2024-05-08T06:12:59.135 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4192768 len 8192 2024-05-08T06:12:59.135 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4192768 len 16384 2024-05-08T06:12:59.135 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4192768 len 16384 2024-05-08T06:12:59.135 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4192768 len 16384 2024-05-08T06:12:59.135 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4192768 len 16384 2024-05-08T06:12:59.135 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4193280 len 1024 2024-05-08T06:12:59.135 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4193280 len 1024 2024-05-08T06:12:59.135 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4193280 len 1024 2024-05-08T06:12:59.135 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4193280 len 1024 2024-05-08T06:12:59.135 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4193280 len 2048 2024-05-08T06:12:59.135 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4193280 len 2048 2024-05-08T06:12:59.135 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4193280 len 2048 2024-05-08T06:12:59.136 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4193280 len 2048 2024-05-08T06:12:59.136 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4193280 len 4096 2024-05-08T06:12:59.136 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4193280 len 4096 2024-05-08T06:12:59.136 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4193280 len 4096 2024-05-08T06:12:59.136 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4193280 len 4096 2024-05-08T06:12:59.136 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4193280 len 8192 2024-05-08T06:12:59.136 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4193280 len 8192 2024-05-08T06:12:59.136 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4193280 len 8192 2024-05-08T06:12:59.136 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4193280 len 8192 2024-05-08T06:12:59.136 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4193280 len 16384 2024-05-08T06:12:59.136 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4193280 len 16384 2024-05-08T06:12:59.137 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4193280 len 16384 2024-05-08T06:12:59.137 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4193280 len 16384 2024-05-08T06:12:59.137 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4193792 len 1024 2024-05-08T06:12:59.137 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4193792 len 1024 2024-05-08T06:12:59.137 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4193792 len 1024 2024-05-08T06:12:59.137 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4193792 len 1024 2024-05-08T06:12:59.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.732+0000 7fd235e00700 1 -- 172.21.15.123:0/2508246729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd23010a490 msgr2=0x7fd2301169a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:59.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.732+0000 7fd235e00700 1 --2- 172.21.15.123:0/2508246729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd23010a490 0x7fd2301169a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd22800b0a0 tx=0x7fd22800b3b0 comp rx=0 tx=0).stop 2024-05-08T06:12:59.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.732+0000 7fd235e00700 1 -- 172.21.15.123:0/2508246729 shutdown_connections 2024-05-08T06:12:59.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.732+0000 7fd235e00700 1 --2- 172.21.15.123:0/2508246729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd23010a490 0x7fd2301169a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:59.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.732+0000 7fd235e00700 1 --2- 172.21.15.123:0/2508246729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd23010fb60 0x7fd23010ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:59.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.732+0000 7fd235e00700 1 --2- 172.21.15.123:0/2508246729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd230109ad0 0x7fd230109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:59.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.732+0000 7fd235e00700 1 -- 172.21.15.123:0/2508246729 >> 172.21.15.123:0/2508246729 conn(0x7fd230076c70 msgr2=0x7fd2301082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:59.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.732+0000 7fd235e00700 1 -- 172.21.15.123:0/2508246729 shutdown_connections 2024-05-08T06:12:59.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.732+0000 7fd235e00700 1 -- 172.21.15.123:0/2508246729 wait complete. 2024-05-08T06:12:59.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.733+0000 7fd235e00700 1 Processor -- start 2024-05-08T06:12:59.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.734+0000 7fd235e00700 1 -- start start 2024-05-08T06:12:59.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.734+0000 7fd235e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd230109ad0 0x7fd2301a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:59.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.734+0000 7fd235e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd23010a490 0x7fd2301a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:59.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.734+0000 7fd235e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd23010fb60 0x7fd2301a4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:59.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.734+0000 7fd235e00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd230119d60 con 0x7fd23010a490 2024-05-08T06:12:59.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.734+0000 7fd235e00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd230119be0 con 0x7fd230109ad0 2024-05-08T06:12:59.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.734+0000 7fd235e00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd230119ee0 con 0x7fd23010fb60 2024-05-08T06:12:59.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.734+0000 7fd22ea00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd230109ad0 0x7fd2301a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:59.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.735+0000 7fd22ea00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd230109ad0 0x7fd2301a01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:38036/0 (socket says 172.21.15.123:38036) 2024-05-08T06:12:59.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.735+0000 7fd22ea00700 1 -- 172.21.15.123:0/3315429631 learned_addr learned my addr 172.21.15.123:0/3315429631 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:59.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.735+0000 7fd22f400700 1 --2- 172.21.15.123:0/3315429631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd23010fb60 0x7fd2301a4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:59.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.735+0000 7fd22e000700 1 --2- 172.21.15.123:0/3315429631 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd23010a490 0x7fd2301a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:59.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.735+0000 7fd22ea00700 1 -- 172.21.15.123:0/3315429631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd23010fb60 msgr2=0x7fd2301a4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:59.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.735+0000 7fd22ea00700 1 --2- 172.21.15.123:0/3315429631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd23010fb60 0x7fd2301a4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:59.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.735+0000 7fd22ea00700 1 -- 172.21.15.123:0/3315429631 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd23010a490 msgr2=0x7fd2301a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:59.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.735+0000 7fd22ea00700 1 --2- 172.21.15.123:0/3315429631 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd23010a490 0x7fd2301a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:59.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.735+0000 7fd22ea00700 1 -- 172.21.15.123:0/3315429631 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2301a5170 con 0x7fd230109ad0 2024-05-08T06:12:59.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.735+0000 7fd22f400700 1 --2- 172.21.15.123:0/3315429631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd23010fb60 0x7fd2301a4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:12:59.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.735+0000 7fd22ea00700 1 --2- 172.21.15.123:0/3315429631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd230109ad0 0x7fd2301a01b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd22000e9c0 tx=0x7fd22000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:59.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.735+0000 7fd234a00700 1 -- 172.21.15.123:0/3315429631 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22000cbd0 con 0x7fd230109ad0 2024-05-08T06:12:59.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.736+0000 7fd234a00700 1 -- 172.21.15.123:0/3315429631 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd22000ee60 con 0x7fd230109ad0 2024-05-08T06:12:59.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.736+0000 7fd235e00700 1 -- 172.21.15.123:0/3315429631 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2301a5460 con 0x7fd230109ad0 2024-05-08T06:12:59.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.736+0000 7fd234a00700 1 -- 172.21.15.123:0/3315429631 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2200055c0 con 0x7fd230109ad0 2024-05-08T06:12:59.737 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.736+0000 7fd235e00700 1 -- 172.21.15.123:0/3315429631 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2301acce0 con 0x7fd230109ad0 2024-05-08T06:12:59.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.739+0000 7fd234a00700 1 -- 172.21.15.123:0/3315429631 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7fd220004750 con 0x7fd230109ad0 2024-05-08T06:12:59.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.741+0000 7fd234a00700 1 --2- 172.21.15.123:0/3315429631 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd2140b2900 0x7fd2140b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:59.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.741+0000 7fd24077bb80 1 Processor -- start 2024-05-08T06:12:59.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.741+0000 7fd234a00700 1 -- 172.21.15.123:0/3315429631 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fd2200e23e0 con 0x7fd230109ad0 2024-05-08T06:12:59.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.741+0000 7fd24077bb80 1 -- start start 2024-05-08T06:12:59.741 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.741+0000 7fd22e000700 1 --2- 172.21.15.123:0/3315429631 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd2140b2900 0x7fd2140b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:59.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.741+0000 7fd24077bb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d55728060 0x560d55728440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:59.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.741+0000 7fd24077bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d55728980 0x560d5572c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:59.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.741+0000 7fd24077bb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d5572c9d0 0x560d5572ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:59.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.741+0000 7fd24077bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560d5567aa20 con 0x560d55728980 2024-05-08T06:12:59.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.742+0000 7fd24077bb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x560d55618760 con 0x560d5572c9d0 2024-05-08T06:12:59.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.742+0000 7fd24077bb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x560d556769e0 con 0x560d55728060 2024-05-08T06:12:59.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.742+0000 7fd22f400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d5572c9d0 0x560d5572ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:59.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.742+0000 7fd22f400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d5572c9d0 0x560d5572ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:38038/0 (socket says 172.21.15.123:38038) 2024-05-08T06:12:59.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.742+0000 7fd22f400700 1 -- 172.21.15.123:0/181180309 learned_addr learned my addr 172.21.15.123:0/181180309 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:59.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.742+0000 7fd22e000700 1 --2- 172.21.15.123:0/181180309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d55728980 0x560d5572c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:59.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.742+0000 7fd22ea00700 1 --2- 172.21.15.123:0/181180309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d55728060 0x560d55728440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:59.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.742+0000 7fd22ea00700 1 -- 172.21.15.123:0/181180309 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d5572c9d0 msgr2=0x560d5572ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:59.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.742+0000 7fd22ea00700 1 --2- 172.21.15.123:0/181180309 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d5572c9d0 0x560d5572ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:59.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.742+0000 7fd22e000700 1 --2- 172.21.15.123:0/3315429631 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd2140b2900 0x7fd2140b4dc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd21c009680 tx=0x7fd21c011720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:59.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.742+0000 7fd22ea00700 1 -- 172.21.15.123:0/181180309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d55728980 msgr2=0x560d5572c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:59.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.742+0000 7fd22ea00700 1 --2- 172.21.15.123:0/181180309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d55728980 0x560d5572c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:59.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.742+0000 7fd22ea00700 1 -- 172.21.15.123:0/181180309 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d555c78d0 con 0x560d55728060 2024-05-08T06:12:59.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.743+0000 7fd22ea00700 1 --2- 172.21.15.123:0/181180309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d55728060 0x560d55728440 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd220009860 tx=0x7fd2200a9120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=690146d2819de44d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:59.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.743+0000 7fd224c00700 1 -- 172.21.15.123:0/181180309 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2200b2160 con 0x560d55728060 2024-05-08T06:12:59.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.743+0000 7fd224c00700 1 -- 172.21.15.123:0/181180309 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2200a94c0 con 0x560d55728060 2024-05-08T06:12:59.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.743+0000 7fd224c00700 1 -- 172.21.15.123:0/181180309 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2200cc300 con 0x560d55728060 2024-05-08T06:12:59.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.744+0000 7fd24077bb80 1 -- 172.21.15.123:0/181180309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d55728060 msgr2=0x560d55728440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:59.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.744+0000 7fd24077bb80 1 --2- 172.21.15.123:0/181180309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d55728060 0x560d55728440 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd220009860 tx=0x7fd2200a9120 comp rx=0 tx=0).stop 2024-05-08T06:12:59.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.744+0000 7fd24077bb80 1 -- 172.21.15.123:0/181180309 shutdown_connections 2024-05-08T06:12:59.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.744+0000 7fd24077bb80 1 --2- 172.21.15.123:0/181180309 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d5572c9d0 0x560d5572ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:59.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.744+0000 7fd24077bb80 1 --2- 172.21.15.123:0/181180309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d55728060 0x560d55728440 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:59.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.744+0000 7fd24077bb80 1 --2- 172.21.15.123:0/181180309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d55728980 0x560d5572c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:59.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.744+0000 7fd24077bb80 1 -- 172.21.15.123:0/181180309 >> 172.21.15.123:0/181180309 conn(0x560d55656b60 msgr2=0x560d557227a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:59.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.744+0000 7fd24077bb80 1 -- 172.21.15.123:0/181180309 shutdown_connections 2024-05-08T06:12:59.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.744+0000 7fd24077bb80 1 -- 172.21.15.123:0/181180309 wait complete. 2024-05-08T06:12:59.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.745+0000 7fd24077bb80 1 -- start start 2024-05-08T06:12:59.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.745+0000 7fd24077bb80 1 Processor -- start 2024-05-08T06:12:59.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.745+0000 7fd24077bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d55728060 0x560d55733ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:59.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.745+0000 7fd24077bb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d55728980 0x560d55731260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:59.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.746+0000 7fd24077bb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d5572c9d0 0x560d557317a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:59.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.746+0000 7fd24077bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560d55734be0 con 0x560d55728060 2024-05-08T06:12:59.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.746+0000 7fd24077bb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x560d555e0b20 con 0x560d55728980 2024-05-08T06:12:59.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.746+0000 7fd24077bb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x560d55734d60 con 0x560d5572c9d0 2024-05-08T06:12:59.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.746+0000 7fd24077bb80 10 client.0 _ll_register_callbacks cb 0x560d55621c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:12:59.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.746+0000 7fd22e000700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d55728980 0x560d55731260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:59.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.746+0000 7fd22e000700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d55728980 0x560d55731260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:38044/0 (socket says 172.21.15.123:38044) 2024-05-08T06:12:59.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.746+0000 7fd22e000700 1 -- 172.21.15.123:0/2006762253 learned_addr learned my addr 172.21.15.123:0/2006762253 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:59.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.746+0000 7fd22ea00700 1 --2- 172.21.15.123:0/2006762253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d55728060 0x560d55733ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:59.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.746+0000 7fd22f400700 1 --2- 172.21.15.123:0/2006762253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d5572c9d0 0x560d557317a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:59.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.746+0000 7fd22e000700 1 -- 172.21.15.123:0/2006762253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d5572c9d0 msgr2=0x560d557317a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:59.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.746+0000 7fd22e000700 1 --2- 172.21.15.123:0/2006762253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d5572c9d0 0x560d557317a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:59.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.746+0000 7fd22e000700 1 -- 172.21.15.123:0/2006762253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d55728060 msgr2=0x560d55733ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:59.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.746+0000 7fd22e000700 1 --2- 172.21.15.123:0/2006762253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d55728060 0x560d55733ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:59.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.746+0000 7fd22e000700 1 -- 172.21.15.123:0/2006762253 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d55640420 con 0x560d55728980 2024-05-08T06:12:59.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.746+0000 7fd22f400700 1 --2- 172.21.15.123:0/2006762253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d5572c9d0 0x560d557317a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:12:59.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.746+0000 7fd22ea00700 1 --2- 172.21.15.123:0/2006762253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d55728060 0x560d55733ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:12:59.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.747+0000 7fd22e000700 1 --2- 172.21.15.123:0/2006762253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d55728980 0x560d55731260 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd21c00f990 tx=0x7fd21c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=225080909d0fc1c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:59.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.747+0000 7fd219600700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-05-08T06:12:59.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.747+0000 7fd219600700 1 -- 172.21.15.123:0/2006762253 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21c02b060 con 0x560d55728980 2024-05-08T06:12:59.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.747+0000 7fd219600700 1 -- 172.21.15.123:0/2006762253 <== mon.2 v2:172.21.15.163:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fd21c01ec70 con 0x560d55728980 2024-05-08T06:12:59.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.747+0000 7fd24077bb80 1 -- 172.21.15.123:0/2006762253 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2200a94c0 con 0x560d55728980 2024-05-08T06:12:59.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.747+0000 7fd219600700 1 -- 172.21.15.123:0/2006762253 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd21c017040 con 0x560d55728980 2024-05-08T06:12:59.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.747+0000 7fd24077bb80 10 client.25709 fetch_fsmap learned FSMap version 24 2024-05-08T06:12:59.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.747+0000 7fd24077bb80 1 -- 172.21.15.123:0/2006762253 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560d5573f510 con 0x560d55728980 2024-05-08T06:12:59.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.748+0000 7fd219600700 1 -- 172.21.15.123:0/2006762253 <== mon.2 v2:172.21.15.163:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2200a94c0 con 0x560d55728980 2024-05-08T06:12:59.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.748+0000 7fd219600700 1 -- 172.21.15.123:0/2006762253 <== mon.2 v2:172.21.15.163:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fd21c025030 con 0x560d55728980 2024-05-08T06:12:59.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.748+0000 7fd24077bb80 10 client.25709 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:12:59.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.748+0000 7fd24077bb80 10 client.25709 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T06:12:59.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.748+0000 7fd24077bb80 20 client.25709 populate_metadata read hostname 'smithi123' 2024-05-08T06:12:59.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.748+0000 7fd24077bb80 1 --2- 172.21.15.123:0/2006762253 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x560d55741b50 0x560d55743f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:59.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.748+0000 7fd24077bb80 4 client.25709 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:12:59.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.748+0000 7fd24077bb80 1 -- 172.21.15.123:0/2006762253 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d556f50a0 con 0x560d55741b50 2024-05-08T06:12:59.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.748+0000 7fd22ea00700 1 --2- 172.21.15.123:0/2006762253 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x560d55741b50 0x560d55743f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:59.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.749+0000 7fd22ea00700 1 --2- 172.21.15.123:0/2006762253 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x560d55741b50 0x560d55743f40 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20175f14bc699da4 server_cookie=26fa8313757d896b in_seq=0 out_seq=0 2024-05-08T06:12:59.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.749+0000 7fd219600700 10 client.25709 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:59.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.750+0000 7fd219600700 1 -- 172.21.15.123:0/2006762253 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560d556f50a0 con 0x560d55741b50 2024-05-08T06:12:59.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.750+0000 7fd219600700 10 client.25709 handle_command_reply: tid=0 2024-05-08T06:12:59.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.751+0000 7fd24077bb80 1 client.25709 shutdown 2024-05-08T06:12:59.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.751+0000 7fd24077bb80 1 -- 172.21.15.123:0/2006762253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d55728980 msgr2=0x560d55731260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:59.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.751+0000 7fd24077bb80 1 --2- 172.21.15.123:0/2006762253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d55728980 0x560d55731260 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd21c00f990 tx=0x7fd21c0047c0 comp rx=0 tx=0).stop 2024-05-08T06:12:59.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.751+0000 7fd24077bb80 1 -- 172.21.15.123:0/2006762253 shutdown_connections 2024-05-08T06:12:59.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.751+0000 7fd24077bb80 1 --2- 172.21.15.123:0/2006762253 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x560d55741b50 0x560d55743f40 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:59.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.751+0000 7fd24077bb80 1 --2- 172.21.15.123:0/2006762253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d55728060 0x560d55733ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:59.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.751+0000 7fd219600700 0 client.25709 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:59.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.751+0000 7fd24077bb80 1 --2- 172.21.15.123:0/2006762253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d5572c9d0 0x560d557317a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:59.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.751+0000 7fd24077bb80 1 --2- 172.21.15.123:0/2006762253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d55728980 0x560d55731260 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:59.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.751+0000 7fd24077bb80 1 -- 172.21.15.123:0/2006762253 >> 172.21.15.123:0/2006762253 conn(0x560d55656b60 msgr2=0x560d55722760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:59.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.751+0000 7fd24077bb80 1 -- 172.21.15.123:0/2006762253 shutdown_connections 2024-05-08T06:12:59.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.751+0000 7fd24077bb80 1 -- 172.21.15.123:0/2006762253 wait complete. 2024-05-08T06:12:59.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.751+0000 7fd24077bb80 20 client.25709 trim_cache size 0 max 16384 2024-05-08T06:12:59.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.776+0000 7fd24077bb80 1 Processor -- start 2024-05-08T06:12:59.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.776+0000 7fd24077bb80 1 -- start start 2024-05-08T06:12:59.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.776+0000 7fd24077bb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d55728060 0x560d55731930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:59.776 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.776+0000 7fd24077bb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d55728980 0x560d557497b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:59.777 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.776+0000 7fd24077bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d5572c9d0 0x560d55749cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:59.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.776+0000 7fd24077bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd2301b6a50 con 0x560d5572c9d0 2024-05-08T06:12:59.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.776+0000 7fd24077bb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd2301b42e0 con 0x560d55728060 2024-05-08T06:12:59.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.776+0000 7fd24077bb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd2301b2130 con 0x560d55728980 2024-05-08T06:12:59.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.777+0000 7fd22ea00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d55728060 0x560d55731930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:59.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.777+0000 7fd22ea00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d55728060 0x560d55731930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:38054/0 (socket says 172.21.15.123:38054) 2024-05-08T06:12:59.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.777+0000 7fd22ea00700 1 -- 172.21.15.123:0/409831292 learned_addr learned my addr 172.21.15.123:0/409831292 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:59.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.777+0000 7fd22e000700 1 --2- 172.21.15.123:0/409831292 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d55728980 0x560d557497b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:59.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.777+0000 7fd22f400700 1 --2- 172.21.15.123:0/409831292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d5572c9d0 0x560d55749cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:59.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.777+0000 7fd22ea00700 1 -- 172.21.15.123:0/409831292 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d55728980 msgr2=0x560d557497b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:59.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.777+0000 7fd22ea00700 1 --2- 172.21.15.123:0/409831292 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d55728980 0x560d557497b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:59.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.777+0000 7fd22ea00700 1 -- 172.21.15.123:0/409831292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d5572c9d0 msgr2=0x560d55749cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:59.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.777+0000 7fd22ea00700 1 --2- 172.21.15.123:0/409831292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d5572c9d0 0x560d55749cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:59.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.777+0000 7fd22ea00700 1 -- 172.21.15.123:0/409831292 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d5571f3f0 con 0x560d55728060 2024-05-08T06:12:59.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.777+0000 7fd22f400700 1 --2- 172.21.15.123:0/409831292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d5572c9d0 0x560d55749cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:12:59.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.777+0000 7fd22ea00700 1 --2- 172.21.15.123:0/409831292 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d55728060 0x560d55731930 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd2200a96b0 tx=0x7fd2200a5870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c68cb902675d1e60 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:59.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.778+0000 7fd21be00700 1 -- 172.21.15.123:0/409831292 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2200a94c0 con 0x560d55728060 2024-05-08T06:12:59.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.778+0000 7fd21be00700 1 -- 172.21.15.123:0/409831292 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2200aa0b0 con 0x560d55728060 2024-05-08T06:12:59.779 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.778+0000 7fd21be00700 1 -- 172.21.15.123:0/409831292 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2200b2380 con 0x560d55728060 2024-05-08T06:12:59.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.779+0000 7fd24077bb80 1 -- 172.21.15.123:0/409831292 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d55728060 msgr2=0x560d55731930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:59.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.779+0000 7fd24077bb80 1 --2- 172.21.15.123:0/409831292 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d55728060 0x560d55731930 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd2200a96b0 tx=0x7fd2200a5870 comp rx=0 tx=0).stop 2024-05-08T06:12:59.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.779+0000 7fd24077bb80 1 -- 172.21.15.123:0/409831292 shutdown_connections 2024-05-08T06:12:59.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.779+0000 7fd24077bb80 1 --2- 172.21.15.123:0/409831292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d5572c9d0 0x560d55749cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:59.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.779+0000 7fd24077bb80 1 --2- 172.21.15.123:0/409831292 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d55728060 0x560d55731930 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:59.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.779+0000 7fd24077bb80 1 --2- 172.21.15.123:0/409831292 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d55728980 0x560d557497b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:59.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.779+0000 7fd24077bb80 1 -- 172.21.15.123:0/409831292 >> 172.21.15.123:0/409831292 conn(0x560d55656b60 msgr2=0x560d55731220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:59.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.779+0000 7fd24077bb80 1 -- 172.21.15.123:0/409831292 shutdown_connections 2024-05-08T06:12:59.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.779+0000 7fd24077bb80 1 -- 172.21.15.123:0/409831292 wait complete. 2024-05-08T06:12:59.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.779+0000 7fd24077bb80 1 -- start start 2024-05-08T06:12:59.780 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.780+0000 7fd24077bb80 1 Processor -- start 2024-05-08T06:12:59.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.781+0000 7fd24077bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d55728060 0x560d5572e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:59.781 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.781+0000 7fd24077bb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d55728980 0x560d5572e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:59.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.781+0000 7fd24077bb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d5572c9d0 0x560d5572ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:59.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.781+0000 7fd24077bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd2301b27a0 con 0x560d55728060 2024-05-08T06:12:59.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.781+0000 7fd24077bb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd2301b3c10 con 0x560d55728980 2024-05-08T06:12:59.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.781+0000 7fd24077bb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd2301b2e70 con 0x560d5572c9d0 2024-05-08T06:12:59.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.781+0000 7fd24077bb80 10 client.0 _ll_register_callbacks cb 0x560d55621c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:12:59.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.781+0000 7fd22e000700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d55728980 0x560d5572e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:59.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.781+0000 7fd22ea00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d55728060 0x560d5572e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:59.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.781+0000 7fd22e000700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d55728980 0x560d5572e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:38056/0 (socket says 172.21.15.123:38056) 2024-05-08T06:12:59.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.781+0000 7fd22e000700 1 -- 172.21.15.123:0/1892391629 learned_addr learned my addr 172.21.15.123:0/1892391629 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:12:59.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.781+0000 7fd22f400700 1 --2- 172.21.15.123:0/1892391629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d5572c9d0 0x560d5572ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:59.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.781+0000 7fd22e000700 1 -- 172.21.15.123:0/1892391629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d5572c9d0 msgr2=0x560d5572ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:59.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.781+0000 7fd22e000700 1 --2- 172.21.15.123:0/1892391629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d5572c9d0 0x560d5572ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:59.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.781+0000 7fd22e000700 1 -- 172.21.15.123:0/1892391629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d55728060 msgr2=0x560d5572e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:12:59.782 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.781+0000 7fd22e000700 1 --2- 172.21.15.123:0/1892391629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d55728060 0x560d5572e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:59.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.781+0000 7fd22e000700 1 -- 172.21.15.123:0/1892391629 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d557ab3e0 con 0x560d55728980 2024-05-08T06:12:59.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.781+0000 7fd22f400700 1 --2- 172.21.15.123:0/1892391629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d5572c9d0 0x560d5572ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:12:59.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.782+0000 7fd22e000700 1 --2- 172.21.15.123:0/1892391629 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d55728980 0x560d5572e6a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x560d5571efd0 tx=0x7fd21c009400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85182471d3a2f15f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:12:59.783 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.782+0000 7fd219600700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-05-08T06:12:59.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.782+0000 7fd219600700 1 -- 172.21.15.123:0/1892391629 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21c036070 con 0x560d55728980 2024-05-08T06:12:59.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.782+0000 7fd24077bb80 1 -- 172.21.15.123:0/1892391629 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2200aa0b0 con 0x560d55728980 2024-05-08T06:12:59.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.782+0000 7fd219600700 1 -- 172.21.15.123:0/1892391629 <== mon.2 v2:172.21.15.163:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fd21c0313d0 con 0x560d55728980 2024-05-08T06:12:59.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.782+0000 7fd219600700 1 -- 172.21.15.123:0/1892391629 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd21c042050 con 0x560d55728980 2024-05-08T06:12:59.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.782+0000 7fd24077bb80 10 client.25715 fetch_fsmap learned FSMap version 24 2024-05-08T06:12:59.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.782+0000 7fd24077bb80 1 -- 172.21.15.123:0/1892391629 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560d557ac700 con 0x560d55728980 2024-05-08T06:12:59.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.783+0000 7fd219600700 1 -- 172.21.15.123:0/1892391629 <== mon.2 v2:172.21.15.163:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2200aa0b0 con 0x560d55728980 2024-05-08T06:12:59.784 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.783+0000 7fd219600700 1 -- 172.21.15.123:0/1892391629 <== mon.2 v2:172.21.15.163:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fd21c025040 con 0x560d55728980 2024-05-08T06:12:59.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.783+0000 7fd24077bb80 10 client.25715 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:12:59.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.783+0000 7fd24077bb80 10 client.25715 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T06:12:59.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.783+0000 7fd24077bb80 20 client.25715 populate_metadata read hostname 'smithi123' 2024-05-08T06:12:59.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.783+0000 7fd24077bb80 1 --2- 172.21.15.123:0/1892391629 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x560d557ace40 0x560d557af260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:12:59.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.783+0000 7fd24077bb80 4 client.25715 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T06:12:59.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.783+0000 7fd24077bb80 1 -- 172.21.15.123:0/1892391629 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560d55688400 con 0x560d557ace40 2024-05-08T06:12:59.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.783+0000 7fd22ea00700 1 --2- 172.21.15.123:0/1892391629 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x560d557ace40 0x560d557af260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:12:59.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.784+0000 7fd22ea00700 1 --2- 172.21.15.123:0/1892391629 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x560d557ace40 0x560d557af260 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1a02c7fef157274 server_cookie=5b86337721226b86 in_seq=0 out_seq=0 2024-05-08T06:12:59.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.784+0000 7fd219600700 10 client.25715 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:59.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.784+0000 7fd219600700 1 -- 172.21.15.123:0/1892391629 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560d55688400 con 0x560d557ace40 2024-05-08T06:12:59.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.784+0000 7fd219600700 10 client.25715 handle_command_reply: tid=0 2024-05-08T06:12:59.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.784+0000 7fd24077bb80 1 client.25715 shutdown 2024-05-08T06:12:59.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.784+0000 7fd24077bb80 1 -- 172.21.15.123:0/1892391629 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d55728980 msgr2=0x560d5572e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:59.785 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.784+0000 7fd24077bb80 1 --2- 172.21.15.123:0/1892391629 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d55728980 0x560d5572e6a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x560d5571efd0 tx=0x7fd21c009400 comp rx=0 tx=0).stop 2024-05-08T06:12:59.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.784+0000 7fd24077bb80 1 -- 172.21.15.123:0/1892391629 shutdown_connections 2024-05-08T06:12:59.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.784+0000 7fd24077bb80 1 --2- 172.21.15.123:0/1892391629 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x560d557ace40 0x560d557af260 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:59.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.784+0000 7fd24077bb80 1 --2- 172.21.15.123:0/1892391629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d55728060 0x560d5572e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:59.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.784+0000 7fd24077bb80 1 --2- 172.21.15.123:0/1892391629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560d5572c9d0 0x560d5572ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:59.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.784+0000 7fd24077bb80 1 --2- 172.21.15.123:0/1892391629 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560d55728980 0x560d5572e6a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:59.786 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.784+0000 7fd219600700 0 client.25715 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:12:59.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.784+0000 7fd24077bb80 1 -- 172.21.15.123:0/1892391629 >> 172.21.15.123:0/1892391629 conn(0x560d55656b60 msgr2=0x560d5571ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:59.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.785+0000 7fd24077bb80 1 -- 172.21.15.123:0/1892391629 shutdown_connections 2024-05-08T06:12:59.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.785+0000 7fd24077bb80 1 -- 172.21.15.123:0/1892391629 wait complete. 2024-05-08T06:12:59.787 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.785+0000 7fd24077bb80 20 client.25715 trim_cache size 0 max 16384 2024-05-08T06:12:59.787 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-05-08T06:12:59.787 INFO:teuthology.orchestra.run.smithi123.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T06:12:59.787 INFO:teuthology.orchestra.run.smithi123.stdout: "scrubs": { 2024-05-08T06:12:59.787 INFO:teuthology.orchestra.run.smithi123.stdout: "a5574e44-adbc-4cfd-b305-22776ae5dc48": { 2024-05-08T06:12:59.787 INFO:teuthology.orchestra.run.smithi123.stdout: "path": "/", 2024-05-08T06:12:59.787 INFO:teuthology.orchestra.run.smithi123.stdout: "tag": "a5574e44-adbc-4cfd-b305-22776ae5dc48", 2024-05-08T06:12:59.787 INFO:teuthology.orchestra.run.smithi123.stdout: "options": "recursive,force" 2024-05-08T06:12:59.788 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:12:59.788 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:12:59.788 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-05-08T06:12:59.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.786+0000 7fd235e00700 1 -- 172.21.15.123:0/3315429631 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd2140b2900 msgr2=0x7fd2140b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:59.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.786+0000 7fd235e00700 1 --2- 172.21.15.123:0/3315429631 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd2140b2900 0x7fd2140b4dc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd21c009680 tx=0x7fd21c011720 comp rx=0 tx=0).stop 2024-05-08T06:12:59.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.786+0000 7fd235e00700 1 -- 172.21.15.123:0/3315429631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd230109ad0 msgr2=0x7fd2301a01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:12:59.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.786+0000 7fd235e00700 1 --2- 172.21.15.123:0/3315429631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd230109ad0 0x7fd2301a01b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd22000e9c0 tx=0x7fd22000ecd0 comp rx=0 tx=0).stop 2024-05-08T06:12:59.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.786+0000 7fd235e00700 1 -- 172.21.15.123:0/3315429631 shutdown_connections 2024-05-08T06:12:59.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.786+0000 7fd235e00700 1 --2- 172.21.15.123:0/3315429631 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd2140b2900 0x7fd2140b4dc0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:59.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.786+0000 7fd235e00700 1 --2- 172.21.15.123:0/3315429631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd23010fb60 0x7fd2301a4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:59.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.786+0000 7fd235e00700 1 --2- 172.21.15.123:0/3315429631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd230109ad0 0x7fd2301a01b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:59.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.786+0000 7fd235e00700 1 --2- 172.21.15.123:0/3315429631 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd23010a490 0x7fd2301a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:12:59.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.786+0000 7fd235e00700 1 -- 172.21.15.123:0/3315429631 >> 172.21.15.123:0/3315429631 conn(0x7fd230076c70 msgr2=0x7fd230077270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:12:59.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.786+0000 7fd235e00700 1 -- 172.21.15.123:0/3315429631 shutdown_connections 2024-05-08T06:12:59.789 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:12:59.786+0000 7fd235e00700 1 -- 172.21.15.123:0/3315429631 wait complete. 2024-05-08T06:12:59.837 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4193792 len 2048 2024-05-08T06:12:59.837 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4193792 len 2048 2024-05-08T06:12:59.837 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4193792 len 2048 2024-05-08T06:12:59.837 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4193792 len 2048 2024-05-08T06:12:59.837 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4193792 len 4096 2024-05-08T06:12:59.837 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4193792 len 4096 2024-05-08T06:12:59.837 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4193792 len 4096 2024-05-08T06:12:59.837 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4193792 len 4096 2024-05-08T06:12:59.837 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4193792 len 8192 2024-05-08T06:12:59.837 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4193792 len 8192 2024-05-08T06:12:59.837 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4193792 len 8192 2024-05-08T06:12:59.837 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4193792 len 8192 2024-05-08T06:12:59.838 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4193792 len 16384 2024-05-08T06:12:59.838 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4193792 len 16384 2024-05-08T06:12:59.838 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4193792 len 16384 2024-05-08T06:12:59.838 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4193792 len 16384 2024-05-08T06:12:59.838 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4194304 len 1024 2024-05-08T06:12:59.838 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4194304 len 1024 2024-05-08T06:12:59.838 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4194304 len 1024 2024-05-08T06:12:59.838 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4194304 len 1024 2024-05-08T06:12:59.838 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4194304 len 2048 2024-05-08T06:12:59.838 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4194304 len 2048 2024-05-08T06:12:59.838 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4194304 len 2048 2024-05-08T06:12:59.838 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4194304 len 2048 2024-05-08T06:12:59.838 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4194304 len 4096 2024-05-08T06:12:59.838 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4194304 len 4096 2024-05-08T06:12:59.839 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4194304 len 4096 2024-05-08T06:12:59.839 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4194304 len 4096 2024-05-08T06:12:59.839 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4194304 len 8192 2024-05-08T06:12:59.839 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4194304 len 8192 2024-05-08T06:12:59.839 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4194304 len 8192 2024-05-08T06:12:59.839 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4194304 len 8192 2024-05-08T06:12:59.839 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4194304 len 16384 2024-05-08T06:12:59.839 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4194304 len 16384 2024-05-08T06:12:59.839 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4194304 len 16384 2024-05-08T06:12:59.839 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4194304 len 16384 2024-05-08T06:12:59.839 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4194816 len 1024 2024-05-08T06:12:59.839 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4194816 len 1024 2024-05-08T06:12:59.839 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4194816 len 1024 2024-05-08T06:12:59.839 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4194816 len 1024 2024-05-08T06:12:59.839 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4194816 len 2048 2024-05-08T06:12:59.840 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4194816 len 2048 2024-05-08T06:12:59.840 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4194816 len 2048 2024-05-08T06:12:59.840 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4194816 len 2048 2024-05-08T06:12:59.840 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4194816 len 4096 2024-05-08T06:12:59.840 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4194816 len 4096 2024-05-08T06:12:59.840 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4194816 len 4096 2024-05-08T06:12:59.840 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4194816 len 4096 2024-05-08T06:12:59.840 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4194816 len 8192 2024-05-08T06:12:59.840 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4194816 len 8192 2024-05-08T06:12:59.840 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4194816 len 8192 2024-05-08T06:12:59.840 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4194816 len 8192 2024-05-08T06:12:59.840 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4194816 len 16384 2024-05-08T06:12:59.840 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4194816 len 16384 2024-05-08T06:12:59.840 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4194816 len 16384 2024-05-08T06:12:59.840 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4194816 len 16384 2024-05-08T06:12:59.840 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4195328 len 1024 2024-05-08T06:12:59.841 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4195328 len 1024 2024-05-08T06:12:59.841 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4195328 len 1024 2024-05-08T06:12:59.841 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4195328 len 1024 2024-05-08T06:12:59.841 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4195328 len 2048 2024-05-08T06:12:59.841 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4195328 len 2048 2024-05-08T06:12:59.841 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4195328 len 2048 2024-05-08T06:12:59.841 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4195328 len 2048 2024-05-08T06:12:59.841 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4195328 len 4096 2024-05-08T06:12:59.841 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4195328 len 4096 2024-05-08T06:12:59.841 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4195328 len 4096 2024-05-08T06:12:59.841 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4195328 len 4096 2024-05-08T06:12:59.841 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4195328 len 8192 2024-05-08T06:12:59.841 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4195328 len 8192 2024-05-08T06:12:59.841 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4195328 len 8192 2024-05-08T06:12:59.841 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4195328 len 8192 2024-05-08T06:12:59.842 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4195328 len 16384 2024-05-08T06:12:59.842 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4195328 len 16384 2024-05-08T06:12:59.842 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4195328 len 16384 2024-05-08T06:12:59.842 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4195328 len 16384 2024-05-08T06:12:59.842 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4195840 len 1024 2024-05-08T06:12:59.842 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4195840 len 1024 2024-05-08T06:12:59.842 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4195840 len 1024 2024-05-08T06:12:59.842 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4195840 len 1024 2024-05-08T06:12:59.842 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4195840 len 2048 2024-05-08T06:13:00.351 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4195840 len 2048 2024-05-08T06:13:00.351 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4195840 len 2048 2024-05-08T06:13:00.352 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4195840 len 2048 2024-05-08T06:13:00.352 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4195840 len 4096 2024-05-08T06:13:00.352 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4195840 len 4096 2024-05-08T06:13:00.352 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4195840 len 4096 2024-05-08T06:13:00.352 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4195840 len 4096 2024-05-08T06:13:00.352 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4195840 len 8192 2024-05-08T06:13:00.352 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4195840 len 8192 2024-05-08T06:13:00.352 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4195840 len 8192 2024-05-08T06:13:00.352 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4195840 len 8192 2024-05-08T06:13:00.352 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4195840 len 16384 2024-05-08T06:13:00.352 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4195840 len 16384 2024-05-08T06:13:00.352 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4195840 len 16384 2024-05-08T06:13:00.352 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4195840 len 16384 2024-05-08T06:13:00.353 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4196352 len 1024 2024-05-08T06:13:00.353 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4196352 len 1024 2024-05-08T06:13:00.353 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4196352 len 1024 2024-05-08T06:13:00.353 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4196352 len 1024 2024-05-08T06:13:00.353 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4196352 len 2048 2024-05-08T06:13:00.353 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4196352 len 2048 2024-05-08T06:13:00.353 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4196352 len 2048 2024-05-08T06:13:00.353 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4196352 len 2048 2024-05-08T06:13:00.353 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4196352 len 4096 2024-05-08T06:13:00.353 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4196352 len 4096 2024-05-08T06:13:00.353 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4196352 len 4096 2024-05-08T06:13:00.353 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4196352 len 4096 2024-05-08T06:13:00.353 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4196352 len 8192 2024-05-08T06:13:00.354 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4196352 len 8192 2024-05-08T06:13:00.354 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4196352 len 8192 2024-05-08T06:13:00.354 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4196352 len 8192 2024-05-08T06:13:00.354 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4196352 len 16384 2024-05-08T06:13:00.354 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4196352 len 16384 2024-05-08T06:13:00.354 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4196352 len 16384 2024-05-08T06:13:00.354 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4196352 len 16384 2024-05-08T06:13:00.354 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4196864 len 1024 2024-05-08T06:13:00.354 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4196864 len 1024 2024-05-08T06:13:00.354 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4196864 len 1024 2024-05-08T06:13:00.354 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4196864 len 1024 2024-05-08T06:13:00.354 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4196864 len 2048 2024-05-08T06:13:00.354 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4196864 len 2048 2024-05-08T06:13:00.355 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4196864 len 2048 2024-05-08T06:13:00.355 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4196864 len 2048 2024-05-08T06:13:00.355 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4196864 len 4096 2024-05-08T06:13:00.355 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4196864 len 4096 2024-05-08T06:13:00.355 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4196864 len 4096 2024-05-08T06:13:00.355 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4196864 len 4096 2024-05-08T06:13:00.355 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4196864 len 8192 2024-05-08T06:13:00.355 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4196864 len 8192 2024-05-08T06:13:00.355 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4196864 len 8192 2024-05-08T06:13:00.355 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4196864 len 8192 2024-05-08T06:13:00.355 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4196864 len 16384 2024-05-08T06:13:00.355 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4196864 len 16384 2024-05-08T06:13:00.355 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4196864 len 16384 2024-05-08T06:13:00.355 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4196864 len 16384 2024-05-08T06:13:00.355 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4197376 len 1024 2024-05-08T06:13:00.356 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4197376 len 1024 2024-05-08T06:13:00.356 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4197376 len 1024 2024-05-08T06:13:00.356 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4197376 len 1024 2024-05-08T06:13:00.356 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4197376 len 2048 2024-05-08T06:13:00.356 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4197376 len 2048 2024-05-08T06:13:00.356 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4197376 len 2048 2024-05-08T06:13:00.356 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4197376 len 2048 2024-05-08T06:13:00.356 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4197376 len 4096 2024-05-08T06:13:00.356 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4197376 len 4096 2024-05-08T06:13:00.356 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4197376 len 4096 2024-05-08T06:13:00.356 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4197376 len 4096 2024-05-08T06:13:00.356 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4197376 len 8192 2024-05-08T06:13:00.356 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4197376 len 8192 2024-05-08T06:13:00.356 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4197376 len 8192 2024-05-08T06:13:00.357 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4197376 len 8192 2024-05-08T06:13:00.357 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4197376 len 16384 2024-05-08T06:13:00.357 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4197376 len 16384 2024-05-08T06:13:00.357 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4197376 len 16384 2024-05-08T06:13:00.357 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4197376 len 16384 2024-05-08T06:13:00.357 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4197888 len 1024 2024-05-08T06:13:00.357 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4197888 len 1024 2024-05-08T06:13:00.357 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4197888 len 1024 2024-05-08T06:13:00.357 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4197888 len 1024 2024-05-08T06:13:00.357 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4197888 len 2048 2024-05-08T06:13:00.357 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4197888 len 2048 2024-05-08T06:13:00.448 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4197888 len 2048 2024-05-08T06:13:00.449 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4197888 len 2048 2024-05-08T06:13:00.449 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4197888 len 4096 2024-05-08T06:13:00.449 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4197888 len 4096 2024-05-08T06:13:00.449 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4197888 len 4096 2024-05-08T06:13:00.449 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4197888 len 4096 2024-05-08T06:13:00.449 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4197888 len 8192 2024-05-08T06:13:00.449 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4197888 len 8192 2024-05-08T06:13:00.449 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4197888 len 8192 2024-05-08T06:13:00.449 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4197888 len 8192 2024-05-08T06:13:00.449 INFO:tasks.workunit.client.0.smithi123.stdout:write_direct buf_align 3584 offset 4197888 len 16384 2024-05-08T06:13:00.449 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4197888 len 16384 2024-05-08T06:13:00.449 INFO:tasks.workunit.client.0.smithi123.stdout:write_sync buf_align 3584 offset 4197888 len 16384 2024-05-08T06:13:00.449 INFO:tasks.workunit.client.0.smithi123.stdout:write_file buf_align 3584 offset 4197888 len 16384 2024-05-08T06:13:00.450 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-08T06:13:00.451 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-08T06:13:00.551 INFO:tasks.workunit:Stopping ['direct_io'] on client.0... 2024-05-08T06:13:00.551 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-08T06:13:00.830 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:00 smithi123 ceph-mon[29787]: pgmap v536: 97 pgs: 97 active+clean; 171 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 69 MiB/s rd, 650 KiB/s wr, 255 op/s 2024-05-08T06:13:00.831 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:00 smithi123 ceph-mon[29787]: scrub summary: idle 2024-05-08T06:13:00.833 INFO:tasks.cephfs.filesystem:scrub status for tag:a5574e44-adbc-4cfd-b305-22776ae5dc48 - {'path': '/', 'tag': 'a5574e44-adbc-4cfd-b305-22776ae5dc48', 'options': 'recursive,force'} 2024-05-08T06:13:01.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:00 smithi163 ceph-mon[34520]: pgmap v536: 97 pgs: 97 active+clean; 171 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 69 MiB/s rd, 650 KiB/s wr, 255 op/s 2024-05-08T06:13:01.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:00 smithi163 ceph-mon[34520]: scrub summary: idle 2024-05-08T06:13:01.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:00 smithi192 ceph-mon[34664]: pgmap v536: 97 pgs: 97 active+clean; 171 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 69 MiB/s rd, 650 KiB/s wr, 255 op/s 2024-05-08T06:13:01.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:00 smithi192 ceph-mon[34664]: scrub summary: idle 2024-05-08T06:13:01.625 DEBUG:teuthology.parallel:result is None 2024-05-08T06:13:01.625 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-08T06:13:01.718 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-08T06:13:01.719 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-05-08T06:13:01.730 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi123.front.sepia.ceph.com 2024-05-08T06:13:01.756 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-05-08T06:13:01.767 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-05-08T06:13:03.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:02 smithi163 ceph-mon[34520]: pgmap v537: 97 pgs: 97 active+clean; 145 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 75 MiB/s rd, 894 KiB/s wr, 316 op/s 2024-05-08T06:13:03.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:02 smithi192 ceph-mon[34664]: pgmap v537: 97 pgs: 97 active+clean; 145 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 75 MiB/s rd, 894 KiB/s wr, 316 op/s 2024-05-08T06:13:03.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:02 smithi123 ceph-mon[29787]: pgmap v537: 97 pgs: 97 active+clean; 145 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 75 MiB/s rd, 894 KiB/s wr, 316 op/s 2024-05-08T06:13:05.008 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:04 smithi123 ceph-mon[29787]: pgmap v538: 97 pgs: 97 active+clean; 217 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 65 MiB/s rd, 692 KiB/s wr, 269 op/s 2024-05-08T06:13:05.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:04 smithi163 ceph-mon[34520]: pgmap v538: 97 pgs: 97 active+clean; 217 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 65 MiB/s rd, 692 KiB/s wr, 269 op/s 2024-05-08T06:13:05.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:04 smithi192 ceph-mon[34664]: pgmap v538: 97 pgs: 97 active+clean; 217 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 65 MiB/s rd, 692 KiB/s wr, 269 op/s 2024-05-08T06:13:06.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:06 smithi123 ceph-mon[29787]: pgmap v539: 97 pgs: 97 active+clean; 141 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 53 MiB/s rd, 638 KiB/s wr, 242 op/s 2024-05-08T06:13:07.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:06 smithi163 ceph-mon[34520]: pgmap v539: 97 pgs: 97 active+clean; 141 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 53 MiB/s rd, 638 KiB/s wr, 242 op/s 2024-05-08T06:13:07.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:06 smithi192 ceph-mon[34664]: pgmap v539: 97 pgs: 97 active+clean; 141 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 53 MiB/s rd, 638 KiB/s wr, 242 op/s 2024-05-08T06:13:09.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:08 smithi163 ceph-mon[34520]: pgmap v540: 97 pgs: 97 active+clean; 89 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 594 KiB/s wr, 225 op/s 2024-05-08T06:13:09.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:08 smithi192 ceph-mon[34664]: pgmap v540: 97 pgs: 97 active+clean; 89 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 594 KiB/s wr, 225 op/s 2024-05-08T06:13:09.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:08 smithi123 ceph-mon[29787]: pgmap v540: 97 pgs: 97 active+clean; 89 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 594 KiB/s wr, 225 op/s 2024-05-08T06:13:11.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:10 smithi163 ceph-mon[34520]: pgmap v541: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 385 KiB/s wr, 170 op/s 2024-05-08T06:13:11.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:10 smithi192 ceph-mon[34664]: pgmap v541: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 385 KiB/s wr, 170 op/s 2024-05-08T06:13:11.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:10 smithi123 ceph-mon[29787]: pgmap v541: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 385 KiB/s wr, 170 op/s 2024-05-08T06:13:13.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:12 smithi163 ceph-mon[34520]: pgmap v542: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 336 KiB/s wr, 130 op/s 2024-05-08T06:13:13.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:12 smithi192 ceph-mon[34664]: pgmap v542: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 336 KiB/s wr, 130 op/s 2024-05-08T06:13:13.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:12 smithi123 ceph-mon[29787]: pgmap v542: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 336 KiB/s wr, 130 op/s 2024-05-08T06:13:15.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:14 smithi163 ceph-mon[34520]: pgmap v543: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 55 KiB/s wr, 44 op/s 2024-05-08T06:13:15.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:14 smithi192 ceph-mon[34664]: pgmap v543: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 55 KiB/s wr, 44 op/s 2024-05-08T06:13:15.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:14 smithi123 ceph-mon[29787]: pgmap v543: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 55 KiB/s wr, 44 op/s 2024-05-08T06:13:17.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:16 smithi163 ceph-mon[34520]: pgmap v544: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 20 KiB/s wr, 27 op/s 2024-05-08T06:13:17.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:16 smithi192 ceph-mon[34664]: pgmap v544: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 20 KiB/s wr, 27 op/s 2024-05-08T06:13:17.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:16 smithi123 ceph-mon[29787]: pgmap v544: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 20 KiB/s wr, 27 op/s 2024-05-08T06:13:19.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:18 smithi163 ceph-mon[34520]: pgmap v545: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-05-08T06:13:19.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:18 smithi192 ceph-mon[34664]: pgmap v545: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-05-08T06:13:19.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:18 smithi123 ceph-mon[29787]: pgmap v545: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-05-08T06:13:21.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:20 smithi163 ceph-mon[34520]: pgmap v546: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 13 op/s 2024-05-08T06:13:21.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:20 smithi192 ceph-mon[34664]: pgmap v546: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 13 op/s 2024-05-08T06:13:21.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:20 smithi123 ceph-mon[29787]: pgmap v546: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 13 op/s 2024-05-08T06:13:23.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:22 smithi163 ceph-mon[34520]: pgmap v547: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-08T06:13:23.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:22 smithi192 ceph-mon[34664]: pgmap v547: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-08T06:13:23.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:22 smithi123 ceph-mon[29787]: pgmap v547: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-08T06:13:25.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:24 smithi163 ceph-mon[34520]: pgmap v548: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-08T06:13:25.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:24 smithi192 ceph-mon[34664]: pgmap v548: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-08T06:13:25.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:24 smithi123 ceph-mon[29787]: pgmap v548: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-08T06:13:27.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:26 smithi163 ceph-mon[34520]: pgmap v549: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-08T06:13:27.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:26 smithi192 ceph-mon[34664]: pgmap v549: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-08T06:13:27.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:26 smithi123 ceph-mon[29787]: pgmap v549: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-08T06:13:29.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:28 smithi163 ceph-mon[34520]: pgmap v550: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-08T06:13:29.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:28 smithi192 ceph-mon[34664]: pgmap v550: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-08T06:13:29.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:28 smithi123 ceph-mon[29787]: pgmap v550: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-08T06:13:30.834 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-08T06:13:31.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:30 smithi163 ceph-mon[34520]: pgmap v551: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-08T06:13:31.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:30 smithi192 ceph-mon[34664]: pgmap v551: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-08T06:13:31.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:30 smithi123 ceph-mon[29787]: pgmap v551: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-08T06:13:31.294 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:13:31.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:31 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:13:32.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:31 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:13:32.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:31 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:13:33.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:32 smithi192 ceph-mon[34664]: pgmap v552: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:13:33.052 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:32 smithi123 ceph-mon[29787]: pgmap v552: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:13:33.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:32 smithi163 ceph-mon[34520]: pgmap v552: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:13:33.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.880+0000 7f9c8e600700 1 -- 172.21.15.123:0/3091838609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c8806bad0 msgr2=0x7f9c8810fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:33.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.880+0000 7f9c8e600700 1 --2- 172.21.15.123:0/3091838609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c8806bad0 0x7f9c8810fa10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9c74009990 tx=0x7f9c74009ca0 comp rx=0 tx=0).stop 2024-05-08T06:13:33.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.880+0000 7f9c8e600700 1 -- 172.21.15.123:0/3091838609 shutdown_connections 2024-05-08T06:13:33.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.880+0000 7f9c8e600700 1 --2- 172.21.15.123:0/3091838609 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c8810ff50 0x7f9c88112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:33.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.880+0000 7f9c8e600700 1 --2- 172.21.15.123:0/3091838609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c8806b1b0 0x7f9c8806b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:33.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.880+0000 7f9c8e600700 1 --2- 172.21.15.123:0/3091838609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c8806bad0 0x7f9c8810fa10 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:33.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.880+0000 7f9c8e600700 1 -- 172.21.15.123:0/3091838609 >> 172.21.15.123:0/3091838609 conn(0x7f9c88079350 msgr2=0x7f9c88079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:13:33.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.880+0000 7f9c8e600700 1 -- 172.21.15.123:0/3091838609 shutdown_connections 2024-05-08T06:13:33.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.880+0000 7f9c8e600700 1 -- 172.21.15.123:0/3091838609 wait complete. 2024-05-08T06:13:33.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.881+0000 7f9c8e600700 1 Processor -- start 2024-05-08T06:13:33.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.882+0000 7f9c8e600700 1 -- start start 2024-05-08T06:13:33.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.882+0000 7f9c8e600700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c8806b1b0 0x7f9c8819be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:33.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.882+0000 7f9c8e600700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c8806bad0 0x7f9c8819c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:33.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.882+0000 7f9c8e600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c8810ff50 0x7f9c881a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:33.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.882+0000 7f9c8e600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9c881158d0 con 0x7f9c8810ff50 2024-05-08T06:13:33.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.882+0000 7f9c8e600700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9c88115750 con 0x7f9c8806bad0 2024-05-08T06:13:33.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.882+0000 7f9c8e600700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9c88115a50 con 0x7f9c8806b1b0 2024-05-08T06:13:33.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.883+0000 7f9c87400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c8806b1b0 0x7f9c8819be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:33.883 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.883+0000 7f9c86a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c8806bad0 0x7f9c8819c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:33.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.883+0000 7f9c87400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c8806b1b0 0x7f9c8819be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:55746/0 (socket says 172.21.15.123:55746) 2024-05-08T06:13:33.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.883+0000 7f9c86a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c8806bad0 0x7f9c8819c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:57332/0 (socket says 172.21.15.123:57332) 2024-05-08T06:13:33.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.883+0000 7f9c87400700 1 -- 172.21.15.123:0/3687096816 learned_addr learned my addr 172.21.15.123:0/3687096816 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:13:33.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.883+0000 7f9c87e00700 1 --2- 172.21.15.123:0/3687096816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c8810ff50 0x7f9c881a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:33.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.883+0000 7f9c87400700 1 -- 172.21.15.123:0/3687096816 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c8806bad0 msgr2=0x7f9c8819c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:33.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.883+0000 7f9c87400700 1 --2- 172.21.15.123:0/3687096816 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c8806bad0 0x7f9c8819c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:33.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.883+0000 7f9c87400700 1 -- 172.21.15.123:0/3687096816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c8810ff50 msgr2=0x7f9c881a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:33.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.883+0000 7f9c87400700 1 --2- 172.21.15.123:0/3687096816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c8810ff50 0x7f9c881a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:33.884 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.883+0000 7f9c87400700 1 -- 172.21.15.123:0/3687096816 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c881a0dc0 con 0x7f9c8806b1b0 2024-05-08T06:13:33.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.883+0000 7f9c87400700 1 --2- 172.21.15.123:0/3687096816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c8806b1b0 0x7f9c8819be00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9c7800e9c0 tx=0x7f9c7800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:13:33.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.884+0000 7f9c7fe00700 1 -- 172.21.15.123:0/3687096816 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7800cbd0 con 0x7f9c8806b1b0 2024-05-08T06:13:33.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.884+0000 7f9c7fe00700 1 -- 172.21.15.123:0/3687096816 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c7800cd70 con 0x7f9c8806b1b0 2024-05-08T06:13:33.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.884+0000 7f9c8e600700 1 -- 172.21.15.123:0/3687096816 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c881a10b0 con 0x7f9c8806b1b0 2024-05-08T06:13:33.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.884+0000 7f9c7fe00700 1 -- 172.21.15.123:0/3687096816 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c78005610 con 0x7f9c8806b1b0 2024-05-08T06:13:33.885 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.884+0000 7f9c8e600700 1 -- 172.21.15.123:0/3687096816 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c881a8930 con 0x7f9c8806b1b0 2024-05-08T06:13:33.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.887+0000 7f9c7fe00700 1 -- 172.21.15.123:0/3687096816 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7f9c78004750 con 0x7f9c8806b1b0 2024-05-08T06:13:33.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.888+0000 7f9c98edeb80 1 Processor -- start 2024-05-08T06:13:33.888 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.888+0000 7f9c98edeb80 1 -- start start 2024-05-08T06:13:33.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.889+0000 7f9c98edeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5a468a060 0x55e5a468a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:33.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.889+0000 7f9c98edeb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e5a468a980 0x55e5a468e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:33.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.889+0000 7f9c98edeb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e5a468e9d0 0x55e5a468ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:33.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.889+0000 7f9c7fe00700 1 --2- 172.21.15.123:0/3687096816 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9c6c0b2900 0x7f9c6c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:33.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.889+0000 7f9c98edeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e5a45dca20 con 0x55e5a468a060 2024-05-08T06:13:33.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.889+0000 7f9c98edeb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55e5a457a760 con 0x55e5a468a980 2024-05-08T06:13:33.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.889+0000 7f9c98edeb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55e5a45d89e0 con 0x55e5a468e9d0 2024-05-08T06:13:33.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.889+0000 7f9c7fe00700 1 -- 172.21.15.123:0/3687096816 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f9c780e2550 con 0x7f9c8806b1b0 2024-05-08T06:13:33.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.889+0000 7f9c86a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e5a468e9d0 0x55e5a468ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:33.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.889+0000 7f9c87400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e5a468a980 0x55e5a468e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:33.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.889+0000 7f9c86a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e5a468e9d0 0x55e5a468ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:55758/0 (socket says 172.21.15.123:55758) 2024-05-08T06:13:33.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.889+0000 7f9c86a00700 1 -- 172.21.15.123:0/2538454494 learned_addr learned my addr 172.21.15.123:0/2538454494 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:13:33.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.889+0000 7f9c87e00700 1 --2- 172.21.15.123:0/2538454494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5a468a060 0x55e5a468a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:33.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.890+0000 7f9c87e00700 1 --2- 172.21.15.123:0/3687096816 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9c6c0b2900 0x7f9c6c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:33.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.890+0000 7f9c86a00700 1 -- 172.21.15.123:0/2538454494 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e5a468a980 msgr2=0x55e5a468e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:13:33.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.890+0000 7f9c86a00700 1 --2- 172.21.15.123:0/2538454494 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e5a468a980 0x55e5a468e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:33.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.890+0000 7f9c86a00700 1 -- 172.21.15.123:0/2538454494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5a468a060 msgr2=0x55e5a468a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:13:33.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.890+0000 7f9c86a00700 1 --2- 172.21.15.123:0/2538454494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5a468a060 0x55e5a468a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:33.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.890+0000 7f9c86a00700 1 -- 172.21.15.123:0/2538454494 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5a45298d0 con 0x55e5a468e9d0 2024-05-08T06:13:33.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.890+0000 7f9c87400700 1 --2- 172.21.15.123:0/2538454494 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e5a468a980 0x55e5a468e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:13:33.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.890+0000 7f9c86a00700 1 --2- 172.21.15.123:0/2538454494 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e5a468e9d0 0x55e5a468ee30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9c74015000 tx=0x7f9c740096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fda480ad7f69d0f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:13:33.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.890+0000 7f9c7cc00700 1 -- 172.21.15.123:0/2538454494 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c74027070 con 0x55e5a468e9d0 2024-05-08T06:13:33.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.890+0000 7f9c7cc00700 1 -- 172.21.15.123:0/2538454494 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c740056d0 con 0x55e5a468e9d0 2024-05-08T06:13:33.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.890+0000 7f9c7cc00700 1 -- 172.21.15.123:0/2538454494 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c74003aa0 con 0x55e5a468e9d0 2024-05-08T06:13:33.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.891+0000 7f9c98edeb80 1 -- 172.21.15.123:0/2538454494 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e5a468e9d0 msgr2=0x55e5a468ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:33.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.891+0000 7f9c87e00700 1 --2- 172.21.15.123:0/3687096816 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9c6c0b2900 0x7f9c6c0b4dc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9c8806ad70 tx=0x7f9c8000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:13:33.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.891+0000 7f9c98edeb80 1 --2- 172.21.15.123:0/2538454494 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e5a468e9d0 0x55e5a468ee30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9c74015000 tx=0x7f9c740096a0 comp rx=0 tx=0).stop 2024-05-08T06:13:33.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.891+0000 7f9c98edeb80 1 -- 172.21.15.123:0/2538454494 shutdown_connections 2024-05-08T06:13:33.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.891+0000 7f9c98edeb80 1 --2- 172.21.15.123:0/2538454494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5a468a060 0x55e5a468a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:33.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.891+0000 7f9c98edeb80 1 --2- 172.21.15.123:0/2538454494 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e5a468e9d0 0x55e5a468ee30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:33.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.891+0000 7f9c98edeb80 1 --2- 172.21.15.123:0/2538454494 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e5a468a980 0x55e5a468e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:33.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.891+0000 7f9c98edeb80 1 -- 172.21.15.123:0/2538454494 >> 172.21.15.123:0/2538454494 conn(0x55e5a45b8b60 msgr2=0x55e5a46847a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:13:33.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.891+0000 7f9c98edeb80 1 -- 172.21.15.123:0/2538454494 shutdown_connections 2024-05-08T06:13:33.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.892+0000 7f9c98edeb80 1 -- 172.21.15.123:0/2538454494 wait complete. 2024-05-08T06:13:33.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.892+0000 7f9c98edeb80 1 -- start start 2024-05-08T06:13:33.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.892+0000 7f9c98edeb80 1 Processor -- start 2024-05-08T06:13:33.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.893+0000 7f9c98edeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5a468a060 0x55e5a4695e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:33.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.893+0000 7f9c98edeb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e5a468a980 0x55e5a4693230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:33.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.893+0000 7f9c98edeb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e5a468e9d0 0x55e5a4693770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:33.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.893+0000 7f9c98edeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e5a4696b80 con 0x55e5a468a060 2024-05-08T06:13:33.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.893+0000 7f9c98edeb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55e5a4542b20 con 0x55e5a468e9d0 2024-05-08T06:13:33.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.893+0000 7f9c98edeb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55e5a4696d00 con 0x55e5a468a980 2024-05-08T06:13:33.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.893+0000 7f9c98edeb80 10 client.0 _ll_register_callbacks cb 0x55e5a4583c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:13:33.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.893+0000 7f9c87400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5a468a060 0x55e5a4695e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:33.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.893+0000 7f9c87400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5a468a060 0x55e5a4695e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59132/0 (socket says 172.21.15.123:59132) 2024-05-08T06:13:33.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.893+0000 7f9c87400700 1 -- 172.21.15.123:0/3730637628 learned_addr learned my addr 172.21.15.123:0/3730637628 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:13:33.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.893+0000 7f9c86a00700 1 --2- 172.21.15.123:0/3730637628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e5a468a980 0x55e5a4693230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:33.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.894+0000 7f9c87e00700 1 --2- 172.21.15.123:0/3730637628 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e5a468e9d0 0x55e5a4693770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:33.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.894+0000 7f9c87400700 1 -- 172.21.15.123:0/3730637628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e5a468a980 msgr2=0x55e5a4693230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:13:33.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.894+0000 7f9c87400700 1 --2- 172.21.15.123:0/3730637628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e5a468a980 0x55e5a4693230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:33.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.894+0000 7f9c87400700 1 -- 172.21.15.123:0/3730637628 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e5a468e9d0 msgr2=0x55e5a4693770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:13:33.896 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.894+0000 7f9c87400700 1 --2- 172.21.15.123:0/3730637628 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e5a468e9d0 0x55e5a4693770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:33.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.894+0000 7f9c87400700 1 -- 172.21.15.123:0/3730637628 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5a45a2420 con 0x55e5a468a060 2024-05-08T06:13:33.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.894+0000 7f9c87400700 1 --2- 172.21.15.123:0/3730637628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5a468a060 0x55e5a4695e90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9c780a6d90 tx=0x7f9c780b2bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9c923cb21784716 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:13:33.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.894+0000 7f9c71600700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-08T06:13:33.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.894+0000 7f9c71600700 1 -- 172.21.15.123:0/3730637628 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c780c3060 con 0x55e5a468a060 2024-05-08T06:13:33.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.894+0000 7f9c71600700 1 -- 172.21.15.123:0/3730637628 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f9c780b2700 con 0x55e5a468a060 2024-05-08T06:13:33.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.894+0000 7f9c98edeb80 1 -- 172.21.15.123:0/3730637628 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c740056d0 con 0x55e5a468a060 2024-05-08T06:13:33.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.895+0000 7f9c71600700 1 -- 172.21.15.123:0/3730637628 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9c780a52b0 con 0x55e5a468a060 2024-05-08T06:13:33.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.895+0000 7f9c98edeb80 10 client.15603 fetch_fsmap learned FSMap version 24 2024-05-08T06:13:33.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.895+0000 7f9c98edeb80 1 -- 172.21.15.123:0/3730637628 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e5a46a1490 con 0x55e5a468a060 2024-05-08T06:13:33.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.895+0000 7f9c71600700 1 -- 172.21.15.123:0/3730637628 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c740056d0 con 0x55e5a468a060 2024-05-08T06:13:33.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.895+0000 7f9c71600700 1 -- 172.21.15.123:0/3730637628 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f9c780e20b0 con 0x55e5a468a060 2024-05-08T06:13:33.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.895+0000 7f9c98edeb80 10 client.15603 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:13:33.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.895+0000 7f9c98edeb80 10 client.15603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T06:13:33.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.895+0000 7f9c98edeb80 20 client.15603 populate_metadata read hostname 'smithi123' 2024-05-08T06:13:33.897 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.895+0000 7f9c98edeb80 1 --2- 172.21.15.123:0/3730637628 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55e5a46a3a20 0x55e5a46a5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:33.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.895+0000 7f9c98edeb80 4 client.15603 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:13:33.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.895+0000 7f9c98edeb80 1 -- 172.21.15.123:0/3730637628 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5a46570a0 con 0x55e5a46a3a20 2024-05-08T06:13:33.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.896+0000 7f9c86a00700 1 --2- 172.21.15.123:0/3730637628 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55e5a46a3a20 0x55e5a46a5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:33.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.897+0000 7f9c86a00700 1 --2- 172.21.15.123:0/3730637628 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55e5a46a3a20 0x55e5a46a5e10 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5aee38df2ff832fa server_cookie=b9f6561ccf488753 in_seq=0 out_seq=0 2024-05-08T06:13:33.898 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.897+0000 7f9c71600700 10 client.15603 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:13:33.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.897+0000 7f9c71600700 1 -- 172.21.15.123:0/3730637628 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e5a46570a0 con 0x55e5a46a3a20 2024-05-08T06:13:33.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.897+0000 7f9c71600700 10 client.15603 handle_command_reply: tid=0 2024-05-08T06:13:33.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.897+0000 7f9c98edeb80 1 client.15603 shutdown 2024-05-08T06:13:33.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.898+0000 7f9c98edeb80 1 -- 172.21.15.123:0/3730637628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5a468a060 msgr2=0x55e5a4695e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:33.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.898+0000 7f9c98edeb80 1 --2- 172.21.15.123:0/3730637628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5a468a060 0x55e5a4695e90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9c780a6d90 tx=0x7f9c780b2bd0 comp rx=0 tx=0).stop 2024-05-08T06:13:33.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.898+0000 7f9c98edeb80 1 -- 172.21.15.123:0/3730637628 shutdown_connections 2024-05-08T06:13:33.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.898+0000 7f9c98edeb80 1 --2- 172.21.15.123:0/3730637628 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55e5a46a3a20 0x55e5a46a5e10 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:33.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.898+0000 7f9c98edeb80 1 --2- 172.21.15.123:0/3730637628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5a468a060 0x55e5a4695e90 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:33.899 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.898+0000 7f9c98edeb80 1 --2- 172.21.15.123:0/3730637628 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e5a468e9d0 0x55e5a4693770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:33.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.898+0000 7f9c71600700 0 client.15603 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:13:33.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.898+0000 7f9c98edeb80 1 --2- 172.21.15.123:0/3730637628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e5a468a980 0x55e5a4693230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:33.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.898+0000 7f9c98edeb80 1 -- 172.21.15.123:0/3730637628 >> 172.21.15.123:0/3730637628 conn(0x55e5a45b8b60 msgr2=0x55e5a467f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:13:33.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.898+0000 7f9c98edeb80 1 -- 172.21.15.123:0/3730637628 shutdown_connections 2024-05-08T06:13:33.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.898+0000 7f9c98edeb80 1 -- 172.21.15.123:0/3730637628 wait complete. 2024-05-08T06:13:33.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.898+0000 7f9c98edeb80 20 client.15603 trim_cache size 0 max 16384 2024-05-08T06:13:33.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.931+0000 7f9c98edeb80 1 Processor -- start 2024-05-08T06:13:33.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.931+0000 7f9c98edeb80 1 -- start start 2024-05-08T06:13:33.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.932+0000 7f9c98edeb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e5a468a060 0x55e5a46931f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:33.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.932+0000 7f9c98edeb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e5a468a980 0x55e5a4693730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:33.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.932+0000 7f9c98edeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5a468e9d0 0x55e5a4691b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:33.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.932+0000 7f9c98edeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9c881b25a0 con 0x55e5a468e9d0 2024-05-08T06:13:33.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.932+0000 7f9c98edeb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9c881afe30 con 0x55e5a468a980 2024-05-08T06:13:33.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.932+0000 7f9c98edeb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9c881adc80 con 0x55e5a468a060 2024-05-08T06:13:33.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.932+0000 7f9c87400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e5a468a060 0x55e5a46931f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:33.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.932+0000 7f9c87e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5a468e9d0 0x55e5a4691b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:33.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.932+0000 7f9c87400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e5a468a060 0x55e5a46931f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:55774/0 (socket says 172.21.15.123:55774) 2024-05-08T06:13:33.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.932+0000 7f9c87400700 1 -- 172.21.15.123:0/4002779135 learned_addr learned my addr 172.21.15.123:0/4002779135 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:13:33.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.932+0000 7f9c86a00700 1 --2- 172.21.15.123:0/4002779135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e5a468a980 0x55e5a4693730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:33.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.933+0000 7f9c87400700 1 -- 172.21.15.123:0/4002779135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e5a468a980 msgr2=0x55e5a4693730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:13:33.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.933+0000 7f9c87400700 1 --2- 172.21.15.123:0/4002779135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e5a468a980 0x55e5a4693730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:33.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.933+0000 7f9c87400700 1 -- 172.21.15.123:0/4002779135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5a468e9d0 msgr2=0x55e5a4691b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:13:33.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.933+0000 7f9c87400700 1 --2- 172.21.15.123:0/4002779135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5a468e9d0 0x55e5a4691b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:33.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.933+0000 7f9c87400700 1 -- 172.21.15.123:0/4002779135 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5a46a6440 con 0x55e5a468a060 2024-05-08T06:13:33.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.933+0000 7f9c87e00700 1 --2- 172.21.15.123:0/4002779135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5a468e9d0 0x55e5a4691b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:13:33.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.933+0000 7f9c87400700 1 --2- 172.21.15.123:0/4002779135 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e5a468a060 0x55e5a46931f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9c78005bf0 tx=0x7f9c780a5a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e390e7c56e808981 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:13:33.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.933+0000 7f9c73e00700 1 -- 172.21.15.123:0/4002779135 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c780b22c0 con 0x55e5a468a060 2024-05-08T06:13:33.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.933+0000 7f9c73e00700 1 -- 172.21.15.123:0/4002779135 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c780aae50 con 0x55e5a468a060 2024-05-08T06:13:33.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.933+0000 7f9c73e00700 1 -- 172.21.15.123:0/4002779135 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c780b2980 con 0x55e5a468a060 2024-05-08T06:13:33.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.934+0000 7f9c98edeb80 1 -- 172.21.15.123:0/4002779135 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e5a468a060 msgr2=0x55e5a46931f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:33.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.934+0000 7f9c98edeb80 1 --2- 172.21.15.123:0/4002779135 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e5a468a060 0x55e5a46931f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9c78005bf0 tx=0x7f9c780a5a60 comp rx=0 tx=0).stop 2024-05-08T06:13:33.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.934+0000 7f9c98edeb80 1 -- 172.21.15.123:0/4002779135 shutdown_connections 2024-05-08T06:13:33.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.934+0000 7f9c98edeb80 1 --2- 172.21.15.123:0/4002779135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5a468e9d0 0x55e5a4691b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:33.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.934+0000 7f9c98edeb80 1 --2- 172.21.15.123:0/4002779135 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e5a468a060 0x55e5a46931f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:33.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.934+0000 7f9c98edeb80 1 --2- 172.21.15.123:0/4002779135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e5a468a980 0x55e5a4693730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:33.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.934+0000 7f9c98edeb80 1 -- 172.21.15.123:0/4002779135 >> 172.21.15.123:0/4002779135 conn(0x55e5a45b8b60 msgr2=0x55e5a468fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:13:33.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.934+0000 7f9c98edeb80 1 -- 172.21.15.123:0/4002779135 shutdown_connections 2024-05-08T06:13:33.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.935+0000 7f9c98edeb80 1 -- 172.21.15.123:0/4002779135 wait complete. 2024-05-08T06:13:33.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.935+0000 7f9c98edeb80 1 -- start start 2024-05-08T06:13:33.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.935+0000 7f9c98edeb80 1 Processor -- start 2024-05-08T06:13:33.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.936+0000 7f9c98edeb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e5a468a060 0x55e5a46aaf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:33.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.936+0000 7f9c98edeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5a468a980 0x55e5a46ab490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:33.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.936+0000 7f9c98edeb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e5a468e9d0 0x55e5a46ab9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:33.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.936+0000 7f9c98edeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9c881ae2f0 con 0x55e5a468a980 2024-05-08T06:13:33.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.936+0000 7f9c98edeb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9c881af760 con 0x55e5a468a060 2024-05-08T06:13:33.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.936+0000 7f9c98edeb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9c881ae9c0 con 0x55e5a468e9d0 2024-05-08T06:13:33.937 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.936+0000 7f9c98edeb80 10 client.0 _ll_register_callbacks cb 0x55e5a4583c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:13:33.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.936+0000 7f9c87400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e5a468a060 0x55e5a46aaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:33.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.936+0000 7f9c87400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e5a468a060 0x55e5a46aaf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:57376/0 (socket says 172.21.15.123:57376) 2024-05-08T06:13:33.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.936+0000 7f9c87400700 1 -- 172.21.15.123:0/1905991066 learned_addr learned my addr 172.21.15.123:0/1905991066 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:13:33.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.936+0000 7f9c87e00700 1 --2- 172.21.15.123:0/1905991066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e5a468e9d0 0x55e5a46ab9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:33.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.937+0000 7f9c87e00700 1 -- 172.21.15.123:0/1905991066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e5a468a060 msgr2=0x55e5a46aaf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:13:33.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.937+0000 7f9c87e00700 1 --2- 172.21.15.123:0/1905991066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e5a468a060 0x55e5a46aaf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:33.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.937+0000 7f9c87e00700 1 -- 172.21.15.123:0/1905991066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5a468a980 msgr2=0x55e5a46ab490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:13:33.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.937+0000 7f9c87e00700 1 --2- 172.21.15.123:0/1905991066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5a468a980 0x55e5a46ab490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:33.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.937+0000 7f9c87e00700 1 -- 172.21.15.123:0/1905991066 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5a470d350 con 0x55e5a468e9d0 2024-05-08T06:13:33.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.937+0000 7f9c87e00700 1 --2- 172.21.15.123:0/1905991066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e5a468e9d0 0x55e5a46ab9d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9c800124b0 tx=0x7f9c80004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec479a1ff094b1c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:13:33.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.937+0000 7f9c71600700 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-05-08T06:13:33.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.938+0000 7f9c71600700 1 -- 172.21.15.123:0/1905991066 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c80009e60 con 0x55e5a468e9d0 2024-05-08T06:13:33.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.938+0000 7f9c71600700 1 -- 172.21.15.123:0/1905991066 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f9c80007830 con 0x55e5a468e9d0 2024-05-08T06:13:33.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.938+0000 7f9c98edeb80 1 -- 172.21.15.123:0/1905991066 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c780aae50 con 0x55e5a468e9d0 2024-05-08T06:13:33.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.938+0000 7f9c71600700 1 -- 172.21.15.123:0/1905991066 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9c80016e50 con 0x55e5a468e9d0 2024-05-08T06:13:33.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.938+0000 7f9c98edeb80 10 client.25717 fetch_fsmap learned FSMap version 24 2024-05-08T06:13:33.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.938+0000 7f9c98edeb80 1 -- 172.21.15.123:0/1905991066 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e5a470e660 con 0x55e5a468e9d0 2024-05-08T06:13:33.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.938+0000 7f9c71600700 1 -- 172.21.15.123:0/1905991066 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c780aae50 con 0x55e5a468e9d0 2024-05-08T06:13:33.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.938+0000 7f9c71600700 1 -- 172.21.15.123:0/1905991066 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f9c8002bbd0 con 0x55e5a468e9d0 2024-05-08T06:13:33.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.939+0000 7f9c98edeb80 10 client.25717 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:13:33.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.939+0000 7f9c98edeb80 10 client.25717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T06:13:33.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.939+0000 7f9c98edeb80 20 client.25717 populate_metadata read hostname 'smithi123' 2024-05-08T06:13:33.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.939+0000 7f9c98edeb80 1 --2- 172.21.15.123:0/1905991066 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55e5a470edc0 0x55e5a47111e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:33.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.939+0000 7f9c98edeb80 4 client.25717 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T06:13:33.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.939+0000 7f9c98edeb80 1 -- 172.21.15.123:0/1905991066 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e5a45ea400 con 0x55e5a470edc0 2024-05-08T06:13:33.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.939+0000 7f9c87400700 1 --2- 172.21.15.123:0/1905991066 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55e5a470edc0 0x55e5a47111e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:33.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.940+0000 7f9c87400700 1 --2- 172.21.15.123:0/1905991066 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55e5a470edc0 0x55e5a47111e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=352a69a892547e0a server_cookie=21c7c7deff078969 in_seq=0 out_seq=0 2024-05-08T06:13:33.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.940+0000 7f9c71600700 10 client.25717 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:13:33.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.940+0000 7f9c71600700 1 -- 172.21.15.123:0/1905991066 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e5a45ea400 con 0x55e5a470edc0 2024-05-08T06:13:33.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.940+0000 7f9c71600700 10 client.25717 handle_command_reply: tid=0 2024-05-08T06:13:33.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.940+0000 7f9c98edeb80 1 client.25717 shutdown 2024-05-08T06:13:33.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.940+0000 7f9c98edeb80 1 -- 172.21.15.123:0/1905991066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e5a468e9d0 msgr2=0x55e5a46ab9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:33.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.940+0000 7f9c98edeb80 1 --2- 172.21.15.123:0/1905991066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e5a468e9d0 0x55e5a46ab9d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9c800124b0 tx=0x7f9c80004b40 comp rx=0 tx=0).stop 2024-05-08T06:13:33.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.940+0000 7f9c98edeb80 1 -- 172.21.15.123:0/1905991066 shutdown_connections 2024-05-08T06:13:33.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.941+0000 7f9c98edeb80 1 --2- 172.21.15.123:0/1905991066 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x55e5a470edc0 0x55e5a47111e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:33.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.941+0000 7f9c98edeb80 1 --2- 172.21.15.123:0/1905991066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e5a468e9d0 0x55e5a46ab9d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:33.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.941+0000 7f9c98edeb80 1 --2- 172.21.15.123:0/1905991066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e5a468a060 0x55e5a46aaf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:33.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.941+0000 7f9c98edeb80 1 --2- 172.21.15.123:0/1905991066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5a468a980 0x55e5a46ab490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:33.942 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.941+0000 7f9c98edeb80 1 -- 172.21.15.123:0/1905991066 >> 172.21.15.123:0/1905991066 conn(0x55e5a45b8b60 msgr2=0x55e5a46906b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:13:33.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.941+0000 7f9c71600700 0 client.25717 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:13:33.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.941+0000 7f9c98edeb80 1 -- 172.21.15.123:0/1905991066 shutdown_connections 2024-05-08T06:13:33.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.941+0000 7f9c98edeb80 1 -- 172.21.15.123:0/1905991066 wait complete. 2024-05-08T06:13:33.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.941+0000 7f9c98edeb80 20 client.25717 trim_cache size 0 max 16384 2024-05-08T06:13:33.943 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-05-08T06:13:33.943 INFO:teuthology.orchestra.run.smithi123.stdout: "status": "no active scrubs running", 2024-05-08T06:13:33.943 INFO:teuthology.orchestra.run.smithi123.stdout: "scrubs": {} 2024-05-08T06:13:33.943 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-05-08T06:13:33.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.942+0000 7f9c8e600700 1 -- 172.21.15.123:0/3687096816 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9c6c0b2900 msgr2=0x7f9c6c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:33.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.942+0000 7f9c8e600700 1 --2- 172.21.15.123:0/3687096816 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9c6c0b2900 0x7f9c6c0b4dc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9c8806ad70 tx=0x7f9c8000b040 comp rx=0 tx=0).stop 2024-05-08T06:13:33.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.942+0000 7f9c8e600700 1 -- 172.21.15.123:0/3687096816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c8806b1b0 msgr2=0x7f9c8819be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:33.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.942+0000 7f9c8e600700 1 --2- 172.21.15.123:0/3687096816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c8806b1b0 0x7f9c8819be00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9c7800e9c0 tx=0x7f9c7800ecd0 comp rx=0 tx=0).stop 2024-05-08T06:13:33.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.942+0000 7f9c8e600700 1 -- 172.21.15.123:0/3687096816 shutdown_connections 2024-05-08T06:13:33.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.942+0000 7f9c8e600700 1 --2- 172.21.15.123:0/3687096816 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f9c6c0b2900 0x7f9c6c0b4dc0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:33.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.942+0000 7f9c8e600700 1 --2- 172.21.15.123:0/3687096816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c8810ff50 0x7f9c881a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:33.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.942+0000 7f9c8e600700 1 --2- 172.21.15.123:0/3687096816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c8806b1b0 0x7f9c8819be00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:33.944 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.942+0000 7f9c8e600700 1 --2- 172.21.15.123:0/3687096816 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c8806bad0 0x7f9c8819c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:33.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.942+0000 7f9c8e600700 1 -- 172.21.15.123:0/3687096816 >> 172.21.15.123:0/3687096816 conn(0x7f9c88079350 msgr2=0x7f9c88111440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:13:33.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.942+0000 7f9c8e600700 1 -- 172.21.15.123:0/3687096816 shutdown_connections 2024-05-08T06:13:33.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:33.942+0000 7f9c8e600700 1 -- 172.21.15.123:0/3687096816 wait complete. 2024-05-08T06:13:34.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:33 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:13:34.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:33 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:13:34.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:33 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:13:34.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:33 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:13:34.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:33 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:13:34.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:33 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:13:34.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:33 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:13:34.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:33 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:13:34.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:33 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:13:34.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:34 smithi123 ceph-mon[29787]: pgmap v553: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:13:34.970 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T06:13:34.970 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T06:13:35.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:34 smithi192 ceph-mon[34664]: pgmap v553: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:13:35.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:34 smithi163 ceph-mon[34520]: pgmap v553: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:13:35.399 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:13:37.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:36 smithi123 ceph-mon[29787]: pgmap v554: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-08T06:13:37.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:36 smithi163 ceph-mon[34520]: pgmap v554: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-08T06:13:37.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:36 smithi192 ceph-mon[34664]: pgmap v554: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-08T06:13:37.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.931+0000 7f3199a00700 1 -- 172.21.15.123:0/4266740643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31941099e0 msgr2=0x7f3194109e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:37.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.931+0000 7f3199a00700 1 --2- 172.21.15.123:0/4266740643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31941099e0 0x7f3194109e60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3180009990 tx=0x7f3180009ca0 comp rx=0 tx=0).stop 2024-05-08T06:13:37.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.931+0000 7f3199a00700 1 -- 172.21.15.123:0/4266740643 shutdown_connections 2024-05-08T06:13:37.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.931+0000 7f3199a00700 1 --2- 172.21.15.123:0/4266740643 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f319410a3a0 0x7f3194116840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:37.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.931+0000 7f3199a00700 1 --2- 172.21.15.123:0/4266740643 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f319410e8b0 0x7f319410ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:37.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.931+0000 7f3199a00700 1 --2- 172.21.15.123:0/4266740643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31941099e0 0x7f3194109e60 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:37.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.931+0000 7f3199a00700 1 -- 172.21.15.123:0/4266740643 >> 172.21.15.123:0/4266740643 conn(0x7f3194079230 msgr2=0x7f31941082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:13:37.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.931+0000 7f3199a00700 1 -- 172.21.15.123:0/4266740643 shutdown_connections 2024-05-08T06:13:37.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.931+0000 7f3199a00700 1 -- 172.21.15.123:0/4266740643 wait complete. 2024-05-08T06:13:37.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.932+0000 7f3199a00700 1 Processor -- start 2024-05-08T06:13:37.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.933+0000 7f3199a00700 1 -- start start 2024-05-08T06:13:37.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.933+0000 7f3199a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31941099e0 0x7f31941a0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:37.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.933+0000 7f3199a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f319410a3a0 0x7f31941a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:37.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.933+0000 7f3199a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f319410e8b0 0x7f31941a4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:37.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.933+0000 7f3199a00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3194119be0 con 0x7f31941099e0 2024-05-08T06:13:37.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.933+0000 7f3199a00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f3194119a60 con 0x7f319410a3a0 2024-05-08T06:13:37.933 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.933+0000 7f3199a00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3194119d60 con 0x7f319410e8b0 2024-05-08T06:13:37.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.933+0000 7f3193e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31941099e0 0x7f31941a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:37.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.933+0000 7f3193e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31941099e0 0x7f31941a0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59190/0 (socket says 172.21.15.123:59190) 2024-05-08T06:13:37.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.933+0000 7f3193e00700 1 -- 172.21.15.123:0/2120470430 learned_addr learned my addr 172.21.15.123:0/2120470430 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:13:37.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.934+0000 7f3193400700 1 --2- 172.21.15.123:0/2120470430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f319410a3a0 0x7f31941a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:37.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.934+0000 7f3199000700 1 --2- 172.21.15.123:0/2120470430 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f319410e8b0 0x7f31941a4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:37.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.934+0000 7f3193e00700 1 -- 172.21.15.123:0/2120470430 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f319410e8b0 msgr2=0x7f31941a4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:37.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.934+0000 7f3193e00700 1 --2- 172.21.15.123:0/2120470430 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f319410e8b0 0x7f31941a4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:37.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.934+0000 7f3193e00700 1 -- 172.21.15.123:0/2120470430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f319410a3a0 msgr2=0x7f31941a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:37.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.934+0000 7f3193e00700 1 --2- 172.21.15.123:0/2120470430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f319410a3a0 0x7f31941a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:37.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.934+0000 7f3193e00700 1 -- 172.21.15.123:0/2120470430 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31941a50d0 con 0x7f31941099e0 2024-05-08T06:13:37.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.934+0000 7f3193e00700 1 --2- 172.21.15.123:0/2120470430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31941099e0 0x7f31941a0110 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f318400e9a0 tx=0x7f318400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:13:37.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.935+0000 7f3190c00700 1 -- 172.21.15.123:0/2120470430 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318400ca10 con 0x7f31941099e0 2024-05-08T06:13:37.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.935+0000 7f3190c00700 1 -- 172.21.15.123:0/2120470430 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f318400cbb0 con 0x7f31941099e0 2024-05-08T06:13:37.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.935+0000 7f3190c00700 1 -- 172.21.15.123:0/2120470430 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3184018680 con 0x7f31941099e0 2024-05-08T06:13:37.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.935+0000 7f3199a00700 1 -- 172.21.15.123:0/2120470430 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31941a53c0 con 0x7f31941099e0 2024-05-08T06:13:37.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.935+0000 7f3199a00700 1 -- 172.21.15.123:0/2120470430 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31941acc40 con 0x7f31941099e0 2024-05-08T06:13:37.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.936+0000 7f3199a00700 1 -- 172.21.15.123:0/2120470430 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f319406a190 con 0x7f31941099e0 2024-05-08T06:13:37.938 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.938+0000 7f3190c00700 1 -- 172.21.15.123:0/2120470430 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7f3184010430 con 0x7f31941099e0 2024-05-08T06:13:37.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.940+0000 7f3190c00700 1 --2- 172.21.15.123:0/2120470430 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f31780b2900 0x7f31780b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:37.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.940+0000 7f3190c00700 1 -- 172.21.15.123:0/2120470430 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f31840e22e0 con 0x7f31941099e0 2024-05-08T06:13:37.940 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.940+0000 7f3193400700 1 --2- 172.21.15.123:0/2120470430 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f31780b2900 0x7f31780b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:37.941 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.941+0000 7f3193400700 1 --2- 172.21.15.123:0/2120470430 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f31780b2900 0x7f31780b4dc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3194109520 tx=0x7f318000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:13:37.945 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:37.945+0000 7f3190c00700 1 -- 172.21.15.123:0/2120470430 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f31840a9fb0 con 0x7f31941099e0 2024-05-08T06:13:38.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:38.232+0000 7f3199a00700 1 -- 172.21.15.123:0/2120470430 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31941a1420 con 0x7f31941099e0 2024-05-08T06:13:38.234 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:38.234+0000 7f3190c00700 1 -- 172.21.15.123:0/2120470430 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11885 (secure 0 0 0) 0x7f31840a98b0 con 0x7f31941099e0 2024-05-08T06:13:38.234 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:13:38.235 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3646476522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3646476522},{"type":"v1","addr":"172.21.15.123:6835","nonce":3646476522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2763706497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2763706497},{"type":"v1","addr":"172.21.15.123:6837","nonce":2763706497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/274635265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":274635265},{"type":"v1","addr":"172.21.15.123:6839","nonce":274635265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/175423115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":175423115},{"type":"v1","addr":"172.21.15.192:6833","nonce":175423115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/3263009472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3263009472},{"type":"v1","addr":"172.21.15.163:6833","nonce":3263009472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3308612826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3308612826},{"type":"v1","addr":"172.21.15.163:6835","nonce":3308612826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T06:05:54.755550+0000","modified":"2024-05-08T06:10:12.600417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24512,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/202513680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":202513680},{"type":"v1","addr":"172.21.15.192:6835","nonce":202513680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/2483545969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2483545969},{"type":"v1","addr":"172.21.15.192:6837","nonce":2483545969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/660511838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":660511838},{"type":"v1","addr":"172.21.15.163:6837","nonce":660511838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.192:6839/3598682360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":3598682360},{"type":"v1","addr":"172.21.15.192:6839","nonce":3598682360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6841/1571726231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":1571726231},{"type":"v1","addr":"172.21.15.123:6841","nonce":1571726231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.163:6839/1568161515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":1568161515},{"type":"v1","addr":"172.21.15.163:6839","nonce":1568161515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24512,24496,24524]},"id":1}]} 2024-05-08T06:13:38.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:38.238+0000 7f3199a00700 1 -- 172.21.15.123:0/2120470430 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f31780b2900 msgr2=0x7f31780b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:38.238 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:38.238+0000 7f3199a00700 1 --2- 172.21.15.123:0/2120470430 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f31780b2900 0x7f31780b4dc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3194109520 tx=0x7f318000b260 comp rx=0 tx=0).stop 2024-05-08T06:13:38.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:38.238+0000 7f3199a00700 1 -- 172.21.15.123:0/2120470430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31941099e0 msgr2=0x7f31941a0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:38.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:38.238+0000 7f3199a00700 1 --2- 172.21.15.123:0/2120470430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31941099e0 0x7f31941a0110 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f318400e9a0 tx=0x7f318400ecb0 comp rx=0 tx=0).stop 2024-05-08T06:13:38.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:38.238+0000 7f3199a00700 1 -- 172.21.15.123:0/2120470430 shutdown_connections 2024-05-08T06:13:38.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:38.238+0000 7f3199a00700 1 --2- 172.21.15.123:0/2120470430 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f31780b2900 0x7f31780b4dc0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:38.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:38.238+0000 7f3199a00700 1 --2- 172.21.15.123:0/2120470430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31941099e0 0x7f31941a0110 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:38.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:38.238+0000 7f3199a00700 1 --2- 172.21.15.123:0/2120470430 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f319410e8b0 0x7f31941a4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:38.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:38.238+0000 7f3199a00700 1 --2- 172.21.15.123:0/2120470430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f319410a3a0 0x7f31941a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:38.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:38.238+0000 7f3199a00700 1 -- 172.21.15.123:0/2120470430 >> 172.21.15.123:0/2120470430 conn(0x7f3194079230 msgr2=0x7f3194079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:13:38.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:38.239+0000 7f3199a00700 1 -- 172.21.15.123:0/2120470430 shutdown_connections 2024-05-08T06:13:38.239 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:38.239+0000 7f3199a00700 1 -- 172.21.15.123:0/2120470430 wait complete. 2024-05-08T06:13:38.240 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 24 2024-05-08T06:13:38.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:38 smithi123 ceph-mon[29787]: pgmap v555: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:13:38.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:38 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/2120470430' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:13:39.102 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:38 smithi192 ceph-mon[34664]: pgmap v555: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:13:39.102 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:38 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/2120470430' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:13:39.188 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-08T06:13:39.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:38 smithi163 ceph-mon[34520]: pgmap v555: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:13:39.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:38 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/2120470430' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:13:39.628 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:13:40.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:40 smithi123 ceph-mon[29787]: pgmap v556: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-05-08T06:13:41.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:40 smithi163 ceph-mon[34520]: pgmap v556: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-05-08T06:13:41.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:40 smithi192 ceph-mon[34664]: pgmap v556: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-05-08T06:13:42.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.093+0000 7fb8f4600700 1 -- 172.21.15.123:0/2188069224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8ec109ad0 msgr2=0x7fb8ec109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:42.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.093+0000 7fb8f4600700 1 --2- 172.21.15.123:0/2188069224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8ec109ad0 0x7fb8ec109f50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8009990 tx=0x7fb8d8009ca0 comp rx=0 tx=0).stop 2024-05-08T06:13:42.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.093+0000 7fb8f4600700 1 -- 172.21.15.123:0/2188069224 shutdown_connections 2024-05-08T06:13:42.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.093+0000 7fb8f4600700 1 --2- 172.21.15.123:0/2188069224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8ec10a490 0x7fb8ec1169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:42.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.093+0000 7fb8f4600700 1 --2- 172.21.15.123:0/2188069224 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8ec10fb60 0x7fb8ec10ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:42.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.093+0000 7fb8f4600700 1 --2- 172.21.15.123:0/2188069224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8ec109ad0 0x7fb8ec109f50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:42.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.093+0000 7fb8f4600700 1 -- 172.21.15.123:0/2188069224 >> 172.21.15.123:0/2188069224 conn(0x7fb8ec076c70 msgr2=0x7fb8ec079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:13:42.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.094+0000 7fb8f4600700 1 -- 172.21.15.123:0/2188069224 shutdown_connections 2024-05-08T06:13:42.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.094+0000 7fb8f4600700 1 -- 172.21.15.123:0/2188069224 wait complete. 2024-05-08T06:13:42.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.095+0000 7fb8f4600700 1 Processor -- start 2024-05-08T06:13:42.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.095+0000 7fb8f4600700 1 -- start start 2024-05-08T06:13:42.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.095+0000 7fb8f4600700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8ec109ad0 0x7fb8ec1a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:42.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.095+0000 7fb8f4600700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8ec10a490 0x7fb8ec1a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:42.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.095+0000 7fb8f4600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8ec10fb60 0x7fb8ec1a4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:42.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.095+0000 7fb8f4600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec119d10 con 0x7fb8ec10fb60 2024-05-08T06:13:42.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.095+0000 7fb8f4600700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec119b90 con 0x7fb8ec10a490 2024-05-08T06:13:42.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.095+0000 7fb8f4600700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec119e90 con 0x7fb8ec109ad0 2024-05-08T06:13:42.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.096+0000 7fb8f1200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8ec10a490 0x7fb8ec1a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:42.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.096+0000 7fb8f2600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8ec10fb60 0x7fb8ec1a4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:42.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.096+0000 7fb8f1200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8ec10a490 0x7fb8ec1a0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:55314/0 (socket says 172.21.15.123:55314) 2024-05-08T06:13:42.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.096+0000 7fb8f1200700 1 -- 172.21.15.123:0/1543483182 learned_addr learned my addr 172.21.15.123:0/1543483182 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:13:42.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.096+0000 7fb8f1c00700 1 --2- 172.21.15.123:0/1543483182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8ec109ad0 0x7fb8ec1a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:42.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.096+0000 7fb8f1200700 1 -- 172.21.15.123:0/1543483182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8ec109ad0 msgr2=0x7fb8ec1a01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:42.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.096+0000 7fb8f1200700 1 --2- 172.21.15.123:0/1543483182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8ec109ad0 0x7fb8ec1a01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:42.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.096+0000 7fb8f1200700 1 -- 172.21.15.123:0/1543483182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8ec10fb60 msgr2=0x7fb8ec1a4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:42.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.096+0000 7fb8f1200700 1 --2- 172.21.15.123:0/1543483182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8ec10fb60 0x7fb8ec1a4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:42.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.096+0000 7fb8f1200700 1 -- 172.21.15.123:0/1543483182 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8ec1a51d0 con 0x7fb8ec10a490 2024-05-08T06:13:42.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.096+0000 7fb8f1200700 1 --2- 172.21.15.123:0/1543483182 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8ec10a490 0x7fb8ec1a0700 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80057f0 tx=0x7fb8d800bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:13:42.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.097+0000 7fb8ea000700 1 -- 172.21.15.123:0/1543483182 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8026070 con 0x7fb8ec10a490 2024-05-08T06:13:42.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.097+0000 7fb8f4600700 1 -- 172.21.15.123:0/1543483182 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8ec1a5460 con 0x7fb8ec10a490 2024-05-08T06:13:42.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.097+0000 7fb8ea000700 1 -- 172.21.15.123:0/1543483182 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8d8021410 con 0x7fb8ec10a490 2024-05-08T06:13:42.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.097+0000 7fb8ea000700 1 -- 172.21.15.123:0/1543483182 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8017650 con 0x7fb8ec10a490 2024-05-08T06:13:42.098 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.097+0000 7fb8f4600700 1 -- 172.21.15.123:0/1543483182 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8ec1acd40 con 0x7fb8ec10a490 2024-05-08T06:13:42.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.100+0000 7fb8ea000700 1 -- 172.21.15.123:0/1543483182 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7fb8d8029070 con 0x7fb8ec10a490 2024-05-08T06:13:42.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.101+0000 7fb8feef3b80 1 Processor -- start 2024-05-08T06:13:42.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.102+0000 7fb8feef3b80 1 -- start start 2024-05-08T06:13:42.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.102+0000 7fb8feef3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563af3e88060 0x563af3e88440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:42.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.102+0000 7fb8feef3b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563af3e88980 0x563af3e8c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:42.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.102+0000 7fb8feef3b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563af3e8c9d0 0x563af3e8ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:42.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.102+0000 7fb8feef3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563af3ddaa20 con 0x563af3e88060 2024-05-08T06:13:42.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.102+0000 7fb8feef3b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x563af3d78760 con 0x563af3e8c9d0 2024-05-08T06:13:42.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.102+0000 7fb8feef3b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x563af3dd69e0 con 0x563af3e88980 2024-05-08T06:13:42.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.102+0000 7fb8ea000700 1 --2- 172.21.15.123:0/1543483182 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fb8d40b2880 0x7fb8d40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:42.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.102+0000 7fb8ea000700 1 -- 172.21.15.123:0/1543483182 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fb8d80e2e50 con 0x7fb8ec10a490 2024-05-08T06:13:42.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.102+0000 7fb8f2600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563af3e88060 0x563af3e88440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:42.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.102+0000 7fb8f1c00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563af3e88980 0x563af3e8c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:42.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.102+0000 7fb8f1200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563af3e8c9d0 0x563af3e8ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:42.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.102+0000 7fb8f1c00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563af3e88980 0x563af3e8c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:46596/0 (socket says 172.21.15.123:46596) 2024-05-08T06:13:42.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.102+0000 7fb8f2600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563af3e88060 0x563af3e88440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59208/0 (socket says 172.21.15.123:59208) 2024-05-08T06:13:42.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.102+0000 7fb8f1c00700 1 -- 172.21.15.123:0/729271770 learned_addr learned my addr 172.21.15.123:0/729271770 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:13:42.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.103+0000 7fb8f1200700 1 -- 172.21.15.123:0/729271770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563af3e88980 msgr2=0x563af3e8c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:13:42.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.103+0000 7fb8f1200700 1 --2- 172.21.15.123:0/729271770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563af3e88980 0x563af3e8c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:42.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.103+0000 7fb8f2600700 1 --2- 172.21.15.123:0/1543483182 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fb8d40b2880 0x7fb8d40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:42.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.103+0000 7fb8f1200700 1 -- 172.21.15.123:0/729271770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563af3e88060 msgr2=0x563af3e88440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:13:42.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.103+0000 7fb8f1200700 1 --2- 172.21.15.123:0/729271770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563af3e88060 0x563af3e88440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:42.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.103+0000 7fb8f1200700 1 -- 172.21.15.123:0/729271770 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563af3d278d0 con 0x563af3e8c9d0 2024-05-08T06:13:42.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.103+0000 7fb8f1200700 1 --2- 172.21.15.123:0/729271770 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563af3e8c9d0 0x563af3e8ce30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80e0990 tx=0x7fb8d8009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edbc7a97ab3abc49 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:13:42.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.103+0000 7fb8dea00700 1 -- 172.21.15.123:0/729271770 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d80aa050 con 0x563af3e8c9d0 2024-05-08T06:13:42.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.104+0000 7fb8dea00700 1 -- 172.21.15.123:0/729271770 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8d80aac40 con 0x563af3e8c9d0 2024-05-08T06:13:42.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.104+0000 7fb8dea00700 1 -- 172.21.15.123:0/729271770 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d80e0a10 con 0x563af3e8c9d0 2024-05-08T06:13:42.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.104+0000 7fb8feef3b80 1 -- 172.21.15.123:0/729271770 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563af3e8c9d0 msgr2=0x563af3e8ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:42.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.104+0000 7fb8feef3b80 1 --2- 172.21.15.123:0/729271770 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563af3e8c9d0 0x563af3e8ce30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80e0990 tx=0x7fb8d8009ca0 comp rx=0 tx=0).stop 2024-05-08T06:13:42.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.104+0000 7fb8f2600700 1 --2- 172.21.15.123:0/1543483182 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fb8d40b2880 0x7fb8d40b4d40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4009e10 tx=0x7fb8e400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:13:42.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.104+0000 7fb8feef3b80 1 -- 172.21.15.123:0/729271770 shutdown_connections 2024-05-08T06:13:42.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.104+0000 7fb8feef3b80 1 --2- 172.21.15.123:0/729271770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563af3e88060 0x563af3e88440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:42.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.104+0000 7fb8feef3b80 1 --2- 172.21.15.123:0/729271770 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563af3e8c9d0 0x563af3e8ce30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:42.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.104+0000 7fb8feef3b80 1 --2- 172.21.15.123:0/729271770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563af3e88980 0x563af3e8c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:42.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.104+0000 7fb8feef3b80 1 -- 172.21.15.123:0/729271770 >> 172.21.15.123:0/729271770 conn(0x563af3db6b60 msgr2=0x563af3e827a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:13:42.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.104+0000 7fb8feef3b80 1 -- 172.21.15.123:0/729271770 shutdown_connections 2024-05-08T06:13:42.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.105+0000 7fb8feef3b80 1 -- 172.21.15.123:0/729271770 wait complete. 2024-05-08T06:13:42.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.105+0000 7fb8feef3b80 1 -- start start 2024-05-08T06:13:42.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.105+0000 7fb8feef3b80 1 Processor -- start 2024-05-08T06:13:42.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.106+0000 7fb8feef3b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563af3e88060 0x563af3e93e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:42.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.106+0000 7fb8feef3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563af3e88980 0x563af3e911a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:42.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.106+0000 7fb8feef3b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563af3e8c9d0 0x563af3e916e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:42.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.106+0000 7fb8feef3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563af3e949a0 con 0x563af3e88980 2024-05-08T06:13:42.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.106+0000 7fb8feef3b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x563af3d40b20 con 0x563af3e8c9d0 2024-05-08T06:13:42.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.106+0000 7fb8feef3b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x563af3e94b20 con 0x563af3e88060 2024-05-08T06:13:42.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.106+0000 7fb8feef3b80 10 client.0 _ll_register_callbacks cb 0x563af3d81c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:13:42.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.106+0000 7fb8f1200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563af3e88980 0x563af3e911a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:42.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.106+0000 7fb8f2600700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563af3e8c9d0 0x563af3e916e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:42.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.106+0000 7fb8f1c00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563af3e88060 0x563af3e93e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:42.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.106+0000 7fb8f2600700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563af3e8c9d0 0x563af3e916e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:55330/0 (socket says 172.21.15.123:55330) 2024-05-08T06:13:42.109 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.106+0000 7fb8f1200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563af3e88980 0x563af3e911a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59216/0 (socket says 172.21.15.123:59216) 2024-05-08T06:13:42.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.106+0000 7fb8f2600700 1 -- 172.21.15.123:0/4113983775 learned_addr learned my addr 172.21.15.123:0/4113983775 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:13:42.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.107+0000 7fb8f1200700 1 -- 172.21.15.123:0/4113983775 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563af3e88060 msgr2=0x563af3e93e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:13:42.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.107+0000 7fb8f1200700 1 --2- 172.21.15.123:0/4113983775 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563af3e88060 0x563af3e93e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:42.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.107+0000 7fb8f1200700 1 -- 172.21.15.123:0/4113983775 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563af3e8c9d0 msgr2=0x563af3e916e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:13:42.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.107+0000 7fb8f1200700 1 --2- 172.21.15.123:0/4113983775 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563af3e8c9d0 0x563af3e916e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:42.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.107+0000 7fb8f1200700 1 -- 172.21.15.123:0/4113983775 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563af3da0420 con 0x563af3e88980 2024-05-08T06:13:42.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.107+0000 7fb8f2600700 1 --2- 172.21.15.123:0/4113983775 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563af3e8c9d0 0x563af3e916e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:13:42.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.107+0000 7fb8f1200700 1 --2- 172.21.15.123:0/4113983775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563af3e88980 0x563af3e911a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80aa1f0 tx=0x7fb8d80b3380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb6572a314ff9b2d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:13:42.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.107+0000 7fb8d3400700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-08T06:13:42.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.107+0000 7fb8d3400700 1 -- 172.21.15.123:0/4113983775 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d80ae8f0 con 0x563af3e88980 2024-05-08T06:13:42.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.107+0000 7fb8d3400700 1 -- 172.21.15.123:0/4113983775 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fb8d80094b0 con 0x563af3e88980 2024-05-08T06:13:42.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.107+0000 7fb8feef3b80 1 -- 172.21.15.123:0/4113983775 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8d80aac40 con 0x563af3e88980 2024-05-08T06:13:42.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.108+0000 7fb8d3400700 1 -- 172.21.15.123:0/4113983775 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb8d80c4050 con 0x563af3e88980 2024-05-08T06:13:42.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.108+0000 7fb8feef3b80 10 client.15624 fetch_fsmap learned FSMap version 24 2024-05-08T06:13:42.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.108+0000 7fb8feef3b80 1 -- 172.21.15.123:0/4113983775 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563af3e949a0 con 0x563af3e88980 2024-05-08T06:13:42.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.108+0000 7fb8d3400700 1 -- 172.21.15.123:0/4113983775 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8d80aac40 con 0x563af3e88980 2024-05-08T06:13:42.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.108+0000 7fb8d3400700 1 -- 172.21.15.123:0/4113983775 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fb8d80b1be0 con 0x563af3e88980 2024-05-08T06:13:42.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.108+0000 7fb8feef3b80 10 client.15624 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:13:42.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.108+0000 7fb8feef3b80 10 client.15624 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T06:13:42.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.108+0000 7fb8feef3b80 20 client.15624 populate_metadata read hostname 'smithi123' 2024-05-08T06:13:42.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.108+0000 7fb8feef3b80 1 --2- 172.21.15.123:0/4113983775 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x563af3ea1400 0x563af3ea37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:42.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.108+0000 7fb8feef3b80 4 client.15624 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:13:42.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.108+0000 7fb8feef3b80 1 -- 172.21.15.123:0/4113983775 --> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563af3e550a0 con 0x563af3ea1400 2024-05-08T06:13:42.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.109+0000 7fb8f1c00700 1 --2- 172.21.15.123:0/4113983775 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x563af3ea1400 0x563af3ea37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:42.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.109+0000 7fb8f1c00700 1 --2- 172.21.15.123:0/4113983775 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x563af3ea1400 0x563af3ea37f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2b6891348e888680 server_cookie=e69ea8835a1b7d8f in_seq=0 out_seq=0 2024-05-08T06:13:42.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.109+0000 7fb8d3400700 10 client.15624 ms_handle_connect on v2:172.21.15.192:6838/3598682360 2024-05-08T06:13:42.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.110+0000 7fb8d3400700 1 -- 172.21.15.123:0/4113983775 <== mds.2 v2:172.21.15.192:6838/3598682360 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563af3e550a0 con 0x563af3ea1400 2024-05-08T06:13:42.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.110+0000 7fb8d3400700 10 client.15624 handle_command_reply: tid=0 2024-05-08T06:13:42.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.110+0000 7fb8feef3b80 1 client.15624 shutdown 2024-05-08T06:13:42.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.111+0000 7fb8feef3b80 1 -- 172.21.15.123:0/4113983775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563af3e88980 msgr2=0x563af3e911a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:42.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.111+0000 7fb8feef3b80 1 --2- 172.21.15.123:0/4113983775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563af3e88980 0x563af3e911a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80aa1f0 tx=0x7fb8d80b3380 comp rx=0 tx=0).stop 2024-05-08T06:13:42.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.111+0000 7fb8feef3b80 1 -- 172.21.15.123:0/4113983775 shutdown_connections 2024-05-08T06:13:42.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.111+0000 7fb8feef3b80 1 --2- 172.21.15.123:0/4113983775 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x563af3ea1400 0x563af3ea37f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:42.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.111+0000 7fb8feef3b80 1 --2- 172.21.15.123:0/4113983775 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563af3e8c9d0 0x563af3e916e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:42.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.111+0000 7fb8feef3b80 1 --2- 172.21.15.123:0/4113983775 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563af3e88060 0x563af3e93e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:42.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.111+0000 7fb8d3400700 0 client.15624 ms_handle_reset on v2:172.21.15.192:6838/3598682360 2024-05-08T06:13:42.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.111+0000 7fb8feef3b80 1 --2- 172.21.15.123:0/4113983775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563af3e88980 0x563af3e911a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:42.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.111+0000 7fb8feef3b80 1 -- 172.21.15.123:0/4113983775 >> 172.21.15.123:0/4113983775 conn(0x563af3db6b60 msgr2=0x563af3e7d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:13:42.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.111+0000 7fb8feef3b80 1 -- 172.21.15.123:0/4113983775 shutdown_connections 2024-05-08T06:13:42.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.111+0000 7fb8feef3b80 1 -- 172.21.15.123:0/4113983775 wait complete. 2024-05-08T06:13:42.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.111+0000 7fb8feef3b80 20 client.15624 trim_cache size 0 max 16384 2024-05-08T06:13:42.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.128+0000 7fb8feef3b80 1 Processor -- start 2024-05-08T06:13:42.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.128+0000 7fb8feef3b80 1 -- start start 2024-05-08T06:13:42.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.128+0000 7fb8feef3b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563af3e88060 0x563af3e9d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:42.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.128+0000 7fb8feef3b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563af3e88980 0x563af3e8ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:42.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.128+0000 7fb8feef3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563af3e8c9d0 0x563af3e901a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:42.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.128+0000 7fb8feef3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec1b6900 con 0x563af3e8c9d0 2024-05-08T06:13:42.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.128+0000 7fb8feef3b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec1b4190 con 0x563af3e88980 2024-05-08T06:13:42.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.128+0000 7fb8feef3b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec1b1fe0 con 0x563af3e88060 2024-05-08T06:13:42.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.129+0000 7fb8f2600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563af3e8c9d0 0x563af3e901a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:42.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.129+0000 7fb8f1c00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563af3e88060 0x563af3e9d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:42.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.129+0000 7fb8f2600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563af3e8c9d0 0x563af3e901a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59218/0 (socket says 172.21.15.123:59218) 2024-05-08T06:13:42.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.129+0000 7fb8f2600700 1 -- 172.21.15.123:0/1126949664 learned_addr learned my addr 172.21.15.123:0/1126949664 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:13:42.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.129+0000 7fb8f1200700 1 --2- 172.21.15.123:0/1126949664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563af3e88980 0x563af3e8ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:42.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.129+0000 7fb8f1c00700 1 -- 172.21.15.123:0/1126949664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563af3e88980 msgr2=0x563af3e8ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:13:42.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.129+0000 7fb8f1c00700 1 --2- 172.21.15.123:0/1126949664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563af3e88980 0x563af3e8ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:42.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.129+0000 7fb8f1c00700 1 -- 172.21.15.123:0/1126949664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563af3e8c9d0 msgr2=0x563af3e901a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:13:42.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.129+0000 7fb8f1c00700 1 --2- 172.21.15.123:0/1126949664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563af3e8c9d0 0x563af3e901a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:42.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.129+0000 7fb8f1c00700 1 -- 172.21.15.123:0/1126949664 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563af3e7ef90 con 0x563af3e88060 2024-05-08T06:13:42.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.129+0000 7fb8f1c00700 1 --2- 172.21.15.123:0/1126949664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563af3e88060 0x563af3e9d9f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x563af3dd6a80 tx=0x7fb8e0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=801b33a4734c285a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:13:42.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.130+0000 7fb8de000700 1 -- 172.21.15.123:0/1126949664 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0003cc0 con 0x563af3e88060 2024-05-08T06:13:42.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.130+0000 7fb8de000700 1 -- 172.21.15.123:0/1126949664 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8e0004ce0 con 0x563af3e88060 2024-05-08T06:13:42.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.130+0000 7fb8de000700 1 -- 172.21.15.123:0/1126949664 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e001ea40 con 0x563af3e88060 2024-05-08T06:13:42.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.130+0000 7fb8feef3b80 1 -- 172.21.15.123:0/1126949664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563af3e88060 msgr2=0x563af3e9d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:42.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.130+0000 7fb8feef3b80 1 --2- 172.21.15.123:0/1126949664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563af3e88060 0x563af3e9d9f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x563af3dd6a80 tx=0x7fb8e0004750 comp rx=0 tx=0).stop 2024-05-08T06:13:42.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.130+0000 7fb8feef3b80 1 -- 172.21.15.123:0/1126949664 shutdown_connections 2024-05-08T06:13:42.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.130+0000 7fb8feef3b80 1 --2- 172.21.15.123:0/1126949664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563af3e8c9d0 0x563af3e901a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:42.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.130+0000 7fb8feef3b80 1 --2- 172.21.15.123:0/1126949664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563af3e88060 0x563af3e9d9f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:42.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.130+0000 7fb8feef3b80 1 --2- 172.21.15.123:0/1126949664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563af3e88980 0x563af3e8ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:42.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.130+0000 7fb8feef3b80 1 -- 172.21.15.123:0/1126949664 >> 172.21.15.123:0/1126949664 conn(0x563af3db6b60 msgr2=0x563af3e8fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:13:42.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.131+0000 7fb8feef3b80 1 -- 172.21.15.123:0/1126949664 shutdown_connections 2024-05-08T06:13:42.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.131+0000 7fb8feef3b80 1 -- 172.21.15.123:0/1126949664 wait complete. 2024-05-08T06:13:42.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.131+0000 7fb8feef3b80 1 -- start start 2024-05-08T06:13:42.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.131+0000 7fb8feef3b80 1 Processor -- start 2024-05-08T06:13:42.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.132+0000 7fb8feef3b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563af3e88060 0x563af3e9a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:42.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.132+0000 7fb8feef3b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563af3e88980 0x563af3e8a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:42.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.132+0000 7fb8feef3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563af3e8c9d0 0x563af3e8a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:42.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.132+0000 7fb8feef3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec1b2650 con 0x563af3e8c9d0 2024-05-08T06:13:42.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.132+0000 7fb8feef3b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec1b3ac0 con 0x563af3e88980 2024-05-08T06:13:42.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.132+0000 7fb8feef3b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec1b2d20 con 0x563af3e88060 2024-05-08T06:13:42.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.132+0000 7fb8feef3b80 10 client.0 _ll_register_callbacks cb 0x563af3d81c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:13:42.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.132+0000 7fb8f2600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563af3e8c9d0 0x563af3e8a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:42.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.132+0000 7fb8f2600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563af3e8c9d0 0x563af3e8a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59234/0 (socket says 172.21.15.123:59234) 2024-05-08T06:13:42.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.132+0000 7fb8f2600700 1 -- 172.21.15.123:0/3024196468 learned_addr learned my addr 172.21.15.123:0/3024196468 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:13:42.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.132+0000 7fb8f1200700 1 --2- 172.21.15.123:0/3024196468 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563af3e88980 0x563af3e8a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:42.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.132+0000 7fb8f1c00700 1 --2- 172.21.15.123:0/3024196468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563af3e88060 0x563af3e9a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:42.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.132+0000 7fb8f2600700 1 -- 172.21.15.123:0/3024196468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563af3e88060 msgr2=0x563af3e9a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:13:42.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.132+0000 7fb8f2600700 1 --2- 172.21.15.123:0/3024196468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563af3e88060 0x563af3e9a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:42.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.132+0000 7fb8f2600700 1 -- 172.21.15.123:0/3024196468 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563af3e88980 msgr2=0x563af3e8a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:13:42.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.132+0000 7fb8f2600700 1 --2- 172.21.15.123:0/3024196468 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563af3e88980 0x563af3e8a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:42.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.132+0000 7fb8f2600700 1 -- 172.21.15.123:0/3024196468 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563af3e8ae70 con 0x563af3e8c9d0 2024-05-08T06:13:42.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.132+0000 7fb8f1c00700 1 --2- 172.21.15.123:0/3024196468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563af3e88060 0x563af3e9a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:13:42.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.132+0000 7fb8f1200700 1 --2- 172.21.15.123:0/3024196468 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563af3e88980 0x563af3e8a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:13:42.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.133+0000 7fb8f2600700 1 --2- 172.21.15.123:0/3024196468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563af3e8c9d0 0x563af3e8a800 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4007e70 tx=0x7fb8e4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910ef7b41717165e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:13:42.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.133+0000 7fb8d3400700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-08T06:13:42.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.133+0000 7fb8d3400700 1 -- 172.21.15.123:0/3024196468 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e401ae20 con 0x563af3e8c9d0 2024-05-08T06:13:42.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.133+0000 7fb8d3400700 1 -- 172.21.15.123:0/3024196468 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fb8e4012040 con 0x563af3e8c9d0 2024-05-08T06:13:42.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.133+0000 7fb8feef3b80 1 -- 172.21.15.123:0/3024196468 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8e0004ce0 con 0x563af3e8c9d0 2024-05-08T06:13:42.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.133+0000 7fb8d3400700 1 -- 172.21.15.123:0/3024196468 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb8e4007ea0 con 0x563af3e8c9d0 2024-05-08T06:13:42.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.133+0000 7fb8feef3b80 10 client.15630 fetch_fsmap learned FSMap version 24 2024-05-08T06:13:42.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.133+0000 7fb8feef3b80 1 -- 172.21.15.123:0/3024196468 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563af3f0bbd0 con 0x563af3e8c9d0 2024-05-08T06:13:42.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.134+0000 7fb8d3400700 1 -- 172.21.15.123:0/3024196468 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8e0004ce0 con 0x563af3e8c9d0 2024-05-08T06:13:42.136 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.134+0000 7fb8d3400700 1 -- 172.21.15.123:0/3024196468 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fb8e4031590 con 0x563af3e8c9d0 2024-05-08T06:13:42.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.134+0000 7fb8feef3b80 10 client.15630 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:13:42.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.134+0000 7fb8feef3b80 10 client.15630 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-08T06:13:42.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.134+0000 7fb8feef3b80 20 client.15630 populate_metadata read hostname 'smithi123' 2024-05-08T06:13:42.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.134+0000 7fb8feef3b80 1 --2- 172.21.15.123:0/3024196468 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x563af3f0c1f0 0x563af3f0e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:42.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.134+0000 7fb8feef3b80 4 client.15630 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T06:13:42.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.134+0000 7fb8feef3b80 1 -- 172.21.15.123:0/3024196468 --> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563af3de8400 con 0x563af3f0c1f0 2024-05-08T06:13:42.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.135+0000 7fb8f1c00700 1 --2- 172.21.15.123:0/3024196468 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x563af3f0c1f0 0x563af3f0e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:42.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.135+0000 7fb8f1c00700 1 --2- 172.21.15.123:0/3024196468 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x563af3f0c1f0 0x563af3f0e610 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a2e6d4c1359af713 server_cookie=6d8dfe59d4354ca3 in_seq=0 out_seq=0 2024-05-08T06:13:42.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.135+0000 7fb8d3400700 10 client.15630 ms_handle_connect on v2:172.21.15.192:6838/3598682360 2024-05-08T06:13:42.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.135+0000 7fb8d3400700 1 -- 172.21.15.123:0/3024196468 <== mds.2 v2:172.21.15.192:6838/3598682360 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563af3de8400 con 0x563af3f0c1f0 2024-05-08T06:13:42.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.135+0000 7fb8d3400700 10 client.15630 handle_command_reply: tid=0 2024-05-08T06:13:42.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.135+0000 7fb8feef3b80 1 client.15630 shutdown 2024-05-08T06:13:42.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.136+0000 7fb8feef3b80 1 -- 172.21.15.123:0/3024196468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563af3e8c9d0 msgr2=0x563af3e8a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:42.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.136+0000 7fb8feef3b80 1 --2- 172.21.15.123:0/3024196468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563af3e8c9d0 0x563af3e8a800 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4007e70 tx=0x7fb8e4004ad0 comp rx=0 tx=0).stop 2024-05-08T06:13:42.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.136+0000 7fb8feef3b80 1 -- 172.21.15.123:0/3024196468 shutdown_connections 2024-05-08T06:13:42.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.136+0000 7fb8feef3b80 1 --2- 172.21.15.123:0/3024196468 >> [v2:172.21.15.192:6838/3598682360,v1:172.21.15.192:6839/3598682360] conn(0x563af3f0c1f0 0x563af3f0e610 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:42.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.136+0000 7fb8feef3b80 1 --2- 172.21.15.123:0/3024196468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563af3e8c9d0 0x563af3e8a800 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:42.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.136+0000 7fb8feef3b80 1 --2- 172.21.15.123:0/3024196468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563af3e88060 0x563af3e9a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:42.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.136+0000 7fb8feef3b80 1 --2- 172.21.15.123:0/3024196468 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563af3e88980 0x563af3e8a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:42.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.136+0000 7fb8feef3b80 1 -- 172.21.15.123:0/3024196468 >> 172.21.15.123:0/3024196468 conn(0x563af3db6b60 msgr2=0x563af3e7d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:13:42.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.136+0000 7fb8d3400700 0 client.15630 ms_handle_reset on v2:172.21.15.192:6838/3598682360 2024-05-08T06:13:42.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.136+0000 7fb8feef3b80 1 -- 172.21.15.123:0/3024196468 shutdown_connections 2024-05-08T06:13:42.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.136+0000 7fb8feef3b80 1 -- 172.21.15.123:0/3024196468 wait complete. 2024-05-08T06:13:42.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.136+0000 7fb8feef3b80 20 client.15630 trim_cache size 0 max 16384 2024-05-08T06:13:42.140 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-05-08T06:13:42.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.137+0000 7fb8f4600700 1 -- 172.21.15.123:0/1543483182 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fb8d40b2880 msgr2=0x7fb8d40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:42.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.137+0000 7fb8f4600700 1 --2- 172.21.15.123:0/1543483182 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fb8d40b2880 0x7fb8d40b4d40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4009e10 tx=0x7fb8e400b040 comp rx=0 tx=0).stop 2024-05-08T06:13:42.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.137+0000 7fb8f4600700 1 -- 172.21.15.123:0/1543483182 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8ec10a490 msgr2=0x7fb8ec1a0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:42.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.137+0000 7fb8f4600700 1 --2- 172.21.15.123:0/1543483182 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8ec10a490 0x7fb8ec1a0700 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80057f0 tx=0x7fb8d800bfd0 comp rx=0 tx=0).stop 2024-05-08T06:13:42.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.137+0000 7fb8f4600700 1 -- 172.21.15.123:0/1543483182 shutdown_connections 2024-05-08T06:13:42.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.137+0000 7fb8f4600700 1 --2- 172.21.15.123:0/1543483182 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fb8d40b2880 0x7fb8d40b4d40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:42.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.137+0000 7fb8f4600700 1 --2- 172.21.15.123:0/1543483182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8ec10fb60 0x7fb8ec1a4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:42.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.137+0000 7fb8f4600700 1 --2- 172.21.15.123:0/1543483182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8ec109ad0 0x7fb8ec1a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:42.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.137+0000 7fb8f4600700 1 --2- 172.21.15.123:0/1543483182 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8ec10a490 0x7fb8ec1a0700 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:42.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.137+0000 7fb8f4600700 1 -- 172.21.15.123:0/1543483182 >> 172.21.15.123:0/1543483182 conn(0x7fb8ec076c70 msgr2=0x7fb8ec077410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:13:42.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.137+0000 7fb8f4600700 1 -- 172.21.15.123:0/1543483182 shutdown_connections 2024-05-08T06:13:42.141 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:42.137+0000 7fb8f4600700 1 -- 172.21.15.123:0/1543483182 wait complete. 2024-05-08T06:13:42.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:42 smithi123 ceph-mon[29787]: pgmap v557: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:13:43.256 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-08T06:13:43.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:42 smithi163 ceph-mon[34520]: pgmap v557: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:13:43.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:42 smithi192 ceph-mon[34664]: pgmap v557: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:13:43.702 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:13:45.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:44 smithi123 ceph-mon[29787]: pgmap v558: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:13:45.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:44 smithi163 ceph-mon[34520]: pgmap v558: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:13:45.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:44 smithi192 ceph-mon[34664]: pgmap v558: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:13:46.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.213+0000 7f2d7fe00700 1 -- 172.21.15.123:0/1289479755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d7806bb40 msgr2=0x7f2d78110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:46.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.213+0000 7f2d7fe00700 1 --2- 172.21.15.123:0/1289479755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d7806bb40 0x7f2d78110330 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2d70009f90 tx=0x7f2d7000b330 comp rx=0 tx=0).stop 2024-05-08T06:13:46.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.213+0000 7f2d7fe00700 1 -- 172.21.15.123:0/1289479755 shutdown_connections 2024-05-08T06:13:46.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.213+0000 7f2d7fe00700 1 --2- 172.21.15.123:0/1289479755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d7806bb40 0x7f2d78110330 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:46.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.213+0000 7f2d7fe00700 1 --2- 172.21.15.123:0/1289479755 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d78109510 0x7f2d781098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:46.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.213+0000 7f2d7fe00700 1 --2- 172.21.15.123:0/1289479755 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d7806b180 0x7f2d7806b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:46.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.213+0000 7f2d7fe00700 1 -- 172.21.15.123:0/1289479755 >> 172.21.15.123:0/1289479755 conn(0x7f2d78079350 msgr2=0x7f2d78079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:13:46.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.213+0000 7f2d7fe00700 1 -- 172.21.15.123:0/1289479755 shutdown_connections 2024-05-08T06:13:46.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.213+0000 7f2d7fe00700 1 -- 172.21.15.123:0/1289479755 wait complete. 2024-05-08T06:13:46.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.214+0000 7f2d7fe00700 1 Processor -- start 2024-05-08T06:13:46.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.214+0000 7f2d7fe00700 1 -- start start 2024-05-08T06:13:46.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.215+0000 7f2d7fe00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d7806b180 0x7f2d781a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:46.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.215+0000 7f2d7fe00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d7806bb40 0x7f2d781a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:46.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.215+0000 7f2d7fe00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d78109510 0x7f2d7819bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:46.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.215+0000 7f2d7fe00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2d78113670 con 0x7f2d7806b180 2024-05-08T06:13:46.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.215+0000 7f2d7fe00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2d781134f0 con 0x7f2d78109510 2024-05-08T06:13:46.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.215+0000 7f2d7fe00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2d781137f0 con 0x7f2d7806bb40 2024-05-08T06:13:46.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.215+0000 7f2d7d400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d7806b180 0x7f2d781a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:46.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.215+0000 7f2d7ca00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d7806bb40 0x7f2d781a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:46.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.215+0000 7f2d7d400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d7806b180 0x7f2d781a1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51652/0 (socket says 172.21.15.123:51652) 2024-05-08T06:13:46.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.215+0000 7f2d7d400700 1 -- 172.21.15.123:0/492926675 learned_addr learned my addr 172.21.15.123:0/492926675 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:13:46.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.215+0000 7f2d7de00700 1 --2- 172.21.15.123:0/492926675 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d78109510 0x7f2d7819bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:46.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.216+0000 7f2d7ca00700 1 -- 172.21.15.123:0/492926675 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d78109510 msgr2=0x7f2d7819bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:46.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.216+0000 7f2d7ca00700 1 --2- 172.21.15.123:0/492926675 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d78109510 0x7f2d7819bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:46.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.216+0000 7f2d7ca00700 1 -- 172.21.15.123:0/492926675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d7806b180 msgr2=0x7f2d781a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:46.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.216+0000 7f2d7ca00700 1 --2- 172.21.15.123:0/492926675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d7806b180 0x7f2d781a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:46.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.216+0000 7f2d7ca00700 1 -- 172.21.15.123:0/492926675 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d7819c410 con 0x7f2d7806bb40 2024-05-08T06:13:46.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.216+0000 7f2d7d400700 1 --2- 172.21.15.123:0/492926675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d7806b180 0x7f2d781a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:13:46.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.216+0000 7f2d7de00700 1 --2- 172.21.15.123:0/492926675 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d78109510 0x7f2d7819bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:13:46.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.216+0000 7f2d7ca00700 1 --2- 172.21.15.123:0/492926675 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d7806bb40 0x7f2d781a1fa0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400e9f0 tx=0x7f2d6400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:13:46.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.216+0000 7f2d76000700 1 -- 172.21.15.123:0/492926675 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6400c9c0 con 0x7f2d7806bb40 2024-05-08T06:13:46.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.216+0000 7f2d76000700 1 -- 172.21.15.123:0/492926675 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d6400cb60 con 0x7f2d7806bb40 2024-05-08T06:13:46.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.216+0000 7f2d7fe00700 1 -- 172.21.15.123:0/492926675 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d7819c700 con 0x7f2d7806bb40 2024-05-08T06:13:46.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.216+0000 7f2d76000700 1 -- 172.21.15.123:0/492926675 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d64018680 con 0x7f2d7806bb40 2024-05-08T06:13:46.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.216+0000 7f2d7fe00700 1 -- 172.21.15.123:0/492926675 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d781a89e0 con 0x7f2d7806bb40 2024-05-08T06:13:46.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.220+0000 7f2d76000700 1 -- 172.21.15.123:0/492926675 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7f2d64010430 con 0x7f2d7806bb40 2024-05-08T06:13:46.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.221+0000 7f2d8a7ffb80 1 Processor -- start 2024-05-08T06:13:46.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.221+0000 7f2d8a7ffb80 1 -- start start 2024-05-08T06:13:46.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.221+0000 7f2d8a7ffb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558796ef5060 0x558796ef5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:46.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.222+0000 7f2d8a7ffb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558796ef5980 0x558796ef9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:46.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.222+0000 7f2d8a7ffb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558796ef99d0 0x558796ef9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:46.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.222+0000 7f2d8a7ffb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558796e47a20 con 0x558796ef5060 2024-05-08T06:13:46.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.222+0000 7f2d76000700 1 --2- 172.21.15.123:0/492926675 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f2d600b2900 0x7f2d600b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:46.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.222+0000 7f2d8a7ffb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x558796de5760 con 0x558796ef99d0 2024-05-08T06:13:46.222 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.222+0000 7f2d8a7ffb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558796e439e0 con 0x558796ef5980 2024-05-08T06:13:46.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.222+0000 7f2d76000700 1 -- 172.21.15.123:0/492926675 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f2d640e2340 con 0x7f2d7806bb40 2024-05-08T06:13:46.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.222+0000 7f2d7de00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558796ef5060 0x558796ef5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:46.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.222+0000 7f2d7d400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558796ef5980 0x558796ef9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:46.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.222+0000 7f2d7d400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558796ef5980 0x558796ef9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:56328/0 (socket says 172.21.15.123:56328) 2024-05-08T06:13:46.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.222+0000 7f2d7d400700 1 -- 172.21.15.123:0/3274468072 learned_addr learned my addr 172.21.15.123:0/3274468072 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:13:46.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.222+0000 7f2d7ca00700 1 --2- 172.21.15.123:0/3274468072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558796ef99d0 0x558796ef9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:46.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.222+0000 7f2d7de00700 1 --2- 172.21.15.123:0/492926675 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f2d600b2900 0x7f2d600b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:46.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.223+0000 7f2d7d400700 1 -- 172.21.15.123:0/3274468072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558796ef99d0 msgr2=0x558796ef9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:13:46.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.223+0000 7f2d7d400700 1 --2- 172.21.15.123:0/3274468072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558796ef99d0 0x558796ef9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:46.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.223+0000 7f2d7d400700 1 -- 172.21.15.123:0/3274468072 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558796ef5060 msgr2=0x558796ef5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:13:46.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.223+0000 7f2d7d400700 1 --2- 172.21.15.123:0/3274468072 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558796ef5060 0x558796ef5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:46.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.223+0000 7f2d7d400700 1 -- 172.21.15.123:0/3274468072 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558796d948d0 con 0x558796ef5980 2024-05-08T06:13:46.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.223+0000 7f2d7de00700 1 --2- 172.21.15.123:0/492926675 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f2d600b2900 0x7f2d600b4dc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2d70009790 tx=0x7f2d70009700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:13:46.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.223+0000 7f2d7d400700 1 --2- 172.21.15.123:0/3274468072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558796ef5980 0x558796ef9490 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c0117e0 tx=0x7f2d6c011af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd09bee54183c000 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:13:46.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.223+0000 7f2d6aa00700 1 -- 172.21.15.123:0/3274468072 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c004830 con 0x558796ef5980 2024-05-08T06:13:46.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.223+0000 7f2d6aa00700 1 -- 172.21.15.123:0/3274468072 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d6c005020 con 0x558796ef5980 2024-05-08T06:13:46.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.223+0000 7f2d6aa00700 1 -- 172.21.15.123:0/3274468072 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c00f720 con 0x558796ef5980 2024-05-08T06:13:46.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.224+0000 7f2d8a7ffb80 1 -- 172.21.15.123:0/3274468072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558796ef5980 msgr2=0x558796ef9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:46.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.224+0000 7f2d8a7ffb80 1 --2- 172.21.15.123:0/3274468072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558796ef5980 0x558796ef9490 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c0117e0 tx=0x7f2d6c011af0 comp rx=0 tx=0).stop 2024-05-08T06:13:46.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.224+0000 7f2d8a7ffb80 1 -- 172.21.15.123:0/3274468072 shutdown_connections 2024-05-08T06:13:46.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.224+0000 7f2d8a7ffb80 1 --2- 172.21.15.123:0/3274468072 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558796ef5060 0x558796ef5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:46.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.224+0000 7f2d8a7ffb80 1 --2- 172.21.15.123:0/3274468072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558796ef99d0 0x558796ef9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:46.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.224+0000 7f2d8a7ffb80 1 --2- 172.21.15.123:0/3274468072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558796ef5980 0x558796ef9490 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:46.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.224+0000 7f2d8a7ffb80 1 -- 172.21.15.123:0/3274468072 >> 172.21.15.123:0/3274468072 conn(0x558796e23b60 msgr2=0x558796eef7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:13:46.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.224+0000 7f2d8a7ffb80 1 -- 172.21.15.123:0/3274468072 shutdown_connections 2024-05-08T06:13:46.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.225+0000 7f2d8a7ffb80 1 -- 172.21.15.123:0/3274468072 wait complete. 2024-05-08T06:13:46.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.225+0000 7f2d8a7ffb80 1 -- start start 2024-05-08T06:13:46.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.225+0000 7f2d8a7ffb80 1 Processor -- start 2024-05-08T06:13:46.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.226+0000 7f2d8a7ffb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558796ef5060 0x558796f00df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:46.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.226+0000 7f2d8a7ffb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558796ef5980 0x558796efe190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:46.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.226+0000 7f2d8a7ffb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558796ef99d0 0x558796efe6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:46.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.226+0000 7f2d8a7ffb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558796f01be0 con 0x558796ef5060 2024-05-08T06:13:46.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.226+0000 7f2d8a7ffb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x558796dadb20 con 0x558796ef5980 2024-05-08T06:13:46.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.226+0000 7f2d8a7ffb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558796f01d60 con 0x558796ef99d0 2024-05-08T06:13:46.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.226+0000 7f2d8a7ffb80 10 client.0 _ll_register_callbacks cb 0x558796deec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:13:46.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.226+0000 7f2d7d400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558796ef5060 0x558796f00df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:46.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.226+0000 7f2d7d400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558796ef5060 0x558796f00df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51670/0 (socket says 172.21.15.123:51670) 2024-05-08T06:13:46.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.226+0000 7f2d7d400700 1 -- 172.21.15.123:0/535261836 learned_addr learned my addr 172.21.15.123:0/535261836 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:13:46.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.226+0000 7f2d7ca00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558796ef5980 0x558796efe190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:46.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.226+0000 7f2d7de00700 1 --2- 172.21.15.123:0/535261836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558796ef99d0 0x558796efe6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:46.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.227+0000 7f2d7ca00700 1 -- 172.21.15.123:0/535261836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558796ef99d0 msgr2=0x558796efe6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:13:46.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.227+0000 7f2d7ca00700 1 --2- 172.21.15.123:0/535261836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558796ef99d0 0x558796efe6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:46.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.227+0000 7f2d7ca00700 1 -- 172.21.15.123:0/535261836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558796ef5060 msgr2=0x558796f00df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:13:46.230 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.227+0000 7f2d7ca00700 1 --2- 172.21.15.123:0/535261836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558796ef5060 0x558796f00df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:46.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.227+0000 7f2d7ca00700 1 -- 172.21.15.123:0/535261836 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558796e0d420 con 0x558796ef5980 2024-05-08T06:13:46.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.227+0000 7f2d7ca00700 1 --2- 172.21.15.123:0/535261836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558796ef5980 0x558796efe190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2d640a6b80 tx=0x7f2d640cc390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df4625526f6376dc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:13:46.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.227+0000 7f2d5f400700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-05-08T06:13:46.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.227+0000 7f2d5f400700 1 -- 172.21.15.123:0/535261836 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d640a50a0 con 0x558796ef5980 2024-05-08T06:13:46.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.227+0000 7f2d5f400700 1 -- 172.21.15.123:0/535261836 <== mon.2 v2:172.21.15.163:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f2d640c4f30 con 0x558796ef5980 2024-05-08T06:13:46.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.227+0000 7f2d8a7ffb80 1 -- 172.21.15.123:0/535261836 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d6c005020 con 0x558796ef5980 2024-05-08T06:13:46.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.227+0000 7f2d5f400700 1 -- 172.21.15.123:0/535261836 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2d640c2250 con 0x558796ef5980 2024-05-08T06:13:46.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.227+0000 7f2d8a7ffb80 10 client.25766 fetch_fsmap learned FSMap version 24 2024-05-08T06:13:46.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.227+0000 7f2d8a7ffb80 1 -- 172.21.15.123:0/535261836 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x558796f0c410 con 0x558796ef5980 2024-05-08T06:13:46.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.228+0000 7f2d5f400700 1 -- 172.21.15.123:0/535261836 <== mon.2 v2:172.21.15.163:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d6c005020 con 0x558796ef5980 2024-05-08T06:13:46.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.228+0000 7f2d5f400700 1 -- 172.21.15.123:0/535261836 <== mon.2 v2:172.21.15.163:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f2d640a9b70 con 0x558796ef5980 2024-05-08T06:13:46.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.228+0000 7f2d8a7ffb80 10 client.25766 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:13:46.231 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.228+0000 7f2d8a7ffb80 10 client.25766 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-08T06:13:46.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.228+0000 7f2d8a7ffb80 20 client.25766 populate_metadata read hostname 'smithi123' 2024-05-08T06:13:46.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.228+0000 7f2d8a7ffb80 1 --2- 172.21.15.123:0/535261836 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x558796f0e9a0 0x558796f10d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:46.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.228+0000 7f2d8a7ffb80 4 client.25766 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:13:46.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.228+0000 7f2d8a7ffb80 1 -- 172.21.15.123:0/535261836 --> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558796ec20a0 con 0x558796f0e9a0 2024-05-08T06:13:46.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.228+0000 7f2d7d400700 1 --2- 172.21.15.123:0/535261836 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x558796f0e9a0 0x558796f10d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:46.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.229+0000 7f2d7d400700 1 --2- 172.21.15.123:0/535261836 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x558796f0e9a0 0x558796f10d90 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f5e515dbce6bb3f5 server_cookie=83f20485fc48339f in_seq=0 out_seq=0 2024-05-08T06:13:46.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.229+0000 7f2d5f400700 10 client.25766 ms_handle_connect on v2:172.21.15.123:6840/1571726231 2024-05-08T06:13:46.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.230+0000 7f2d5f400700 1 -- 172.21.15.123:0/535261836 <== mds.1 v2:172.21.15.123:6840/1571726231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558796ec20a0 con 0x558796f0e9a0 2024-05-08T06:13:46.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.230+0000 7f2d5f400700 10 client.25766 handle_command_reply: tid=0 2024-05-08T06:13:46.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.230+0000 7f2d8a7ffb80 1 client.25766 shutdown 2024-05-08T06:13:46.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.230+0000 7f2d8a7ffb80 1 -- 172.21.15.123:0/535261836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558796ef5980 msgr2=0x558796efe190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:46.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.230+0000 7f2d8a7ffb80 1 --2- 172.21.15.123:0/535261836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558796ef5980 0x558796efe190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2d640a6b80 tx=0x7f2d640cc390 comp rx=0 tx=0).stop 2024-05-08T06:13:46.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.230+0000 7f2d8a7ffb80 1 -- 172.21.15.123:0/535261836 shutdown_connections 2024-05-08T06:13:46.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.230+0000 7f2d8a7ffb80 1 --2- 172.21.15.123:0/535261836 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x558796f0e9a0 0x558796f10d90 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:46.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.230+0000 7f2d8a7ffb80 1 --2- 172.21.15.123:0/535261836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558796ef5060 0x558796f00df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:46.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.230+0000 7f2d8a7ffb80 1 --2- 172.21.15.123:0/535261836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558796ef99d0 0x558796efe6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:46.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.230+0000 7f2d8a7ffb80 1 --2- 172.21.15.123:0/535261836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558796ef5980 0x558796efe190 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:46.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.230+0000 7f2d8a7ffb80 1 -- 172.21.15.123:0/535261836 >> 172.21.15.123:0/535261836 conn(0x558796e23b60 msgr2=0x558796eea2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:13:46.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.230+0000 7f2d5f400700 0 client.25766 ms_handle_reset on v2:172.21.15.123:6840/1571726231 2024-05-08T06:13:46.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.230+0000 7f2d8a7ffb80 1 -- 172.21.15.123:0/535261836 shutdown_connections 2024-05-08T06:13:46.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.230+0000 7f2d8a7ffb80 1 -- 172.21.15.123:0/535261836 wait complete. 2024-05-08T06:13:46.233 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.231+0000 7f2d8a7ffb80 20 client.25766 trim_cache size 0 max 16384 2024-05-08T06:13:46.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.247+0000 7f2d8a7ffb80 1 Processor -- start 2024-05-08T06:13:46.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.247+0000 7f2d8a7ffb80 1 -- start start 2024-05-08T06:13:46.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.247+0000 7f2d8a7ffb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558796ef5060 0x558796f0aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:46.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.247+0000 7f2d8a7ffb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558796ef5980 0x558796eea130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:46.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.247+0000 7f2d8a7ffb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558796ef99d0 0x558796f16620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:46.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.247+0000 7f2d8a7ffb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2d781b2630 con 0x558796ef5980 2024-05-08T06:13:46.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.247+0000 7f2d8a7ffb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2d781afec0 con 0x558796ef99d0 2024-05-08T06:13:46.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.247+0000 7f2d8a7ffb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2d781add10 con 0x558796ef5060 2024-05-08T06:13:46.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.248+0000 7f2d7ca00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558796ef5980 0x558796eea130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:46.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.248+0000 7f2d7d400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558796ef5060 0x558796f0aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:46.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.248+0000 7f2d7d400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558796ef5060 0x558796f0aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:56338/0 (socket says 172.21.15.123:56338) 2024-05-08T06:13:46.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.248+0000 7f2d7d400700 1 -- 172.21.15.123:0/631101879 learned_addr learned my addr 172.21.15.123:0/631101879 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:13:46.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.248+0000 7f2d7de00700 1 --2- 172.21.15.123:0/631101879 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558796ef99d0 0x558796f16620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:46.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.248+0000 7f2d7d400700 1 -- 172.21.15.123:0/631101879 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558796ef99d0 msgr2=0x558796f16620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:13:46.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.248+0000 7f2d7d400700 1 --2- 172.21.15.123:0/631101879 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558796ef99d0 0x558796f16620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:46.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.248+0000 7f2d7d400700 1 -- 172.21.15.123:0/631101879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558796ef5980 msgr2=0x558796eea130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:13:46.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.248+0000 7f2d7d400700 1 --2- 172.21.15.123:0/631101879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558796ef5980 0x558796eea130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:46.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.248+0000 7f2d7d400700 1 -- 172.21.15.123:0/631101879 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558796ee7e40 con 0x558796ef5060 2024-05-08T06:13:46.249 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.249+0000 7f2d7d400700 1 --2- 172.21.15.123:0/631101879 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558796ef5060 0x558796f0aca0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c011e40 tx=0x7f2d6c0096f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41b60cfaa74539e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:13:46.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.249+0000 7f2d6a000700 1 -- 172.21.15.123:0/631101879 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c003700 con 0x558796ef5060 2024-05-08T06:13:46.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.249+0000 7f2d6a000700 1 -- 172.21.15.123:0/631101879 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d6c003d50 con 0x558796ef5060 2024-05-08T06:13:46.250 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.249+0000 7f2d6a000700 1 -- 172.21.15.123:0/631101879 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c029740 con 0x558796ef5060 2024-05-08T06:13:46.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.249+0000 7f2d8a7ffb80 1 -- 172.21.15.123:0/631101879 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558796ef5060 msgr2=0x558796f0aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:46.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.249+0000 7f2d8a7ffb80 1 --2- 172.21.15.123:0/631101879 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558796ef5060 0x558796f0aca0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c011e40 tx=0x7f2d6c0096f0 comp rx=0 tx=0).stop 2024-05-08T06:13:46.252 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.250+0000 7f2d8a7ffb80 1 -- 172.21.15.123:0/631101879 shutdown_connections 2024-05-08T06:13:46.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.250+0000 7f2d8a7ffb80 1 --2- 172.21.15.123:0/631101879 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558796ef99d0 0x558796f16620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:46.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.250+0000 7f2d8a7ffb80 1 --2- 172.21.15.123:0/631101879 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558796ef5060 0x558796f0aca0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:46.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.250+0000 7f2d8a7ffb80 1 --2- 172.21.15.123:0/631101879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558796ef5980 0x558796eea130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:46.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.250+0000 7f2d8a7ffb80 1 -- 172.21.15.123:0/631101879 >> 172.21.15.123:0/631101879 conn(0x558796e23b60 msgr2=0x558796ee9a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:13:46.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.250+0000 7f2d8a7ffb80 1 -- 172.21.15.123:0/631101879 shutdown_connections 2024-05-08T06:13:46.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.250+0000 7f2d8a7ffb80 1 -- 172.21.15.123:0/631101879 wait complete. 2024-05-08T06:13:46.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.250+0000 7f2d8a7ffb80 1 -- start start 2024-05-08T06:13:46.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.250+0000 7f2d8a7ffb80 1 Processor -- start 2024-05-08T06:13:46.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.251+0000 7f2d8a7ffb80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558796ef5060 0x558796f02e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:46.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.251+0000 7f2d8a7ffb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558796ef5980 0x558796f033b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:46.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.251+0000 7f2d8a7ffb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558796ef99d0 0x558796f038f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:46.253 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.251+0000 7f2d8a7ffb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2d781ae380 con 0x558796ef99d0 2024-05-08T06:13:46.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.251+0000 7f2d8a7ffb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2d781af7f0 con 0x558796ef5980 2024-05-08T06:13:46.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.251+0000 7f2d8a7ffb80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2d781aea50 con 0x558796ef5060 2024-05-08T06:13:46.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.251+0000 7f2d8a7ffb80 10 client.0 _ll_register_callbacks cb 0x558796deec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:13:46.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.251+0000 7f2d7de00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558796ef99d0 0x558796f038f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:46.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.251+0000 7f2d7de00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558796ef99d0 0x558796f038f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51700/0 (socket says 172.21.15.123:51700) 2024-05-08T06:13:46.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.251+0000 7f2d7de00700 1 -- 172.21.15.123:0/1855598862 learned_addr learned my addr 172.21.15.123:0/1855598862 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:13:46.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.251+0000 7f2d7ca00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558796ef5980 0x558796f033b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:46.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.251+0000 7f2d7d400700 1 --2- 172.21.15.123:0/1855598862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558796ef5060 0x558796f02e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:46.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.252+0000 7f2d7ca00700 1 -- 172.21.15.123:0/1855598862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558796ef5060 msgr2=0x558796f02e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:13:46.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.252+0000 7f2d7ca00700 1 --2- 172.21.15.123:0/1855598862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558796ef5060 0x558796f02e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:46.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.252+0000 7f2d7ca00700 1 -- 172.21.15.123:0/1855598862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558796ef99d0 msgr2=0x558796f038f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:13:46.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.252+0000 7f2d7ca00700 1 --2- 172.21.15.123:0/1855598862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558796ef99d0 0x558796f038f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:46.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.252+0000 7f2d7ca00700 1 -- 172.21.15.123:0/1855598862 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558796f78430 con 0x558796ef5980 2024-05-08T06:13:46.254 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.252+0000 7f2d7de00700 1 --2- 172.21.15.123:0/1855598862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558796ef99d0 0x558796f038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:13:46.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.252+0000 7f2d7ca00700 1 --2- 172.21.15.123:0/1855598862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558796ef5980 0x558796f033b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2d640c4d70 tx=0x7f2d640b2510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bef0a91458192e79 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:13:46.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.252+0000 7f2d5f400700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-05-08T06:13:46.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.252+0000 7f2d5f400700 1 -- 172.21.15.123:0/1855598862 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d640a57d0 con 0x558796ef5980 2024-05-08T06:13:46.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.252+0000 7f2d8a7ffb80 1 -- 172.21.15.123:0/1855598862 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d6c003d50 con 0x558796ef5980 2024-05-08T06:13:46.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.252+0000 7f2d5f400700 1 -- 172.21.15.123:0/1855598862 <== mon.2 v2:172.21.15.163:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f2d640a5f00 con 0x558796ef5980 2024-05-08T06:13:46.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.252+0000 7f2d5f400700 1 -- 172.21.15.123:0/1855598862 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2d64088020 con 0x558796ef5980 2024-05-08T06:13:46.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.253+0000 7f2d8a7ffb80 10 client.25772 fetch_fsmap learned FSMap version 24 2024-05-08T06:13:46.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.253+0000 7f2d8a7ffb80 1 -- 172.21.15.123:0/1855598862 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x558796f79730 con 0x558796ef5980 2024-05-08T06:13:46.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.253+0000 7f2d5f400700 1 -- 172.21.15.123:0/1855598862 <== mon.2 v2:172.21.15.163:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d6c003d50 con 0x558796ef5980 2024-05-08T06:13:46.256 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.253+0000 7f2d5f400700 1 -- 172.21.15.123:0/1855598862 <== mon.2 v2:172.21.15.163:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f2d640a8b70 con 0x558796ef5980 2024-05-08T06:13:46.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.253+0000 7f2d8a7ffb80 10 client.25772 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:13:46.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.253+0000 7f2d8a7ffb80 10 client.25772 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-08T06:13:46.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.253+0000 7f2d8a7ffb80 20 client.25772 populate_metadata read hostname 'smithi123' 2024-05-08T06:13:46.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.253+0000 7f2d8a7ffb80 1 --2- 172.21.15.123:0/1855598862 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x558796f79e40 0x558796f7c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:46.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.253+0000 7f2d8a7ffb80 4 client.25772 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T06:13:46.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.253+0000 7f2d8a7ffb80 1 -- 172.21.15.123:0/1855598862 --> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558796e55400 con 0x558796f79e40 2024-05-08T06:13:46.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.253+0000 7f2d7d400700 1 --2- 172.21.15.123:0/1855598862 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x558796f79e40 0x558796f7c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:46.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.254+0000 7f2d7d400700 1 --2- 172.21.15.123:0/1855598862 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x558796f79e40 0x558796f7c260 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d79781edd94bc66e server_cookie=252f9423988fa634 in_seq=0 out_seq=0 2024-05-08T06:13:46.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.254+0000 7f2d5f400700 10 client.25772 ms_handle_connect on v2:172.21.15.123:6840/1571726231 2024-05-08T06:13:46.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.254+0000 7f2d5f400700 1 -- 172.21.15.123:0/1855598862 <== mds.1 v2:172.21.15.123:6840/1571726231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558796e55400 con 0x558796f79e40 2024-05-08T06:13:46.257 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.254+0000 7f2d5f400700 10 client.25772 handle_command_reply: tid=0 2024-05-08T06:13:46.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.254+0000 7f2d8a7ffb80 1 client.25772 shutdown 2024-05-08T06:13:46.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.254+0000 7f2d8a7ffb80 1 -- 172.21.15.123:0/1855598862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558796ef5980 msgr2=0x558796f033b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:46.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.254+0000 7f2d8a7ffb80 1 --2- 172.21.15.123:0/1855598862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558796ef5980 0x558796f033b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2d640c4d70 tx=0x7f2d640b2510 comp rx=0 tx=0).stop 2024-05-08T06:13:46.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.254+0000 7f2d8a7ffb80 1 -- 172.21.15.123:0/1855598862 shutdown_connections 2024-05-08T06:13:46.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.254+0000 7f2d8a7ffb80 1 --2- 172.21.15.123:0/1855598862 >> [v2:172.21.15.123:6840/1571726231,v1:172.21.15.123:6841/1571726231] conn(0x558796f79e40 0x558796f7c260 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:46.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.255+0000 7f2d8a7ffb80 1 --2- 172.21.15.123:0/1855598862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558796ef99d0 0x558796f038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:46.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.255+0000 7f2d8a7ffb80 1 --2- 172.21.15.123:0/1855598862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558796ef5060 0x558796f02e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:46.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.255+0000 7f2d8a7ffb80 1 --2- 172.21.15.123:0/1855598862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558796ef5980 0x558796f033b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:46.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.255+0000 7f2d5f400700 0 client.25772 ms_handle_reset on v2:172.21.15.123:6840/1571726231 2024-05-08T06:13:46.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.255+0000 7f2d8a7ffb80 1 -- 172.21.15.123:0/1855598862 >> 172.21.15.123:0/1855598862 conn(0x558796e23b60 msgr2=0x558796eea2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:13:46.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.255+0000 7f2d8a7ffb80 1 -- 172.21.15.123:0/1855598862 shutdown_connections 2024-05-08T06:13:46.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.255+0000 7f2d8a7ffb80 1 -- 172.21.15.123:0/1855598862 wait complete. 2024-05-08T06:13:46.258 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.255+0000 7f2d8a7ffb80 20 client.25772 trim_cache size 0 max 16384 2024-05-08T06:13:46.259 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-05-08T06:13:46.259 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.256+0000 7f2d7fe00700 1 -- 172.21.15.123:0/492926675 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f2d600b2900 msgr2=0x7f2d600b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:46.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.256+0000 7f2d7fe00700 1 --2- 172.21.15.123:0/492926675 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f2d600b2900 0x7f2d600b4dc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2d70009790 tx=0x7f2d70009700 comp rx=0 tx=0).stop 2024-05-08T06:13:46.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.256+0000 7f2d7fe00700 1 -- 172.21.15.123:0/492926675 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d7806bb40 msgr2=0x7f2d781a1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:46.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.256+0000 7f2d7fe00700 1 --2- 172.21.15.123:0/492926675 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d7806bb40 0x7f2d781a1fa0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400e9f0 tx=0x7f2d6400ed00 comp rx=0 tx=0).stop 2024-05-08T06:13:46.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.256+0000 7f2d7fe00700 1 -- 172.21.15.123:0/492926675 shutdown_connections 2024-05-08T06:13:46.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.256+0000 7f2d7fe00700 1 --2- 172.21.15.123:0/492926675 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f2d600b2900 0x7f2d600b4dc0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:46.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.256+0000 7f2d7fe00700 1 --2- 172.21.15.123:0/492926675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d7806b180 0x7f2d781a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:46.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.256+0000 7f2d7fe00700 1 --2- 172.21.15.123:0/492926675 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d78109510 0x7f2d7819bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:46.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.256+0000 7f2d7fe00700 1 --2- 172.21.15.123:0/492926675 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d7806bb40 0x7f2d781a1fa0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:46.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.256+0000 7f2d7fe00700 1 -- 172.21.15.123:0/492926675 >> 172.21.15.123:0/492926675 conn(0x7f2d78079350 msgr2=0x7f2d7810e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:13:46.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.256+0000 7f2d7fe00700 1 -- 172.21.15.123:0/492926675 shutdown_connections 2024-05-08T06:13:46.260 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:46.256+0000 7f2d7fe00700 1 -- 172.21.15.123:0/492926675 wait complete. 2024-05-08T06:13:46.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:46 smithi123 ceph-mon[29787]: pgmap v559: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T06:13:47.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:46 smithi163 ceph-mon[34520]: pgmap v559: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T06:13:47.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:46 smithi192 ceph-mon[34664]: pgmap v559: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T06:13:47.314 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-08T06:13:47.793 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:13:49.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:48 smithi123 ceph-mon[29787]: pgmap v560: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:13:49.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:48 smithi163 ceph-mon[34520]: pgmap v560: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:13:49.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:48 smithi192 ceph-mon[34664]: pgmap v560: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:13:50.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.399+0000 7efe82800700 1 -- 172.21.15.123:0/2552421664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe7c10b6c0 msgr2=0x7efe7c10baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:50.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.399+0000 7efe82800700 1 --2- 172.21.15.123:0/2552421664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe7c10b6c0 0x7efe7c10baa0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7efe6c009990 tx=0x7efe6c009ca0 comp rx=0 tx=0).stop 2024-05-08T06:13:50.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.400+0000 7efe82800700 1 -- 172.21.15.123:0/2552421664 shutdown_connections 2024-05-08T06:13:50.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.400+0000 7efe82800700 1 --2- 172.21.15.123:0/2552421664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe7c10b6c0 0x7efe7c10baa0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:50.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.400+0000 7efe82800700 1 --2- 172.21.15.123:0/2552421664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe7c106080 0x7efe7c112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:50.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.400+0000 7efe82800700 1 --2- 172.21.15.123:0/2552421664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe7c1056c0 0x7efe7c105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:50.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.400+0000 7efe82800700 1 -- 172.21.15.123:0/2552421664 >> 172.21.15.123:0/2552421664 conn(0x7efe7c1011b0 msgr2=0x7efe7c1035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:13:50.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.400+0000 7efe82800700 1 -- 172.21.15.123:0/2552421664 shutdown_connections 2024-05-08T06:13:50.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.400+0000 7efe82800700 1 -- 172.21.15.123:0/2552421664 wait complete. 2024-05-08T06:13:50.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.401+0000 7efe82800700 1 Processor -- start 2024-05-08T06:13:50.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.401+0000 7efe82800700 1 -- start start 2024-05-08T06:13:50.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.402+0000 7efe82800700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe7c1056c0 0x7efe7c19bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:50.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.402+0000 7efe82800700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe7c106080 0x7efe7c19c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:50.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.402+0000 7efe82800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe7c10b6c0 0x7efe7c1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:50.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.402+0000 7efe82800700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efe7c1158c0 con 0x7efe7c10b6c0 2024-05-08T06:13:50.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.402+0000 7efe82800700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7efe7c115740 con 0x7efe7c1056c0 2024-05-08T06:13:50.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.402+0000 7efe82800700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7efe7c115a40 con 0x7efe7c106080 2024-05-08T06:13:50.403 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.402+0000 7efe7b400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe7c1056c0 0x7efe7c19bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:50.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.402+0000 7efe7be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe7c10b6c0 0x7efe7c1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:50.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.402+0000 7efe7b400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe7c1056c0 0x7efe7c19bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:46680/0 (socket says 172.21.15.123:46680) 2024-05-08T06:13:50.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.402+0000 7efe7aa00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe7c106080 0x7efe7c19c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:50.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.402+0000 7efe7b400700 1 -- 172.21.15.123:0/3133793287 learned_addr learned my addr 172.21.15.123:0/3133793287 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:13:50.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.403+0000 7efe7b400700 1 -- 172.21.15.123:0/3133793287 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe7c106080 msgr2=0x7efe7c19c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:50.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.403+0000 7efe7b400700 1 --2- 172.21.15.123:0/3133793287 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe7c106080 0x7efe7c19c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:50.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.403+0000 7efe7b400700 1 -- 172.21.15.123:0/3133793287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe7c10b6c0 msgr2=0x7efe7c1a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:50.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.403+0000 7efe7b400700 1 --2- 172.21.15.123:0/3133793287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe7c10b6c0 0x7efe7c1a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:50.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.403+0000 7efe7b400700 1 -- 172.21.15.123:0/3133793287 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe7c1a0d60 con 0x7efe7c1056c0 2024-05-08T06:13:50.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.403+0000 7efe7b400700 1 --2- 172.21.15.123:0/3133793287 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe7c1056c0 0x7efe7c19bd50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efe6c000c00 tx=0x7efe6c00bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:13:50.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.403+0000 7efe73e00700 1 -- 172.21.15.123:0/3133793287 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c027070 con 0x7efe7c1056c0 2024-05-08T06:13:50.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.403+0000 7efe73e00700 1 -- 172.21.15.123:0/3133793287 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe6c022470 con 0x7efe7c1056c0 2024-05-08T06:13:50.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.403+0000 7efe82800700 1 -- 172.21.15.123:0/3133793287 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe7c1a0ff0 con 0x7efe7c1056c0 2024-05-08T06:13:50.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.403+0000 7efe73e00700 1 -- 172.21.15.123:0/3133793287 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c017650 con 0x7efe7c1056c0 2024-05-08T06:13:50.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.404+0000 7efe82800700 1 -- 172.21.15.123:0/3133793287 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe7c1a88d0 con 0x7efe7c1056c0 2024-05-08T06:13:50.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.407+0000 7efe73e00700 1 -- 172.21.15.123:0/3133793287 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7efe6c02a070 con 0x7efe7c1056c0 2024-05-08T06:13:50.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.408+0000 7efe8d145b80 1 Processor -- start 2024-05-08T06:13:50.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.408+0000 7efe8d145b80 1 -- start start 2024-05-08T06:13:50.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.408+0000 7efe8d145b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55907602f060 0x55907602f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:50.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.408+0000 7efe8d145b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55907602f980 0x559076033490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:50.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.409+0000 7efe8d145b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5590760339d0 0x559076033e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:50.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.409+0000 7efe8d145b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559075f81a20 con 0x55907602f980 2024-05-08T06:13:50.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.409+0000 7efe8d145b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x559075f1f760 con 0x5590760339d0 2024-05-08T06:13:50.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.409+0000 7efe8d145b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x559075f7d9e0 con 0x55907602f060 2024-05-08T06:13:50.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.409+0000 7efe7be00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55907602f060 0x55907602f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:50.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.409+0000 7efe7b400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55907602f980 0x559076033490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:50.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.409+0000 7efe7be00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55907602f060 0x55907602f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.123:56390/0 (socket says 172.21.15.123:56390) 2024-05-08T06:13:50.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.409+0000 7efe7be00700 1 -- 172.21.15.123:0/4124325943 learned_addr learned my addr 172.21.15.123:0/4124325943 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:13:50.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.409+0000 7efe7aa00700 1 --2- 172.21.15.123:0/4124325943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5590760339d0 0x559076033e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:50.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.409+0000 7efe73e00700 1 --2- 172.21.15.123:0/3133793287 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7efe600b2880 0x7efe600b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:50.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.409+0000 7efe7be00700 1 -- 172.21.15.123:0/4124325943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5590760339d0 msgr2=0x559076033e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:13:50.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.409+0000 7efe7be00700 1 --2- 172.21.15.123:0/4124325943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5590760339d0 0x559076033e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:50.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.409+0000 7efe7be00700 1 -- 172.21.15.123:0/4124325943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55907602f980 msgr2=0x559076033490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:13:50.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.409+0000 7efe7be00700 1 --2- 172.21.15.123:0/4124325943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55907602f980 0x559076033490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:50.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.409+0000 7efe7be00700 1 -- 172.21.15.123:0/4124325943 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559075ece8d0 con 0x55907602f060 2024-05-08T06:13:50.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.410+0000 7efe73e00700 1 -- 172.21.15.123:0/3133793287 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7efe6c0e2ed0 con 0x7efe7c1056c0 2024-05-08T06:13:50.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.410+0000 7efe7be00700 1 --2- 172.21.15.123:0/3133793287 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7efe600b2880 0x7efe600b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:50.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.410+0000 7efe7be00700 1 --2- 172.21.15.123:0/4124325943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55907602f060 0x55907602f440 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efe74011680 tx=0x7efe74009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a39d86ef6798031a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:13:50.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.411+0000 7efe70c00700 1 -- 172.21.15.123:0/4124325943 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7401b040 con 0x55907602f060 2024-05-08T06:13:50.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.411+0000 7efe70c00700 1 -- 172.21.15.123:0/4124325943 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe7401c030 con 0x55907602f060 2024-05-08T06:13:50.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.411+0000 7efe70c00700 1 -- 172.21.15.123:0/4124325943 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe740156b0 con 0x55907602f060 2024-05-08T06:13:50.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.411+0000 7efe8d145b80 1 -- 172.21.15.123:0/4124325943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55907602f060 msgr2=0x55907602f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:50.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.411+0000 7efe7be00700 1 --2- 172.21.15.123:0/3133793287 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7efe600b2880 0x7efe600b4d40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7efe740094f0 tx=0x7efe74023000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:13:50.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.411+0000 7efe8d145b80 1 --2- 172.21.15.123:0/4124325943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55907602f060 0x55907602f440 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efe74011680 tx=0x7efe74009920 comp rx=0 tx=0).stop 2024-05-08T06:13:50.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.412+0000 7efe8d145b80 1 -- 172.21.15.123:0/4124325943 shutdown_connections 2024-05-08T06:13:50.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.412+0000 7efe8d145b80 1 --2- 172.21.15.123:0/4124325943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5590760339d0 0x559076033e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:50.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.412+0000 7efe8d145b80 1 --2- 172.21.15.123:0/4124325943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55907602f060 0x55907602f440 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:50.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.412+0000 7efe8d145b80 1 --2- 172.21.15.123:0/4124325943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55907602f980 0x559076033490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:50.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.412+0000 7efe8d145b80 1 -- 172.21.15.123:0/4124325943 >> 172.21.15.123:0/4124325943 conn(0x559075f5db60 msgr2=0x5590760297a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:13:50.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.412+0000 7efe8d145b80 1 -- 172.21.15.123:0/4124325943 shutdown_connections 2024-05-08T06:13:50.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.412+0000 7efe8d145b80 1 -- 172.21.15.123:0/4124325943 wait complete. 2024-05-08T06:13:50.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.412+0000 7efe8d145b80 1 -- start start 2024-05-08T06:13:50.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.413+0000 7efe8d145b80 1 Processor -- start 2024-05-08T06:13:50.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.413+0000 7efe8d145b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55907602f060 0x55907603ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:50.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.413+0000 7efe8d145b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55907602f980 0x5590760381f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:50.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.413+0000 7efe8d145b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590760339d0 0x559076038730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:50.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.413+0000 7efe8d145b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55907603bc30 con 0x5590760339d0 2024-05-08T06:13:50.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.413+0000 7efe8d145b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x559075ee7b20 con 0x55907602f060 2024-05-08T06:13:50.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.413+0000 7efe8d145b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55907603bdb0 con 0x55907602f980 2024-05-08T06:13:50.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.413+0000 7efe8d145b80 10 client.0 _ll_register_callbacks cb 0x559075f28c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:13:50.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.413+0000 7efe7be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590760339d0 0x559076038730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:50.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.413+0000 7efe7b400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55907602f060 0x55907603ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:50.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.413+0000 7efe7aa00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55907602f980 0x5590760381f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:50.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.413+0000 7efe7b400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55907602f060 0x55907603ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:46692/0 (socket says 172.21.15.123:46692) 2024-05-08T06:13:50.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.413+0000 7efe7b400700 1 -- 172.21.15.123:0/958299241 learned_addr learned my addr 172.21.15.123:0/958299241 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:13:50.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.413+0000 7efe7be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590760339d0 0x559076038730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51738/0 (socket says 172.21.15.123:51738) 2024-05-08T06:13:50.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.414+0000 7efe7b400700 1 -- 172.21.15.123:0/958299241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55907602f980 msgr2=0x5590760381f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:13:50.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.414+0000 7efe7b400700 1 --2- 172.21.15.123:0/958299241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55907602f980 0x5590760381f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:50.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.414+0000 7efe7b400700 1 -- 172.21.15.123:0/958299241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590760339d0 msgr2=0x559076038730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:13:50.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.414+0000 7efe7b400700 1 --2- 172.21.15.123:0/958299241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590760339d0 0x559076038730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:50.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.414+0000 7efe7b400700 1 -- 172.21.15.123:0/958299241 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559075f47420 con 0x55907602f060 2024-05-08T06:13:50.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.414+0000 7efe7aa00700 1 --2- 172.21.15.123:0/958299241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55907602f980 0x5590760381f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:13:50.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.414+0000 7efe7b400700 1 --2- 172.21.15.123:0/958299241 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55907602f060 0x55907603ade0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efe6c0e0980 tx=0x7efe6c022c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=775f69f454ce80de server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:13:50.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.414+0000 7efe65600700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-05-08T06:13:50.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.414+0000 7efe65600700 1 -- 172.21.15.123:0/958299241 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c0a63f0 con 0x55907602f060 2024-05-08T06:13:50.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.414+0000 7efe65600700 1 -- 172.21.15.123:0/958299241 <== mon.2 v2:172.21.15.163:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7efe6c0b1bd0 con 0x55907602f060 2024-05-08T06:13:50.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.414+0000 7efe8d145b80 1 -- 172.21.15.123:0/958299241 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe7401c030 con 0x55907602f060 2024-05-08T06:13:50.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.415+0000 7efe65600700 1 -- 172.21.15.123:0/958299241 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7efe6c0a9ca0 con 0x55907602f060 2024-05-08T06:13:50.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.415+0000 7efe8d145b80 10 client.25784 fetch_fsmap learned FSMap version 24 2024-05-08T06:13:50.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.415+0000 7efe8d145b80 1 -- 172.21.15.123:0/958299241 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5590760464a0 con 0x55907602f060 2024-05-08T06:13:50.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.415+0000 7efe65600700 1 -- 172.21.15.123:0/958299241 <== mon.2 v2:172.21.15.163:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe7401c030 con 0x55907602f060 2024-05-08T06:13:50.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.415+0000 7efe65600700 1 -- 172.21.15.123:0/958299241 <== mon.2 v2:172.21.15.163:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7efe6c0b2e70 con 0x55907602f060 2024-05-08T06:13:50.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.415+0000 7efe8d145b80 10 client.25784 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:13:50.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.415+0000 7efe8d145b80 10 client.25784 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T06:13:50.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.415+0000 7efe8d145b80 20 client.25784 populate_metadata read hostname 'smithi123' 2024-05-08T06:13:50.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.415+0000 7efe8d145b80 1 --2- 172.21.15.123:0/958299241 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5590760489c0 0x55907604adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:50.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.415+0000 7efe8d145b80 4 client.25784 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T06:13:50.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.415+0000 7efe8d145b80 1 -- 172.21.15.123:0/958299241 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559075ffc0a0 con 0x5590760489c0 2024-05-08T06:13:50.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.416+0000 7efe7aa00700 1 --2- 172.21.15.123:0/958299241 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5590760489c0 0x55907604adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:50.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.416+0000 7efe7aa00700 1 --2- 172.21.15.123:0/958299241 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5590760489c0 0x55907604adf0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dea9f665565401f server_cookie=d7493bfeb4e24208 in_seq=0 out_seq=0 2024-05-08T06:13:50.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.416+0000 7efe65600700 10 client.25784 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:13:50.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.417+0000 7efe65600700 1 -- 172.21.15.123:0/958299241 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559075ffc0a0 con 0x5590760489c0 2024-05-08T06:13:50.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.417+0000 7efe65600700 10 client.25784 handle_command_reply: tid=0 2024-05-08T06:13:50.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.417+0000 7efe8d145b80 1 client.25784 shutdown 2024-05-08T06:13:50.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.417+0000 7efe8d145b80 1 -- 172.21.15.123:0/958299241 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55907602f060 msgr2=0x55907603ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:50.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.417+0000 7efe8d145b80 1 --2- 172.21.15.123:0/958299241 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55907602f060 0x55907603ade0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efe6c0e0980 tx=0x7efe6c022c20 comp rx=0 tx=0).stop 2024-05-08T06:13:50.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.418+0000 7efe8d145b80 1 -- 172.21.15.123:0/958299241 shutdown_connections 2024-05-08T06:13:50.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.418+0000 7efe8d145b80 1 --2- 172.21.15.123:0/958299241 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5590760489c0 0x55907604adf0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:50.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.418+0000 7efe8d145b80 1 --2- 172.21.15.123:0/958299241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590760339d0 0x559076038730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:50.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.418+0000 7efe8d145b80 1 --2- 172.21.15.123:0/958299241 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55907602f060 0x55907603ade0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:50.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.418+0000 7efe8d145b80 1 --2- 172.21.15.123:0/958299241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55907602f980 0x5590760381f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:50.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.418+0000 7efe65600700 0 client.25784 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:13:50.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.418+0000 7efe8d145b80 1 -- 172.21.15.123:0/958299241 >> 172.21.15.123:0/958299241 conn(0x559075f5db60 msgr2=0x5590760242b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:13:50.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.418+0000 7efe8d145b80 1 -- 172.21.15.123:0/958299241 shutdown_connections 2024-05-08T06:13:50.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.418+0000 7efe8d145b80 1 -- 172.21.15.123:0/958299241 wait complete. 2024-05-08T06:13:50.420 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.418+0000 7efe8d145b80 20 client.25784 trim_cache size 0 max 16384 2024-05-08T06:13:50.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.434+0000 7efe8d145b80 1 Processor -- start 2024-05-08T06:13:50.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.434+0000 7efe8d145b80 1 -- start start 2024-05-08T06:13:50.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.435+0000 7efe8d145b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55907602f060 0x559076037b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:50.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.435+0000 7efe8d145b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55907602f980 0x559076034dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:50.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.435+0000 7efe8d145b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590760339d0 0x559076024130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:50.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.435+0000 7efe8d145b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efe7c1b2560 con 0x5590760339d0 2024-05-08T06:13:50.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.435+0000 7efe8d145b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7efe7c1afdf0 con 0x55907602f060 2024-05-08T06:13:50.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.435+0000 7efe8d145b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7efe7c1adc40 con 0x55907602f980 2024-05-08T06:13:50.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.435+0000 7efe7b400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55907602f060 0x559076037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:50.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.435+0000 7efe7b400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55907602f060 0x559076037b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:46702/0 (socket says 172.21.15.123:46702) 2024-05-08T06:13:50.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.435+0000 7efe7be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590760339d0 0x559076024130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:50.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.435+0000 7efe7b400700 1 -- 172.21.15.123:0/3051533387 learned_addr learned my addr 172.21.15.123:0/3051533387 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:13:50.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.435+0000 7efe7aa00700 1 --2- 172.21.15.123:0/3051533387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55907602f980 0x559076034dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:50.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.436+0000 7efe7b400700 1 -- 172.21.15.123:0/3051533387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55907602f980 msgr2=0x559076034dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:13:50.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.436+0000 7efe7b400700 1 --2- 172.21.15.123:0/3051533387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55907602f980 0x559076034dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:50.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.436+0000 7efe7b400700 1 -- 172.21.15.123:0/3051533387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590760339d0 msgr2=0x559076024130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:13:50.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.436+0000 7efe7b400700 1 --2- 172.21.15.123:0/3051533387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590760339d0 0x559076024130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:50.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.436+0000 7efe7b400700 1 -- 172.21.15.123:0/3051533387 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590760263f0 con 0x55907602f060 2024-05-08T06:13:50.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.436+0000 7efe7aa00700 1 --2- 172.21.15.123:0/3051533387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55907602f980 0x559076034dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:13:50.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.436+0000 7efe7b400700 1 --2- 172.21.15.123:0/3051533387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55907602f060 0x559076037b50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efe6c0098e0 tx=0x7efe6c009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20618748c46bd8f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:13:50.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.436+0000 7efe67e00700 1 -- 172.21.15.123:0/3051533387 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c0a9e80 con 0x55907602f060 2024-05-08T06:13:50.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.436+0000 7efe67e00700 1 -- 172.21.15.123:0/3051533387 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe6c0aa040 con 0x55907602f060 2024-05-08T06:13:50.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.436+0000 7efe67e00700 1 -- 172.21.15.123:0/3051533387 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c0aa300 con 0x55907602f060 2024-05-08T06:13:50.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.437+0000 7efe8d145b80 1 -- 172.21.15.123:0/3051533387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55907602f060 msgr2=0x559076037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:50.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.437+0000 7efe8d145b80 1 --2- 172.21.15.123:0/3051533387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55907602f060 0x559076037b50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efe6c0098e0 tx=0x7efe6c009ca0 comp rx=0 tx=0).stop 2024-05-08T06:13:50.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.437+0000 7efe8d145b80 1 -- 172.21.15.123:0/3051533387 shutdown_connections 2024-05-08T06:13:50.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.437+0000 7efe8d145b80 1 --2- 172.21.15.123:0/3051533387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590760339d0 0x559076024130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:50.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.437+0000 7efe8d145b80 1 --2- 172.21.15.123:0/3051533387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55907602f060 0x559076037b50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:50.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.437+0000 7efe8d145b80 1 --2- 172.21.15.123:0/3051533387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55907602f980 0x559076034dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:50.439 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.437+0000 7efe8d145b80 1 -- 172.21.15.123:0/3051533387 >> 172.21.15.123:0/3051533387 conn(0x559075f5db60 msgr2=0x559076023a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:13:50.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.437+0000 7efe8d145b80 1 -- 172.21.15.123:0/3051533387 shutdown_connections 2024-05-08T06:13:50.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.437+0000 7efe8d145b80 1 -- 172.21.15.123:0/3051533387 wait complete. 2024-05-08T06:13:50.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.437+0000 7efe8d145b80 1 -- start start 2024-05-08T06:13:50.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.438+0000 7efe8d145b80 1 Processor -- start 2024-05-08T06:13:50.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.438+0000 7efe8d145b80 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55907602f060 0x559076041220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:50.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.438+0000 7efe8d145b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55907602f980 0x559076041760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:50.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.438+0000 7efe8d145b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590760339d0 0x55907604f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:50.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.438+0000 7efe8d145b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efe7c1ae2b0 con 0x5590760339d0 2024-05-08T06:13:50.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.438+0000 7efe8d145b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7efe7c1af720 con 0x55907602f980 2024-05-08T06:13:50.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.438+0000 7efe8d145b80 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7efe7c1ae980 con 0x55907602f060 2024-05-08T06:13:50.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.438+0000 7efe8d145b80 10 client.0 _ll_register_callbacks cb 0x559075f28c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T06:13:50.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.439+0000 7efe7aa00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55907602f980 0x559076041760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:50.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.439+0000 7efe7be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590760339d0 0x55907604f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:50.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.439+0000 7efe7b400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55907602f060 0x559076041220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:50.440 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.439+0000 7efe7be00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590760339d0 0x55907604f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51746/0 (socket says 172.21.15.123:51746) 2024-05-08T06:13:50.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.439+0000 7efe7be00700 1 -- 172.21.15.123:0/1683195040 learned_addr learned my addr 172.21.15.123:0/1683195040 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:13:50.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.439+0000 7efe7aa00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55907602f980 0x559076041760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:46706/0 (socket says 172.21.15.123:46706) 2024-05-08T06:13:50.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.439+0000 7efe7aa00700 1 -- 172.21.15.123:0/1683195040 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55907602f060 msgr2=0x559076041220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:13:50.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.439+0000 7efe7aa00700 1 --2- 172.21.15.123:0/1683195040 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55907602f060 0x559076041220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:50.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.439+0000 7efe7aa00700 1 -- 172.21.15.123:0/1683195040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590760339d0 msgr2=0x55907604f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T06:13:50.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.439+0000 7efe7aa00700 1 --2- 172.21.15.123:0/1683195040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590760339d0 0x55907604f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:50.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.439+0000 7efe7aa00700 1 -- 172.21.15.123:0/1683195040 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559076050000 con 0x55907602f980 2024-05-08T06:13:50.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.439+0000 7efe7be00700 1 --2- 172.21.15.123:0/1683195040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590760339d0 0x55907604f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:13:50.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.439+0000 7efe7aa00700 1 --2- 172.21.15.123:0/1683195040 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55907602f980 0x559076041760 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55907604b630 tx=0x7efe68004480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb9329be86ea60c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:13:50.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.439+0000 7efe65600700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-05-08T06:13:50.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.440+0000 7efe65600700 1 -- 172.21.15.123:0/1683195040 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe68022070 con 0x55907602f980 2024-05-08T06:13:50.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.440+0000 7efe8d145b80 1 -- 172.21.15.123:0/1683195040 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe6c0aa040 con 0x55907602f980 2024-05-08T06:13:50.443 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.440+0000 7efe65600700 1 -- 172.21.15.123:0/1683195040 <== mon.2 v2:172.21.15.163:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7efe68004830 con 0x55907602f980 2024-05-08T06:13:50.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.440+0000 7efe65600700 1 -- 172.21.15.123:0/1683195040 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7efe68004a30 con 0x55907602f980 2024-05-08T06:13:50.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.440+0000 7efe8d145b80 10 client.25790 fetch_fsmap learned FSMap version 24 2024-05-08T06:13:50.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.440+0000 7efe8d145b80 1 -- 172.21.15.123:0/1683195040 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5590760b2e10 con 0x55907602f980 2024-05-08T06:13:50.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.440+0000 7efe65600700 1 -- 172.21.15.123:0/1683195040 <== mon.2 v2:172.21.15.163:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe6c0aa040 con 0x55907602f980 2024-05-08T06:13:50.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.440+0000 7efe65600700 1 -- 172.21.15.123:0/1683195040 <== mon.2 v2:172.21.15.163:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7efe68025020 con 0x55907602f980 2024-05-08T06:13:50.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.441+0000 7efe8d145b80 10 client.25790 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T06:13:50.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.441+0000 7efe8d145b80 10 client.25790 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T06:13:50.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.441+0000 7efe8d145b80 20 client.25790 populate_metadata read hostname 'smithi123' 2024-05-08T06:13:50.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.441+0000 7efe8d145b80 1 --2- 172.21.15.123:0/1683195040 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5590760b3520 0x5590760b5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:50.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.441+0000 7efe8d145b80 4 client.25790 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T06:13:50.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.441+0000 7efe8d145b80 1 -- 172.21.15.123:0/1683195040 --> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559075f8f400 con 0x5590760b3520 2024-05-08T06:13:50.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.441+0000 7efe7b400700 1 --2- 172.21.15.123:0/1683195040 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5590760b3520 0x5590760b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:50.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.441+0000 7efe7b400700 1 --2- 172.21.15.123:0/1683195040 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5590760b3520 0x5590760b5940 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=963659ea3bbb62e9 server_cookie=3aee69d20911cc73 in_seq=0 out_seq=0 2024-05-08T06:13:50.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.441+0000 7efe65600700 10 client.25790 ms_handle_connect on v2:172.21.15.163:6838/1568161515 2024-05-08T06:13:50.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.442+0000 7efe65600700 1 -- 172.21.15.123:0/1683195040 <== mds.0 v2:172.21.15.163:6838/1568161515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559075f8f400 con 0x5590760b3520 2024-05-08T06:13:50.444 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.442+0000 7efe65600700 10 client.25790 handle_command_reply: tid=0 2024-05-08T06:13:50.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.442+0000 7efe8d145b80 1 client.25790 shutdown 2024-05-08T06:13:50.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.442+0000 7efe8d145b80 1 -- 172.21.15.123:0/1683195040 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55907602f980 msgr2=0x559076041760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:50.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.442+0000 7efe8d145b80 1 --2- 172.21.15.123:0/1683195040 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55907602f980 0x559076041760 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55907604b630 tx=0x7efe68004480 comp rx=0 tx=0).stop 2024-05-08T06:13:50.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.442+0000 7efe8d145b80 1 -- 172.21.15.123:0/1683195040 shutdown_connections 2024-05-08T06:13:50.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.442+0000 7efe8d145b80 1 --2- 172.21.15.123:0/1683195040 >> [v2:172.21.15.163:6838/1568161515,v1:172.21.15.163:6839/1568161515] conn(0x5590760b3520 0x5590760b5940 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:50.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.442+0000 7efe8d145b80 1 --2- 172.21.15.123:0/1683195040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590760339d0 0x55907604f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:50.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.442+0000 7efe8d145b80 1 --2- 172.21.15.123:0/1683195040 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55907602f060 0x559076041220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:50.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.442+0000 7efe65600700 0 client.25790 ms_handle_reset on v2:172.21.15.163:6838/1568161515 2024-05-08T06:13:50.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.442+0000 7efe8d145b80 1 --2- 172.21.15.123:0/1683195040 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55907602f980 0x559076041760 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:50.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.442+0000 7efe8d145b80 1 -- 172.21.15.123:0/1683195040 >> 172.21.15.123:0/1683195040 conn(0x559075f5db60 msgr2=0x5590760242b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:13:50.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.442+0000 7efe8d145b80 1 -- 172.21.15.123:0/1683195040 shutdown_connections 2024-05-08T06:13:50.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.442+0000 7efe8d145b80 1 -- 172.21.15.123:0/1683195040 wait complete. 2024-05-08T06:13:50.445 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.442+0000 7efe8d145b80 20 client.25790 trim_cache size 0 max 16384 2024-05-08T06:13:50.446 INFO:teuthology.orchestra.run.smithi123.stdout:[] 2024-05-08T06:13:50.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.443+0000 7efe82800700 1 -- 172.21.15.123:0/3133793287 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7efe600b2880 msgr2=0x7efe600b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:50.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.443+0000 7efe82800700 1 --2- 172.21.15.123:0/3133793287 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7efe600b2880 0x7efe600b4d40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7efe740094f0 tx=0x7efe74023000 comp rx=0 tx=0).stop 2024-05-08T06:13:50.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.443+0000 7efe82800700 1 -- 172.21.15.123:0/3133793287 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe7c1056c0 msgr2=0x7efe7c19bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:50.446 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.443+0000 7efe82800700 1 --2- 172.21.15.123:0/3133793287 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe7c1056c0 0x7efe7c19bd50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efe6c000c00 tx=0x7efe6c00bfd0 comp rx=0 tx=0).stop 2024-05-08T06:13:50.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.443+0000 7efe82800700 1 -- 172.21.15.123:0/3133793287 shutdown_connections 2024-05-08T06:13:50.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.443+0000 7efe82800700 1 --2- 172.21.15.123:0/3133793287 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7efe600b2880 0x7efe600b4d40 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:50.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.443+0000 7efe82800700 1 --2- 172.21.15.123:0/3133793287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe7c10b6c0 0x7efe7c1a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:50.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.443+0000 7efe82800700 1 --2- 172.21.15.123:0/3133793287 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe7c1056c0 0x7efe7c19bd50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:50.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.443+0000 7efe82800700 1 --2- 172.21.15.123:0/3133793287 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe7c106080 0x7efe7c19c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:50.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.443+0000 7efe82800700 1 -- 172.21.15.123:0/3133793287 >> 172.21.15.123:0/3133793287 conn(0x7efe7c1011b0 msgr2=0x7efe7c101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:13:50.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.443+0000 7efe82800700 1 -- 172.21.15.123:0/3133793287 shutdown_connections 2024-05-08T06:13:50.447 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:50.443+0000 7efe82800700 1 -- 172.21.15.123:0/3133793287 wait complete. 2024-05-08T06:13:51.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:50 smithi123 ceph-mon[29787]: pgmap v561: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-05-08T06:13:51.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:50 smithi163 ceph-mon[34520]: pgmap v561: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-05-08T06:13:51.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:50 smithi192 ceph-mon[34664]: pgmap v561: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-05-08T06:13:51.523 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-05-08T06:13:51.524 INFO:tasks.fwd_scrub:done joining 2024-05-08T06:13:51.524 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-08T06:13:51.535 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T06:13:51.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.748+0000 7faa80a00640 1 -- 172.21.15.123:0/57932538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa7c10a940 msgr2=0x7faa7c111260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:51.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.748+0000 7faa80a00640 1 --2- 172.21.15.123:0/57932538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa7c10a940 0x7faa7c111260 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7faa74009ca0 tx=0x7faa7402a8c0 comp rx=0 tx=0).stop 2024-05-08T06:13:51.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.748+0000 7faa80a00640 1 -- 172.21.15.123:0/57932538 shutdown_connections 2024-05-08T06:13:51.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.748+0000 7faa80a00640 1 --2- 172.21.15.123:0/57932538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa7c10a940 0x7faa7c111260 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:51.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.748+0000 7faa80a00640 1 --2- 172.21.15.123:0/57932538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa7c109f80 0x7faa7c10a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:51.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.748+0000 7faa80a00640 1 --2- 172.21.15.123:0/57932538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa7c108e30 0x7faa7c109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:51.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.748+0000 7faa80a00640 1 -- 172.21.15.123:0/57932538 >> 172.21.15.123:0/57932538 conn(0x7faa7c077df0 msgr2=0x7faa7c07a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:13:51.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.748+0000 7faa80a00640 1 -- 172.21.15.123:0/57932538 shutdown_connections 2024-05-08T06:13:51.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.748+0000 7faa80a00640 1 -- 172.21.15.123:0/57932538 wait complete. 2024-05-08T06:13:51.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.749+0000 7faa80a00640 1 Processor -- start 2024-05-08T06:13:51.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.749+0000 7faa80a00640 1 -- start start 2024-05-08T06:13:51.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.749+0000 7faa80a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa7c108e30 0x7faa7c19f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:51.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.749+0000 7faa80a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa7c109f80 0x7faa7c19f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:51.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.749+0000 7faa80a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa7c10a940 0x7faa7c1a65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:51.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.749+0000 7faa80a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faa7c114980 con 0x7faa7c108e30 2024-05-08T06:13:51.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.749+0000 7faa80a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7faa7c114800 con 0x7faa7c10a940 2024-05-08T06:13:51.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.749+0000 7faa80a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7faa7c114b00 con 0x7faa7c109f80 2024-05-08T06:13:51.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.750+0000 7faa7b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa7c108e30 0x7faa7c19f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:51.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.750+0000 7faa7b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa7c108e30 0x7faa7c19f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51756/0 (socket says 172.21.15.123:51756) 2024-05-08T06:13:51.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.750+0000 7faa7b400640 1 -- 172.21.15.123:0/1089706387 learned_addr learned my addr 172.21.15.123:0/1089706387 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:13:51.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.750+0000 7faa7be00640 1 --2- 172.21.15.123:0/1089706387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa7c10a940 0x7faa7c1a65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:51.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.750+0000 7faa7aa00640 1 --2- 172.21.15.123:0/1089706387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa7c109f80 0x7faa7c19f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:51.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.750+0000 7faa7b400640 1 -- 172.21.15.123:0/1089706387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa7c109f80 msgr2=0x7faa7c19f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:51.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.750+0000 7faa7b400640 1 --2- 172.21.15.123:0/1089706387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa7c109f80 0x7faa7c19f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:51.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.750+0000 7faa7b400640 1 -- 172.21.15.123:0/1089706387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa7c10a940 msgr2=0x7faa7c1a65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:51.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.750+0000 7faa7b400640 1 --2- 172.21.15.123:0/1089706387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa7c10a940 0x7faa7c1a65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:51.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.750+0000 7faa7b400640 1 -- 172.21.15.123:0/1089706387 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa7c1a6c40 con 0x7faa7c108e30 2024-05-08T06:13:51.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.750+0000 7faa7b400640 1 --2- 172.21.15.123:0/1089706387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa7c108e30 0x7faa7c19f020 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7faa6c00e9f0 tx=0x7faa6c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:13:51.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.750+0000 7faa73e00640 1 -- 172.21.15.123:0/1089706387 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c00cd80 con 0x7faa7c108e30 2024-05-08T06:13:51.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.751+0000 7faa73e00640 1 -- 172.21.15.123:0/1089706387 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa6c0054d0 con 0x7faa7c108e30 2024-05-08T06:13:51.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.751+0000 7faa80a00640 1 -- 172.21.15.123:0/1089706387 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa7c1a6f30 con 0x7faa7c108e30 2024-05-08T06:13:51.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.751+0000 7faa73e00640 1 -- 172.21.15.123:0/1089706387 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c010690 con 0x7faa7c108e30 2024-05-08T06:13:51.752 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.751+0000 7faa80a00640 1 -- 172.21.15.123:0/1089706387 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa7c1a7390 con 0x7faa7c108e30 2024-05-08T06:13:51.753 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.752+0000 7faa80a00640 1 -- 172.21.15.123:0/1089706387 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa7c110140 con 0x7faa7c108e30 2024-05-08T06:13:51.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.754+0000 7faa73e00640 1 -- 172.21.15.123:0/1089706387 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7faa6c004e90 con 0x7faa7c108e30 2024-05-08T06:13:51.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.756+0000 7faa73e00640 1 --2- 172.21.15.123:0/1089706387 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7faa600b27f0 0x7faa600b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:51.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.756+0000 7faa73e00640 1 -- 172.21.15.123:0/1089706387 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7faa6c0e2650 con 0x7faa7c108e30 2024-05-08T06:13:51.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.756+0000 7faa7aa00640 1 --2- 172.21.15.123:0/1089706387 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7faa600b27f0 0x7faa600b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:51.756 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.757+0000 7faa7aa00640 1 --2- 172.21.15.123:0/1089706387 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7faa600b27f0 0x7faa600b4cb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7faa680095f0 tx=0x7faa68005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:13:51.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.760+0000 7faa73e00640 1 -- 172.21.15.123:0/1089706387 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faa6c0aa320 con 0x7faa7c108e30 2024-05-08T06:13:51.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.974+0000 7faa80a00640 1 -- 172.21.15.123:0/1089706387 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faa7c0636f0 con 0x7faa7c108e30 2024-05-08T06:13:51.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.978+0000 7faa73e00640 1 -- 172.21.15.123:0/1089706387 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22719 (secure 0 0 0) 0x7faa6c0a9c20 con 0x7faa7c108e30 2024-05-08T06:13:51.978 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:13:51.979 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":82,"fsid":"85cfb2e6-0cff-11ef-bc97-c7b262605968","created":"2024-05-08T05:56:12.544750+0000","modified":"2024-05-08T06:06:32.471993+0000","last_up_change":"2024-05-08T06:04:33.654727+0000","last_in_change":"2024-05-08T06:04:13.809050+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T06:00:17.050518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T06:05:44.815334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T06:05:49.604457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a14fde9b-bfb1-40a7-a105-305e8981a778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6803","nonce":3608618058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6805","nonce":3608618058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6809","nonce":3608618058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6807","nonce":3608618058}]},"public_addr":"172.21.15.123:6803/3608618058","cluster_addr":"172.21.15.123:6805/3608618058","heartbeat_back_addr":"172.21.15.123:6809/3608618058","heartbeat_front_addr":"172.21.15.123:6807/3608618058","state":["exists","up"]},{"osd":1,"uuid":"e023ae3d-a82e-4bb6-8fef-889dd07fe183","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6801","nonce":1884753449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6803","nonce":1884753449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6807","nonce":1884753449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6805","nonce":1884753449}]},"public_addr":"172.21.15.163:6801/1884753449","cluster_addr":"172.21.15.163:6803/1884753449","heartbeat_back_addr":"172.21.15.163:6807/1884753449","heartbeat_front_addr":"172.21.15.163:6805/1884753449","state":["exists","up"]},{"osd":2,"uuid":"69f4cfca-63cb-4862-9480-d79891b0f338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6801","nonce":4161570404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6803","nonce":4161570404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6807","nonce":4161570404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6805","nonce":4161570404}]},"public_addr":"172.21.15.192:6801/4161570404","cluster_addr":"172.21.15.192:6803/4161570404","heartbeat_back_addr":"172.21.15.192:6807/4161570404","heartbeat_front_addr":"172.21.15.192:6805/4161570404","state":["exists","up"]},{"osd":3,"uuid":"71bb5ecd-c88c-45fb-8b51-0eefb53941c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6811","nonce":1243003258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6813","nonce":1243003258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6817","nonce":1243003258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6815","nonce":1243003258}]},"public_addr":"172.21.15.123:6811/1243003258","cluster_addr":"172.21.15.123:6813/1243003258","heartbeat_back_addr":"172.21.15.123:6817/1243003258","heartbeat_front_addr":"172.21.15.123:6815/1243003258","state":["exists","up"]},{"osd":4,"uuid":"5a7a3462-e74f-4d15-a85c-709b5d423745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6809","nonce":591680273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6811","nonce":591680273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6815","nonce":591680273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6813","nonce":591680273}]},"public_addr":"172.21.15.163:6809/591680273","cluster_addr":"172.21.15.163:6811/591680273","heartbeat_back_addr":"172.21.15.163:6815/591680273","heartbeat_front_addr":"172.21.15.163:6813/591680273","state":["exists","up"]},{"osd":5,"uuid":"85e3f1ab-f8c6-4025-bab9-9ddb6bd3e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6809","nonce":1734821180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6811","nonce":1734821180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6815","nonce":1734821180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6813","nonce":1734821180}]},"public_addr":"172.21.15.192:6809/1734821180","cluster_addr":"172.21.15.192:6811/1734821180","heartbeat_back_addr":"172.21.15.192:6815/1734821180","heartbeat_front_addr":"172.21.15.192:6813/1734821180","state":["exists","up"]},{"osd":6,"uuid":"f21a5624-6f8f-48be-bc6d-0a883d6b7b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6819","nonce":1002138878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6821","nonce":1002138878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6825","nonce":1002138878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6823","nonce":1002138878}]},"public_addr":"172.21.15.123:6819/1002138878","cluster_addr":"172.21.15.123:6821/1002138878","heartbeat_back_addr":"172.21.15.123:6825/1002138878","heartbeat_front_addr":"172.21.15.123:6823/1002138878","state":["exists","up"]},{"osd":7,"uuid":"6eb16729-d260-4041-9829-3e868795b111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6817","nonce":354411978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6819","nonce":354411978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6823","nonce":354411978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6821","nonce":354411978}]},"public_addr":"172.21.15.163:6817/354411978","cluster_addr":"172.21.15.163:6819/354411978","heartbeat_back_addr":"172.21.15.163:6823/354411978","heartbeat_front_addr":"172.21.15.163:6821/354411978","state":["exists","up"]},{"osd":8,"uuid":"122a281e-18b8-4728-97db-458e7432b2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6817","nonce":3669753689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6819","nonce":3669753689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6823","nonce":3669753689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6821","nonce":3669753689}]},"public_addr":"172.21.15.192:6817/3669753689","cluster_addr":"172.21.15.192:6819/3669753689","heartbeat_back_addr":"172.21.15.192:6823/3669753689","heartbeat_front_addr":"172.21.15.192:6821/3669753689","state":["exists","up"]},{"osd":9,"uuid":"9fa022a9-b873-4e07-806c-641a706a150f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6827","nonce":3104436473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6829","nonce":3104436473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6833","nonce":3104436473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6831","nonce":3104436473}]},"public_addr":"172.21.15.123:6827/3104436473","cluster_addr":"172.21.15.123:6829/3104436473","heartbeat_back_addr":"172.21.15.123:6833/3104436473","heartbeat_front_addr":"172.21.15.123:6831/3104436473","state":["exists","up"]},{"osd":10,"uuid":"fb59339b-e11b-4ffc-b202-7f67cb21da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6825","nonce":2628487883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6827","nonce":2628487883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6831","nonce":2628487883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6829","nonce":2628487883}]},"public_addr":"172.21.15.163:6825/2628487883","cluster_addr":"172.21.15.163:6827/2628487883","heartbeat_back_addr":"172.21.15.163:6831/2628487883","heartbeat_front_addr":"172.21.15.163:6829/2628487883","state":["exists","up"]},{"osd":11,"uuid":"be7a5f36-c1be-41d7-80ee-004d0546eb9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6825","nonce":1339159347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6827","nonce":1339159347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6831","nonce":1339159347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6829","nonce":1339159347}]},"public_addr":"172.21.15.192:6825/1339159347","cluster_addr":"172.21.15.192:6827/1339159347","heartbeat_back_addr":"172.21.15.192:6831/1339159347","heartbeat_front_addr":"172.21.15.192:6829/1339159347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:59:24.923234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:59:49.867071+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:00:13.454062+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:00:46.983594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:01:12.820452+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:01:37.451382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:02:13.325181+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:02:39.453436+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:03:03.837509+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:03:39.460579+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:04:06.314082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:04:31.723154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6801/3057688649":"2024-05-09T05:57:30.887936+0000","172.21.15.123:6800/3057688649":"2024-05-09T05:57:30.887936+0000","172.21.15.123:6800/2769724312":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/399884043":"2024-05-09T05:57:30.887936+0000","172.21.15.123:0/3406061897":"2024-05-09T05:57:30.887936+0000","172.21.15.123:0/644457268":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/3055092197":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/343390357":"2024-05-09T05:57:30.887936+0000","172.21.15.123:6801/2769724312":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/2263171562":"2024-05-09T05:56:55.849036+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T06:13:51.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.981+0000 7faa80a00640 1 -- 172.21.15.123:0/1089706387 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7faa600b27f0 msgr2=0x7faa600b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:51.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.981+0000 7faa80a00640 1 --2- 172.21.15.123:0/1089706387 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7faa600b27f0 0x7faa600b4cb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7faa680095f0 tx=0x7faa68005f90 comp rx=0 tx=0).stop 2024-05-08T06:13:51.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.981+0000 7faa80a00640 1 -- 172.21.15.123:0/1089706387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa7c108e30 msgr2=0x7faa7c19f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:51.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.981+0000 7faa80a00640 1 --2- 172.21.15.123:0/1089706387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa7c108e30 0x7faa7c19f020 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7faa6c00e9f0 tx=0x7faa6c00edf0 comp rx=0 tx=0).stop 2024-05-08T06:13:51.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.981+0000 7faa80a00640 1 -- 172.21.15.123:0/1089706387 shutdown_connections 2024-05-08T06:13:51.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.981+0000 7faa80a00640 1 --2- 172.21.15.123:0/1089706387 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7faa600b27f0 0x7faa600b4cb0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:51.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.981+0000 7faa80a00640 1 --2- 172.21.15.123:0/1089706387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa7c10a940 0x7faa7c1a65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:51.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.981+0000 7faa80a00640 1 --2- 172.21.15.123:0/1089706387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa7c109f80 0x7faa7c19f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:51.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.981+0000 7faa80a00640 1 --2- 172.21.15.123:0/1089706387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa7c108e30 0x7faa7c19f020 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:51.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.981+0000 7faa80a00640 1 -- 172.21.15.123:0/1089706387 >> 172.21.15.123:0/1089706387 conn(0x7faa7c077df0 msgr2=0x7faa7c10d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:13:51.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.981+0000 7faa80a00640 1 -- 172.21.15.123:0/1089706387 shutdown_connections 2024-05-08T06:13:51.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:51.981+0000 7faa80a00640 1 -- 172.21.15.123:0/1089706387 wait complete. 2024-05-08T06:13:52.002 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T06:00:17.050518+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T06:05:44.815334+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T06:05:49.604457+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T06:13:52.002 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T06:13:52.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.209+0000 7fd949e00640 1 -- 172.21.15.123:0/2980433010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd944103be0 msgr2=0x7fd944104060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:52.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.209+0000 7fd949e00640 1 --2- 172.21.15.123:0/2980433010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd944103be0 0x7fd944104060 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd930009900 tx=0x7fd93002a3f0 comp rx=0 tx=0).stop 2024-05-08T06:13:52.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.209+0000 7fd949e00640 1 -- 172.21.15.123:0/2980433010 shutdown_connections 2024-05-08T06:13:52.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.209+0000 7fd949e00640 1 --2- 172.21.15.123:0/2980433010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9441045a0 0x7fd94410ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:52.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.209+0000 7fd949e00640 1 --2- 172.21.15.123:0/2980433010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd944103be0 0x7fd944104060 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:52.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.209+0000 7fd949e00640 1 --2- 172.21.15.123:0/2980433010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9441029f0 0x7fd944102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:52.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.209+0000 7fd949e00640 1 -- 172.21.15.123:0/2980433010 >> 172.21.15.123:0/2980433010 conn(0x7fd9440fdf60 msgr2=0x7fd9441003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:13:52.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.209+0000 7fd949e00640 1 -- 172.21.15.123:0/2980433010 shutdown_connections 2024-05-08T06:13:52.210 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.209+0000 7fd949e00640 1 -- 172.21.15.123:0/2980433010 wait complete. 2024-05-08T06:13:52.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.210+0000 7fd949e00640 1 Processor -- start 2024-05-08T06:13:52.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.210+0000 7fd949e00640 1 -- start start 2024-05-08T06:13:52.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.211+0000 7fd949e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9441029f0 0x7fd944198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:52.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.211+0000 7fd949e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd944103be0 0x7fd9441991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:52.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.211+0000 7fd949e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9441045a0 0x7fd9441a0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:52.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.211+0000 7fd949e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd94410e570 con 0x7fd9441029f0 2024-05-08T06:13:52.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.211+0000 7fd949e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd94410e3f0 con 0x7fd9441045a0 2024-05-08T06:13:52.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.211+0000 7fd949e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd94410e6f0 con 0x7fd944103be0 2024-05-08T06:13:52.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.211+0000 7fd942a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9441029f0 0x7fd944198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:52.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.211+0000 7fd942a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9441029f0 0x7fd944198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51776/0 (socket says 172.21.15.123:51776) 2024-05-08T06:13:52.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.211+0000 7fd942a00640 1 -- 172.21.15.123:0/3073981986 learned_addr learned my addr 172.21.15.123:0/3073981986 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:13:52.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.211+0000 7fd943400640 1 --2- 172.21.15.123:0/3073981986 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9441045a0 0x7fd9441a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:52.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.211+0000 7fd942000640 1 --2- 172.21.15.123:0/3073981986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd944103be0 0x7fd9441991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:52.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.211+0000 7fd942a00640 1 -- 172.21.15.123:0/3073981986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd944103be0 msgr2=0x7fd9441991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:52.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.211+0000 7fd942a00640 1 --2- 172.21.15.123:0/3073981986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd944103be0 0x7fd9441991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:52.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.211+0000 7fd942a00640 1 -- 172.21.15.123:0/3073981986 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9441045a0 msgr2=0x7fd9441a0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:52.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.211+0000 7fd942a00640 1 --2- 172.21.15.123:0/3073981986 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9441045a0 0x7fd9441a0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:52.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.211+0000 7fd942a00640 1 -- 172.21.15.123:0/3073981986 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9441a08b0 con 0x7fd9441029f0 2024-05-08T06:13:52.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.211+0000 7fd942000640 1 --2- 172.21.15.123:0/3073981986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd944103be0 0x7fd9441991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:13:52.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.212+0000 7fd942a00640 1 --2- 172.21.15.123:0/3073981986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9441029f0 0x7fd944198c70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd93400e9c0 tx=0x7fd93400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:13:52.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.212+0000 7fd948a00640 1 -- 172.21.15.123:0/3073981986 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93400cde0 con 0x7fd9441029f0 2024-05-08T06:13:52.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.212+0000 7fd948a00640 1 -- 172.21.15.123:0/3073981986 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9340108e0 con 0x7fd9441029f0 2024-05-08T06:13:52.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.212+0000 7fd949e00640 1 -- 172.21.15.123:0/3073981986 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9441a0ba0 con 0x7fd9441029f0 2024-05-08T06:13:52.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.212+0000 7fd948a00640 1 -- 172.21.15.123:0/3073981986 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9340188d0 con 0x7fd9441029f0 2024-05-08T06:13:52.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.212+0000 7fd949e00640 1 -- 172.21.15.123:0/3073981986 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9441a10c0 con 0x7fd9441029f0 2024-05-08T06:13:52.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.213+0000 7fd949e00640 1 -- 172.21.15.123:0/3073981986 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd944109de0 con 0x7fd9441029f0 2024-05-08T06:13:52.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.215+0000 7fd948a00640 1 -- 172.21.15.123:0/3073981986 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7fd93401f020 con 0x7fd9441029f0 2024-05-08T06:13:52.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.217+0000 7fd948a00640 1 --2- 172.21.15.123:0/3073981986 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd9280b27f0 0x7fd9280b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:52.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.217+0000 7fd948a00640 1 -- 172.21.15.123:0/3073981986 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fd9340e2400 con 0x7fd9441029f0 2024-05-08T06:13:52.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.217+0000 7fd942000640 1 --2- 172.21.15.123:0/3073981986 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd9280b27f0 0x7fd9280b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:52.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.218+0000 7fd942000640 1 --2- 172.21.15.123:0/3073981986 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd9280b27f0 0x7fd9280b4cb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd930000c00 tx=0x7fd9300314c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:13:52.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.221+0000 7fd948a00640 1 -- 172.21.15.123:0/3073981986 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd9340aa130 con 0x7fd9441029f0 2024-05-08T06:13:52.432 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.432+0000 7fd949e00640 1 -- 172.21.15.123:0/3073981986 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd94402df10 con 0x7fd9441029f0 2024-05-08T06:13:52.433 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.433+0000 7fd948a00640 1 -- 172.21.15.123:0/3073981986 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fd9340a9a30 con 0x7fd9441029f0 2024-05-08T06:13:52.433 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2024-05-08T06:13:52.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.436+0000 7fd949e00640 1 -- 172.21.15.123:0/3073981986 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd9280b27f0 msgr2=0x7fd9280b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:52.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.436+0000 7fd949e00640 1 --2- 172.21.15.123:0/3073981986 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd9280b27f0 0x7fd9280b4cb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd930000c00 tx=0x7fd9300314c0 comp rx=0 tx=0).stop 2024-05-08T06:13:52.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.436+0000 7fd949e00640 1 -- 172.21.15.123:0/3073981986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9441029f0 msgr2=0x7fd944198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:52.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.436+0000 7fd949e00640 1 --2- 172.21.15.123:0/3073981986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9441029f0 0x7fd944198c70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd93400e9c0 tx=0x7fd93400ee80 comp rx=0 tx=0).stop 2024-05-08T06:13:52.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.436+0000 7fd949e00640 1 -- 172.21.15.123:0/3073981986 shutdown_connections 2024-05-08T06:13:52.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.436+0000 7fd949e00640 1 --2- 172.21.15.123:0/3073981986 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fd9280b27f0 0x7fd9280b4cb0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:52.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.436+0000 7fd949e00640 1 --2- 172.21.15.123:0/3073981986 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9441045a0 0x7fd9441a0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:52.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.436+0000 7fd949e00640 1 --2- 172.21.15.123:0/3073981986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd944103be0 0x7fd9441991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:52.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.436+0000 7fd949e00640 1 --2- 172.21.15.123:0/3073981986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9441029f0 0x7fd944198c70 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:52.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.436+0000 7fd949e00640 1 -- 172.21.15.123:0/3073981986 >> 172.21.15.123:0/3073981986 conn(0x7fd9440fdf60 msgr2=0x7fd944106fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:13:52.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.437+0000 7fd949e00640 1 -- 172.21.15.123:0/3073981986 shutdown_connections 2024-05-08T06:13:52.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.437+0000 7fd949e00640 1 -- 172.21.15.123:0/3073981986 wait complete. 2024-05-08T06:13:52.454 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T06:13:52.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.651+0000 7f5eca000640 1 -- 172.21.15.123:0/774608223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ec41045b0 msgr2=0x7f5ec410ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:52.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.651+0000 7f5eca000640 1 --2- 172.21.15.123:0/774608223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ec41045b0 0x7f5ec410ae40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc009ca0 tx=0x7f5ebc02a8c0 comp rx=0 tx=0).stop 2024-05-08T06:13:52.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.651+0000 7f5eca000640 1 -- 172.21.15.123:0/774608223 shutdown_connections 2024-05-08T06:13:52.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.651+0000 7f5eca000640 1 --2- 172.21.15.123:0/774608223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ec41045b0 0x7f5ec410ae40 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:52.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.651+0000 7f5eca000640 1 --2- 172.21.15.123:0/774608223 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ec4103bf0 0x7f5ec4104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:52.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.651+0000 7f5eca000640 1 --2- 172.21.15.123:0/774608223 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ec4102a00 0x7f5ec4102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:52.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.651+0000 7f5eca000640 1 -- 172.21.15.123:0/774608223 >> 172.21.15.123:0/774608223 conn(0x7f5ec40fdf90 msgr2=0x7f5ec41003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:13:52.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.651+0000 7f5eca000640 1 -- 172.21.15.123:0/774608223 shutdown_connections 2024-05-08T06:13:52.653 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.652+0000 7f5eca000640 1 -- 172.21.15.123:0/774608223 wait complete. 2024-05-08T06:13:52.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.652+0000 7f5eca000640 1 Processor -- start 2024-05-08T06:13:52.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.653+0000 7f5eca000640 1 -- start start 2024-05-08T06:13:52.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.653+0000 7f5eca000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ec4102a00 0x7f5ec4076760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:52.655 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.653+0000 7f5eca000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ec4103bf0 0x7f5ec4074da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:52.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.653+0000 7f5eca000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ec41045b0 0x7f5ec40752e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:52.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.653+0000 7f5eca000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec410e580 con 0x7f5ec41045b0 2024-05-08T06:13:52.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.653+0000 7f5eca000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec410e400 con 0x7f5ec4102a00 2024-05-08T06:13:52.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.653+0000 7f5eca000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec410e700 con 0x7f5ec4103bf0 2024-05-08T06:13:52.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.653+0000 7f5ec3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ec41045b0 0x7f5ec40752e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:52.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.653+0000 7f5ec3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ec41045b0 0x7f5ec40752e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51792/0 (socket says 172.21.15.123:51792) 2024-05-08T06:13:52.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.653+0000 7f5ec3e00640 1 -- 172.21.15.123:0/3394650494 learned_addr learned my addr 172.21.15.123:0/3394650494 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:13:52.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.653+0000 7f5ec2a00640 1 --2- 172.21.15.123:0/3394650494 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ec4103bf0 0x7f5ec4074da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:52.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.653+0000 7f5ec3400640 1 --2- 172.21.15.123:0/3394650494 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ec4102a00 0x7f5ec4076760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:52.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.654+0000 7f5ec3e00640 1 -- 172.21.15.123:0/3394650494 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ec4103bf0 msgr2=0x7f5ec4074da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:52.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.654+0000 7f5ec3e00640 1 --2- 172.21.15.123:0/3394650494 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ec4103bf0 0x7f5ec4074da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:52.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.654+0000 7f5ec3e00640 1 -- 172.21.15.123:0/3394650494 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ec4102a00 msgr2=0x7f5ec4076760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:52.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.654+0000 7f5ec3e00640 1 --2- 172.21.15.123:0/3394650494 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ec4102a00 0x7f5ec4076760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:52.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.654+0000 7f5ec3e00640 1 -- 172.21.15.123:0/3394650494 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec4075b70 con 0x7f5ec41045b0 2024-05-08T06:13:52.656 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.654+0000 7f5ec3e00640 1 --2- 172.21.15.123:0/3394650494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ec41045b0 0x7f5ec40752e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc033040 tx=0x7f5ebc0316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:13:52.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.654+0000 7f5ebbe00640 1 -- 172.21.15.123:0/3394650494 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc003b70 con 0x7f5ec41045b0 2024-05-08T06:13:52.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.654+0000 7f5eca000640 1 -- 172.21.15.123:0/3394650494 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ec41a2d50 con 0x7f5ec41045b0 2024-05-08T06:13:52.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.654+0000 7f5ebbe00640 1 -- 172.21.15.123:0/3394650494 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ebc003d10 con 0x7f5ec41045b0 2024-05-08T06:13:52.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.654+0000 7f5ebbe00640 1 -- 172.21.15.123:0/3394650494 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc02fb40 con 0x7f5ec41045b0 2024-05-08T06:13:52.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.654+0000 7f5eca000640 1 -- 172.21.15.123:0/3394650494 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ec41a3210 con 0x7f5ec41045b0 2024-05-08T06:13:52.657 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.656+0000 7f5eca000640 1 -- 172.21.15.123:0/3394650494 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ec4109df0 con 0x7f5ec41045b0 2024-05-08T06:13:52.658 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.658+0000 7f5ebbe00640 1 -- 172.21.15.123:0/3394650494 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7f5ebc004610 con 0x7f5ec41045b0 2024-05-08T06:13:52.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.659+0000 7f5ebbe00640 1 --2- 172.21.15.123:0/3394650494 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5ea80b27f0 0x7f5ea80b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:52.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.659+0000 7f5ebbe00640 1 -- 172.21.15.123:0/3394650494 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f5ebc103650 con 0x7f5ec41045b0 2024-05-08T06:13:52.660 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.660+0000 7f5ec3400640 1 --2- 172.21.15.123:0/3394650494 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5ea80b27f0 0x7f5ea80b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:52.661 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.660+0000 7f5ec3400640 1 --2- 172.21.15.123:0/3394650494 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5ea80b27f0 0x7f5ea80b4cb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4103a50 tx=0x7f5eb4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:13:52.664 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.663+0000 7f5ebbe00640 1 -- 172.21.15.123:0/3394650494 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5ebc0cb320 con 0x7f5ec41045b0 2024-05-08T06:13:52.874 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.873+0000 7f5eca000640 1 -- 172.21.15.123:0/3394650494 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5ec40635b0 con 0x7f5ec41045b0 2024-05-08T06:13:52.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.874+0000 7f5ebbe00640 1 -- 172.21.15.123:0/3394650494 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) ==== 104+0+11 (secure 0 0 0) 0x7f5ebc0d33a0 con 0x7f5ec41045b0 2024-05-08T06:13:52.875 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2024-05-08T06:13:52.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.876+0000 7f5eca000640 1 -- 172.21.15.123:0/3394650494 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5ea80b27f0 msgr2=0x7f5ea80b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:52.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.876+0000 7f5eca000640 1 --2- 172.21.15.123:0/3394650494 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5ea80b27f0 0x7f5ea80b4cb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4103a50 tx=0x7f5eb4005f90 comp rx=0 tx=0).stop 2024-05-08T06:13:52.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.876+0000 7f5eca000640 1 -- 172.21.15.123:0/3394650494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ec41045b0 msgr2=0x7f5ec40752e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:52.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.876+0000 7f5eca000640 1 --2- 172.21.15.123:0/3394650494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ec41045b0 0x7f5ec40752e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc033040 tx=0x7f5ebc0316f0 comp rx=0 tx=0).stop 2024-05-08T06:13:52.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.877+0000 7f5eca000640 1 -- 172.21.15.123:0/3394650494 shutdown_connections 2024-05-08T06:13:52.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.877+0000 7f5eca000640 1 --2- 172.21.15.123:0/3394650494 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f5ea80b27f0 0x7f5ea80b4cb0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:52.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.877+0000 7f5eca000640 1 --2- 172.21.15.123:0/3394650494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ec41045b0 0x7f5ec40752e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:52.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.877+0000 7f5eca000640 1 --2- 172.21.15.123:0/3394650494 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ec4103bf0 0x7f5ec4074da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:52.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.877+0000 7f5eca000640 1 --2- 172.21.15.123:0/3394650494 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ec4102a00 0x7f5ec4076760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:52.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.877+0000 7f5eca000640 1 -- 172.21.15.123:0/3394650494 >> 172.21.15.123:0/3394650494 conn(0x7f5ec40fdf90 msgr2=0x7f5ec4106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:13:52.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.877+0000 7f5eca000640 1 -- 172.21.15.123:0/3394650494 shutdown_connections 2024-05-08T06:13:52.879 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:52.877+0000 7f5eca000640 1 -- 172.21.15.123:0/3394650494 wait complete. 2024-05-08T06:13:52.896 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T06:13:53.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.102+0000 7f51ea600640 1 -- 172.21.15.123:0/18989664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51e4103350 msgr2=0x7f51e4105740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:53.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.102+0000 7f51ea600640 1 --2- 172.21.15.123:0/18989664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51e4103350 0x7f51e4105740 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f51d8009900 tx=0x7f51d802a410 comp rx=0 tx=0).stop 2024-05-08T06:13:53.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.102+0000 7f51ea600640 1 -- 172.21.15.123:0/18989664 shutdown_connections 2024-05-08T06:13:53.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.102+0000 7f51ea600640 1 --2- 172.21.15.123:0/18989664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51e41085b0 0x7f51e410aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:53.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.102+0000 7f51ea600640 1 --2- 172.21.15.123:0/18989664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51e4105c80 0x7f51e4108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:53.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.102+0000 7f51ea600640 1 --2- 172.21.15.123:0/18989664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51e4103350 0x7f51e4105740 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:53.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.102+0000 7f51ea600640 1 -- 172.21.15.123:0/18989664 >> 172.21.15.123:0/18989664 conn(0x7f51e40fcd60 msgr2=0x7f51e40ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:13:53.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.102+0000 7f51ea600640 1 -- 172.21.15.123:0/18989664 shutdown_connections 2024-05-08T06:13:53.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.102+0000 7f51ea600640 1 -- 172.21.15.123:0/18989664 wait complete. 2024-05-08T06:13:53.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.103+0000 7f51ea600640 1 Processor -- start 2024-05-08T06:13:53.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.104+0000 7f51ea600640 1 -- start start 2024-05-08T06:13:53.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.104+0000 7f51ea600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51e4103350 0x7f51e4102cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:53.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.104+0000 7f51ea600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51e4105c80 0x7f51e4101300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:53.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.104+0000 7f51ea600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51e41085b0 0x7f51e4101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:53.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.104+0000 7f51ea600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f51e410e480 con 0x7f51e4103350 2024-05-08T06:13:53.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.104+0000 7f51ea600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f51e410e300 con 0x7f51e4105c80 2024-05-08T06:13:53.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.104+0000 7f51ea600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f51e410e600 con 0x7f51e41085b0 2024-05-08T06:13:53.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.104+0000 7f51e9200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51e4103350 0x7f51e4102cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:53.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.104+0000 7f51e9200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51e4103350 0x7f51e4102cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51812/0 (socket says 172.21.15.123:51812) 2024-05-08T06:13:53.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.104+0000 7f51e9200640 1 -- 172.21.15.123:0/2782919950 learned_addr learned my addr 172.21.15.123:0/2782919950 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:13:53.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.104+0000 7f51e3e00640 1 --2- 172.21.15.123:0/2782919950 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51e4105c80 0x7f51e4101300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:53.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.104+0000 7f51e9c00640 1 --2- 172.21.15.123:0/2782919950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51e41085b0 0x7f51e4101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:53.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.105+0000 7f51e9200640 1 -- 172.21.15.123:0/2782919950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51e41085b0 msgr2=0x7f51e4101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:53.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.105+0000 7f51e9200640 1 --2- 172.21.15.123:0/2782919950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51e41085b0 0x7f51e4101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:53.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.105+0000 7f51e9200640 1 -- 172.21.15.123:0/2782919950 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51e4105c80 msgr2=0x7f51e4101300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:53.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.105+0000 7f51e9200640 1 --2- 172.21.15.123:0/2782919950 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51e4105c80 0x7f51e4101300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:53.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.105+0000 7f51e9200640 1 -- 172.21.15.123:0/2782919950 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51e41020d0 con 0x7f51e4103350 2024-05-08T06:13:53.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.105+0000 7f51e9c00640 1 --2- 172.21.15.123:0/2782919950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51e41085b0 0x7f51e4101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:13:53.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.105+0000 7f51e9200640 1 --2- 172.21.15.123:0/2782919950 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51e4103350 0x7f51e4102cc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f51d8005ab0 tx=0x7f51d8003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:13:53.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.105+0000 7f51e1600640 1 -- 172.21.15.123:0/2782919950 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d803f070 con 0x7f51e4103350 2024-05-08T06:13:53.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.105+0000 7f51ea600640 1 -- 172.21.15.123:0/2782919950 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51e41a2c10 con 0x7f51e4103350 2024-05-08T06:13:53.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.105+0000 7f51e1600640 1 -- 172.21.15.123:0/2782919950 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51d8031c50 con 0x7f51e4103350 2024-05-08T06:13:53.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.105+0000 7f51e1600640 1 -- 172.21.15.123:0/2782919950 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d8035030 con 0x7f51e4103350 2024-05-08T06:13:53.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.106+0000 7f51ea600640 1 -- 172.21.15.123:0/2782919950 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51e41a31b0 con 0x7f51e4103350 2024-05-08T06:13:53.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.107+0000 7f51ea600640 1 -- 172.21.15.123:0/2782919950 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51e40fe850 con 0x7f51e4103350 2024-05-08T06:13:53.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.109+0000 7f51e1600640 1 -- 172.21.15.123:0/2782919950 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7f51d8042070 con 0x7f51e4103350 2024-05-08T06:13:53.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.111+0000 7f51e1600640 1 --2- 172.21.15.123:0/2782919950 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f51cc0b2870 0x7f51cc0b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:53.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.111+0000 7f51e1600640 1 -- 172.21.15.123:0/2782919950 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f51d8102a70 con 0x7f51e4103350 2024-05-08T06:13:53.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.111+0000 7f51e3e00640 1 --2- 172.21.15.123:0/2782919950 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f51cc0b2870 0x7f51cc0b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:53.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.112+0000 7f51e3e00640 1 --2- 172.21.15.123:0/2782919950 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f51cc0b2870 0x7f51cc0b4d30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f51e4102a30 tx=0x7f51d0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:13:53.116 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.115+0000 7f51e1600640 1 -- 172.21.15.123:0/2782919950 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f51d80ca740 con 0x7f51e4103350 2024-05-08T06:13:53.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:52 smithi123 ceph-mon[29787]: pgmap v562: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:13:53.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:52 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/1089706387' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:13:53.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:52 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/3073981986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T06:13:53.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:52 smithi163 ceph-mon[34520]: pgmap v562: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:13:53.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:52 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/1089706387' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:13:53.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:52 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/3073981986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T06:13:53.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:52 smithi192 ceph-mon[34664]: pgmap v562: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:13:53.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:52 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/1089706387' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:13:53.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:52 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/3073981986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T06:13:53.323 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.322+0000 7f51ea600640 1 -- 172.21.15.123:0/2782919950 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f51e40635b0 con 0x7f51e4103350 2024-05-08T06:13:53.324 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.323+0000 7f51e1600640 1 -- 172.21.15.123:0/2782919950 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) ==== 100+0+11 (secure 0 0 0) 0x7f51d80ca040 con 0x7f51e4103350 2024-05-08T06:13:53.324 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2024-05-08T06:13:53.327 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.326+0000 7f51ea600640 1 -- 172.21.15.123:0/2782919950 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f51cc0b2870 msgr2=0x7f51cc0b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:53.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.326+0000 7f51ea600640 1 --2- 172.21.15.123:0/2782919950 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f51cc0b2870 0x7f51cc0b4d30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f51e4102a30 tx=0x7f51d0005f90 comp rx=0 tx=0).stop 2024-05-08T06:13:53.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.326+0000 7f51ea600640 1 -- 172.21.15.123:0/2782919950 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51e4103350 msgr2=0x7f51e4102cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:53.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.326+0000 7f51ea600640 1 --2- 172.21.15.123:0/2782919950 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51e4103350 0x7f51e4102cc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f51d8005ab0 tx=0x7f51d8003b90 comp rx=0 tx=0).stop 2024-05-08T06:13:53.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.326+0000 7f51ea600640 1 -- 172.21.15.123:0/2782919950 shutdown_connections 2024-05-08T06:13:53.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.326+0000 7f51ea600640 1 --2- 172.21.15.123:0/2782919950 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f51cc0b2870 0x7f51cc0b4d30 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:53.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.326+0000 7f51ea600640 1 --2- 172.21.15.123:0/2782919950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51e41085b0 0x7f51e4101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:53.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.326+0000 7f51ea600640 1 --2- 172.21.15.123:0/2782919950 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51e4105c80 0x7f51e4101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:53.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.326+0000 7f51ea600640 1 --2- 172.21.15.123:0/2782919950 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51e4103350 0x7f51e4102cc0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:53.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.326+0000 7f51ea600640 1 -- 172.21.15.123:0/2782919950 >> 172.21.15.123:0/2782919950 conn(0x7f51e40fcd60 msgr2=0x7f51e4103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:13:53.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.327+0000 7f51ea600640 1 -- 172.21.15.123:0/2782919950 shutdown_connections 2024-05-08T06:13:53.328 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.327+0000 7f51ea600640 1 -- 172.21.15.123:0/2782919950 wait complete. 2024-05-08T06:13:53.347 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-08T06:13:53.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.546+0000 7fb7c5400640 1 -- 172.21.15.123:0/133663201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7c010a200 msgr2=0x7fb7c010a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:53.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.546+0000 7fb7c5400640 1 --2- 172.21.15.123:0/133663201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7c010a200 0x7fb7c010a680 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8009900 tx=0x7fb7a802a400 comp rx=0 tx=0).stop 2024-05-08T06:13:53.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.546+0000 7fb7c5400640 1 -- 172.21.15.123:0/133663201 shutdown_connections 2024-05-08T06:13:53.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.546+0000 7fb7c5400640 1 --2- 172.21.15.123:0/133663201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7c010abc0 0x7fb7c0111450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:53.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.546+0000 7fb7c5400640 1 --2- 172.21.15.123:0/133663201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7c010a200 0x7fb7c010a680 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:53.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.546+0000 7fb7c5400640 1 --2- 172.21.15.123:0/133663201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7c0109010 0x7fb7c0109430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:53.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.546+0000 7fb7c5400640 1 -- 172.21.15.123:0/133663201 >> 172.21.15.123:0/133663201 conn(0x7fb7c0077fd0 msgr2=0x7fb7c007a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:13:53.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.547+0000 7fb7c5400640 1 -- 172.21.15.123:0/133663201 shutdown_connections 2024-05-08T06:13:53.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.547+0000 7fb7c5400640 1 -- 172.21.15.123:0/133663201 wait complete. 2024-05-08T06:13:53.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.548+0000 7fb7c5400640 1 Processor -- start 2024-05-08T06:13:53.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.548+0000 7fb7c5400640 1 -- start start 2024-05-08T06:13:53.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.548+0000 7fb7c5400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7c0109010 0x7fb7c019f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:53.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.548+0000 7fb7c5400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7c010a200 0x7fb7c019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:53.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.548+0000 7fb7c5400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7c010abc0 0x7fb7c01a6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:53.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.548+0000 7fb7c5400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c0114b50 con 0x7fb7c010a200 2024-05-08T06:13:53.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.548+0000 7fb7c5400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c01149d0 con 0x7fb7c0109010 2024-05-08T06:13:53.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.548+0000 7fb7c5400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c0114cd0 con 0x7fb7c010abc0 2024-05-08T06:13:53.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.549+0000 7fb7be800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7c0109010 0x7fb7c019f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:53.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.549+0000 7fb7b5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7c010a200 0x7fb7c019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:53.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.549+0000 7fb7be800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7c0109010 0x7fb7c019f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:46820/0 (socket says 172.21.15.123:46820) 2024-05-08T06:13:53.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.549+0000 7fb7be800640 1 -- 172.21.15.123:0/1172555814 learned_addr learned my addr 172.21.15.123:0/1172555814 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:13:53.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.549+0000 7fb7bf200640 1 --2- 172.21.15.123:0/1172555814 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7c010abc0 0x7fb7c01a6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:53.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.549+0000 7fb7be800640 1 -- 172.21.15.123:0/1172555814 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7c010abc0 msgr2=0x7fb7c01a6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:53.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.549+0000 7fb7be800640 1 --2- 172.21.15.123:0/1172555814 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7c010abc0 0x7fb7c01a6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:53.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.549+0000 7fb7be800640 1 -- 172.21.15.123:0/1172555814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7c010a200 msgr2=0x7fb7c019f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:53.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.549+0000 7fb7be800640 1 --2- 172.21.15.123:0/1172555814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7c010a200 0x7fb7c019f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:53.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.549+0000 7fb7be800640 1 -- 172.21.15.123:0/1172555814 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7c01a6e70 con 0x7fb7c0109010 2024-05-08T06:13:53.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.549+0000 7fb7b5e00640 1 --2- 172.21.15.123:0/1172555814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7c010a200 0x7fb7c019f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:13:53.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.549+0000 7fb7be800640 1 --2- 172.21.15.123:0/1172555814 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7c0109010 0x7fb7c019f250 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000e9c0 tx=0x7fb7b000edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:13:53.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.550+0000 7fb7b7e00640 1 -- 172.21.15.123:0/1172555814 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b000ccc0 con 0x7fb7c0109010 2024-05-08T06:13:53.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.550+0000 7fb7c5400640 1 -- 172.21.15.123:0/1172555814 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7c01a7160 con 0x7fb7c0109010 2024-05-08T06:13:53.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.550+0000 7fb7b7e00640 1 -- 172.21.15.123:0/1172555814 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7b00035f0 con 0x7fb7c0109010 2024-05-08T06:13:53.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.550+0000 7fb7b7e00640 1 -- 172.21.15.123:0/1172555814 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b000ce60 con 0x7fb7c0109010 2024-05-08T06:13:53.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.550+0000 7fb7c5400640 1 -- 172.21.15.123:0/1172555814 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7c01a76d0 con 0x7fb7c0109010 2024-05-08T06:13:53.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.551+0000 7fb7c5400640 1 -- 172.21.15.123:0/1172555814 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7c0110450 con 0x7fb7c0109010 2024-05-08T06:13:53.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.553+0000 7fb7b7e00640 1 -- 172.21.15.123:0/1172555814 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7fb7b00037a0 con 0x7fb7c0109010 2024-05-08T06:13:53.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.555+0000 7fb7b7e00640 1 --2- 172.21.15.123:0/1172555814 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fb7a40b27f0 0x7fb7a40b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:13:53.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.555+0000 7fb7b7e00640 1 -- 172.21.15.123:0/1172555814 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fb7b00e25f0 con 0x7fb7c0109010 2024-05-08T06:13:53.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.555+0000 7fb7b5e00640 1 --2- 172.21.15.123:0/1172555814 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fb7a40b27f0 0x7fb7a40b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:13:53.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.556+0000 7fb7b5e00640 1 --2- 172.21.15.123:0/1172555814 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fb7a40b27f0 0x7fb7a40b4cb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8000c00 tx=0x7fb7a80314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:13:53.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.559+0000 7fb7b7e00640 1 -- 172.21.15.123:0/1172555814 <== mon.2 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb7b00aa2c0 con 0x7fb7c0109010 2024-05-08T06:13:53.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.821+0000 7fb7c5400640 1 -- 172.21.15.123:0/1172555814 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fb7c00635b0 con 0x7fb7c0109010 2024-05-08T06:13:53.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.833+0000 7fb7b7e00640 1 -- 172.21.15.123:0/1172555814 <== mon.2 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+815651 (secure 0 0 0) 0x7fb7b00a9bc0 con 0x7fb7c0109010 2024-05-08T06:13:53.844 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:13:53.845 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-05-08T06:13:53.845 INFO:teuthology.orchestra.run.smithi123.stdout: "epoch": 20, 2024-05-08T06:13:53.845 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.845 INFO:teuthology.orchestra.run.smithi123.stdout: "active_gid": 14152, 2024-05-08T06:13:53.845 INFO:teuthology.orchestra.run.smithi123.stdout: "active_name": "x", 2024-05-08T06:13:53.845 INFO:teuthology.orchestra.run.smithi123.stdout: "active_addrs": { 2024-05-08T06:13:53.845 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-05-08T06:13:53.845 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:53.845 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-05-08T06:13:53.845 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6800", 2024-05-08T06:13:53.845 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 4169246006 2024-05-08T06:13:53.845 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.845 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:53.845 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-05-08T06:13:53.846 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:6801", 2024-05-08T06:13:53.846 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 4169246006 2024-05-08T06:13:53.846 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:53.846 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-05-08T06:13:53.846 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.846 INFO:teuthology.orchestra.run.smithi123.stdout: "active_addr": "172.21.15.123:6801/4169246006", 2024-05-08T06:13:53.846 INFO:teuthology.orchestra.run.smithi123.stdout: "active_change": "2024-05-08T05:57:30.888145+0000", 2024-05-08T06:13:53.846 INFO:teuthology.orchestra.run.smithi123.stdout: "active_mgr_features": 4540701547738038271, 2024-05-08T06:13:53.846 INFO:teuthology.orchestra.run.smithi123.stdout: "available": true, 2024-05-08T06:13:53.846 INFO:teuthology.orchestra.run.smithi123.stdout: "standbys": [ 2024-05-08T06:13:53.846 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:53.846 INFO:teuthology.orchestra.run.smithi123.stdout: "gid": 24109, 2024-05-08T06:13:53.847 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "z", 2024-05-08T06:13:53.847 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr_features": 4540701547738038271, 2024-05-08T06:13:53.847 INFO:teuthology.orchestra.run.smithi123.stdout: "available_modules": [ 2024-05-08T06:13:53.847 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:53.847 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "alerts", 2024-05-08T06:13:53.847 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:53.847 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:53.847 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:53.847 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-05-08T06:13:53.847 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-05-08T06:13:53.847 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:53.847 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.848 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.848 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-05-08T06:13:53.848 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.848 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.848 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.848 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "How frequently to reexamine health status", 2024-05-08T06:13:53.848 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.848 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.848 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.848 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.848 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:53.848 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:53.848 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.848 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.849 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.849 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:53.849 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.849 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.849 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:53.849 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:53.849 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:53.849 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:53.849 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:53.849 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:53.849 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:53.849 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:53.849 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.849 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.850 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.850 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.850 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.850 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:53.850 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:53.850 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.850 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.850 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.850 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:53.850 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.850 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.850 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.850 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.850 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.850 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.851 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.851 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.851 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:53.851 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:53.851 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.851 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.851 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.851 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:53.851 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.851 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.851 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:53.851 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:53.851 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:53.851 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:53.851 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:53.852 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:53.852 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:53.852 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:53.852 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.852 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.852 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.852 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.852 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.852 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:53.852 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:53.852 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.852 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.852 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.852 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:53.853 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.853 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.853 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.853 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.853 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.853 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.853 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.853 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.853 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_destination": { 2024-05-08T06:13:53.853 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_destination", 2024-05-08T06:13:53.853 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.853 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.853 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.853 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:53.853 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.854 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.854 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.854 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Email address to send alerts to", 2024-05-08T06:13:53.854 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.854 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.854 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.854 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.854 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_from_name": { 2024-05-08T06:13:53.854 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_from_name", 2024-05-08T06:13:53.854 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.854 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.854 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.854 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "Ceph", 2024-05-08T06:13:53.854 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.854 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.855 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.855 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Email From: name", 2024-05-08T06:13:53.855 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.855 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.855 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.855 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.855 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_host": { 2024-05-08T06:13:53.855 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_host", 2024-05-08T06:13:53.855 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.855 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.855 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.855 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:53.855 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.855 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.855 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.856 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "SMTP server", 2024-05-08T06:13:53.856 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.856 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.856 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.856 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.856 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_password": { 2024-05-08T06:13:53.856 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_password", 2024-05-08T06:13:53.856 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.856 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.856 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.856 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:53.856 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.856 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.856 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.856 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Password to authenticate with", 2024-05-08T06:13:53.856 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.857 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.857 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.857 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.857 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_port": { 2024-05-08T06:13:53.857 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_port", 2024-05-08T06:13:53.857 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:53.857 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.857 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.857 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "465", 2024-05-08T06:13:53.857 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.857 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.857 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.857 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "SMTP port", 2024-05-08T06:13:53.857 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.857 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.857 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.858 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.858 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_sender": { 2024-05-08T06:13:53.858 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_sender", 2024-05-08T06:13:53.858 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.858 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.858 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.858 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:53.858 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.858 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.858 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.858 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "SMTP envelope sender", 2024-05-08T06:13:53.858 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.858 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.858 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.858 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.859 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_ssl": { 2024-05-08T06:13:53.859 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_ssl", 2024-05-08T06:13:53.859 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.859 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.859 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.859 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:53.859 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.859 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.859 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.859 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-08T06:13:53.859 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.859 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.859 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.859 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.859 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_user": { 2024-05-08T06:13:53.859 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_user", 2024-05-08T06:13:53.860 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.860 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.860 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.860 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:53.860 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.860 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.860 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.860 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "User to authenticate as", 2024-05-08T06:13:53.860 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.860 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.860 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.860 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.860 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:53.860 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:53.860 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:53.860 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:53.861 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.861 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:53.861 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.861 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.861 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.861 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.861 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.861 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.861 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.861 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:53.861 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:53.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.861 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:53.861 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "balancer", 2024-05-08T06:13:53.861 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:53.861 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:53.862 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:53.862 INFO:teuthology.orchestra.run.smithi123.stdout: "active": { 2024-05-08T06:13:53.862 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "active", 2024-05-08T06:13:53.862 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.862 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.862 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.862 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:53.862 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.862 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.862 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.862 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "automatically balance PGs across cluster", 2024-05-08T06:13:53.862 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.862 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.862 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.862 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.862 INFO:teuthology.orchestra.run.smithi123.stdout: "begin_time": { 2024-05-08T06:13:53.863 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "begin_time", 2024-05-08T06:13:53.863 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.863 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.863 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.863 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0000", 2024-05-08T06:13:53.863 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.863 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.863 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.863 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "beginning time of day to automatically balance", 2024-05-08T06:13:53.863 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T06:13:53.863 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.863 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.863 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.863 INFO:teuthology.orchestra.run.smithi123.stdout: "begin_weekday": { 2024-05-08T06:13:53.863 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "begin_weekday", 2024-05-08T06:13:53.864 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-05-08T06:13:53.864 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.864 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.864 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:53.864 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "0", 2024-05-08T06:13:53.864 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "6", 2024-05-08T06:13:53.864 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.864 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-08T06:13:53.864 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T06:13:53.864 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.864 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.864 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.864 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_compat_max_iterations": { 2024-05-08T06:13:53.864 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crush_compat_max_iterations", 2024-05-08T06:13:53.864 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-05-08T06:13:53.865 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.865 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.865 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "25", 2024-05-08T06:13:53.865 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1", 2024-05-08T06:13:53.865 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "250", 2024-05-08T06:13:53.865 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.865 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-08T06:13:53.865 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.865 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.865 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.865 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.865 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_compat_metrics": { 2024-05-08T06:13:53.865 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crush_compat_metrics", 2024-05-08T06:13:53.865 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.866 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.866 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.866 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "pgs,objects,bytes", 2024-05-08T06:13:53.866 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.866 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.866 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.866 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-08T06:13:53.866 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-08T06:13:53.866 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.866 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.866 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.866 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_compat_step": { 2024-05-08T06:13:53.866 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crush_compat_step", 2024-05-08T06:13:53.866 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-05-08T06:13:53.867 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.867 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.867 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0.5", 2024-05-08T06:13:53.867 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "0.001", 2024-05-08T06:13:53.867 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "0.999", 2024-05-08T06:13:53.867 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.867 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "aggressiveness of optimization", 2024-05-08T06:13:53.867 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-08T06:13:53.867 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.867 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.867 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.867 INFO:teuthology.orchestra.run.smithi123.stdout: "end_time": { 2024-05-08T06:13:53.867 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "end_time", 2024-05-08T06:13:53.867 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.868 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.868 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.868 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "2359", 2024-05-08T06:13:53.868 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.868 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.868 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.868 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "ending time of day to automatically balance", 2024-05-08T06:13:53.868 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T06:13:53.868 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.868 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.868 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.868 INFO:teuthology.orchestra.run.smithi123.stdout: "end_weekday": { 2024-05-08T06:13:53.868 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "end_weekday", 2024-05-08T06:13:53.868 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-05-08T06:13:53.869 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.869 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.869 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:53.869 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "0", 2024-05-08T06:13:53.869 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "6", 2024-05-08T06:13:53.869 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.869 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-08T06:13:53.869 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T06:13:53.869 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.869 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.869 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.869 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:53.869 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:53.869 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.870 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.870 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.870 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:53.870 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.870 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.870 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:53.870 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:53.870 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:53.870 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:53.870 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:53.870 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:53.870 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:53.870 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:53.870 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.871 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.871 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.871 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.871 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.871 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:53.871 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:53.871 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.871 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.871 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.871 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:53.871 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.871 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.871 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.871 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.871 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.871 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.872 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.872 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.872 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:53.872 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:53.872 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.872 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.872 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.872 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:53.872 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.872 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.872 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:53.872 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:53.872 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:53.872 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:53.872 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:53.872 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:53.873 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:53.873 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:53.873 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.873 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.873 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.873 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.873 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.873 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:53.873 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:53.873 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.873 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.873 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.873 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:53.873 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.873 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.873 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.873 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.874 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.874 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.874 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.874 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.874 INFO:teuthology.orchestra.run.smithi123.stdout: "min_score": { 2024-05-08T06:13:53.874 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "min_score", 2024-05-08T06:13:53.874 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-05-08T06:13:53.874 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.874 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.874 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:53.874 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.874 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.874 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.875 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-08T06:13:53.875 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.875 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.875 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.875 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.875 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": { 2024-05-08T06:13:53.875 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mode", 2024-05-08T06:13:53.875 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.875 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.875 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.875 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "upmap", 2024-05-08T06:13:53.875 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.875 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.876 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:53.876 INFO:teuthology.orchestra.run.smithi123.stdout: "crush-compat", 2024-05-08T06:13:53.876 INFO:teuthology.orchestra.run.smithi123.stdout: "none", 2024-05-08T06:13:53.876 INFO:teuthology.orchestra.run.smithi123.stdout: "read", 2024-05-08T06:13:53.876 INFO:teuthology.orchestra.run.smithi123.stdout: "upmap", 2024-05-08T06:13:53.876 INFO:teuthology.orchestra.run.smithi123.stdout: "upmap-read" 2024-05-08T06:13:53.876 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:53.876 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Balancer mode", 2024-05-08T06:13:53.876 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.876 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.876 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.876 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.876 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_ids": { 2024-05-08T06:13:53.876 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "pool_ids", 2024-05-08T06:13:53.876 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.877 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.877 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.877 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:53.877 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.877 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.877 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.877 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-08T06:13:53.877 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.877 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.877 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.877 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.877 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-05-08T06:13:53.877 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-05-08T06:13:53.877 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:53.877 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.878 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.878 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-05-08T06:13:53.878 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.878 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.878 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.878 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-08T06:13:53.878 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.878 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.878 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.878 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.878 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:53.878 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:53.878 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:53.878 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:53.879 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.879 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:53.879 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.879 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.879 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.879 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.879 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.879 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.879 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.879 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.879 INFO:teuthology.orchestra.run.smithi123.stdout: "upmap_max_deviation": { 2024-05-08T06:13:53.879 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "upmap_max_deviation", 2024-05-08T06:13:53.879 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:53.879 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.879 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.879 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-05-08T06:13:53.880 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1", 2024-05-08T06:13:53.880 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.880 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.880 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-08T06:13:53.880 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-08T06:13:53.880 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.880 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.880 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.880 INFO:teuthology.orchestra.run.smithi123.stdout: "upmap_max_optimizations": { 2024-05-08T06:13:53.880 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "upmap_max_optimizations", 2024-05-08T06:13:53.880 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-05-08T06:13:53.880 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.880 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.881 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-05-08T06:13:53.881 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.881 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.881 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.881 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-08T06:13:53.881 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.881 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.881 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.881 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:53.881 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:53.881 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.881 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:53.881 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cephadm", 2024-05-08T06:13:53.881 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:53.881 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:53.882 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:53.882 INFO:teuthology.orchestra.run.smithi123.stdout: "agent_down_multiplier": { 2024-05-08T06:13:53.882 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "agent_down_multiplier", 2024-05-08T06:13:53.882 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-05-08T06:13:53.882 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.882 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.882 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "3.0", 2024-05-08T06:13:53.882 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.882 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.882 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.882 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-08T06:13:53.882 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.882 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.882 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.883 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.883 INFO:teuthology.orchestra.run.smithi123.stdout: "agent_refresh_rate": { 2024-05-08T06:13:53.883 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "agent_refresh_rate", 2024-05-08T06:13:53.883 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:53.883 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.883 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.883 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "20", 2024-05-08T06:13:53.883 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.883 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.883 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.883 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-08T06:13:53.883 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.883 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.884 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.884 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.884 INFO:teuthology.orchestra.run.smithi123.stdout: "agent_starting_port": { 2024-05-08T06:13:53.884 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "agent_starting_port", 2024-05-08T06:13:53.884 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:53.884 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.884 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.884 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "4721", 2024-05-08T06:13:53.884 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.884 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.884 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.884 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-08T06:13:53.884 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.884 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.884 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.885 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.885 INFO:teuthology.orchestra.run.smithi123.stdout: "allow_ptrace": { 2024-05-08T06:13:53.885 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "allow_ptrace", 2024-05-08T06:13:53.885 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.885 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.885 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.885 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:53.885 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.885 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.885 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.885 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-08T06:13:53.885 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-08T06:13:53.885 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.885 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.885 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.886 INFO:teuthology.orchestra.run.smithi123.stdout: "autotune_interval": { 2024-05-08T06:13:53.886 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "autotune_interval", 2024-05-08T06:13:53.886 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:53.886 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.886 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.886 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-05-08T06:13:53.886 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.886 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.886 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.886 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-08T06:13:53.886 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.886 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.886 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.887 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.887 INFO:teuthology.orchestra.run.smithi123.stdout: "autotune_memory_target_ratio": { 2024-05-08T06:13:53.887 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "autotune_memory_target_ratio", 2024-05-08T06:13:53.887 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-05-08T06:13:53.887 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.887 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.887 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0.7", 2024-05-08T06:13:53.887 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.887 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.887 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.887 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-08T06:13:53.887 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.887 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.888 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.888 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.888 INFO:teuthology.orchestra.run.smithi123.stdout: "cephadm_log_destination": { 2024-05-08T06:13:53.888 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cephadm_log_destination", 2024-05-08T06:13:53.888 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.888 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.888 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.888 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:53.888 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.888 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.888 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:53.888 INFO:teuthology.orchestra.run.smithi123.stdout: "file", 2024-05-08T06:13:53.888 INFO:teuthology.orchestra.run.smithi123.stdout: "file,syslog", 2024-05-08T06:13:53.888 INFO:teuthology.orchestra.run.smithi123.stdout: "syslog" 2024-05-08T06:13:53.888 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:53.888 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-08T06:13:53.889 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.889 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.889 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.889 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.889 INFO:teuthology.orchestra.run.smithi123.stdout: "cgroups_split": { 2024-05-08T06:13:53.889 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cgroups_split", 2024-05-08T06:13:53.889 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.889 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.889 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.889 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:53.889 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.889 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.889 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.889 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-08T06:13:53.890 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.890 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.890 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.890 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.890 INFO:teuthology.orchestra.run.smithi123.stdout: "config_checks_enabled": { 2024-05-08T06:13:53.890 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "config_checks_enabled", 2024-05-08T06:13:53.890 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.890 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.890 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.890 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:53.890 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.890 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.890 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.890 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-08T06:13:53.890 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.891 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.891 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.891 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.891 INFO:teuthology.orchestra.run.smithi123.stdout: "config_dashboard": { 2024-05-08T06:13:53.891 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "config_dashboard", 2024-05-08T06:13:53.891 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.891 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.891 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.891 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:53.891 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.891 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.891 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.891 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-08T06:13:53.891 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.891 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.892 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.892 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.892 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_alertmanager": { 2024-05-08T06:13:53.892 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_alertmanager", 2024-05-08T06:13:53.892 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.892 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.892 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.892 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-08T06:13:53.892 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.892 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.892 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.892 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Prometheus container image", 2024-05-08T06:13:53.892 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.893 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.893 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.893 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.893 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_base": { 2024-05-08T06:13:53.893 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_base", 2024-05-08T06:13:53.893 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.893 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.893 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.893 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-08T06:13:53.893 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.893 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.893 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.893 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Container image name, without the tag", 2024-05-08T06:13:53.894 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.894 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.894 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.894 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.894 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_elasticsearch": { 2024-05-08T06:13:53.894 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_elasticsearch", 2024-05-08T06:13:53.894 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.894 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.894 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.894 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-08T06:13:53.894 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.894 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.894 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.894 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "elasticsearch container image", 2024-05-08T06:13:53.895 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.895 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.895 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.895 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.895 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_grafana": { 2024-05-08T06:13:53.895 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_grafana", 2024-05-08T06:13:53.895 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.895 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.895 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.895 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-08T06:13:53.895 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.895 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.895 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.895 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Prometheus container image", 2024-05-08T06:13:53.896 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.896 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.896 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.896 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.896 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_haproxy": { 2024-05-08T06:13:53.896 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_haproxy", 2024-05-08T06:13:53.896 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.896 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.896 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.896 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-08T06:13:53.896 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.896 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.896 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.896 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "HAproxy container image", 2024-05-08T06:13:53.897 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.897 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.897 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.897 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.897 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_jaeger_agent": { 2024-05-08T06:13:53.897 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_jaeger_agent", 2024-05-08T06:13:53.897 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.897 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.897 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.897 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-08T06:13:53.897 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.897 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.897 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.897 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Jaeger agent container image", 2024-05-08T06:13:53.898 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.898 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.898 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.898 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.898 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_jaeger_collector": { 2024-05-08T06:13:53.898 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_jaeger_collector", 2024-05-08T06:13:53.898 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.898 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.898 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.898 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-08T06:13:53.898 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.898 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.898 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.899 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Jaeger collector container image", 2024-05-08T06:13:53.899 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.899 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.899 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.899 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.899 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_jaeger_query": { 2024-05-08T06:13:53.899 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_jaeger_query", 2024-05-08T06:13:53.899 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.899 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.899 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.899 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-08T06:13:53.899 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.899 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.899 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.900 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Jaeger query container image", 2024-05-08T06:13:53.900 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.900 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.900 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.900 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.900 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_keepalived": { 2024-05-08T06:13:53.900 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_keepalived", 2024-05-08T06:13:53.900 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.900 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.900 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.900 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-08T06:13:53.900 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.900 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.900 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.901 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Keepalived container image", 2024-05-08T06:13:53.901 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.901 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.901 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.901 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.901 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_loki": { 2024-05-08T06:13:53.901 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_loki", 2024-05-08T06:13:53.901 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.901 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.901 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.901 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-08T06:13:53.901 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.901 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.901 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.902 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Loki container image", 2024-05-08T06:13:53.902 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.902 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.902 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.902 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.902 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_node_exporter": { 2024-05-08T06:13:53.902 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_node_exporter", 2024-05-08T06:13:53.902 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.902 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.902 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.902 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-08T06:13:53.902 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.902 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.903 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.903 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Prometheus container image", 2024-05-08T06:13:53.903 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.903 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.903 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.903 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.903 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_nvmeof": { 2024-05-08T06:13:53.903 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_nvmeof", 2024-05-08T06:13:53.903 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.903 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.903 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.903 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-08T06:13:53.903 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.903 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.903 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.903 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Nvme-of container image", 2024-05-08T06:13:53.904 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.904 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.904 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.904 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.904 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_prometheus": { 2024-05-08T06:13:53.904 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_prometheus", 2024-05-08T06:13:53.904 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.904 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.904 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.904 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-08T06:13:53.904 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.904 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.904 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.904 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Prometheus container image", 2024-05-08T06:13:53.904 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.904 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.905 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.905 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.905 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_promtail": { 2024-05-08T06:13:53.905 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_promtail", 2024-05-08T06:13:53.905 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.905 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.905 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.905 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-08T06:13:53.905 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.905 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.905 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.905 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Promtail container image", 2024-05-08T06:13:53.905 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.905 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.905 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.905 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.906 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_samba": { 2024-05-08T06:13:53.906 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_samba", 2024-05-08T06:13:53.906 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.906 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.906 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.906 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-08T06:13:53.906 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.906 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.906 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.906 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Samba/SMB container image", 2024-05-08T06:13:53.906 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.906 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.906 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.907 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.907 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_snmp_gateway": { 2024-05-08T06:13:53.907 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_snmp_gateway", 2024-05-08T06:13:53.907 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.907 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.907 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.907 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-08T06:13:53.907 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.907 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.907 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.907 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "SNMP Gateway container image", 2024-05-08T06:13:53.907 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.907 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.907 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.907 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.907 INFO:teuthology.orchestra.run.smithi123.stdout: "container_init": { 2024-05-08T06:13:53.908 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_init", 2024-05-08T06:13:53.908 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.908 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.908 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.908 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:53.908 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.908 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.908 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.908 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Run podman/docker with `--init`", 2024-05-08T06:13:53.908 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.908 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.908 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.908 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.908 INFO:teuthology.orchestra.run.smithi123.stdout: "daemon_cache_timeout": { 2024-05-08T06:13:53.908 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "daemon_cache_timeout", 2024-05-08T06:13:53.908 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:53.909 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.909 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.909 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-05-08T06:13:53.909 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.909 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.909 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.909 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-08T06:13:53.909 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.909 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.909 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.909 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.909 INFO:teuthology.orchestra.run.smithi123.stdout: "default_cephadm_command_timeout": { 2024-05-08T06:13:53.909 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "default_cephadm_command_timeout", 2024-05-08T06:13:53.909 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:53.909 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.909 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.910 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "900", 2024-05-08T06:13:53.910 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.910 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.910 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.910 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-08T06:13:53.910 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.910 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.910 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.910 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.910 INFO:teuthology.orchestra.run.smithi123.stdout: "default_registry": { 2024-05-08T06:13:53.910 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "default_registry", 2024-05-08T06:13:53.910 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.910 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.910 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.911 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "docker.io", 2024-05-08T06:13:53.911 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.911 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.911 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.911 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-08T06:13:53.911 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.911 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.911 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.911 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.911 INFO:teuthology.orchestra.run.smithi123.stdout: "device_cache_timeout": { 2024-05-08T06:13:53.911 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "device_cache_timeout", 2024-05-08T06:13:53.911 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:53.911 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.911 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.912 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "1800", 2024-05-08T06:13:53.912 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.912 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.912 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.912 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "seconds to cache device inventory", 2024-05-08T06:13:53.912 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.912 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.912 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.912 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.912 INFO:teuthology.orchestra.run.smithi123.stdout: "device_enhanced_scan": { 2024-05-08T06:13:53.912 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "device_enhanced_scan", 2024-05-08T06:13:53.912 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.912 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.912 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.912 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:53.912 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.913 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.913 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.913 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-08T06:13:53.913 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.913 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.913 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.913 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.913 INFO:teuthology.orchestra.run.smithi123.stdout: "facts_cache_timeout": { 2024-05-08T06:13:53.913 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "facts_cache_timeout", 2024-05-08T06:13:53.913 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:53.913 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.913 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.913 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-05-08T06:13:53.913 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.913 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.913 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.913 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "seconds to cache host facts data", 2024-05-08T06:13:53.914 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.914 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.914 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.914 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.914 INFO:teuthology.orchestra.run.smithi123.stdout: "grafana_dashboards_path": { 2024-05-08T06:13:53.914 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "grafana_dashboards_path", 2024-05-08T06:13:53.914 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.914 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.914 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.914 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-08T06:13:53.914 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.914 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.914 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.914 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-08T06:13:53.915 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.915 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.915 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.915 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.915 INFO:teuthology.orchestra.run.smithi123.stdout: "host_check_interval": { 2024-05-08T06:13:53.915 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "host_check_interval", 2024-05-08T06:13:53.915 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:53.915 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.915 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.915 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-05-08T06:13:53.915 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.915 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.915 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.915 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to perform a host check", 2024-05-08T06:13:53.915 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.915 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.915 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.916 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.916 INFO:teuthology.orchestra.run.smithi123.stdout: "hw_monitoring": { 2024-05-08T06:13:53.916 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "hw_monitoring", 2024-05-08T06:13:53.916 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.916 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.916 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.916 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:53.916 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.916 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.916 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.916 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-08T06:13:53.916 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.916 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.916 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.916 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.916 INFO:teuthology.orchestra.run.smithi123.stdout: "inventory_list_all": { 2024-05-08T06:13:53.917 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "inventory_list_all", 2024-05-08T06:13:53.917 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.917 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.917 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.917 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:53.917 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.917 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.917 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.917 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-08T06:13:53.917 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.917 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.917 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.917 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.917 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:53.917 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:53.917 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.918 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.918 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.918 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:53.918 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.918 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.918 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:53.918 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:53.918 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:53.918 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:53.918 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:53.918 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:53.918 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:53.918 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:53.918 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.919 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.919 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.919 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.919 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.919 INFO:teuthology.orchestra.run.smithi123.stdout: "log_refresh_metadata": { 2024-05-08T06:13:53.919 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_refresh_metadata", 2024-05-08T06:13:53.919 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.919 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.919 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.919 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:53.919 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.919 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.919 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.919 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-08T06:13:53.919 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.919 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.920 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.920 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.920 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:53.920 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:53.920 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.920 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.920 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.920 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:53.920 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.920 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.920 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.920 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-08T06:13:53.920 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.920 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.920 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.921 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.921 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:53.921 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:53.921 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.921 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.921 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.921 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:53.921 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.921 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.921 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:53.921 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:53.921 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:53.921 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:53.921 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:53.922 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:53.922 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:53.922 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:53.922 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.922 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.922 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.922 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.922 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.922 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:53.922 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:53.922 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.922 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.922 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.923 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:53.923 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.923 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.923 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.923 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.923 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.923 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.923 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.923 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.923 INFO:teuthology.orchestra.run.smithi123.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-08T06:13:53.923 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-08T06:13:53.923 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.923 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.924 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.924 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:53.924 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.924 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.924 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.924 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-08T06:13:53.924 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.924 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.924 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.924 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.924 INFO:teuthology.orchestra.run.smithi123.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-08T06:13:53.924 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-08T06:13:53.924 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.924 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.925 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.925 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "*", 2024-05-08T06:13:53.925 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.925 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.925 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.925 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-08T06:13:53.925 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.925 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.925 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.925 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.925 INFO:teuthology.orchestra.run.smithi123.stdout: "max_count_per_host": { 2024-05-08T06:13:53.925 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_count_per_host", 2024-05-08T06:13:53.925 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:53.925 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.925 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.926 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-05-08T06:13:53.926 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.926 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.926 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.926 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "max number of daemons per service per host", 2024-05-08T06:13:53.926 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.926 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.926 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.926 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.926 INFO:teuthology.orchestra.run.smithi123.stdout: "max_osd_draining_count": { 2024-05-08T06:13:53.926 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_osd_draining_count", 2024-05-08T06:13:53.926 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:53.926 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.926 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.927 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-05-08T06:13:53.927 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.927 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.927 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.927 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-08T06:13:53.927 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.927 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.927 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.927 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.927 INFO:teuthology.orchestra.run.smithi123.stdout: "migration_current": { 2024-05-08T06:13:53.927 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "migration_current", 2024-05-08T06:13:53.927 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:53.927 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.927 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.928 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:53.928 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.928 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.928 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.928 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "internal - do not modify", 2024-05-08T06:13:53.928 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.928 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.928 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.928 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.928 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": { 2024-05-08T06:13:53.928 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mode", 2024-05-08T06:13:53.928 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.928 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.928 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.928 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "root", 2024-05-08T06:13:53.928 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.929 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.929 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:53.929 INFO:teuthology.orchestra.run.smithi123.stdout: "cephadm-package", 2024-05-08T06:13:53.929 INFO:teuthology.orchestra.run.smithi123.stdout: "root" 2024-05-08T06:13:53.929 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:53.929 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "mode for remote execution of cephadm", 2024-05-08T06:13:53.929 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.929 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.929 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.929 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.929 INFO:teuthology.orchestra.run.smithi123.stdout: "oob_default_addr": { 2024-05-08T06:13:53.929 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "oob_default_addr", 2024-05-08T06:13:53.929 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.929 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.930 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.930 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "169.254.1.1", 2024-05-08T06:13:53.930 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.930 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.930 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.930 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-08T06:13:53.930 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.930 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.930 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.930 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.930 INFO:teuthology.orchestra.run.smithi123.stdout: "prometheus_alerts_path": { 2024-05-08T06:13:53.930 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "prometheus_alerts_path", 2024-05-08T06:13:53.930 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.931 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.931 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.931 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-08T06:13:53.931 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.931 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.931 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.931 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-08T06:13:53.931 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.931 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.931 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.931 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.931 INFO:teuthology.orchestra.run.smithi123.stdout: "registry_insecure": { 2024-05-08T06:13:53.931 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "registry_insecure", 2024-05-08T06:13:53.931 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.931 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.932 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.932 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:53.932 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.932 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.932 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.932 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-08T06:13:53.932 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.932 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.932 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.932 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.932 INFO:teuthology.orchestra.run.smithi123.stdout: "registry_password": { 2024-05-08T06:13:53.932 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "registry_password", 2024-05-08T06:13:53.932 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.932 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.932 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.932 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:53.933 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.933 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.933 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.933 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-08T06:13:53.933 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.933 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.933 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.933 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.933 INFO:teuthology.orchestra.run.smithi123.stdout: "registry_url": { 2024-05-08T06:13:53.933 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "registry_url", 2024-05-08T06:13:53.933 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.933 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.933 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.933 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:53.933 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.933 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.934 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.934 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-08T06:13:53.934 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.934 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.934 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.934 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.934 INFO:teuthology.orchestra.run.smithi123.stdout: "registry_username": { 2024-05-08T06:13:53.934 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "registry_username", 2024-05-08T06:13:53.934 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.934 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.934 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.934 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:53.934 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.934 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.934 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.935 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-08T06:13:53.935 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.935 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.935 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.935 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.935 INFO:teuthology.orchestra.run.smithi123.stdout: "secure_monitoring_stack": { 2024-05-08T06:13:53.935 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "secure_monitoring_stack", 2024-05-08T06:13:53.935 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.935 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.935 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.935 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:53.935 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.935 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.935 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.935 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-08T06:13:53.935 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.936 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.936 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.936 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.936 INFO:teuthology.orchestra.run.smithi123.stdout: "service_discovery_port": { 2024-05-08T06:13:53.936 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "service_discovery_port", 2024-05-08T06:13:53.936 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:53.936 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.936 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.936 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8765", 2024-05-08T06:13:53.936 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.936 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.936 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.936 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "cephadm service discovery port", 2024-05-08T06:13:53.936 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.936 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.936 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.937 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.937 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:53.937 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:53.937 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:53.937 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:53.937 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.937 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:53.937 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.937 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.937 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.937 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.937 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.937 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.937 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.937 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.937 INFO:teuthology.orchestra.run.smithi123.stdout: "ssh_config_file": { 2024-05-08T06:13:53.938 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ssh_config_file", 2024-05-08T06:13:53.938 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.938 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.938 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.938 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:53.938 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.938 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.938 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.938 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-08T06:13:53.938 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.938 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.938 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.938 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.938 INFO:teuthology.orchestra.run.smithi123.stdout: "use_agent": { 2024-05-08T06:13:53.938 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "use_agent", 2024-05-08T06:13:53.939 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.939 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.939 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.939 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:53.939 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.939 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.939 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.939 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-08T06:13:53.939 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.939 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.939 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.939 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.939 INFO:teuthology.orchestra.run.smithi123.stdout: "use_repo_digest": { 2024-05-08T06:13:53.939 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "use_repo_digest", 2024-05-08T06:13:53.939 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.940 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.940 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.940 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:53.940 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.940 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.940 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.940 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-08T06:13:53.940 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.940 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.940 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.940 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.940 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_on_failed_host_check": { 2024-05-08T06:13:53.940 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_on_failed_host_check", 2024-05-08T06:13:53.940 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.940 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.940 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.941 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:53.941 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.941 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.941 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.941 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "raise a health warning if the host check fails", 2024-05-08T06:13:53.941 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.941 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.941 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.941 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.941 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_on_stray_daemons": { 2024-05-08T06:13:53.941 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_on_stray_daemons", 2024-05-08T06:13:53.941 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.941 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.941 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.941 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:53.942 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.942 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.942 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.942 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-08T06:13:53.942 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.942 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.942 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.942 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.942 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_on_stray_hosts": { 2024-05-08T06:13:53.942 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_on_stray_hosts", 2024-05-08T06:13:53.942 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.942 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.942 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.942 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:53.943 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.943 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.943 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.943 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-08T06:13:53.943 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.943 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.943 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.943 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:53.943 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:53.943 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.943 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:53.943 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crash", 2024-05-08T06:13:53.943 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:53.943 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:53.943 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:53.943 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:53.944 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:53.944 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.944 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.944 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.944 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:53.944 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.944 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.944 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:53.944 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:53.944 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:53.944 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:53.944 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:53.944 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:53.944 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:53.944 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:53.945 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.945 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.945 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.945 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.945 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.945 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:53.945 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:53.945 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.945 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.945 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.945 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:53.945 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.945 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.945 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.946 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.946 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.946 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.946 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.946 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.946 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:53.946 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:53.946 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.946 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.946 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.946 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:53.946 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.946 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.946 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:53.947 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:53.947 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:53.947 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:53.947 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:53.947 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:53.947 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:53.947 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:53.947 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.947 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.947 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.947 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.947 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.947 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:53.947 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:53.947 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.947 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.947 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.948 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:53.948 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.948 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.948 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.948 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.948 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.948 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.948 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.948 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.948 INFO:teuthology.orchestra.run.smithi123.stdout: "retain_interval": { 2024-05-08T06:13:53.948 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "retain_interval", 2024-05-08T06:13:53.948 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:53.948 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.948 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.949 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "31536000", 2024-05-08T06:13:53.949 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.949 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.949 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.949 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-08T06:13:53.949 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.949 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.949 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.949 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.949 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:53.949 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:53.949 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:53.949 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:53.949 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.949 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:53.950 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.950 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.950 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.950 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.950 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.950 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.950 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.950 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.950 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_recent_interval": { 2024-05-08T06:13:53.950 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_recent_interval", 2024-05-08T06:13:53.950 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:53.950 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.950 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.950 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "1209600", 2024-05-08T06:13:53.951 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.951 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.951 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.951 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-08T06:13:53.951 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.951 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.951 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.951 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:53.951 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:53.951 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.951 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:53.951 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "dashboard", 2024-05-08T06:13:53.951 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:53.951 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:53.951 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:53.951 INFO:teuthology.orchestra.run.smithi123.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-08T06:13:53.952 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-08T06:13:53.952 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:53.952 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.952 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.952 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-05-08T06:13:53.952 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.952 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.952 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.952 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.952 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.952 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.952 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.952 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.952 INFO:teuthology.orchestra.run.smithi123.stdout: "ALERTMANAGER_API_HOST": { 2024-05-08T06:13:53.953 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-08T06:13:53.953 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.953 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.953 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.953 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:53.953 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.953 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.953 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.953 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.953 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.953 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.953 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.954 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.954 INFO:teuthology.orchestra.run.smithi123.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-08T06:13:53.954 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-08T06:13:53.954 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.954 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.954 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.954 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:53.954 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.954 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.954 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.954 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.954 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.954 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.955 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.955 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.955 INFO:teuthology.orchestra.run.smithi123.stdout: "AUDIT_API_ENABLED": { 2024-05-08T06:13:53.955 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "AUDIT_API_ENABLED", 2024-05-08T06:13:53.955 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.955 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.955 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.955 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:53.955 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.955 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.955 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.955 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.955 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.955 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.955 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.956 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.956 INFO:teuthology.orchestra.run.smithi123.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-08T06:13:53.956 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-08T06:13:53.956 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.956 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.956 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.956 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:53.956 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.956 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.956 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.956 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.956 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.956 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.956 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.956 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.956 INFO:teuthology.orchestra.run.smithi123.stdout: "ENABLE_BROWSABLE_API": { 2024-05-08T06:13:53.957 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-08T06:13:53.957 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.957 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.957 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.957 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:53.957 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.957 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.957 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.957 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.957 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.957 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.957 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.957 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.957 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-08T06:13:53.957 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-08T06:13:53.957 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.957 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.958 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.958 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:53.958 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.958 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.958 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.958 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.958 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.958 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.958 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.958 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.958 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-08T06:13:53.958 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-08T06:13:53.958 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.959 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.959 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.959 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:53.959 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.959 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.959 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.959 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.959 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.959 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.959 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.959 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.959 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-08T06:13:53.959 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-08T06:13:53.959 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.959 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.959 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.959 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:53.960 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.960 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.960 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.960 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.960 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.960 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.960 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.960 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.960 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-08T06:13:53.960 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-08T06:13:53.960 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.960 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.960 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.960 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:53.960 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.961 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.961 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.961 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.961 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.961 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.961 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.961 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.961 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-08T06:13:53.961 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-08T06:13:53.961 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.961 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.961 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.961 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:53.961 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.961 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.961 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.962 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.962 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.962 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.962 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.962 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.962 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-08T06:13:53.962 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-08T06:13:53.962 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.962 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.962 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.962 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:53.962 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.962 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.962 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.963 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.963 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.963 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.963 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.963 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.963 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-08T06:13:53.963 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-08T06:13:53.963 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.963 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.963 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.963 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:53.963 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.963 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.963 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.963 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.963 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.964 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.964 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.964 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.964 INFO:teuthology.orchestra.run.smithi123.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-08T06:13:53.964 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-08T06:13:53.964 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.964 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.964 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.964 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:53.964 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.964 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.964 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.964 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.964 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.964 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.964 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.965 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.965 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_API_PASSWORD": { 2024-05-08T06:13:53.965 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-08T06:13:53.965 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.965 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.965 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.965 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "admin", 2024-05-08T06:13:53.965 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.965 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.965 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.965 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.965 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.965 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.965 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.965 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.965 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-08T06:13:53.966 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-08T06:13:53.966 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.966 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.966 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.966 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:53.966 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.966 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.966 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.966 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.966 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.966 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.966 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.966 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.966 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_API_URL": { 2024-05-08T06:13:53.967 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_API_URL", 2024-05-08T06:13:53.967 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.967 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.967 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.967 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:53.967 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.967 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.967 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.967 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.967 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.967 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.967 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.967 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.967 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_API_USERNAME": { 2024-05-08T06:13:53.967 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-08T06:13:53.967 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.968 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.968 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.968 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "admin", 2024-05-08T06:13:53.968 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.968 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.968 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.968 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.968 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.968 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.968 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.968 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.968 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-08T06:13:53.968 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-08T06:13:53.968 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.969 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.969 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.969 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:53.969 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.969 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.969 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.969 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.969 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.969 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.969 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.969 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.969 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-08T06:13:53.969 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-08T06:13:53.969 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.969 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.969 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.969 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:53.970 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.970 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.970 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.970 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.970 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.970 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.970 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.970 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.970 INFO:teuthology.orchestra.run.smithi123.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-08T06:13:53.970 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-08T06:13:53.970 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.970 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.970 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.970 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:53.971 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.971 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.971 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.971 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.971 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.971 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.971 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.971 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.971 INFO:teuthology.orchestra.run.smithi123.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-08T06:13:53.971 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-08T06:13:53.971 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.971 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.971 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.971 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:53.971 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.971 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.972 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.972 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.972 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.972 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.972 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.972 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.972 INFO:teuthology.orchestra.run.smithi123.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-08T06:13:53.972 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-08T06:13:53.972 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.972 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.972 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.972 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "[]", 2024-05-08T06:13:53.972 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.972 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.972 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.972 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.973 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.973 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.973 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.973 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.973 INFO:teuthology.orchestra.run.smithi123.stdout: "MULTICLUSTER_CONFIG": { 2024-05-08T06:13:53.973 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-08T06:13:53.973 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.973 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.973 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.973 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "{}", 2024-05-08T06:13:53.973 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.973 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.973 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.973 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.973 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.973 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.973 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.974 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.974 INFO:teuthology.orchestra.run.smithi123.stdout: "PROMETHEUS_API_HOST": { 2024-05-08T06:13:53.974 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-08T06:13:53.974 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.974 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.974 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.974 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:53.974 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.974 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.974 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.974 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.974 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.974 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.974 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.974 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.974 INFO:teuthology.orchestra.run.smithi123.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-08T06:13:53.975 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-08T06:13:53.975 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.975 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.975 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.975 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:53.975 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.975 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.975 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.975 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.975 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.975 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.975 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.975 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.975 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-08T06:13:53.975 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-08T06:13:53.976 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.976 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.976 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.976 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:53.976 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.976 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.976 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.976 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.976 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.976 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.976 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.976 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.976 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-08T06:13:53.976 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-08T06:13:53.976 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.976 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.977 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.977 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:53.977 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.977 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.977 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.977 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.977 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.977 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.977 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.977 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.977 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-08T06:13:53.977 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-08T06:13:53.977 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.977 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.977 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.977 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:53.978 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.978 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.978 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.978 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.978 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.978 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.978 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.978 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.978 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-08T06:13:53.978 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-08T06:13:53.978 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.978 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.978 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.978 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:53.979 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.979 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.979 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.979 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.979 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.979 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.979 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.979 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.979 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-08T06:13:53.979 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-08T06:13:53.979 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.979 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.979 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.979 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:53.979 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.980 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.980 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.980 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.980 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.980 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.980 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.980 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.980 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-08T06:13:53.980 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-08T06:13:53.980 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.980 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.980 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.980 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:53.980 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.980 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.981 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.981 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.981 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.981 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.981 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.981 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.981 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-08T06:13:53.981 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-08T06:13:53.981 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.981 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.981 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.981 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:53.981 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.982 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.982 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.982 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.982 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.982 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.982 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.982 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.982 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_ENABLED": { 2024-05-08T06:13:53.982 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-08T06:13:53.982 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.982 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.982 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.982 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:53.983 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.983 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.983 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.983 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.983 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.983 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.983 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.983 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.983 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-08T06:13:53.983 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-08T06:13:53.983 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.983 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.983 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.983 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-08T06:13:53.983 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.984 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.984 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.984 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.984 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.984 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.984 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.984 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.984 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-08T06:13:53.984 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-08T06:13:53.984 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:53.984 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.984 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.984 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-05-08T06:13:53.984 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.985 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.985 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.985 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.985 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.985 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.985 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.985 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.985 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-08T06:13:53.985 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-08T06:13:53.985 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:53.985 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.985 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.985 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8", 2024-05-08T06:13:53.985 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.985 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.986 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.986 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.986 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.986 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.986 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.986 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.986 INFO:teuthology.orchestra.run.smithi123.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-08T06:13:53.986 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-08T06:13:53.986 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:53.986 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.986 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.986 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "45", 2024-05-08T06:13:53.986 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.986 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.987 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.987 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.987 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.987 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.987 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.987 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.987 INFO:teuthology.orchestra.run.smithi123.stdout: "RGW_API_ACCESS_KEY": { 2024-05-08T06:13:53.987 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-08T06:13:53.987 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.987 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.987 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.987 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:53.987 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.987 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.987 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.988 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.988 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.988 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.988 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.988 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.988 INFO:teuthology.orchestra.run.smithi123.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-08T06:13:53.988 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-08T06:13:53.988 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.988 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.988 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.988 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "admin", 2024-05-08T06:13:53.988 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.988 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.988 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.988 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.989 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.989 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.989 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.989 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.989 INFO:teuthology.orchestra.run.smithi123.stdout: "RGW_API_SECRET_KEY": { 2024-05-08T06:13:53.989 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-08T06:13:53.989 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.989 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.989 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.989 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:53.989 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.989 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.989 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.989 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.989 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.989 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.990 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.990 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.990 INFO:teuthology.orchestra.run.smithi123.stdout: "RGW_API_SSL_VERIFY": { 2024-05-08T06:13:53.990 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-08T06:13:53.990 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.990 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.990 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.990 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:53.990 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.990 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.990 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.990 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.990 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.990 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.991 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.991 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.991 INFO:teuthology.orchestra.run.smithi123.stdout: "UNSAFE_TLS_v1_2": { 2024-05-08T06:13:53.991 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-08T06:13:53.991 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.991 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.991 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.991 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:53.991 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.991 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.991 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.991 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.991 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.991 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.992 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.992 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.992 INFO:teuthology.orchestra.run.smithi123.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-08T06:13:53.992 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-08T06:13:53.992 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:53.992 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.992 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.992 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:53.992 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.992 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.992 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.992 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.992 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.992 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.992 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.992 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.993 INFO:teuthology.orchestra.run.smithi123.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-08T06:13:53.993 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-08T06:13:53.993 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:53.993 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.993 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.993 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-05-08T06:13:53.993 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.993 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.993 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.993 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.993 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.993 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.993 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.993 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.993 INFO:teuthology.orchestra.run.smithi123.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-08T06:13:53.993 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-08T06:13:53.993 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:53.994 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.994 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.994 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-05-08T06:13:53.994 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.994 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.994 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.994 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.994 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.994 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.994 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.994 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.994 INFO:teuthology.orchestra.run.smithi123.stdout: "cross_origin_url": { 2024-05-08T06:13:53.994 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cross_origin_url", 2024-05-08T06:13:53.994 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.994 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.995 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.995 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:53.995 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.995 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.995 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.995 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.995 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.995 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.995 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.995 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.995 INFO:teuthology.orchestra.run.smithi123.stdout: "crt_file": { 2024-05-08T06:13:53.995 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crt_file", 2024-05-08T06:13:53.995 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.995 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.995 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.995 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:53.996 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.996 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.996 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.996 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.996 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.996 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.996 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.996 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.996 INFO:teuthology.orchestra.run.smithi123.stdout: "debug": { 2024-05-08T06:13:53.996 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "debug", 2024-05-08T06:13:53.996 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:53.996 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.996 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.996 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:53.996 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.996 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.997 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.997 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Enable/disable debug options", 2024-05-08T06:13:53.997 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.997 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.997 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.997 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.997 INFO:teuthology.orchestra.run.smithi123.stdout: "jwt_token_ttl": { 2024-05-08T06:13:53.997 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "jwt_token_ttl", 2024-05-08T06:13:53.997 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:53.997 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.997 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.997 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "28800", 2024-05-08T06:13:53.997 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.997 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.997 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.997 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.998 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.998 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.998 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.998 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.998 INFO:teuthology.orchestra.run.smithi123.stdout: "key_file": { 2024-05-08T06:13:53.998 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "key_file", 2024-05-08T06:13:53.998 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.998 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.998 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:53.998 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:53.998 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.998 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.998 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:53.998 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:53.998 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:53.999 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:53.999 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:53.999 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:53.999 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:53.999 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:53.999 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:53.999 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:53.999 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:53.999 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:53.999 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:53.999 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:53.999 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:53.999 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:53.999 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:53.999 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:53.999 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.000 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.000 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.000 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.000 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.000 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.000 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.000 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.000 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.000 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.000 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.000 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.000 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.000 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.000 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.000 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.000 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.001 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.001 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.001 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.001 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.001 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.001 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.001 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.001 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.001 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.001 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.001 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.001 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.001 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.001 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.001 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.001 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.002 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.002 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.002 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.002 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.002 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.002 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.002 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.002 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.002 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.002 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.002 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.002 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.002 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.002 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.003 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.003 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.003 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.003 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.003 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.003 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.003 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.003 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.003 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.003 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.003 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.003 INFO:teuthology.orchestra.run.smithi123.stdout: "motd": { 2024-05-08T06:13:54.003 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "motd", 2024-05-08T06:13:54.003 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.003 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.003 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.004 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.004 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.004 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.004 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.004 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "The message of the day", 2024-05-08T06:13:54.004 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.004 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.004 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.004 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.004 INFO:teuthology.orchestra.run.smithi123.stdout: "redirect_resolve_ip_addr": { 2024-05-08T06:13:54.004 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "redirect_resolve_ip_addr", 2024-05-08T06:13:54.004 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.004 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.004 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.004 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.005 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.005 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.005 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.005 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.005 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.005 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.005 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.005 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.005 INFO:teuthology.orchestra.run.smithi123.stdout: "server_addr": { 2024-05-08T06:13:54.005 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_addr", 2024-05-08T06:13:54.005 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.005 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.005 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.005 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "::", 2024-05-08T06:13:54.005 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.006 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.006 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.006 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.006 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.006 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.006 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.006 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.006 INFO:teuthology.orchestra.run.smithi123.stdout: "server_port": { 2024-05-08T06:13:54.006 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_port", 2024-05-08T06:13:54.006 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.006 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.006 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.006 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8080", 2024-05-08T06:13:54.006 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.006 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.007 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.007 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.007 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.007 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.007 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.007 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.007 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.007 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.007 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.007 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.007 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.007 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.007 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.007 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.007 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.007 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.008 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.008 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.008 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.008 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.008 INFO:teuthology.orchestra.run.smithi123.stdout: "ssl": { 2024-05-08T06:13:54.008 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ssl", 2024-05-08T06:13:54.008 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.008 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.008 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.008 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.008 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.008 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.008 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.008 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.008 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.008 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.008 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.009 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.009 INFO:teuthology.orchestra.run.smithi123.stdout: "ssl_server_port": { 2024-05-08T06:13:54.009 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ssl_server_port", 2024-05-08T06:13:54.009 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.009 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.009 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.009 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8443", 2024-05-08T06:13:54.009 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.009 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.009 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.009 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.009 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.009 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.009 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.009 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.010 INFO:teuthology.orchestra.run.smithi123.stdout: "standby_behaviour": { 2024-05-08T06:13:54.010 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "standby_behaviour", 2024-05-08T06:13:54.010 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.010 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.010 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.010 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "redirect", 2024-05-08T06:13:54.010 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.010 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.010 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.010 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.010 INFO:teuthology.orchestra.run.smithi123.stdout: "redirect" 2024-05-08T06:13:54.010 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.010 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.010 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.011 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.011 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.011 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.011 INFO:teuthology.orchestra.run.smithi123.stdout: "standby_error_status_code": { 2024-05-08T06:13:54.011 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "standby_error_status_code", 2024-05-08T06:13:54.011 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.011 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.011 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.011 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "500", 2024-05-08T06:13:54.011 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "400", 2024-05-08T06:13:54.011 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "599", 2024-05-08T06:13:54.011 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.011 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.011 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.012 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.012 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.012 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.012 INFO:teuthology.orchestra.run.smithi123.stdout: "url_prefix": { 2024-05-08T06:13:54.012 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "url_prefix", 2024-05-08T06:13:54.012 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.012 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.012 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.012 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.012 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.012 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.012 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.012 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.012 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.012 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.013 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.013 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.013 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.013 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.013 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.013 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "devicehealth", 2024-05-08T06:13:54.013 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.013 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.013 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.013 INFO:teuthology.orchestra.run.smithi123.stdout: "enable_monitoring": { 2024-05-08T06:13:54.013 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "enable_monitoring", 2024-05-08T06:13:54.013 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.013 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.013 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.014 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.014 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.014 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.014 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.014 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "monitor device health metrics", 2024-05-08T06:13:54.014 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.014 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.014 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.014 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.014 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.014 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.014 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.014 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.014 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.014 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.015 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.015 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.015 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.015 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.015 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.015 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.015 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.015 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.015 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.015 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.015 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.015 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.015 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.015 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.015 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.015 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.016 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.016 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.016 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.016 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.016 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.016 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.016 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.016 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.016 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.016 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.016 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.016 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.016 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.016 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.016 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.017 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.017 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.017 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.017 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.017 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.017 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.017 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.017 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.017 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.017 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.017 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.017 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.017 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.017 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.017 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.017 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.018 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.018 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.018 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.018 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.018 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.018 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.018 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.018 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.018 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.018 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.018 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.018 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.018 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.018 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.019 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.019 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.019 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.019 INFO:teuthology.orchestra.run.smithi123.stdout: "mark_out_threshold": { 2024-05-08T06:13:54.019 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mark_out_threshold", 2024-05-08T06:13:54.019 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.019 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.019 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.019 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "2419200", 2024-05-08T06:13:54.019 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.019 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.019 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.019 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-08T06:13:54.019 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.019 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.020 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.020 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.020 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_name": { 2024-05-08T06:13:54.020 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "pool_name", 2024-05-08T06:13:54.020 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.020 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.020 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.020 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "device_health_metrics", 2024-05-08T06:13:54.020 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.020 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.020 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.020 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-08T06:13:54.020 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.020 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.020 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.021 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.021 INFO:teuthology.orchestra.run.smithi123.stdout: "retention_period": { 2024-05-08T06:13:54.021 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "retention_period", 2024-05-08T06:13:54.021 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.021 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.021 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.021 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "15552000", 2024-05-08T06:13:54.021 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.021 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.021 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.021 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how long to retain device health metrics", 2024-05-08T06:13:54.021 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.021 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.021 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.022 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.022 INFO:teuthology.orchestra.run.smithi123.stdout: "scrape_frequency": { 2024-05-08T06:13:54.022 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "scrape_frequency", 2024-05-08T06:13:54.022 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.022 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.022 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.022 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "86400", 2024-05-08T06:13:54.022 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.022 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.022 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.022 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-08T06:13:54.022 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.022 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.022 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.023 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.023 INFO:teuthology.orchestra.run.smithi123.stdout: "self_heal": { 2024-05-08T06:13:54.023 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "self_heal", 2024-05-08T06:13:54.023 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.023 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.023 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.023 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.023 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.023 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.023 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.023 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-08T06:13:54.023 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.023 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.023 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.023 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.024 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-05-08T06:13:54.024 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-05-08T06:13:54.024 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.024 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.024 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.024 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-05-08T06:13:54.024 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.024 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.024 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.024 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to wake up and check device health", 2024-05-08T06:13:54.024 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.024 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.024 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.024 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.024 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.024 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.025 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.025 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.025 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.025 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.025 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.025 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.025 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.025 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.025 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.025 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.025 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.025 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.025 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_threshold": { 2024-05-08T06:13:54.025 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_threshold", 2024-05-08T06:13:54.026 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.026 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.026 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.026 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "7257600", 2024-05-08T06:13:54.026 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.026 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.026 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.026 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-08T06:13:54.026 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.026 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.026 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.026 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.026 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.026 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.027 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.027 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "diskprediction_local", 2024-05-08T06:13:54.027 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.027 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.027 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.027 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.027 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.027 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.027 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.027 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.027 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.027 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.027 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.027 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.027 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.028 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.028 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.028 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.028 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.028 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.028 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.028 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.028 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.028 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.028 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.028 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.028 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.028 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.028 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.028 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.028 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.029 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.029 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.029 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.029 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.029 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.029 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.029 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.029 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.029 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.029 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.029 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.029 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.029 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.029 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.029 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.030 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.030 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.030 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.030 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.030 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.030 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.030 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.030 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.030 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.030 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.030 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.030 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.030 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.030 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.030 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.031 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.031 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.031 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.031 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.031 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.031 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.031 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.031 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.031 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.031 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.031 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.031 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.031 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.031 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.031 INFO:teuthology.orchestra.run.smithi123.stdout: "predict_interval": { 2024-05-08T06:13:54.031 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "predict_interval", 2024-05-08T06:13:54.032 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.032 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.032 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.032 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "86400", 2024-05-08T06:13:54.032 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.032 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.032 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.032 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.032 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.032 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.032 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.032 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.032 INFO:teuthology.orchestra.run.smithi123.stdout: "predictor_model": { 2024-05-08T06:13:54.033 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "predictor_model", 2024-05-08T06:13:54.033 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.033 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.033 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.033 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "prophetstor", 2024-05-08T06:13:54.033 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.033 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.033 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.033 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.033 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.033 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.033 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.033 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.033 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-05-08T06:13:54.034 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-05-08T06:13:54.034 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.034 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.034 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.034 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-05-08T06:13:54.034 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.034 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.034 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.034 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.034 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.034 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.034 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.034 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.034 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.034 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.035 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.035 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.035 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.035 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.035 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.035 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.035 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.035 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.035 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.035 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.035 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.035 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.035 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.035 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.035 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.036 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "influx", 2024-05-08T06:13:54.036 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": false, 2024-05-08T06:13:54.036 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "influxdb python module not found", 2024-05-08T06:13:54.036 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.036 INFO:teuthology.orchestra.run.smithi123.stdout: "batch_size": { 2024-05-08T06:13:54.036 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "batch_size", 2024-05-08T06:13:54.036 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.036 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.036 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.036 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5000", 2024-05-08T06:13:54.036 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.036 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.036 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.036 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-08T06:13:54.036 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.036 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.037 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.037 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.037 INFO:teuthology.orchestra.run.smithi123.stdout: "database": { 2024-05-08T06:13:54.037 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "database", 2024-05-08T06:13:54.037 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.037 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.037 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.037 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "ceph", 2024-05-08T06:13:54.037 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.037 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.037 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.037 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-08T06:13:54.037 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.037 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.037 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.037 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.037 INFO:teuthology.orchestra.run.smithi123.stdout: "hostname": { 2024-05-08T06:13:54.038 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "hostname", 2024-05-08T06:13:54.038 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.038 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.038 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.038 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.038 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.038 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.038 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.038 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "InfluxDB server hostname", 2024-05-08T06:13:54.038 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.038 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.038 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.038 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.038 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-05-08T06:13:54.038 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-05-08T06:13:54.039 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.039 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.039 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.039 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "30", 2024-05-08T06:13:54.039 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "5", 2024-05-08T06:13:54.039 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.039 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.039 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-08T06:13:54.039 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.039 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.039 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.039 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.039 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.039 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.039 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.039 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.040 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.040 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.040 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.040 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.040 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.040 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.040 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.040 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.040 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.040 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.040 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.040 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.040 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.040 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.040 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.040 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.041 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.041 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.041 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.041 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.041 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.041 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.041 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.041 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.041 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.041 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.041 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.041 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.041 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.041 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.041 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.041 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.041 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.042 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.042 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.042 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.042 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.042 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.042 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.042 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.042 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.042 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.042 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.042 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.042 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.042 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.042 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.042 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.043 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.043 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.043 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.043 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.043 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.043 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.043 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.043 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.043 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.043 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.043 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.043 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.044 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.044 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.044 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.044 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.044 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.044 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.044 INFO:teuthology.orchestra.run.smithi123.stdout: "password": { 2024-05-08T06:13:54.044 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "password", 2024-05-08T06:13:54.044 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.044 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.044 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.044 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.044 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.044 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.044 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.045 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "password of InfluxDB server user", 2024-05-08T06:13:54.045 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.045 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.045 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.045 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.045 INFO:teuthology.orchestra.run.smithi123.stdout: "port": { 2024-05-08T06:13:54.045 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "port", 2024-05-08T06:13:54.045 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.045 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.045 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.045 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8086", 2024-05-08T06:13:54.045 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.045 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.045 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.045 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "InfluxDB server port", 2024-05-08T06:13:54.046 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.046 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.046 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.046 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.046 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.046 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.046 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.046 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.046 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.046 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.046 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.046 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.046 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.047 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.047 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.047 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.047 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.047 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.047 INFO:teuthology.orchestra.run.smithi123.stdout: "ssl": { 2024-05-08T06:13:54.047 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ssl", 2024-05-08T06:13:54.047 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.047 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.047 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.047 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "false", 2024-05-08T06:13:54.047 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.047 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.047 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.047 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T06:13:54.047 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.048 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.048 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.048 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.048 INFO:teuthology.orchestra.run.smithi123.stdout: "threads": { 2024-05-08T06:13:54.048 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "threads", 2024-05-08T06:13:54.048 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.048 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.048 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.048 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-05-08T06:13:54.048 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1", 2024-05-08T06:13:54.048 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "32", 2024-05-08T06:13:54.048 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.048 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-08T06:13:54.048 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.049 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.049 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.049 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.049 INFO:teuthology.orchestra.run.smithi123.stdout: "username": { 2024-05-08T06:13:54.049 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "username", 2024-05-08T06:13:54.049 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.049 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.049 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.049 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.049 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.049 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.049 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.049 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "username of InfluxDB server user", 2024-05-08T06:13:54.049 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.049 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.050 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.050 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.050 INFO:teuthology.orchestra.run.smithi123.stdout: "verify_ssl": { 2024-05-08T06:13:54.050 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "verify_ssl", 2024-05-08T06:13:54.050 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.050 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.050 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.050 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "true", 2024-05-08T06:13:54.050 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.050 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.050 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.050 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T06:13:54.050 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.050 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.050 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.051 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.051 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.051 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.051 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.051 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "insights", 2024-05-08T06:13:54.051 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.051 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.051 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.051 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.051 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.051 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.051 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.051 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.051 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.052 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.052 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.052 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.052 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.052 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.052 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.052 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.052 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.052 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.052 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.052 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.052 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.052 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.052 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.052 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.052 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.053 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.053 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.053 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.053 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.053 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.053 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.053 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.053 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.053 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.053 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.053 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.053 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.053 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.053 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.054 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.054 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.054 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.054 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.054 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.054 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.054 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.054 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.054 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.054 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.054 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.054 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.054 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.054 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.054 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.055 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.055 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.055 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.055 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.055 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.055 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.055 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.055 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.055 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.055 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.055 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.055 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.055 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.055 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.056 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.056 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.056 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.056 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.056 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.056 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.056 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.056 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.056 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.056 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.056 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.056 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.056 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.056 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.056 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.056 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.057 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.057 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.057 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.057 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.057 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.057 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.057 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "iostat", 2024-05-08T06:13:54.057 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.057 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.057 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.057 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.057 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.057 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.057 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.057 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.057 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.058 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.058 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.058 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.058 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.058 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.058 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.058 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.058 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.058 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.058 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.058 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.058 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.058 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.058 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.059 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.059 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.059 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.059 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.059 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.059 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.059 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.059 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.059 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.059 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.059 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.059 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.059 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.059 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.059 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.059 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.059 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.060 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.060 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.060 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.060 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.060 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.060 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.060 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.060 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.060 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.060 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.060 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.060 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.060 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.060 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.060 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.060 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.061 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.061 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.061 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.061 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.061 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.061 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.061 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.061 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.061 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.061 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.061 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.061 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.061 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.061 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.061 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.061 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.061 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.062 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.062 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.062 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.062 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.062 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.062 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.062 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.062 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.062 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.062 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.062 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.062 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.062 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.062 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.062 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.063 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.063 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.063 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "k8sevents", 2024-05-08T06:13:54.063 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.063 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.063 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.063 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_event_retention_days": { 2024-05-08T06:13:54.063 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ceph_event_retention_days", 2024-05-08T06:13:54.063 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.063 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.063 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.063 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "7", 2024-05-08T06:13:54.063 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.063 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.063 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.063 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-08T06:13:54.063 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.064 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.064 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.064 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.064 INFO:teuthology.orchestra.run.smithi123.stdout: "config_check_secs": { 2024-05-08T06:13:54.064 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "config_check_secs", 2024-05-08T06:13:54.064 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.064 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.064 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.064 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-05-08T06:13:54.064 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "10", 2024-05-08T06:13:54.064 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.064 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.064 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-08T06:13:54.064 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.064 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.065 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.065 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.065 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.065 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.065 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.065 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.065 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.065 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.065 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.065 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.065 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.065 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.065 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.065 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.065 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.065 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.066 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.066 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.066 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.066 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.066 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.066 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.066 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.066 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.066 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.066 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.066 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.066 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.066 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.066 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.066 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.067 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.067 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.067 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.067 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.067 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.067 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.067 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.067 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.067 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.067 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.067 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.067 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.067 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.067 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.068 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.068 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.068 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.068 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.068 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.068 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.068 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.068 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.068 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.068 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.068 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.068 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.068 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.069 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.069 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.069 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.069 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.069 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.069 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.069 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.069 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.069 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.069 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.069 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.069 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.069 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.070 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.070 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.070 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.070 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.070 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.070 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.070 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.070 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.070 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.070 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.070 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.070 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.070 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.070 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.071 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.071 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.071 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.071 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.071 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "localpool", 2024-05-08T06:13:54.071 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.071 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.071 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.071 INFO:teuthology.orchestra.run.smithi123.stdout: "failure_domain": { 2024-05-08T06:13:54.071 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "failure_domain", 2024-05-08T06:13:54.071 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.071 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.071 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.071 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "host", 2024-05-08T06:13:54.071 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.071 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.072 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.072 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "failure domain for any created local pool", 2024-05-08T06:13:54.072 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-08T06:13:54.072 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.072 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.072 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.072 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.072 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.072 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.072 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.072 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.072 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.072 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.072 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.072 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.073 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.073 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.073 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.073 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.073 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.073 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.073 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.073 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.073 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.073 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.073 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.073 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.073 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.073 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.073 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.073 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.074 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.074 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.074 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.074 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.074 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.074 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.074 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.074 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.074 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.074 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.074 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.074 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.074 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.074 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.075 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.075 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.075 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.075 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.075 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.075 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.075 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.075 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.075 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.075 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.075 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.075 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.075 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.075 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.075 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.076 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.076 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.076 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.076 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.076 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.076 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.076 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.076 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.076 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.076 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.076 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.076 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.076 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.076 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.076 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.077 INFO:teuthology.orchestra.run.smithi123.stdout: "min_size": { 2024-05-08T06:13:54.077 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "min_size", 2024-05-08T06:13:54.077 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.077 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.077 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.077 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.077 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.077 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.077 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.077 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "default min_size for any created local pool", 2024-05-08T06:13:54.077 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-08T06:13:54.077 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.077 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.077 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.077 INFO:teuthology.orchestra.run.smithi123.stdout: "num_rep": { 2024-05-08T06:13:54.077 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "num_rep", 2024-05-08T06:13:54.078 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.078 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.078 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.078 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "3", 2024-05-08T06:13:54.078 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.078 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.078 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.078 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "default replica count for any created local pool", 2024-05-08T06:13:54.078 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.078 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.078 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.078 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.078 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num": { 2024-05-08T06:13:54.078 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "pg_num", 2024-05-08T06:13:54.079 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.079 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.079 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.079 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "128", 2024-05-08T06:13:54.079 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.079 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.079 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.079 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "default pg_num for any created local pool", 2024-05-08T06:13:54.079 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.079 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.079 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.079 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.079 INFO:teuthology.orchestra.run.smithi123.stdout: "prefix": { 2024-05-08T06:13:54.079 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "prefix", 2024-05-08T06:13:54.079 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.080 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.080 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.080 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.080 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.080 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.080 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.080 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "name prefix for any created local pool", 2024-05-08T06:13:54.080 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.080 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.080 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.080 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.080 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.080 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.080 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.081 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.081 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.081 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.081 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.081 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.081 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.081 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.081 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.081 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.081 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.081 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.081 INFO:teuthology.orchestra.run.smithi123.stdout: "subtree": { 2024-05-08T06:13:54.081 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "subtree", 2024-05-08T06:13:54.081 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.082 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.082 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.082 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "rack", 2024-05-08T06:13:54.082 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.082 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.082 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.082 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-08T06:13:54.082 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-08T06:13:54.082 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.082 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.082 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.082 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.083 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.083 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.083 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mds_autoscaler", 2024-05-08T06:13:54.083 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.083 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.083 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.083 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.083 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.083 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.083 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.083 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.083 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.083 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.083 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.083 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.084 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.084 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.084 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.084 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.084 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.084 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.084 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.084 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.084 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.084 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.084 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.084 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.084 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.084 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.084 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.085 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.085 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.085 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.085 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.085 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.085 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.085 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.085 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.085 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.085 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.085 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.085 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.085 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.085 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.085 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.085 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.085 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.086 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.086 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.086 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.086 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.086 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.086 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.086 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.086 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.086 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.086 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.086 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.086 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.086 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.087 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.087 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.087 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.087 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.087 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.087 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.087 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.087 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.087 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.087 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.087 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.087 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.087 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.087 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.087 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.088 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.088 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.088 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.088 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.088 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.088 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.088 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.088 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.088 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.088 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.088 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.088 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.088 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.088 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.089 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.089 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.089 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.089 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.089 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mirroring", 2024-05-08T06:13:54.089 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.089 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.089 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.089 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.089 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.089 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.089 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.089 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.089 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.089 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.089 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.090 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.090 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.090 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.090 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.090 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.090 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.090 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.090 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.090 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.090 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.090 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.090 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.090 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.090 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.090 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.091 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.091 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.091 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.091 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.091 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.091 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.091 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.091 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.091 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.091 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.091 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.091 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.091 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.092 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.092 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.092 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.092 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.092 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.092 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.092 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.092 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.092 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.092 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.092 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.092 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.092 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.092 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.092 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.093 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.093 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.093 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.093 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.093 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.093 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.093 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.093 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.093 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.093 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.093 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.093 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.093 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.093 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.093 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.094 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.094 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.094 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.094 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.094 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.094 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.094 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.094 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.094 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.094 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.094 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.094 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.094 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.095 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.095 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.095 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.095 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.095 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.095 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.095 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.095 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.095 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "nfs", 2024-05-08T06:13:54.095 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.095 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.095 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.095 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.095 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.095 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.096 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.096 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.096 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.096 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.096 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.096 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.096 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.096 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.096 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.096 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.096 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.096 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.096 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.096 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.097 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.097 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.097 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.097 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.097 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.097 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.097 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.097 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.097 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.097 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.097 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.097 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.097 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.097 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.097 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.097 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.098 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.098 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.098 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.098 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.098 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.098 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.098 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.098 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.098 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.098 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.098 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.098 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.098 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.098 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.099 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.099 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.099 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.099 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.099 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.099 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.099 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.099 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.099 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.099 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.099 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.099 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.099 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.099 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.099 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.100 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.100 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.100 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.100 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.100 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.100 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.100 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.100 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.100 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.100 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.100 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.100 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.100 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.100 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.100 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.100 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.100 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.101 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.101 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.101 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.101 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.101 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.101 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.101 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.101 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.101 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "orchestrator", 2024-05-08T06:13:54.101 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.101 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.101 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.101 INFO:teuthology.orchestra.run.smithi123.stdout: "fail_fs": { 2024-05-08T06:13:54.101 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "fail_fs", 2024-05-08T06:13:54.102 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.102 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.102 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.102 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.102 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.102 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.102 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.102 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-08T06:13:54.102 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.102 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.102 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.102 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.102 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.102 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.102 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.103 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.103 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.103 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.103 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.103 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.103 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.103 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.103 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.103 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.103 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.103 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.103 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.103 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.103 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.103 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.103 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.104 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.104 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.104 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.104 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.104 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.104 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.104 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.104 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.104 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.104 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.104 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.104 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.104 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.104 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.105 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.105 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.105 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.105 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.105 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.105 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.105 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.105 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.105 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.105 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.105 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.105 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.105 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.105 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.106 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.106 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.106 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.106 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.106 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.106 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.106 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.106 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.106 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.106 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.106 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.106 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.106 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.106 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.107 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.107 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.107 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.107 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.107 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.107 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.107 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.107 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.107 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.107 INFO:teuthology.orchestra.run.smithi123.stdout: "orchestrator": { 2024-05-08T06:13:54.107 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "orchestrator", 2024-05-08T06:13:54.107 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.107 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.107 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.107 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.107 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.108 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.108 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.108 INFO:teuthology.orchestra.run.smithi123.stdout: "cephadm", 2024-05-08T06:13:54.108 INFO:teuthology.orchestra.run.smithi123.stdout: "rook", 2024-05-08T06:13:54.108 INFO:teuthology.orchestra.run.smithi123.stdout: "test_orchestrator" 2024-05-08T06:13:54.108 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.108 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Orchestrator backend", 2024-05-08T06:13:54.108 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.108 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.108 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.108 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.108 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.108 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.108 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.108 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.108 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.109 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.109 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.109 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.109 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.109 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.109 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.109 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.109 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.109 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.109 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.109 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.109 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.109 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "osd_perf_query", 2024-05-08T06:13:54.109 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.109 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.110 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.110 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.110 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.110 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.110 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.110 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.110 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.110 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.110 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.110 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.110 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.110 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.110 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.111 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.111 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.111 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.111 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.111 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.111 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.111 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.111 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.111 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.111 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.111 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.111 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.111 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.111 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.111 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.111 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.112 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.112 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.112 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.112 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.112 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.112 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.112 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.112 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.112 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.112 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.112 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.112 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.112 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.112 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.112 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.112 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.113 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.113 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.113 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.113 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.113 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.113 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.113 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.113 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.113 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.113 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.113 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.113 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.113 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.113 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.113 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.113 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.114 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.114 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.114 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.114 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.114 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.114 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.114 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.114 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.114 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.114 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.114 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.114 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.114 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.114 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.114 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.115 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.115 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.115 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.115 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.115 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.115 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.115 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.115 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.115 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.115 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.115 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.115 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.115 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "osd_support", 2024-05-08T06:13:54.115 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.115 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.116 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.116 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.116 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.116 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.116 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.116 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.116 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.116 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.116 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.116 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.116 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.116 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.116 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.116 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.116 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.116 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.116 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.117 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.117 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.117 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.117 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.117 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.117 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.117 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.117 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.117 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.117 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.117 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.117 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.117 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.117 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.117 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.118 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.118 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.118 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.118 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.118 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.118 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.118 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.118 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.118 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.118 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.118 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.118 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.118 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.119 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.119 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.119 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.119 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.119 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.119 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.119 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.119 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.119 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.119 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.119 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.119 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.119 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.119 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.119 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.119 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.119 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.120 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.120 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.120 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.120 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.120 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.120 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.120 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.120 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.120 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.120 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.120 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.120 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.120 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.120 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.121 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.121 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.121 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.121 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.121 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.121 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.121 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.121 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.121 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.121 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.121 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.121 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.121 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "pg_autoscaler", 2024-05-08T06:13:54.121 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.121 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.121 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.122 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.122 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.122 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.122 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.122 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.122 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.122 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.122 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.122 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.122 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.122 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.122 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.122 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.122 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.122 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.123 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.123 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.123 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.123 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.123 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.123 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.123 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.123 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.123 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.123 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.123 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.123 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.123 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.123 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.123 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.123 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.123 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.124 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.124 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.124 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.124 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.124 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.124 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.124 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.124 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.124 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.124 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.124 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.124 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.124 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.124 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.124 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.124 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.125 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.125 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.125 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.125 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.125 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.125 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.125 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.125 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.125 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.125 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.125 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.125 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.125 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.125 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.125 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.125 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.126 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.126 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.126 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.126 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.126 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.126 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.126 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-05-08T06:13:54.126 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-05-08T06:13:54.126 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.126 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.126 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.126 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-05-08T06:13:54.126 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.127 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.127 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.127 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.127 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.127 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.127 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.127 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.127 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.127 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.127 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.127 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.127 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.127 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.127 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.127 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.127 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.128 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.128 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.128 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.128 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.128 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.128 INFO:teuthology.orchestra.run.smithi123.stdout: "threshold": { 2024-05-08T06:13:54.128 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "threshold", 2024-05-08T06:13:54.128 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-05-08T06:13:54.128 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.128 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.128 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "3.0", 2024-05-08T06:13:54.128 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1.0", 2024-05-08T06:13:54.128 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.128 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.128 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "scaling threshold", 2024-05-08T06:13:54.128 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-08T06:13:54.129 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.129 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.129 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.129 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.129 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.129 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.129 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "progress", 2024-05-08T06:13:54.129 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.129 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.129 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.129 INFO:teuthology.orchestra.run.smithi123.stdout: "allow_pg_recovery_event": { 2024-05-08T06:13:54.129 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "allow_pg_recovery_event", 2024-05-08T06:13:54.129 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.129 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.130 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.130 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.130 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.130 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.130 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.130 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-08T06:13:54.130 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.130 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.130 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.130 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.130 INFO:teuthology.orchestra.run.smithi123.stdout: "enabled": { 2024-05-08T06:13:54.130 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "enabled", 2024-05-08T06:13:54.130 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.130 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.130 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.130 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.131 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.131 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.131 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.131 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.131 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.131 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.131 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.131 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.131 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.131 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.131 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.131 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.131 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.131 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.131 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.132 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.132 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.132 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.132 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.132 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.132 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.132 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.132 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.132 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.132 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.132 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.132 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.132 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.132 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.132 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.132 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.133 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.133 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.133 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.133 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.133 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.133 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.133 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.133 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.133 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.133 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.133 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.133 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.133 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.133 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.133 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.133 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.134 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.134 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.134 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.134 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.134 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.134 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.134 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.134 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.134 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.134 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.134 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.134 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.134 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.134 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.134 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.134 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.135 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.135 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.135 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.135 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.135 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.135 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.135 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.135 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.135 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.135 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.135 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.135 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.135 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.135 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.135 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.135 INFO:teuthology.orchestra.run.smithi123.stdout: "max_completed_events": { 2024-05-08T06:13:54.136 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_completed_events", 2024-05-08T06:13:54.136 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.136 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.136 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.136 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "50", 2024-05-08T06:13:54.136 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.136 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.136 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.136 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "number of past completed events to remember", 2024-05-08T06:13:54.136 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.136 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.136 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.136 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.136 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-05-08T06:13:54.136 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-05-08T06:13:54.137 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.137 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.137 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.137 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-05-08T06:13:54.137 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.137 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.137 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.137 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how long the module is going to sleep", 2024-05-08T06:13:54.137 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.137 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.137 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.137 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.137 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.137 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.137 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.137 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.137 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.138 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.138 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.138 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.138 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.138 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.138 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.138 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.138 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.138 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.138 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.138 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.138 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.138 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "prometheus", 2024-05-08T06:13:54.138 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.138 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.138 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.139 INFO:teuthology.orchestra.run.smithi123.stdout: "cache": { 2024-05-08T06:13:54.139 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cache", 2024-05-08T06:13:54.139 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.139 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.139 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.139 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.139 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.139 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.139 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.139 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.139 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.139 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.139 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.139 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.139 INFO:teuthology.orchestra.run.smithi123.stdout: "exclude_perf_counters": { 2024-05-08T06:13:54.139 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "exclude_perf_counters", 2024-05-08T06:13:54.140 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.140 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.140 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.140 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.140 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.140 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.140 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.140 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-08T06:13:54.140 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-08T06:13:54.140 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.140 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.140 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.140 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.140 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.140 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.140 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.140 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.141 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.141 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.141 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.141 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.141 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.141 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.141 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.141 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.141 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.141 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.141 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.141 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.142 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.142 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.142 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.142 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.142 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.142 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.142 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.142 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.142 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.142 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.142 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.142 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.142 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.142 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.142 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.143 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.143 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.143 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.143 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.143 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.143 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.143 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.143 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.143 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.143 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.143 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.143 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.143 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.143 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.143 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.143 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.144 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.144 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.144 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.144 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.144 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.144 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.144 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.144 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.144 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.144 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.144 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.144 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.144 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.144 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.144 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.144 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.145 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.145 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.145 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.145 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.145 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.145 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.145 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_stats_pools": { 2024-05-08T06:13:54.145 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rbd_stats_pools", 2024-05-08T06:13:54.145 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.145 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.145 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.145 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.145 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.145 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.145 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.145 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.146 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.146 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.146 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.146 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.146 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-08T06:13:54.146 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-08T06:13:54.146 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.146 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.146 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.146 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "300", 2024-05-08T06:13:54.146 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.146 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.146 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.146 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.146 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.146 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.147 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.147 INFO:teuthology.orchestra.run.smithi123.stdout: "scrape_interval": { 2024-05-08T06:13:54.147 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "scrape_interval", 2024-05-08T06:13:54.147 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-05-08T06:13:54.147 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.147 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.147 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "15.0", 2024-05-08T06:13:54.147 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.147 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.147 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.147 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.147 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.147 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.147 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.148 INFO:teuthology.orchestra.run.smithi123.stdout: "server_addr": { 2024-05-08T06:13:54.148 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_addr", 2024-05-08T06:13:54.148 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.148 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.148 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.148 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "::", 2024-05-08T06:13:54.148 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.148 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.148 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.148 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-08T06:13:54.148 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.148 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.148 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.149 INFO:teuthology.orchestra.run.smithi123.stdout: "server_port": { 2024-05-08T06:13:54.149 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_port", 2024-05-08T06:13:54.149 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.149 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.149 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.149 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "9283", 2024-05-08T06:13:54.149 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.149 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.149 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.149 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-08T06:13:54.149 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.149 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.149 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.149 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.149 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.149 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.150 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.150 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.150 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.150 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.150 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.150 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.150 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.150 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.150 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.150 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.150 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.150 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.150 INFO:teuthology.orchestra.run.smithi123.stdout: "stale_cache_strategy": { 2024-05-08T06:13:54.150 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "stale_cache_strategy", 2024-05-08T06:13:54.150 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.150 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.151 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.151 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "log", 2024-05-08T06:13:54.151 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.151 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.151 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.151 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.151 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.151 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.151 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.151 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.151 INFO:teuthology.orchestra.run.smithi123.stdout: "standby_behaviour": { 2024-05-08T06:13:54.151 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "standby_behaviour", 2024-05-08T06:13:54.151 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.151 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.151 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.151 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "default", 2024-05-08T06:13:54.151 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.152 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.152 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.152 INFO:teuthology.orchestra.run.smithi123.stdout: "default", 2024-05-08T06:13:54.152 INFO:teuthology.orchestra.run.smithi123.stdout: "error" 2024-05-08T06:13:54.152 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.152 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.152 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.152 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.152 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.152 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.152 INFO:teuthology.orchestra.run.smithi123.stdout: "standby_error_status_code": { 2024-05-08T06:13:54.152 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "standby_error_status_code", 2024-05-08T06:13:54.152 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.152 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.152 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.152 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "500", 2024-05-08T06:13:54.153 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "400", 2024-05-08T06:13:54.153 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "599", 2024-05-08T06:13:54.153 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.153 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.153 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.153 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.153 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.153 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.153 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.153 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.153 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.153 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rbd_support", 2024-05-08T06:13:54.153 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.153 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.153 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.153 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.154 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.154 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.154 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.154 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.154 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.154 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.154 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.154 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.154 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.154 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.154 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.154 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.154 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.154 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.154 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.154 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.155 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.155 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.155 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.155 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.155 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.155 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.155 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.155 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.155 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.155 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.155 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.155 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.155 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.155 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.155 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.155 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.156 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.156 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.156 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.156 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.156 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.156 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.156 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.156 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.156 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.156 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.156 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.156 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.156 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.156 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.156 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.156 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.157 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.157 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.157 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.157 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.157 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.157 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.157 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.157 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.157 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.157 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.157 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.157 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.157 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.157 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.157 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.157 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.157 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.158 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.158 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.158 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.158 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.158 INFO:teuthology.orchestra.run.smithi123.stdout: "max_concurrent_snap_create": { 2024-05-08T06:13:54.158 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_concurrent_snap_create", 2024-05-08T06:13:54.158 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.158 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.158 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.158 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-05-08T06:13:54.158 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.158 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.158 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.158 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.158 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.158 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.159 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.159 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.159 INFO:teuthology.orchestra.run.smithi123.stdout: "mirror_snapshot_schedule": { 2024-05-08T06:13:54.159 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mirror_snapshot_schedule", 2024-05-08T06:13:54.159 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.159 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.159 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.159 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.159 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.159 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.159 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.159 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.159 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.160 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.160 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.160 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.160 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.160 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.160 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.160 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.160 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.160 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.160 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.160 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.160 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.160 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.160 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.160 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.160 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.161 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.161 INFO:teuthology.orchestra.run.smithi123.stdout: "trash_purge_schedule": { 2024-05-08T06:13:54.161 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "trash_purge_schedule", 2024-05-08T06:13:54.161 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.161 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.161 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.161 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.161 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.161 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.161 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.161 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.161 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.161 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.161 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.161 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.162 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.162 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.162 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.162 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "restful", 2024-05-08T06:13:54.162 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.162 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.162 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.162 INFO:teuthology.orchestra.run.smithi123.stdout: "enable_auth": { 2024-05-08T06:13:54.162 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "enable_auth", 2024-05-08T06:13:54.162 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.162 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.162 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.162 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.162 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.162 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.162 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.163 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.163 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.163 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.163 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.163 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.163 INFO:teuthology.orchestra.run.smithi123.stdout: "key_file": { 2024-05-08T06:13:54.163 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "key_file", 2024-05-08T06:13:54.163 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.163 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.163 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.163 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.163 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.163 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.163 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.163 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.163 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.164 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.164 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.164 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.164 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.164 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.164 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.164 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.164 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.164 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.164 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.164 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.164 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.164 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.164 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.164 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.164 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.165 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.165 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.165 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.165 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.165 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.165 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.165 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.165 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.165 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.165 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.165 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.165 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.165 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.165 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.165 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.165 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.166 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.166 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.166 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.166 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.166 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.166 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.166 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.166 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.166 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.166 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.166 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.166 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.166 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.166 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.166 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.166 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.167 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.167 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.167 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.167 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.167 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.167 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.167 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.167 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.167 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.167 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.167 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.167 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.167 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.167 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.167 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.168 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.168 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.168 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.168 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.168 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.168 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.168 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.168 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.168 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.168 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.168 INFO:teuthology.orchestra.run.smithi123.stdout: "server_addr": { 2024-05-08T06:13:54.168 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_addr", 2024-05-08T06:13:54.168 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.168 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.168 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.168 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.169 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.169 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.169 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.169 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.169 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.169 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.169 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.169 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.169 INFO:teuthology.orchestra.run.smithi123.stdout: "server_port": { 2024-05-08T06:13:54.169 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_port", 2024-05-08T06:13:54.169 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.169 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.169 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.169 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.169 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.170 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.170 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.170 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.170 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.170 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.170 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.170 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.170 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.170 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.170 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.170 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.170 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.170 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.170 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.170 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.170 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.171 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.171 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.171 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.171 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.171 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.171 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.171 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.171 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.171 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rgw", 2024-05-08T06:13:54.171 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.171 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.171 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.171 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.171 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.172 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.172 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.172 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.172 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.172 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.172 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.172 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.172 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.172 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.172 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.172 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.172 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.172 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.172 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.172 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.172 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.173 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.173 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.173 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.173 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.173 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.173 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.173 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.173 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.173 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.173 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.173 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.173 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.173 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.173 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.173 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.173 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.174 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.174 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.174 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.174 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.174 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.174 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.174 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.174 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.174 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.174 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.174 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.174 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.174 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.174 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.174 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.175 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.175 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.175 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.175 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.175 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.175 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.175 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.175 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.175 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.175 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.175 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.175 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.175 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.176 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.176 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.176 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.176 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.176 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.176 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.176 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.176 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.176 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.176 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.176 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.176 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.176 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.176 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.177 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.177 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.177 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.177 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.177 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.177 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.177 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.177 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.177 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.177 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.177 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.177 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rook", 2024-05-08T06:13:54.177 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.177 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.178 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.178 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.178 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.178 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.178 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.178 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.178 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.178 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.178 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.178 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.178 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.178 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.178 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.178 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.179 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.179 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.179 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.179 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.179 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.179 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.179 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.179 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.179 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.179 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.179 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.179 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.179 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.179 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.180 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.180 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.180 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.180 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.180 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.180 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.180 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.180 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.180 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.180 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.180 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.180 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.180 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.181 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.181 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.181 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.181 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.181 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.181 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.181 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.181 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.181 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.181 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.181 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.181 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.181 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.181 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.182 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.182 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.182 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.182 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.182 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.182 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.182 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.182 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.182 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.182 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.182 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.182 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.182 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.182 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.183 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.183 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.183 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.183 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.183 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.183 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.183 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.183 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.183 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.183 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.183 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.183 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.183 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.183 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.184 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.184 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.184 INFO:teuthology.orchestra.run.smithi123.stdout: "storage_class": { 2024-05-08T06:13:54.184 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "storage_class", 2024-05-08T06:13:54.184 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.184 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.184 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.184 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "local", 2024-05-08T06:13:54.184 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.184 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.184 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.184 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-08T06:13:54.184 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.184 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.185 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.185 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.185 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.185 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.185 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.185 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "selftest", 2024-05-08T06:13:54.185 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.185 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.185 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.185 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.185 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.185 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.185 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.185 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.186 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.186 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.186 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.186 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.186 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.186 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.186 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.186 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.186 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.186 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.186 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.186 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.186 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.187 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.187 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.187 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.187 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.187 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.187 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.187 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.187 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.187 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.187 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.187 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.187 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.187 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.187 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.188 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.188 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.188 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.188 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.188 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.188 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.188 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.188 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.188 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.188 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.188 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.188 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.188 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.188 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.188 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.188 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.189 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.189 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.189 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.189 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.189 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.189 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.189 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.189 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.189 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.189 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.189 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.189 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.189 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.189 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.189 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.190 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.190 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.190 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.190 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.190 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.190 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.190 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.190 INFO:teuthology.orchestra.run.smithi123.stdout: "roption1": { 2024-05-08T06:13:54.190 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "roption1", 2024-05-08T06:13:54.190 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.190 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.190 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.190 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.190 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.191 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.191 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.191 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.191 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.191 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.191 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.191 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.191 INFO:teuthology.orchestra.run.smithi123.stdout: "roption2": { 2024-05-08T06:13:54.191 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "roption2", 2024-05-08T06:13:54.191 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.191 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.191 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.191 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "xyz", 2024-05-08T06:13:54.191 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.191 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.192 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.192 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.192 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.192 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.192 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.192 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.192 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption1": { 2024-05-08T06:13:54.192 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption1", 2024-05-08T06:13:54.192 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.192 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.192 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.192 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.192 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.192 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.192 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.193 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.193 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.193 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.193 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.193 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.193 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption2": { 2024-05-08T06:13:54.193 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption2", 2024-05-08T06:13:54.193 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.193 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.193 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.193 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.193 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.193 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.193 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.193 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.194 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.194 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.194 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.194 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.194 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption3": { 2024-05-08T06:13:54.194 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption3", 2024-05-08T06:13:54.194 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-05-08T06:13:54.194 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.194 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.194 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.194 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.194 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.194 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.194 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.194 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.195 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.195 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.195 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.195 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption4": { 2024-05-08T06:13:54.195 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption4", 2024-05-08T06:13:54.195 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.195 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.195 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.195 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.195 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.195 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.195 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.195 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.195 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.196 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.196 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.196 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.196 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption5": { 2024-05-08T06:13:54.196 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption5", 2024-05-08T06:13:54.196 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.196 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.196 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.196 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.196 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.196 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.196 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.196 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.197 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.197 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.197 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.197 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.197 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption6": { 2024-05-08T06:13:54.197 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption6", 2024-05-08T06:13:54.197 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.197 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.197 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.197 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.197 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.197 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.197 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.197 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.198 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.198 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.198 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.198 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.198 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption7": { 2024-05-08T06:13:54.198 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption7", 2024-05-08T06:13:54.198 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.198 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.198 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.198 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.198 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1", 2024-05-08T06:13:54.198 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "42", 2024-05-08T06:13:54.199 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.199 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.199 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.199 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.199 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.199 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.199 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.199 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.199 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.199 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.199 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.199 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.199 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.199 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.200 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.200 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.200 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.200 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.200 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.200 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.200 INFO:teuthology.orchestra.run.smithi123.stdout: "testkey": { 2024-05-08T06:13:54.200 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "testkey", 2024-05-08T06:13:54.200 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.200 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.200 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.200 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.200 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.200 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.201 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.201 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.201 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.201 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.201 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.201 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.201 INFO:teuthology.orchestra.run.smithi123.stdout: "testlkey": { 2024-05-08T06:13:54.201 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "testlkey", 2024-05-08T06:13:54.201 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.201 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.201 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.201 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.201 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.201 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.201 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.202 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.202 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.202 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.202 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.202 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.202 INFO:teuthology.orchestra.run.smithi123.stdout: "testnewline": { 2024-05-08T06:13:54.202 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "testnewline", 2024-05-08T06:13:54.202 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.202 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.202 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.202 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.202 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.202 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.202 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.202 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.203 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.203 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.203 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.203 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.203 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.203 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.203 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.203 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smb", 2024-05-08T06:13:54.203 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.203 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.203 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.203 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.203 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.204 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.204 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.204 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.204 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.204 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.204 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.204 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.204 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.204 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.204 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.204 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.204 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.204 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.205 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.205 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.205 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.205 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.205 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.205 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.205 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.205 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.205 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.205 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.205 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.205 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.205 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.205 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.206 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.206 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.206 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.206 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.206 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.206 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.206 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.206 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.206 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.206 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.206 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.206 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.206 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.207 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.207 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.207 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.207 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.207 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.207 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.207 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.207 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.207 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.207 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.207 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.207 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.207 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.207 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.208 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.208 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.208 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.208 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.208 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.208 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.208 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.208 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.208 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.208 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.208 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.208 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.208 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.208 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.208 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.209 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.209 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.209 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.209 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.209 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.209 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.209 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.209 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.209 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.209 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.209 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.209 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.209 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.210 INFO:teuthology.orchestra.run.smithi123.stdout: "update_orchestration": { 2024-05-08T06:13:54.210 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "update_orchestration", 2024-05-08T06:13:54.210 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.210 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.210 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.210 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.210 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.210 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.210 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.210 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-08T06:13:54.210 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.210 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.210 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.210 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.211 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.211 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.211 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.211 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "snap_schedule", 2024-05-08T06:13:54.211 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.211 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.211 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.211 INFO:teuthology.orchestra.run.smithi123.stdout: "allow_m_granularity": { 2024-05-08T06:13:54.211 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "allow_m_granularity", 2024-05-08T06:13:54.211 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.211 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.211 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.211 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.212 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.212 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.212 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.212 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "allow minute scheduled snapshots", 2024-05-08T06:13:54.212 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.212 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.212 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.212 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.212 INFO:teuthology.orchestra.run.smithi123.stdout: "dump_on_update": { 2024-05-08T06:13:54.212 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "dump_on_update", 2024-05-08T06:13:54.212 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.212 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.212 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.212 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.213 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.213 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.213 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.213 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "dump database to debug log on update", 2024-05-08T06:13:54.213 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.213 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.213 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.213 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.213 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.213 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.213 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.213 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.213 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.214 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.214 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.214 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.214 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.214 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.214 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.214 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.214 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.214 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.214 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.214 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.214 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.214 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.215 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.215 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.215 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.215 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.215 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.215 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.215 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.215 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.215 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.215 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.215 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.215 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.215 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.215 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.216 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.216 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.216 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.216 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.216 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.216 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.216 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.216 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.216 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.216 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.216 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.216 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.216 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.216 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.217 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.217 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.217 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.217 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.217 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.217 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.217 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.217 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.217 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.217 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.217 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.217 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.217 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.218 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.218 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.218 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.218 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.218 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.218 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.218 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.218 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.218 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.218 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.218 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.218 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.218 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.218 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.219 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.219 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.219 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.219 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.219 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.219 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.219 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.219 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.219 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.219 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.219 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.219 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.219 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.219 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.219 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "stats", 2024-05-08T06:13:54.220 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.220 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.220 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.220 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.220 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.220 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.220 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.220 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.220 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.220 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.220 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.220 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.220 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.220 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.220 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.221 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.221 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.221 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.221 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.221 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.221 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.221 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.221 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.221 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.221 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.221 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.221 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.221 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.221 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.221 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.222 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.222 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.222 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.222 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.222 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.222 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.222 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.222 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.222 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.222 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.222 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.222 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.222 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.223 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.223 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.223 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.223 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.223 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.223 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.223 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.223 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.223 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.223 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.223 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.223 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.223 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.223 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.223 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.223 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.224 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.224 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.224 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.224 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.224 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.224 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.224 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.224 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.224 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.224 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.224 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.224 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.224 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.225 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.225 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.225 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.225 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.225 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.225 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.225 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.225 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.225 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.225 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.225 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.225 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.225 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.226 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.226 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.226 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.226 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.226 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.226 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "status", 2024-05-08T06:13:54.226 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.226 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.226 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.226 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.226 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.226 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.226 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.227 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.227 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.227 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.227 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.227 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.227 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.227 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.227 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.227 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.227 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.227 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.227 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.227 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.228 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.228 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.228 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.228 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.228 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.228 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.228 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.228 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.228 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.228 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.228 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.228 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.228 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.228 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.229 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.229 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.229 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.229 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.229 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.229 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.229 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.229 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.229 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.229 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.229 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.229 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.229 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.230 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.230 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.230 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.230 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.230 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.230 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.230 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.230 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.230 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.230 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.230 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.230 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.230 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.230 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.231 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.231 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.231 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.231 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.231 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.231 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.231 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.231 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.231 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.231 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.231 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.231 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.231 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.231 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.231 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.232 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.232 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.232 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.232 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.232 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.232 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.232 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.232 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.232 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.232 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.232 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.232 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.232 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.232 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.232 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "telegraf", 2024-05-08T06:13:54.233 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.233 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.233 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.233 INFO:teuthology.orchestra.run.smithi123.stdout: "address": { 2024-05-08T06:13:54.233 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "address", 2024-05-08T06:13:54.233 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.233 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.233 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.233 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-08T06:13:54.233 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.233 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.233 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.233 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.233 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.233 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.233 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.234 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.234 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-05-08T06:13:54.234 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-05-08T06:13:54.234 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.234 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.234 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.234 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "15", 2024-05-08T06:13:54.234 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.234 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.234 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.234 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.234 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.234 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.234 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.235 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.235 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.235 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.235 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.235 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.235 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.235 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.235 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.235 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.235 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.235 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.235 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.235 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.235 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.235 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.236 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.236 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.236 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.236 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.236 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.236 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.236 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.236 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.236 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.236 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.236 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.236 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.236 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.236 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.236 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.236 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.237 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.237 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.237 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.237 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.237 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.237 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.237 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.237 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.237 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.237 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.237 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.237 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.237 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.237 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.237 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.237 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.238 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.238 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.238 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.238 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.238 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.238 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.238 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.238 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.238 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.238 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.238 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.238 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.238 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.238 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.239 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.239 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.239 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.239 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.239 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.239 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.239 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.239 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.239 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.239 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.239 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.239 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.239 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.239 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.239 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.239 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.240 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.240 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.240 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.240 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.240 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.240 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.240 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.240 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.240 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.240 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.240 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.240 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "telemetry", 2024-05-08T06:13:54.240 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.240 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.240 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.240 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_basic": { 2024-05-08T06:13:54.240 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_basic", 2024-05-08T06:13:54.241 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.241 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.241 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.241 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.241 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.241 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.241 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.241 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-08T06:13:54.241 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.241 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.241 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.241 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.241 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_crash": { 2024-05-08T06:13:54.241 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_crash", 2024-05-08T06:13:54.241 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.241 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.242 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.242 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.242 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.242 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.242 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.242 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-08T06:13:54.242 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.242 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.242 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.242 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.242 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_device": { 2024-05-08T06:13:54.242 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_device", 2024-05-08T06:13:54.242 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.242 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.242 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.243 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.243 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.243 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.243 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.243 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-08T06:13:54.243 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.243 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.243 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.243 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.243 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_ident": { 2024-05-08T06:13:54.243 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_ident", 2024-05-08T06:13:54.243 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.243 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.243 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.243 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.243 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.244 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.244 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.244 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-08T06:13:54.244 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.244 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.244 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.244 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.244 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_perf": { 2024-05-08T06:13:54.244 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_perf", 2024-05-08T06:13:54.244 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.244 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.244 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.244 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.244 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.244 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.244 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.244 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-08T06:13:54.245 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.245 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.245 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.245 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.245 INFO:teuthology.orchestra.run.smithi123.stdout: "contact": { 2024-05-08T06:13:54.245 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "contact", 2024-05-08T06:13:54.245 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.245 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.245 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.245 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.245 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.245 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.245 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.245 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.245 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.245 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.245 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.246 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.246 INFO:teuthology.orchestra.run.smithi123.stdout: "description": { 2024-05-08T06:13:54.246 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "description", 2024-05-08T06:13:54.246 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.246 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.246 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.246 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.246 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.246 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.246 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.246 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.246 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.246 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.246 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.247 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.247 INFO:teuthology.orchestra.run.smithi123.stdout: "device_url": { 2024-05-08T06:13:54.247 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "device_url", 2024-05-08T06:13:54.247 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.247 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.247 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.247 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-08T06:13:54.247 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.247 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.247 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.247 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.247 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.247 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.247 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.247 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.247 INFO:teuthology.orchestra.run.smithi123.stdout: "enabled": { 2024-05-08T06:13:54.248 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "enabled", 2024-05-08T06:13:54.248 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.248 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.248 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.248 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.248 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.248 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.248 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.248 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.248 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.248 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.248 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.248 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.248 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-05-08T06:13:54.248 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-05-08T06:13:54.248 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.249 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.249 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.249 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "24", 2024-05-08T06:13:54.249 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "8", 2024-05-08T06:13:54.249 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.249 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.249 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.249 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.249 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.249 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.249 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.249 INFO:teuthology.orchestra.run.smithi123.stdout: "last_opt_revision": { 2024-05-08T06:13:54.249 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "last_opt_revision", 2024-05-08T06:13:54.249 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.249 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.249 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.250 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "1", 2024-05-08T06:13:54.250 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.250 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.250 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.250 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.250 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.250 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.250 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.250 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.250 INFO:teuthology.orchestra.run.smithi123.stdout: "leaderboard": { 2024-05-08T06:13:54.250 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "leaderboard", 2024-05-08T06:13:54.250 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.250 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.250 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.251 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.251 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.251 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.251 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.251 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.251 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.251 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.251 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.251 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.251 INFO:teuthology.orchestra.run.smithi123.stdout: "leaderboard_description": { 2024-05-08T06:13:54.251 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "leaderboard_description", 2024-05-08T06:13:54.251 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.251 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.251 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.251 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.252 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.252 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.252 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.252 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.252 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.252 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.252 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.252 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.252 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.252 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.252 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.252 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.252 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.252 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.252 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.252 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.253 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.253 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.253 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.253 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.253 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.253 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.253 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.253 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.253 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.253 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.253 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.253 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.253 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.253 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.253 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.253 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.254 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.254 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.254 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.254 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.254 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.254 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.254 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.254 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.254 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.254 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.254 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.254 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.254 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.254 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.255 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.255 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.255 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.255 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.255 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.255 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.255 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.255 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.255 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.255 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.255 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.255 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.255 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.255 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.255 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.255 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.255 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.256 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.256 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.256 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.256 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.256 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.256 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.256 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.256 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.256 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.256 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.256 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.256 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.256 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.256 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.257 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.257 INFO:teuthology.orchestra.run.smithi123.stdout: "organization": { 2024-05-08T06:13:54.257 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "organization", 2024-05-08T06:13:54.257 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.257 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.257 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.257 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.257 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.257 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.257 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.257 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.257 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.257 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.257 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.257 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.257 INFO:teuthology.orchestra.run.smithi123.stdout: "proxy": { 2024-05-08T06:13:54.258 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "proxy", 2024-05-08T06:13:54.258 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.258 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.258 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.258 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.258 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.258 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.258 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.258 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.258 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.258 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.258 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.258 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.258 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.258 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.259 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.259 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.259 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.259 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.259 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.259 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.259 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.259 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.259 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.259 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.259 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.259 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.259 INFO:teuthology.orchestra.run.smithi123.stdout: "url": { 2024-05-08T06:13:54.259 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "url", 2024-05-08T06:13:54.259 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.259 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.260 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.260 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-08T06:13:54.260 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.260 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.260 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.260 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.260 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.260 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.260 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.260 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.260 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.260 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.260 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.260 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "test_orchestrator", 2024-05-08T06:13:54.260 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.260 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.261 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.261 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.261 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.261 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.261 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.261 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.261 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.261 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.261 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.261 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.261 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.261 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.261 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.261 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.261 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.261 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.262 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.262 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.262 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.262 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.262 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.262 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.262 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.262 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.262 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.262 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.262 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.262 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.262 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.262 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.262 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.263 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.263 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.263 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.263 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.263 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.263 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.263 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.263 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.263 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.263 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.263 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.263 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.263 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.263 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.263 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.263 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.264 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.264 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.264 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.264 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.264 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.264 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.264 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.264 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.264 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.264 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.264 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.264 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.264 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.264 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.264 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.264 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.265 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.265 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.265 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.265 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.265 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.265 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.265 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.265 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.265 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.265 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.265 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.265 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.265 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.265 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.265 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.265 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.266 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.266 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.266 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.266 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.266 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.266 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.266 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.266 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.266 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.266 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "volumes", 2024-05-08T06:13:54.266 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.266 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.266 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.266 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.266 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.267 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.267 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.267 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.267 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.267 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.267 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.267 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.267 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.267 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.267 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.267 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.267 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.267 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.267 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.267 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.267 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.268 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.268 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.268 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.268 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.268 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.268 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.268 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.268 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.268 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.268 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.268 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.268 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.268 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.268 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.268 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.268 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.269 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.269 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.269 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.269 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.269 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.269 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.269 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.269 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.269 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.269 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.269 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.269 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.269 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.269 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.269 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.270 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.270 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.270 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.270 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.270 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.270 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.270 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.270 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.270 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.270 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.270 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.270 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.270 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.270 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.271 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.271 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.271 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.271 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.271 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.271 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.271 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.271 INFO:teuthology.orchestra.run.smithi123.stdout: "max_concurrent_clones": { 2024-05-08T06:13:54.271 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_concurrent_clones", 2024-05-08T06:13:54.271 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.271 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.271 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.271 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "4", 2024-05-08T06:13:54.271 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.271 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.272 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.272 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-08T06:13:54.272 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.272 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.272 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.272 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.272 INFO:teuthology.orchestra.run.smithi123.stdout: "periodic_async_work": { 2024-05-08T06:13:54.272 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "periodic_async_work", 2024-05-08T06:13:54.272 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.272 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.272 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.272 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.272 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.272 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.273 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.273 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Periodically check for async work", 2024-05-08T06:13:54.273 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.273 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.273 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.273 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.273 INFO:teuthology.orchestra.run.smithi123.stdout: "snapshot_clone_delay": { 2024-05-08T06:13:54.273 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "snapshot_clone_delay", 2024-05-08T06:13:54.273 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.273 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.273 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.273 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.273 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.273 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.273 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.273 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-08T06:13:54.274 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.274 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.274 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.274 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.274 INFO:teuthology.orchestra.run.smithi123.stdout: "snapshot_clone_no_wait": { 2024-05-08T06:13:54.274 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "snapshot_clone_no_wait", 2024-05-08T06:13:54.274 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.274 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.274 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.274 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.274 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.274 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.274 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.274 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-08T06:13:54.274 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.274 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.275 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.275 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.275 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.275 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.275 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.275 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.275 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.275 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.275 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.275 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.275 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.275 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.275 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.275 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.275 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.275 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.276 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.276 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.276 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.276 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "zabbix", 2024-05-08T06:13:54.276 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.276 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.276 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.276 INFO:teuthology.orchestra.run.smithi123.stdout: "discovery_interval": { 2024-05-08T06:13:54.276 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "discovery_interval", 2024-05-08T06:13:54.276 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-05-08T06:13:54.276 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.276 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.276 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "100", 2024-05-08T06:13:54.276 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.276 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.277 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.277 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.277 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.277 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.277 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.277 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.277 INFO:teuthology.orchestra.run.smithi123.stdout: "identifier": { 2024-05-08T06:13:54.277 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "identifier", 2024-05-08T06:13:54.277 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.277 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.277 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.277 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.277 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.277 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.277 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.278 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.278 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.278 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.278 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.278 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.278 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-05-08T06:13:54.278 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-05-08T06:13:54.278 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.278 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.278 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.278 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-05-08T06:13:54.278 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.278 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.278 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.279 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.279 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.279 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.279 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.279 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.279 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.279 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.279 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.279 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.279 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.279 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.279 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.279 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.279 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.279 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.279 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.280 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.280 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.280 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.280 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.280 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.280 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.280 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.280 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.280 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.280 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.280 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.280 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.280 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.280 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.280 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.280 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.281 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.281 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.281 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.281 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.281 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.281 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.281 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.281 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.281 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.281 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.281 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.281 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.281 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.281 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.281 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.282 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.282 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.282 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.282 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.282 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.282 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.282 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.282 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.282 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.282 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.282 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.282 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.282 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.282 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.283 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.283 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.283 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.283 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.283 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.283 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.283 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.283 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.283 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.283 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.283 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.283 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.283 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.283 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.284 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.284 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.284 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.284 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.284 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.284 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.284 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.284 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.284 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.284 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.284 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.284 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.284 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.284 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.284 INFO:teuthology.orchestra.run.smithi123.stdout: "zabbix_host": { 2024-05-08T06:13:54.284 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "zabbix_host", 2024-05-08T06:13:54.285 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.285 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.285 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.285 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.285 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.285 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.285 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.285 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.285 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.285 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.285 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.285 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.285 INFO:teuthology.orchestra.run.smithi123.stdout: "zabbix_port": { 2024-05-08T06:13:54.285 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "zabbix_port", 2024-05-08T06:13:54.285 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.285 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.286 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.286 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10051", 2024-05-08T06:13:54.286 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.286 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.286 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.286 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.286 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.286 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.286 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.286 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.286 INFO:teuthology.orchestra.run.smithi123.stdout: "zabbix_sender": { 2024-05-08T06:13:54.286 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "zabbix_sender", 2024-05-08T06:13:54.286 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.286 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.286 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.287 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-08T06:13:54.287 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.287 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.287 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.287 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.287 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.287 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.287 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.287 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.287 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.287 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.287 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-05-08T06:13:54.287 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.287 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.287 INFO:teuthology.orchestra.run.smithi123.stdout: "gid": 24113, 2024-05-08T06:13:54.287 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "y", 2024-05-08T06:13:54.288 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr_features": 4540701547738038271, 2024-05-08T06:13:54.288 INFO:teuthology.orchestra.run.smithi123.stdout: "available_modules": [ 2024-05-08T06:13:54.288 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.288 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "alerts", 2024-05-08T06:13:54.288 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.288 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.288 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.288 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-05-08T06:13:54.288 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-05-08T06:13:54.288 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.288 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.288 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.288 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-05-08T06:13:54.288 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.288 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.288 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.289 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "How frequently to reexamine health status", 2024-05-08T06:13:54.289 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.289 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.289 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.289 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.289 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.289 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.289 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.289 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.289 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.289 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.289 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.289 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.289 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.289 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.289 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.290 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.290 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.290 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.290 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.290 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.290 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.290 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.290 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.290 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.290 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.290 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.290 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.290 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.290 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.290 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.291 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.291 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.291 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.291 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.291 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.291 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.291 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.291 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.291 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.291 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.291 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.291 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.291 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.291 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.291 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.292 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.292 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.292 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.292 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.292 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.292 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.292 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.292 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.292 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.292 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.292 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.292 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.292 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.292 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.292 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.292 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.292 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.293 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.293 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.293 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.293 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.293 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.293 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.293 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.293 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.293 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.293 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.293 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.293 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.293 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_destination": { 2024-05-08T06:13:54.293 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_destination", 2024-05-08T06:13:54.293 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.293 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.294 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.294 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.294 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.294 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.294 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.294 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Email address to send alerts to", 2024-05-08T06:13:54.294 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.294 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.294 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.294 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.294 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_from_name": { 2024-05-08T06:13:54.294 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_from_name", 2024-05-08T06:13:54.294 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.294 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.294 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.294 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "Ceph", 2024-05-08T06:13:54.295 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.295 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.295 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.295 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Email From: name", 2024-05-08T06:13:54.295 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.295 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.295 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.295 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.295 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_host": { 2024-05-08T06:13:54.295 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_host", 2024-05-08T06:13:54.295 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.295 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.295 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.295 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.295 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.295 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.295 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.296 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "SMTP server", 2024-05-08T06:13:54.296 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.296 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.296 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.296 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.296 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_password": { 2024-05-08T06:13:54.296 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_password", 2024-05-08T06:13:54.296 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.296 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.296 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.296 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.296 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.296 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.296 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.296 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Password to authenticate with", 2024-05-08T06:13:54.296 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.297 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.297 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.297 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.297 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_port": { 2024-05-08T06:13:54.297 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_port", 2024-05-08T06:13:54.297 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.297 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.297 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.297 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "465", 2024-05-08T06:13:54.297 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.297 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.297 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.297 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "SMTP port", 2024-05-08T06:13:54.297 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.297 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.297 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.298 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.298 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_sender": { 2024-05-08T06:13:54.298 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_sender", 2024-05-08T06:13:54.298 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.298 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.298 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.298 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.298 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.298 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.298 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.298 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "SMTP envelope sender", 2024-05-08T06:13:54.298 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.298 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.298 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.298 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.298 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_ssl": { 2024-05-08T06:13:54.299 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_ssl", 2024-05-08T06:13:54.299 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.299 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.299 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.299 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.299 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.299 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.299 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.299 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-08T06:13:54.299 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.299 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.299 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.299 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.299 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_user": { 2024-05-08T06:13:54.299 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_user", 2024-05-08T06:13:54.299 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.300 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.300 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.300 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.300 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.300 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.300 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.300 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "User to authenticate as", 2024-05-08T06:13:54.300 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.300 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.300 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.300 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.300 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.300 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.301 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.301 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.301 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.301 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.301 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.301 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.301 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.301 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.301 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.301 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.301 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.301 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.301 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.301 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.301 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.301 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "balancer", 2024-05-08T06:13:54.302 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.302 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.302 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.302 INFO:teuthology.orchestra.run.smithi123.stdout: "active": { 2024-05-08T06:13:54.302 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "active", 2024-05-08T06:13:54.302 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.302 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.302 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.302 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.302 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.302 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.302 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.302 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "automatically balance PGs across cluster", 2024-05-08T06:13:54.303 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.303 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.303 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.303 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.303 INFO:teuthology.orchestra.run.smithi123.stdout: "begin_time": { 2024-05-08T06:13:54.303 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "begin_time", 2024-05-08T06:13:54.303 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.303 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.303 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.303 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0000", 2024-05-08T06:13:54.303 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.303 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.303 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.303 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "beginning time of day to automatically balance", 2024-05-08T06:13:54.303 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T06:13:54.303 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.304 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.304 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.304 INFO:teuthology.orchestra.run.smithi123.stdout: "begin_weekday": { 2024-05-08T06:13:54.304 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "begin_weekday", 2024-05-08T06:13:54.304 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-05-08T06:13:54.304 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.304 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.304 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.304 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "0", 2024-05-08T06:13:54.304 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "6", 2024-05-08T06:13:54.304 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.304 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-08T06:13:54.304 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T06:13:54.304 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.304 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.304 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.305 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_compat_max_iterations": { 2024-05-08T06:13:54.305 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crush_compat_max_iterations", 2024-05-08T06:13:54.305 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-05-08T06:13:54.305 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.305 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.305 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "25", 2024-05-08T06:13:54.305 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1", 2024-05-08T06:13:54.305 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "250", 2024-05-08T06:13:54.305 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.305 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-08T06:13:54.305 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.305 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.305 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.305 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.305 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_compat_metrics": { 2024-05-08T06:13:54.305 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crush_compat_metrics", 2024-05-08T06:13:54.305 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.306 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.306 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.306 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "pgs,objects,bytes", 2024-05-08T06:13:54.306 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.306 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.306 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.306 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-08T06:13:54.306 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-08T06:13:54.306 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.306 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.306 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.306 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_compat_step": { 2024-05-08T06:13:54.306 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crush_compat_step", 2024-05-08T06:13:54.306 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-05-08T06:13:54.306 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.306 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.307 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0.5", 2024-05-08T06:13:54.307 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "0.001", 2024-05-08T06:13:54.307 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "0.999", 2024-05-08T06:13:54.307 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.307 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "aggressiveness of optimization", 2024-05-08T06:13:54.307 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-08T06:13:54.307 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.307 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.307 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.307 INFO:teuthology.orchestra.run.smithi123.stdout: "end_time": { 2024-05-08T06:13:54.307 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "end_time", 2024-05-08T06:13:54.307 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.307 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.307 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.307 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "2359", 2024-05-08T06:13:54.308 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.308 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.308 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.308 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "ending time of day to automatically balance", 2024-05-08T06:13:54.308 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T06:13:54.308 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.308 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.308 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.308 INFO:teuthology.orchestra.run.smithi123.stdout: "end_weekday": { 2024-05-08T06:13:54.308 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "end_weekday", 2024-05-08T06:13:54.308 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-05-08T06:13:54.308 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.308 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.308 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.308 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "0", 2024-05-08T06:13:54.308 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "6", 2024-05-08T06:13:54.309 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.309 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-08T06:13:54.309 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T06:13:54.309 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.309 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.309 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.309 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.309 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.309 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.309 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.309 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.309 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.309 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.309 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.309 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.309 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.310 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.310 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.310 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.310 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.310 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.310 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.310 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.310 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.310 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.310 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.310 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.310 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.310 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.310 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.310 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.311 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.311 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.311 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.311 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.311 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.311 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.311 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.311 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.311 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.311 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.311 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.311 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.311 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.311 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.312 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.312 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.312 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.312 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.312 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.312 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.312 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.312 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.312 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.312 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.312 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.312 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.312 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.312 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.312 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.313 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.313 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.313 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.313 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.313 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.313 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.313 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.313 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.313 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.313 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.313 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.313 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.313 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.313 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.313 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.314 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.314 INFO:teuthology.orchestra.run.smithi123.stdout: "min_score": { 2024-05-08T06:13:54.314 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "min_score", 2024-05-08T06:13:54.314 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-05-08T06:13:54.314 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.314 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.314 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.314 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.314 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.314 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.314 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-08T06:13:54.314 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.314 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.314 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.314 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.315 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": { 2024-05-08T06:13:54.315 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mode", 2024-05-08T06:13:54.315 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.315 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.315 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.315 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "upmap", 2024-05-08T06:13:54.315 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.315 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.315 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.315 INFO:teuthology.orchestra.run.smithi123.stdout: "crush-compat", 2024-05-08T06:13:54.315 INFO:teuthology.orchestra.run.smithi123.stdout: "none", 2024-05-08T06:13:54.315 INFO:teuthology.orchestra.run.smithi123.stdout: "read", 2024-05-08T06:13:54.315 INFO:teuthology.orchestra.run.smithi123.stdout: "upmap", 2024-05-08T06:13:54.315 INFO:teuthology.orchestra.run.smithi123.stdout: "upmap-read" 2024-05-08T06:13:54.315 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.315 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Balancer mode", 2024-05-08T06:13:54.316 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.316 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.316 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.316 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.316 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_ids": { 2024-05-08T06:13:54.316 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "pool_ids", 2024-05-08T06:13:54.316 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.316 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.316 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.316 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.316 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.316 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.316 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.316 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-08T06:13:54.316 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.316 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.317 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.317 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.317 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-05-08T06:13:54.317 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-05-08T06:13:54.317 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.317 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.317 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.317 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-05-08T06:13:54.317 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.317 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.317 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.317 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-08T06:13:54.317 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.317 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.317 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.317 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.318 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.318 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.318 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.318 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.318 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.318 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.318 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.318 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.318 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.318 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.318 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.318 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.318 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.318 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.318 INFO:teuthology.orchestra.run.smithi123.stdout: "upmap_max_deviation": { 2024-05-08T06:13:54.318 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "upmap_max_deviation", 2024-05-08T06:13:54.319 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.319 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.319 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.319 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-05-08T06:13:54.319 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1", 2024-05-08T06:13:54.319 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.319 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.319 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-08T06:13:54.319 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-08T06:13:54.319 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.319 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.319 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.319 INFO:teuthology.orchestra.run.smithi123.stdout: "upmap_max_optimizations": { 2024-05-08T06:13:54.319 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "upmap_max_optimizations", 2024-05-08T06:13:54.319 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-05-08T06:13:54.320 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.320 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.320 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-05-08T06:13:54.320 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.320 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.320 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.320 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-08T06:13:54.320 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.320 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.320 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.320 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.320 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.320 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.320 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.320 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cephadm", 2024-05-08T06:13:54.321 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.321 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.321 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.321 INFO:teuthology.orchestra.run.smithi123.stdout: "agent_down_multiplier": { 2024-05-08T06:13:54.321 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "agent_down_multiplier", 2024-05-08T06:13:54.321 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-05-08T06:13:54.321 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.321 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.321 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "3.0", 2024-05-08T06:13:54.321 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.321 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.321 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.321 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-08T06:13:54.321 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.321 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.321 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.322 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.322 INFO:teuthology.orchestra.run.smithi123.stdout: "agent_refresh_rate": { 2024-05-08T06:13:54.322 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "agent_refresh_rate", 2024-05-08T06:13:54.322 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.322 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.322 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.322 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "20", 2024-05-08T06:13:54.322 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.322 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.322 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.322 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-08T06:13:54.322 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.322 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.322 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.322 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.323 INFO:teuthology.orchestra.run.smithi123.stdout: "agent_starting_port": { 2024-05-08T06:13:54.323 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "agent_starting_port", 2024-05-08T06:13:54.323 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.323 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.323 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.323 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "4721", 2024-05-08T06:13:54.323 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.323 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.323 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.323 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-08T06:13:54.323 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.323 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.323 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.323 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.323 INFO:teuthology.orchestra.run.smithi123.stdout: "allow_ptrace": { 2024-05-08T06:13:54.323 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "allow_ptrace", 2024-05-08T06:13:54.324 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.324 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.324 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.324 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.324 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.324 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.324 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.324 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-08T06:13:54.324 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-08T06:13:54.324 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.324 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.324 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.324 INFO:teuthology.orchestra.run.smithi123.stdout: "autotune_interval": { 2024-05-08T06:13:54.324 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "autotune_interval", 2024-05-08T06:13:54.324 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.324 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.325 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.325 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-05-08T06:13:54.325 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.325 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.325 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.325 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-08T06:13:54.325 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.325 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.325 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.325 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.325 INFO:teuthology.orchestra.run.smithi123.stdout: "autotune_memory_target_ratio": { 2024-05-08T06:13:54.325 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "autotune_memory_target_ratio", 2024-05-08T06:13:54.325 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-05-08T06:13:54.325 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.325 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.325 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0.7", 2024-05-08T06:13:54.325 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.326 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.326 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.326 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-08T06:13:54.326 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.326 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.326 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.326 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.326 INFO:teuthology.orchestra.run.smithi123.stdout: "cephadm_log_destination": { 2024-05-08T06:13:54.326 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cephadm_log_destination", 2024-05-08T06:13:54.326 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.326 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.326 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.326 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.326 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.326 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.327 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.327 INFO:teuthology.orchestra.run.smithi123.stdout: "file", 2024-05-08T06:13:54.327 INFO:teuthology.orchestra.run.smithi123.stdout: "file,syslog", 2024-05-08T06:13:54.327 INFO:teuthology.orchestra.run.smithi123.stdout: "syslog" 2024-05-08T06:13:54.327 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.327 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-08T06:13:54.327 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.327 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.327 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.327 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.327 INFO:teuthology.orchestra.run.smithi123.stdout: "cgroups_split": { 2024-05-08T06:13:54.327 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cgroups_split", 2024-05-08T06:13:54.327 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.327 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.327 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.327 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.327 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.328 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.328 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.328 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-08T06:13:54.328 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.328 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.328 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.328 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.328 INFO:teuthology.orchestra.run.smithi123.stdout: "config_checks_enabled": { 2024-05-08T06:13:54.328 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "config_checks_enabled", 2024-05-08T06:13:54.328 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.328 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.328 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.328 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.328 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.328 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.328 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.328 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-08T06:13:54.329 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.329 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.329 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.329 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.329 INFO:teuthology.orchestra.run.smithi123.stdout: "config_dashboard": { 2024-05-08T06:13:54.329 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "config_dashboard", 2024-05-08T06:13:54.329 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.329 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.329 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.329 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.329 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.329 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.329 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.329 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-08T06:13:54.329 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.330 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.330 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.330 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.330 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_alertmanager": { 2024-05-08T06:13:54.330 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_alertmanager", 2024-05-08T06:13:54.330 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.330 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.330 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.330 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-08T06:13:54.330 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.330 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.330 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.330 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Prometheus container image", 2024-05-08T06:13:54.330 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.330 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.330 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.331 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.331 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_base": { 2024-05-08T06:13:54.331 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_base", 2024-05-08T06:13:54.331 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.331 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.331 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.331 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-08T06:13:54.331 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.331 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.331 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.331 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Container image name, without the tag", 2024-05-08T06:13:54.331 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.331 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.331 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.331 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.332 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_elasticsearch": { 2024-05-08T06:13:54.332 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_elasticsearch", 2024-05-08T06:13:54.332 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.332 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.332 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.332 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-08T06:13:54.332 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.332 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.332 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.332 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "elasticsearch container image", 2024-05-08T06:13:54.332 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.332 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.332 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.332 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.332 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_grafana": { 2024-05-08T06:13:54.332 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_grafana", 2024-05-08T06:13:54.333 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.333 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.333 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.333 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-08T06:13:54.333 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.333 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.333 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.333 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Prometheus container image", 2024-05-08T06:13:54.333 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.333 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.333 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.333 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.333 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_haproxy": { 2024-05-08T06:13:54.333 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_haproxy", 2024-05-08T06:13:54.333 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.333 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.334 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.334 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-08T06:13:54.334 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.334 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.334 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.334 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "HAproxy container image", 2024-05-08T06:13:54.334 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.334 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.334 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.334 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.334 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_jaeger_agent": { 2024-05-08T06:13:54.334 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_jaeger_agent", 2024-05-08T06:13:54.334 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.334 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.335 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.335 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-08T06:13:54.335 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.335 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.335 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.335 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Jaeger agent container image", 2024-05-08T06:13:54.335 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.335 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.335 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.335 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.335 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_jaeger_collector": { 2024-05-08T06:13:54.335 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_jaeger_collector", 2024-05-08T06:13:54.335 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.335 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.335 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.335 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-08T06:13:54.336 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.336 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.336 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.336 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Jaeger collector container image", 2024-05-08T06:13:54.336 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.336 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.336 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.336 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.336 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_jaeger_query": { 2024-05-08T06:13:54.336 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_jaeger_query", 2024-05-08T06:13:54.336 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.336 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.336 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.336 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-08T06:13:54.336 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.336 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.337 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.337 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Jaeger query container image", 2024-05-08T06:13:54.337 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.337 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.337 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.337 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.337 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_keepalived": { 2024-05-08T06:13:54.337 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_keepalived", 2024-05-08T06:13:54.337 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.337 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.337 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.337 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-08T06:13:54.337 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.337 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.337 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.338 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Keepalived container image", 2024-05-08T06:13:54.338 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.338 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.338 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.338 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.338 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_loki": { 2024-05-08T06:13:54.338 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_loki", 2024-05-08T06:13:54.338 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.338 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.338 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.338 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-08T06:13:54.338 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.338 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.338 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.338 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Loki container image", 2024-05-08T06:13:54.338 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.339 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.339 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.339 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.339 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_node_exporter": { 2024-05-08T06:13:54.339 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_node_exporter", 2024-05-08T06:13:54.339 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.339 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.339 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.339 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-08T06:13:54.339 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.339 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.339 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.339 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Prometheus container image", 2024-05-08T06:13:54.339 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.339 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.339 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.340 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.340 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_nvmeof": { 2024-05-08T06:13:54.340 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_nvmeof", 2024-05-08T06:13:54.340 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.340 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.340 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.340 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-08T06:13:54.340 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.340 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.340 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.340 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Nvme-of container image", 2024-05-08T06:13:54.340 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.340 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.340 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.340 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.340 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_prometheus": { 2024-05-08T06:13:54.341 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_prometheus", 2024-05-08T06:13:54.341 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.341 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.341 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.341 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-08T06:13:54.341 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.341 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.341 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.341 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Prometheus container image", 2024-05-08T06:13:54.341 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.341 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.341 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.341 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.341 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_promtail": { 2024-05-08T06:13:54.341 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_promtail", 2024-05-08T06:13:54.342 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.342 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.342 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.342 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-08T06:13:54.342 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.342 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.342 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.342 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Promtail container image", 2024-05-08T06:13:54.342 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.342 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.342 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.342 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.342 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_samba": { 2024-05-08T06:13:54.342 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_samba", 2024-05-08T06:13:54.342 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.343 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.343 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.343 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-08T06:13:54.343 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.343 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.343 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.343 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Samba/SMB container image", 2024-05-08T06:13:54.343 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.343 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.343 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.343 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.343 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_snmp_gateway": { 2024-05-08T06:13:54.343 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_snmp_gateway", 2024-05-08T06:13:54.343 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.343 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.343 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.344 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-08T06:13:54.344 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.344 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.344 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.344 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "SNMP Gateway container image", 2024-05-08T06:13:54.344 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.344 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.344 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.344 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.344 INFO:teuthology.orchestra.run.smithi123.stdout: "container_init": { 2024-05-08T06:13:54.344 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_init", 2024-05-08T06:13:54.344 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.344 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.344 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.344 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.344 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.345 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.345 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.345 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Run podman/docker with `--init`", 2024-05-08T06:13:54.345 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.345 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.345 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.345 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.345 INFO:teuthology.orchestra.run.smithi123.stdout: "daemon_cache_timeout": { 2024-05-08T06:13:54.345 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "daemon_cache_timeout", 2024-05-08T06:13:54.345 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.345 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.345 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.345 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-05-08T06:13:54.345 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.345 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.345 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.346 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-08T06:13:54.346 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.346 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.346 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.346 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.346 INFO:teuthology.orchestra.run.smithi123.stdout: "default_cephadm_command_timeout": { 2024-05-08T06:13:54.346 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "default_cephadm_command_timeout", 2024-05-08T06:13:54.346 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.346 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.346 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.346 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "900", 2024-05-08T06:13:54.346 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.346 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.346 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.346 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-08T06:13:54.346 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.347 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.347 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.347 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.347 INFO:teuthology.orchestra.run.smithi123.stdout: "default_registry": { 2024-05-08T06:13:54.347 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "default_registry", 2024-05-08T06:13:54.347 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.347 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.347 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.347 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "docker.io", 2024-05-08T06:13:54.347 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.347 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.347 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.347 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-08T06:13:54.347 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.347 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.347 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.348 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.348 INFO:teuthology.orchestra.run.smithi123.stdout: "device_cache_timeout": { 2024-05-08T06:13:54.348 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "device_cache_timeout", 2024-05-08T06:13:54.348 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.348 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.348 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.348 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "1800", 2024-05-08T06:13:54.348 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.348 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.348 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.348 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "seconds to cache device inventory", 2024-05-08T06:13:54.348 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.348 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.348 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.348 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.348 INFO:teuthology.orchestra.run.smithi123.stdout: "device_enhanced_scan": { 2024-05-08T06:13:54.349 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "device_enhanced_scan", 2024-05-08T06:13:54.349 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.349 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.349 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.349 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.349 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.349 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.349 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.349 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-08T06:13:54.349 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.349 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.349 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.349 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.349 INFO:teuthology.orchestra.run.smithi123.stdout: "facts_cache_timeout": { 2024-05-08T06:13:54.349 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "facts_cache_timeout", 2024-05-08T06:13:54.349 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.350 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.350 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.350 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-05-08T06:13:54.350 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.350 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.350 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.350 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "seconds to cache host facts data", 2024-05-08T06:13:54.350 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.350 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.350 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.350 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.350 INFO:teuthology.orchestra.run.smithi123.stdout: "grafana_dashboards_path": { 2024-05-08T06:13:54.350 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "grafana_dashboards_path", 2024-05-08T06:13:54.350 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.350 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.351 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.351 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-08T06:13:54.351 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.351 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.351 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.351 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-08T06:13:54.351 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.351 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.351 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.351 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.351 INFO:teuthology.orchestra.run.smithi123.stdout: "host_check_interval": { 2024-05-08T06:13:54.351 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "host_check_interval", 2024-05-08T06:13:54.351 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.351 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.352 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.352 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-05-08T06:13:54.352 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.352 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.352 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.352 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to perform a host check", 2024-05-08T06:13:54.352 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.352 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.352 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.352 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.352 INFO:teuthology.orchestra.run.smithi123.stdout: "hw_monitoring": { 2024-05-08T06:13:54.352 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "hw_monitoring", 2024-05-08T06:13:54.352 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.352 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.353 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.353 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.353 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.353 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.353 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.353 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-08T06:13:54.353 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.353 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.353 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.353 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.353 INFO:teuthology.orchestra.run.smithi123.stdout: "inventory_list_all": { 2024-05-08T06:13:54.353 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "inventory_list_all", 2024-05-08T06:13:54.353 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.353 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.354 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.354 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.354 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.354 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.354 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.354 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-08T06:13:54.354 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.354 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.354 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.354 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.354 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.354 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.354 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.354 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.355 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.355 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.355 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.355 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.355 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.355 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.355 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.355 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.355 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.355 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.355 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.355 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.355 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.355 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.355 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.356 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.356 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.356 INFO:teuthology.orchestra.run.smithi123.stdout: "log_refresh_metadata": { 2024-05-08T06:13:54.356 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_refresh_metadata", 2024-05-08T06:13:54.356 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.356 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.356 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.356 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.356 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.356 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.356 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.356 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-08T06:13:54.356 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.356 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.356 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.357 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.357 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.357 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.357 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.357 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.357 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.357 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.357 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.357 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.357 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.357 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-08T06:13:54.357 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.357 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.357 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.358 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.358 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.358 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.358 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.358 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.358 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.358 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.358 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.358 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.358 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.358 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.358 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.358 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.358 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.358 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.359 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.359 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.359 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.359 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.359 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.359 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.359 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.359 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.359 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.359 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.359 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.359 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.359 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.359 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.359 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.359 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.360 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.360 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.360 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.360 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.360 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.360 INFO:teuthology.orchestra.run.smithi123.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-08T06:13:54.360 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-08T06:13:54.360 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.360 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.360 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.360 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.360 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.360 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.360 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.360 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-08T06:13:54.361 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.361 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.361 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.361 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.361 INFO:teuthology.orchestra.run.smithi123.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-08T06:13:54.361 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-08T06:13:54.361 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.361 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.361 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.361 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "*", 2024-05-08T06:13:54.361 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.361 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.361 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.361 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-08T06:13:54.362 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.362 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.362 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.362 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.362 INFO:teuthology.orchestra.run.smithi123.stdout: "max_count_per_host": { 2024-05-08T06:13:54.362 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_count_per_host", 2024-05-08T06:13:54.362 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.362 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.362 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.362 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-05-08T06:13:54.362 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.362 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.363 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.363 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "max number of daemons per service per host", 2024-05-08T06:13:54.363 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.363 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.363 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.363 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.363 INFO:teuthology.orchestra.run.smithi123.stdout: "max_osd_draining_count": { 2024-05-08T06:13:54.363 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_osd_draining_count", 2024-05-08T06:13:54.363 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.363 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.363 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.363 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-05-08T06:13:54.363 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.363 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.363 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.363 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-08T06:13:54.364 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.364 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.364 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.364 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.364 INFO:teuthology.orchestra.run.smithi123.stdout: "migration_current": { 2024-05-08T06:13:54.364 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "migration_current", 2024-05-08T06:13:54.364 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.364 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.364 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.364 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.364 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.364 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.364 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.365 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "internal - do not modify", 2024-05-08T06:13:54.365 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.365 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.365 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.365 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.365 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": { 2024-05-08T06:13:54.365 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mode", 2024-05-08T06:13:54.365 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.365 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.365 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.365 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "root", 2024-05-08T06:13:54.365 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.365 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.366 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.366 INFO:teuthology.orchestra.run.smithi123.stdout: "cephadm-package", 2024-05-08T06:13:54.366 INFO:teuthology.orchestra.run.smithi123.stdout: "root" 2024-05-08T06:13:54.366 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.366 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "mode for remote execution of cephadm", 2024-05-08T06:13:54.366 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.366 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.366 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.366 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.366 INFO:teuthology.orchestra.run.smithi123.stdout: "oob_default_addr": { 2024-05-08T06:13:54.366 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "oob_default_addr", 2024-05-08T06:13:54.366 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.366 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.366 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.366 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "169.254.1.1", 2024-05-08T06:13:54.367 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.367 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.367 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.367 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-08T06:13:54.367 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.367 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.367 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.367 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.367 INFO:teuthology.orchestra.run.smithi123.stdout: "prometheus_alerts_path": { 2024-05-08T06:13:54.367 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "prometheus_alerts_path", 2024-05-08T06:13:54.367 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.367 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.367 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.368 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-08T06:13:54.368 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.368 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.368 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.368 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-08T06:13:54.368 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.368 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.368 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.368 INFO:teuthology.orchestra.run.smithi123.stdout: "registry_insecure": { 2024-05-08T06:13:54.368 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "registry_insecure", 2024-05-08T06:13:54.368 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.368 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.368 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.369 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.369 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.369 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.369 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.369 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-08T06:13:54.369 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.369 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.369 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.369 INFO:teuthology.orchestra.run.smithi123.stdout: "registry_password": { 2024-05-08T06:13:54.369 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "registry_password", 2024-05-08T06:13:54.369 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.369 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.369 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.370 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.370 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.370 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.370 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.370 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-08T06:13:54.370 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.370 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.370 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.370 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.370 INFO:teuthology.orchestra.run.smithi123.stdout: "registry_url": { 2024-05-08T06:13:54.370 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "registry_url", 2024-05-08T06:13:54.370 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.370 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.371 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.371 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.371 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.371 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.371 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.371 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-08T06:13:54.371 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.371 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.371 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.371 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.371 INFO:teuthology.orchestra.run.smithi123.stdout: "registry_username": { 2024-05-08T06:13:54.371 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "registry_username", 2024-05-08T06:13:54.371 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.371 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.372 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.372 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.372 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.372 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.372 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.372 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-08T06:13:54.372 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.372 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.372 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.372 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.372 INFO:teuthology.orchestra.run.smithi123.stdout: "secure_monitoring_stack": { 2024-05-08T06:13:54.372 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "secure_monitoring_stack", 2024-05-08T06:13:54.372 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.373 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.373 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.373 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.373 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.373 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.373 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.373 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-08T06:13:54.373 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.373 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.373 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.373 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.373 INFO:teuthology.orchestra.run.smithi123.stdout: "service_discovery_port": { 2024-05-08T06:13:54.373 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "service_discovery_port", 2024-05-08T06:13:54.373 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.373 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.374 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.374 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8765", 2024-05-08T06:13:54.374 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.374 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.374 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.374 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "cephadm service discovery port", 2024-05-08T06:13:54.374 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.374 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.374 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.374 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.374 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.374 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.374 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.374 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.374 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.375 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.375 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.375 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.375 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.375 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.375 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.375 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.375 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.375 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.375 INFO:teuthology.orchestra.run.smithi123.stdout: "ssh_config_file": { 2024-05-08T06:13:54.375 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ssh_config_file", 2024-05-08T06:13:54.375 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.375 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.375 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.375 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.375 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.376 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.376 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.376 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-08T06:13:54.376 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.376 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.376 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.376 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.376 INFO:teuthology.orchestra.run.smithi123.stdout: "use_agent": { 2024-05-08T06:13:54.376 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "use_agent", 2024-05-08T06:13:54.376 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.376 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.376 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.376 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.376 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.376 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.377 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.377 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-08T06:13:54.377 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.377 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.377 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.377 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.377 INFO:teuthology.orchestra.run.smithi123.stdout: "use_repo_digest": { 2024-05-08T06:13:54.377 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "use_repo_digest", 2024-05-08T06:13:54.377 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.377 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.377 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.377 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.377 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.378 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.378 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.378 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-08T06:13:54.378 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.378 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.378 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.378 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.378 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_on_failed_host_check": { 2024-05-08T06:13:54.378 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_on_failed_host_check", 2024-05-08T06:13:54.378 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.378 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.378 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.378 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.379 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.379 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.379 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.379 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "raise a health warning if the host check fails", 2024-05-08T06:13:54.379 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.379 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.379 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.379 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.379 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_on_stray_daemons": { 2024-05-08T06:13:54.379 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_on_stray_daemons", 2024-05-08T06:13:54.379 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.379 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.379 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.379 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.379 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.379 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.380 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.380 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-08T06:13:54.380 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.380 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.380 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.380 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.380 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_on_stray_hosts": { 2024-05-08T06:13:54.380 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_on_stray_hosts", 2024-05-08T06:13:54.380 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.380 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.380 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.380 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.380 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.380 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.380 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.380 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-08T06:13:54.381 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.381 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.381 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.381 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.381 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.381 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.381 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.381 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crash", 2024-05-08T06:13:54.381 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.381 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.381 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.381 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.381 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.381 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.382 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.382 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.382 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.382 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.382 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.382 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.382 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.382 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.382 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.382 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.382 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.382 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.382 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.382 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.383 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.383 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.383 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.383 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.383 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.383 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.383 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.383 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.383 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.383 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.383 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.383 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.383 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.383 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.383 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.384 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.384 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.384 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.384 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.384 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.384 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.384 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.384 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.384 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.384 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.384 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.384 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.384 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.384 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.385 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.385 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.385 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.385 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.385 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.385 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.385 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.385 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.385 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.385 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.385 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.385 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.385 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.385 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.385 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.386 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.386 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.386 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.386 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.386 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.386 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.386 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.386 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.386 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.386 INFO:teuthology.orchestra.run.smithi123.stdout: "retain_interval": { 2024-05-08T06:13:54.386 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "retain_interval", 2024-05-08T06:13:54.386 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.386 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.386 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.386 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "31536000", 2024-05-08T06:13:54.387 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.387 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.387 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.387 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-08T06:13:54.387 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.387 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.387 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.387 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.387 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.387 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.387 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.387 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.387 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.387 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.387 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.388 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.388 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.388 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.388 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.388 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.388 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.388 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.388 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_recent_interval": { 2024-05-08T06:13:54.388 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_recent_interval", 2024-05-08T06:13:54.388 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.388 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.388 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.388 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "1209600", 2024-05-08T06:13:54.388 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.388 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.388 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.388 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-08T06:13:54.389 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.389 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.389 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.389 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.389 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.389 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.389 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.389 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "dashboard", 2024-05-08T06:13:54.389 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.389 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.389 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.389 INFO:teuthology.orchestra.run.smithi123.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-08T06:13:54.389 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-08T06:13:54.389 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.389 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.389 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.390 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-05-08T06:13:54.390 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.390 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.390 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.390 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.390 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.390 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.390 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.390 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.390 INFO:teuthology.orchestra.run.smithi123.stdout: "ALERTMANAGER_API_HOST": { 2024-05-08T06:13:54.390 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-08T06:13:54.390 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.390 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.390 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.390 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.391 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.391 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.391 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.391 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.391 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.391 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.391 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.391 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.391 INFO:teuthology.orchestra.run.smithi123.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-08T06:13:54.391 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-08T06:13:54.391 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.391 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.391 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.391 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.391 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.392 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.392 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.392 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.392 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.392 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.392 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.392 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.392 INFO:teuthology.orchestra.run.smithi123.stdout: "AUDIT_API_ENABLED": { 2024-05-08T06:13:54.392 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "AUDIT_API_ENABLED", 2024-05-08T06:13:54.392 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.392 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.392 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.392 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.392 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.392 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.392 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.393 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.393 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.393 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.393 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.393 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.393 INFO:teuthology.orchestra.run.smithi123.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-08T06:13:54.393 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-08T06:13:54.393 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.393 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.393 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.393 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.393 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.393 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.393 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.394 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.394 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.394 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.394 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.394 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.394 INFO:teuthology.orchestra.run.smithi123.stdout: "ENABLE_BROWSABLE_API": { 2024-05-08T06:13:54.394 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-08T06:13:54.394 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.394 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.394 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.394 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.394 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.394 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.395 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.395 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.395 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.395 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.395 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.395 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.395 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-08T06:13:54.395 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-08T06:13:54.395 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.395 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.395 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.395 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.395 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.395 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.395 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.395 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.395 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.396 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.396 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.396 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.396 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-08T06:13:54.396 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-08T06:13:54.396 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.396 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.396 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.396 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.396 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.396 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.396 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.396 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.396 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.396 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.396 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.397 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.397 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-08T06:13:54.397 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-08T06:13:54.397 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.397 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.397 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.397 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.397 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.397 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.397 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.397 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.397 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.397 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.397 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.397 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.397 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-08T06:13:54.397 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-08T06:13:54.398 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.398 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.398 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.398 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.398 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.398 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.398 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.398 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.398 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.398 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.398 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.398 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.398 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-08T06:13:54.398 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-08T06:13:54.399 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.399 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.399 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.399 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.399 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.399 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.399 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.399 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.399 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.399 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.399 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.399 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.399 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-08T06:13:54.399 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-08T06:13:54.399 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.399 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.400 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.400 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.400 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.400 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.400 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.400 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.400 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.400 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.400 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.400 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.400 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-08T06:13:54.400 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-08T06:13:54.400 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.400 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.400 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.400 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.401 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.401 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.401 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.401 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.401 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.401 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.401 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.401 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.401 INFO:teuthology.orchestra.run.smithi123.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-08T06:13:54.401 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-08T06:13:54.401 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.401 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.401 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.401 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.402 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.402 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.402 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.402 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.402 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.402 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.402 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.402 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.402 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_API_PASSWORD": { 2024-05-08T06:13:54.402 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-08T06:13:54.402 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.402 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.402 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.402 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "admin", 2024-05-08T06:13:54.403 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.403 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.403 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.403 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.403 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.403 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.403 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.403 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.403 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-08T06:13:54.403 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-08T06:13:54.403 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.403 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.403 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.403 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.404 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.404 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.404 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.404 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.404 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.404 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.404 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.404 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.404 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_API_URL": { 2024-05-08T06:13:54.404 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_API_URL", 2024-05-08T06:13:54.404 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.404 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.404 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.404 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.405 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.405 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.405 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.405 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.405 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.405 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.405 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.405 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.405 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_API_USERNAME": { 2024-05-08T06:13:54.405 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-08T06:13:54.405 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.405 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.405 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.405 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "admin", 2024-05-08T06:13:54.406 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.406 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.406 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.406 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.406 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.406 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.406 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.406 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.406 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-08T06:13:54.406 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-08T06:13:54.406 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.406 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.406 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.406 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.407 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.407 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.407 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.407 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.407 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.407 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.407 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.407 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.407 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-08T06:13:54.407 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-08T06:13:54.407 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.407 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.407 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.407 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.407 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.407 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.408 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.408 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.408 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.408 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.408 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.408 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.408 INFO:teuthology.orchestra.run.smithi123.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-08T06:13:54.408 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-08T06:13:54.408 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.408 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.408 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.408 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.408 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.408 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.408 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.409 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.409 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.409 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.409 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.409 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.409 INFO:teuthology.orchestra.run.smithi123.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-08T06:13:54.409 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-08T06:13:54.409 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.409 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.409 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.409 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.409 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.409 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.409 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.409 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.409 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.409 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.410 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.410 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.410 INFO:teuthology.orchestra.run.smithi123.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-08T06:13:54.410 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-08T06:13:54.410 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.410 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.410 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.410 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "[]", 2024-05-08T06:13:54.410 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.410 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.410 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.410 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.410 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.410 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.411 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.411 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.411 INFO:teuthology.orchestra.run.smithi123.stdout: "MULTICLUSTER_CONFIG": { 2024-05-08T06:13:54.411 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-08T06:13:54.411 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.411 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.411 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.411 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "{}", 2024-05-08T06:13:54.411 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.411 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.411 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.411 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.411 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.411 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.411 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.411 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.412 INFO:teuthology.orchestra.run.smithi123.stdout: "PROMETHEUS_API_HOST": { 2024-05-08T06:13:54.412 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-08T06:13:54.412 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.412 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.412 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.412 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.412 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.412 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.412 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.412 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.412 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.412 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.412 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.412 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.412 INFO:teuthology.orchestra.run.smithi123.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-08T06:13:54.412 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-08T06:13:54.413 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.413 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.413 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.413 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.413 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.413 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.413 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.413 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.413 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.413 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.413 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.413 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.413 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-08T06:13:54.413 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-08T06:13:54.413 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.414 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.414 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.414 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.414 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.414 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.414 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.414 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.414 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.414 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.414 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.414 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.414 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-08T06:13:54.414 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-08T06:13:54.414 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.415 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.415 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.415 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.415 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.415 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.415 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.415 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.415 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.415 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.415 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.415 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.415 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-08T06:13:54.415 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-08T06:13:54.415 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.415 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.415 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.416 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.416 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.416 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.416 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.416 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.416 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.416 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.416 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.416 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.416 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-08T06:13:54.416 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-08T06:13:54.416 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.416 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.416 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.416 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.417 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.417 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.417 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.417 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.417 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.417 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.417 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.417 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.417 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-08T06:13:54.417 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-08T06:13:54.417 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.417 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.417 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.417 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.417 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.418 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.418 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.418 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.418 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.418 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.418 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.418 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.418 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-08T06:13:54.418 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-08T06:13:54.418 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.418 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.418 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.418 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.418 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.418 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.419 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.419 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.419 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.419 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.419 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.419 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.419 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-08T06:13:54.419 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-08T06:13:54.419 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.419 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.419 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.419 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.419 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.419 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.420 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.420 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.420 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.420 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.420 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.420 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.420 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_ENABLED": { 2024-05-08T06:13:54.420 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-08T06:13:54.420 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.420 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.420 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.420 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.420 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.420 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.420 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.420 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.421 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.421 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.421 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.421 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.421 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-08T06:13:54.421 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-08T06:13:54.421 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.421 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.421 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.421 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-08T06:13:54.421 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.421 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.421 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.421 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.421 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.422 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.422 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.422 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.422 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-08T06:13:54.422 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-08T06:13:54.422 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.422 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.422 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.422 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-05-08T06:13:54.422 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.422 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.422 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.422 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.422 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.422 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.423 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.423 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.423 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-08T06:13:54.423 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-08T06:13:54.423 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.423 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.423 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.423 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8", 2024-05-08T06:13:54.423 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.423 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.423 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.423 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.423 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.423 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.423 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.423 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.424 INFO:teuthology.orchestra.run.smithi123.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-08T06:13:54.424 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-08T06:13:54.424 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.424 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.424 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.424 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "45", 2024-05-08T06:13:54.424 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.424 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.424 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.424 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.424 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.424 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.424 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.424 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.424 INFO:teuthology.orchestra.run.smithi123.stdout: "RGW_API_ACCESS_KEY": { 2024-05-08T06:13:54.424 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-08T06:13:54.425 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.425 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.425 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.425 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.425 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.425 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.425 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.425 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.425 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.425 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.425 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.425 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.425 INFO:teuthology.orchestra.run.smithi123.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-08T06:13:54.425 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-08T06:13:54.425 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.426 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.426 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.426 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "admin", 2024-05-08T06:13:54.426 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.426 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.426 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.426 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.426 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.426 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.426 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.426 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.426 INFO:teuthology.orchestra.run.smithi123.stdout: "RGW_API_SECRET_KEY": { 2024-05-08T06:13:54.426 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-08T06:13:54.426 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.426 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.427 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.427 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.427 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.427 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.427 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.427 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.427 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.427 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.427 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.427 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.427 INFO:teuthology.orchestra.run.smithi123.stdout: "RGW_API_SSL_VERIFY": { 2024-05-08T06:13:54.427 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-08T06:13:54.427 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.427 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.427 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.427 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.428 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.428 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.428 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.428 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.428 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.428 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.428 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.428 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.428 INFO:teuthology.orchestra.run.smithi123.stdout: "UNSAFE_TLS_v1_2": { 2024-05-08T06:13:54.428 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-08T06:13:54.428 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.428 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.428 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.428 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.428 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.428 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.429 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.429 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.429 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.429 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.429 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.429 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.429 INFO:teuthology.orchestra.run.smithi123.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-08T06:13:54.429 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-08T06:13:54.429 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.429 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.429 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.429 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.429 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.429 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.429 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.430 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.430 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.430 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.430 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.430 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.430 INFO:teuthology.orchestra.run.smithi123.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-08T06:13:54.430 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-08T06:13:54.430 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.430 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.430 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.430 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-05-08T06:13:54.430 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.430 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.430 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.430 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.431 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.431 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.431 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.431 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.431 INFO:teuthology.orchestra.run.smithi123.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-08T06:13:54.431 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-08T06:13:54.431 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.431 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.431 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.431 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-05-08T06:13:54.431 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.431 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.431 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.431 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.431 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.432 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.432 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.432 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.432 INFO:teuthology.orchestra.run.smithi123.stdout: "cross_origin_url": { 2024-05-08T06:13:54.432 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cross_origin_url", 2024-05-08T06:13:54.432 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.432 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.432 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.432 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.432 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.432 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.432 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.432 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.432 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.432 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.432 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.433 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.433 INFO:teuthology.orchestra.run.smithi123.stdout: "crt_file": { 2024-05-08T06:13:54.433 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crt_file", 2024-05-08T06:13:54.433 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.433 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.433 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.433 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.433 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.433 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.433 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.433 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.433 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.433 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.433 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.433 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.433 INFO:teuthology.orchestra.run.smithi123.stdout: "debug": { 2024-05-08T06:13:54.434 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "debug", 2024-05-08T06:13:54.434 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.434 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.434 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.434 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.434 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.434 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.434 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.434 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Enable/disable debug options", 2024-05-08T06:13:54.434 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.434 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.434 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.434 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.434 INFO:teuthology.orchestra.run.smithi123.stdout: "jwt_token_ttl": { 2024-05-08T06:13:54.434 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "jwt_token_ttl", 2024-05-08T06:13:54.434 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.435 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.435 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.435 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "28800", 2024-05-08T06:13:54.435 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.435 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.435 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.435 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.435 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.435 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.435 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.435 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.435 INFO:teuthology.orchestra.run.smithi123.stdout: "key_file": { 2024-05-08T06:13:54.435 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "key_file", 2024-05-08T06:13:54.435 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.436 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.436 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.436 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.436 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.436 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.436 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.436 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.436 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.436 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.436 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.436 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.436 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.436 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.436 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.437 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.437 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.437 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.437 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.437 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.437 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.437 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.437 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.437 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.437 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.437 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.437 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.437 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.437 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.437 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.437 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.438 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.438 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.438 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.438 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.438 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.438 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.438 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.438 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.438 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.438 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.438 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.438 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.438 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.438 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.438 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.439 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.439 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.439 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.439 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.439 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.439 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.439 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.439 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.439 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.439 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.439 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.439 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.439 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.439 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.439 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.439 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.440 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.440 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.440 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.440 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.440 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.440 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.440 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.440 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.440 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.440 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.440 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.440 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.440 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.440 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.440 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.440 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.441 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.441 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.441 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.441 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.441 INFO:teuthology.orchestra.run.smithi123.stdout: "motd": { 2024-05-08T06:13:54.441 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "motd", 2024-05-08T06:13:54.441 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.441 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.441 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.441 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.441 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.441 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.441 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.441 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "The message of the day", 2024-05-08T06:13:54.441 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.441 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.442 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.442 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.442 INFO:teuthology.orchestra.run.smithi123.stdout: "redirect_resolve_ip_addr": { 2024-05-08T06:13:54.442 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "redirect_resolve_ip_addr", 2024-05-08T06:13:54.442 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.442 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.442 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.442 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.442 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.442 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.442 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.442 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.442 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.442 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.442 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.442 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.443 INFO:teuthology.orchestra.run.smithi123.stdout: "server_addr": { 2024-05-08T06:13:54.443 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_addr", 2024-05-08T06:13:54.443 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.443 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.443 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.443 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "::", 2024-05-08T06:13:54.443 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.443 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.443 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.443 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.443 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.443 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.443 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.443 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.444 INFO:teuthology.orchestra.run.smithi123.stdout: "server_port": { 2024-05-08T06:13:54.444 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_port", 2024-05-08T06:13:54.444 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.444 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.444 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.444 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8080", 2024-05-08T06:13:54.444 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.444 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.444 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.444 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.444 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.444 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.444 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.444 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.445 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.445 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.445 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.445 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.445 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.445 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.445 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.445 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.445 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.445 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.445 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.445 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.445 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.445 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.445 INFO:teuthology.orchestra.run.smithi123.stdout: "ssl": { 2024-05-08T06:13:54.446 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ssl", 2024-05-08T06:13:54.446 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.446 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.446 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.446 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.446 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.446 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.446 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.446 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.446 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.446 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.446 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.446 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.446 INFO:teuthology.orchestra.run.smithi123.stdout: "ssl_server_port": { 2024-05-08T06:13:54.446 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ssl_server_port", 2024-05-08T06:13:54.447 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.447 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.447 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.447 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8443", 2024-05-08T06:13:54.447 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.447 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.447 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.447 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.447 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.447 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.447 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.447 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.447 INFO:teuthology.orchestra.run.smithi123.stdout: "standby_behaviour": { 2024-05-08T06:13:54.447 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "standby_behaviour", 2024-05-08T06:13:54.447 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.447 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.448 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.448 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "redirect", 2024-05-08T06:13:54.448 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.448 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.448 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.448 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.448 INFO:teuthology.orchestra.run.smithi123.stdout: "redirect" 2024-05-08T06:13:54.448 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.448 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.448 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.448 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.448 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.448 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.448 INFO:teuthology.orchestra.run.smithi123.stdout: "standby_error_status_code": { 2024-05-08T06:13:54.448 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "standby_error_status_code", 2024-05-08T06:13:54.448 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.448 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.449 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.449 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "500", 2024-05-08T06:13:54.449 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "400", 2024-05-08T06:13:54.449 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "599", 2024-05-08T06:13:54.449 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.449 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.449 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.449 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.449 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.449 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.449 INFO:teuthology.orchestra.run.smithi123.stdout: "url_prefix": { 2024-05-08T06:13:54.449 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "url_prefix", 2024-05-08T06:13:54.449 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.449 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.449 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.450 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.450 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.450 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.450 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.450 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.450 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.450 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.450 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.450 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.450 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.450 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.450 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.450 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "devicehealth", 2024-05-08T06:13:54.450 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.450 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.450 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.451 INFO:teuthology.orchestra.run.smithi123.stdout: "enable_monitoring": { 2024-05-08T06:13:54.451 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "enable_monitoring", 2024-05-08T06:13:54.451 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.451 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.451 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.451 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.451 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.451 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.451 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.451 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "monitor device health metrics", 2024-05-08T06:13:54.451 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.451 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.451 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.451 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.451 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.451 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.452 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.452 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.452 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.452 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.452 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.452 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.452 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.452 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.452 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.452 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.452 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.452 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.452 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.452 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.452 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.452 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.453 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.453 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.453 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.453 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.453 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.453 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.453 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.453 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.453 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.453 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.453 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.453 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.453 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.453 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.453 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.453 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.454 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.454 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.454 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.454 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.454 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.454 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.454 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.454 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.454 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.454 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.454 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.454 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.454 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.454 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.454 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.455 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.455 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.455 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.455 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.455 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.455 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.455 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.455 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.455 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.455 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.455 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.455 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.455 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.455 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.455 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.455 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.456 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.456 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.456 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.456 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.456 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.456 INFO:teuthology.orchestra.run.smithi123.stdout: "mark_out_threshold": { 2024-05-08T06:13:54.456 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mark_out_threshold", 2024-05-08T06:13:54.456 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.456 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.456 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.456 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "2419200", 2024-05-08T06:13:54.456 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.456 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.456 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.456 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-08T06:13:54.456 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.456 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.457 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.457 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.457 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_name": { 2024-05-08T06:13:54.457 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "pool_name", 2024-05-08T06:13:54.457 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.457 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.457 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.457 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "device_health_metrics", 2024-05-08T06:13:54.457 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.457 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.457 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.457 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-08T06:13:54.457 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.457 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.457 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.457 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.458 INFO:teuthology.orchestra.run.smithi123.stdout: "retention_period": { 2024-05-08T06:13:54.458 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "retention_period", 2024-05-08T06:13:54.458 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.458 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.458 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.458 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "15552000", 2024-05-08T06:13:54.458 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.458 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.458 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.458 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how long to retain device health metrics", 2024-05-08T06:13:54.458 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.458 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.458 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.458 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.458 INFO:teuthology.orchestra.run.smithi123.stdout: "scrape_frequency": { 2024-05-08T06:13:54.459 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "scrape_frequency", 2024-05-08T06:13:54.459 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.459 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.459 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.459 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "86400", 2024-05-08T06:13:54.459 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.459 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.459 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.459 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-08T06:13:54.459 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.459 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.459 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.459 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.459 INFO:teuthology.orchestra.run.smithi123.stdout: "self_heal": { 2024-05-08T06:13:54.459 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "self_heal", 2024-05-08T06:13:54.459 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.460 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.460 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.460 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.460 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.460 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.460 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.460 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-08T06:13:54.460 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.460 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.460 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.460 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.460 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-05-08T06:13:54.460 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-05-08T06:13:54.460 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.460 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.460 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.460 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-05-08T06:13:54.461 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.461 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.461 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.461 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to wake up and check device health", 2024-05-08T06:13:54.461 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.461 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.461 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.461 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.461 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.461 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.461 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.461 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.462 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.462 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.462 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.462 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.462 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.462 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.462 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.462 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.462 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.462 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.462 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_threshold": { 2024-05-08T06:13:54.462 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_threshold", 2024-05-08T06:13:54.462 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.463 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.463 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.463 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "7257600", 2024-05-08T06:13:54.463 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.463 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.463 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.463 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-08T06:13:54.463 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.463 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.463 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.463 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.463 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.463 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.463 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.463 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "diskprediction_local", 2024-05-08T06:13:54.463 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.463 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.464 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.464 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.464 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.464 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.464 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.464 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.464 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.464 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.464 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.464 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.464 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.464 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.464 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.464 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.464 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.465 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.465 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.465 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.465 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.465 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.465 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.465 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.465 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.465 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.465 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.465 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.465 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.465 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.465 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.465 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.465 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.466 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.466 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.466 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.466 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.466 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.466 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.466 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.466 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.466 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.466 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.466 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.466 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.466 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.467 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.467 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.467 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.467 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.467 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.467 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.467 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.467 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.467 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.467 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.467 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.467 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.467 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.467 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.467 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.467 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.468 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.468 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.468 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.468 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.468 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.468 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.468 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.468 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.468 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.468 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.468 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.468 INFO:teuthology.orchestra.run.smithi123.stdout: "predict_interval": { 2024-05-08T06:13:54.468 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "predict_interval", 2024-05-08T06:13:54.468 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.469 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.469 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.469 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "86400", 2024-05-08T06:13:54.469 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.469 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.469 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.469 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.469 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.469 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.469 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.469 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.469 INFO:teuthology.orchestra.run.smithi123.stdout: "predictor_model": { 2024-05-08T06:13:54.469 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "predictor_model", 2024-05-08T06:13:54.469 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.469 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.470 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.470 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "prophetstor", 2024-05-08T06:13:54.470 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.470 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.470 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.470 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.470 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.470 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.470 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.470 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.470 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-05-08T06:13:54.470 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-05-08T06:13:54.470 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.471 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.471 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.471 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-05-08T06:13:54.471 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.471 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.471 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.471 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.471 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.471 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.471 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.471 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.471 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.471 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.471 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.471 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.472 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.472 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.472 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.472 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.472 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.472 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.472 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.472 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.472 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.472 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.472 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.472 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.472 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.472 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "influx", 2024-05-08T06:13:54.472 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": false, 2024-05-08T06:13:54.472 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "influxdb python module not found", 2024-05-08T06:13:54.473 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.473 INFO:teuthology.orchestra.run.smithi123.stdout: "batch_size": { 2024-05-08T06:13:54.473 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "batch_size", 2024-05-08T06:13:54.473 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.473 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.473 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.473 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5000", 2024-05-08T06:13:54.473 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.473 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.473 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.473 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-08T06:13:54.473 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.473 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.473 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.473 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.473 INFO:teuthology.orchestra.run.smithi123.stdout: "database": { 2024-05-08T06:13:54.474 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "database", 2024-05-08T06:13:54.474 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.474 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.474 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.474 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "ceph", 2024-05-08T06:13:54.474 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.474 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.474 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.474 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-08T06:13:54.474 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.474 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.474 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.474 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.474 INFO:teuthology.orchestra.run.smithi123.stdout: "hostname": { 2024-05-08T06:13:54.474 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "hostname", 2024-05-08T06:13:54.475 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.475 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.475 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.475 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.475 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.475 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.475 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.475 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "InfluxDB server hostname", 2024-05-08T06:13:54.475 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.475 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.475 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.475 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.475 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-05-08T06:13:54.475 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-05-08T06:13:54.475 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.475 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.476 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.476 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "30", 2024-05-08T06:13:54.476 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "5", 2024-05-08T06:13:54.476 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.476 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.476 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-08T06:13:54.476 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.476 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.476 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.476 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.476 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.476 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.476 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.476 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.476 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.476 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.477 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.477 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.477 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.477 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.477 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.477 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.477 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.477 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.477 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.477 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.477 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.477 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.477 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.477 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.478 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.478 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.478 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.478 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.478 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.478 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.478 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.478 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.478 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.478 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.478 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.478 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.478 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.478 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.478 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.479 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.479 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.479 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.479 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.479 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.479 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.479 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.479 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.479 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.479 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.479 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.479 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.479 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.479 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.479 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.480 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.480 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.480 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.480 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.480 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.480 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.480 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.480 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.480 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.480 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.480 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.480 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.480 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.480 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.480 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.480 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.481 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.481 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.481 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.481 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.481 INFO:teuthology.orchestra.run.smithi123.stdout: "password": { 2024-05-08T06:13:54.481 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "password", 2024-05-08T06:13:54.481 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.481 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.481 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.481 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.481 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.481 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.481 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.481 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "password of InfluxDB server user", 2024-05-08T06:13:54.481 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.482 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.482 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.482 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.482 INFO:teuthology.orchestra.run.smithi123.stdout: "port": { 2024-05-08T06:13:54.482 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "port", 2024-05-08T06:13:54.482 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.482 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.482 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.482 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8086", 2024-05-08T06:13:54.482 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.482 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.482 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.482 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "InfluxDB server port", 2024-05-08T06:13:54.482 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.482 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.483 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.483 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.483 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.483 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.483 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.483 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.483 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.483 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.483 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.483 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.483 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.483 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.483 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.483 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.483 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.484 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.484 INFO:teuthology.orchestra.run.smithi123.stdout: "ssl": { 2024-05-08T06:13:54.484 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ssl", 2024-05-08T06:13:54.484 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.484 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.484 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.484 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "false", 2024-05-08T06:13:54.484 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.484 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.484 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.484 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T06:13:54.484 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.484 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.484 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.485 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.485 INFO:teuthology.orchestra.run.smithi123.stdout: "threads": { 2024-05-08T06:13:54.485 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "threads", 2024-05-08T06:13:54.485 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.485 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.485 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.485 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-05-08T06:13:54.485 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1", 2024-05-08T06:13:54.485 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "32", 2024-05-08T06:13:54.485 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.485 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-08T06:13:54.485 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.485 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.486 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.486 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.486 INFO:teuthology.orchestra.run.smithi123.stdout: "username": { 2024-05-08T06:13:54.486 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "username", 2024-05-08T06:13:54.486 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.486 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.486 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.486 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.486 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.486 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.486 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.486 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "username of InfluxDB server user", 2024-05-08T06:13:54.486 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.486 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.487 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.487 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.487 INFO:teuthology.orchestra.run.smithi123.stdout: "verify_ssl": { 2024-05-08T06:13:54.487 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "verify_ssl", 2024-05-08T06:13:54.487 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.487 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.487 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.487 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "true", 2024-05-08T06:13:54.487 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.487 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.487 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.487 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T06:13:54.487 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.487 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.487 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.488 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.488 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.488 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.488 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.488 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "insights", 2024-05-08T06:13:54.488 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.488 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.488 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.488 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.488 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.488 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.488 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.488 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.488 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.489 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.489 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.489 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.489 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.489 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.489 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.489 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.489 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.489 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.489 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.489 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.489 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.489 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.489 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.489 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.490 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.490 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.490 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.490 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.490 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.490 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.490 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.490 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.490 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.490 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.490 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.490 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.490 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.490 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.490 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.491 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.491 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.491 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.491 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.491 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.491 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.491 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.491 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.491 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.491 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.491 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.491 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.491 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.491 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.491 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.491 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.492 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.492 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.492 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.492 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.492 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.492 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.492 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.492 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.492 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.492 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.492 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.492 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.492 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.492 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.492 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.493 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.493 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.493 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.493 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.493 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.493 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.493 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.493 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.493 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.493 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.493 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.493 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.493 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.493 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.493 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.493 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.493 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.494 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.494 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.494 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.494 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "iostat", 2024-05-08T06:13:54.494 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.494 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.494 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.494 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.494 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.494 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.494 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.494 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.494 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.494 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.494 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.495 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.495 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.495 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.495 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.495 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.495 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.495 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.495 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.495 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.495 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.495 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.495 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.495 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.495 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.496 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.496 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.496 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.496 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.496 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.496 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.496 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.496 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.496 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.496 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.496 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.496 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.496 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.496 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.496 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.497 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.497 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.497 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.497 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.497 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.497 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.497 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.497 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.497 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.497 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.497 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.497 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.497 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.497 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.497 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.498 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.498 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.498 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.498 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.498 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.498 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.498 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.498 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.498 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.498 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.498 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.498 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.498 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.499 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.499 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.499 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.499 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.499 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.499 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.499 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.499 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.499 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.499 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.499 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.499 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.499 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.499 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.499 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.500 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.500 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.500 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.500 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.500 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.500 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.500 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.500 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "k8sevents", 2024-05-08T06:13:54.500 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.500 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.500 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.500 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_event_retention_days": { 2024-05-08T06:13:54.500 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ceph_event_retention_days", 2024-05-08T06:13:54.500 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.500 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.501 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.501 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "7", 2024-05-08T06:13:54.501 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.501 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.501 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.501 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-08T06:13:54.501 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.501 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.501 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.501 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.501 INFO:teuthology.orchestra.run.smithi123.stdout: "config_check_secs": { 2024-05-08T06:13:54.501 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "config_check_secs", 2024-05-08T06:13:54.501 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.501 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.501 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.501 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-05-08T06:13:54.502 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "10", 2024-05-08T06:13:54.502 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.502 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.502 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-08T06:13:54.502 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.502 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.502 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.502 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.502 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.502 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.502 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.502 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.502 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.502 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.503 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.503 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.503 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.503 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.503 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.503 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.503 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.503 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.503 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.503 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.503 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.503 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.503 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.503 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.503 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.504 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.504 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.504 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.504 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.504 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.504 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.504 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.504 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.504 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.504 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.504 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.504 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.504 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.504 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.504 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.504 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.505 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.505 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.505 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.505 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.505 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.505 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.505 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.505 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.505 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.505 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.505 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.505 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.505 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.505 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.505 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.506 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.506 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.506 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.506 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.506 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.506 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.506 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.506 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.506 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.506 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.506 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.506 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.506 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.506 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.506 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.507 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.507 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.507 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.507 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.507 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.507 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.507 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.507 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.507 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.507 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.507 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.507 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.507 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.507 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.507 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.508 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.508 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.508 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.508 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.508 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.508 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "localpool", 2024-05-08T06:13:54.508 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.508 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.508 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.508 INFO:teuthology.orchestra.run.smithi123.stdout: "failure_domain": { 2024-05-08T06:13:54.508 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "failure_domain", 2024-05-08T06:13:54.508 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.508 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.508 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.509 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "host", 2024-05-08T06:13:54.509 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.509 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.509 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.509 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "failure domain for any created local pool", 2024-05-08T06:13:54.509 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-08T06:13:54.509 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.509 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.509 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.509 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.509 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.509 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.509 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.509 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.509 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.509 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.510 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.510 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.510 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.510 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.510 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.510 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.510 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.510 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.510 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.510 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.510 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.510 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.510 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.510 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.510 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.510 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.511 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.511 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.511 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.511 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.511 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.511 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.511 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.511 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.511 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.511 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.511 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.511 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.511 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.511 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.512 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.512 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.512 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.512 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.512 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.512 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.512 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.512 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.512 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.512 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.512 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.512 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.512 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.513 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.513 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.513 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.513 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.513 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.513 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.513 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.513 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.513 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.513 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.513 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.513 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.513 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.514 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.514 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.514 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.514 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.514 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.514 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.514 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.514 INFO:teuthology.orchestra.run.smithi123.stdout: "min_size": { 2024-05-08T06:13:54.514 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "min_size", 2024-05-08T06:13:54.514 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.514 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.514 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.514 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.514 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.514 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.515 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.515 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "default min_size for any created local pool", 2024-05-08T06:13:54.515 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-08T06:13:54.515 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.515 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.515 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.515 INFO:teuthology.orchestra.run.smithi123.stdout: "num_rep": { 2024-05-08T06:13:54.515 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "num_rep", 2024-05-08T06:13:54.515 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.515 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.515 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.515 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "3", 2024-05-08T06:13:54.515 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.515 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.515 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.515 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "default replica count for any created local pool", 2024-05-08T06:13:54.516 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.516 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.516 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.516 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.516 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num": { 2024-05-08T06:13:54.516 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "pg_num", 2024-05-08T06:13:54.516 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.516 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.516 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.516 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "128", 2024-05-08T06:13:54.516 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.516 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.516 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.516 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "default pg_num for any created local pool", 2024-05-08T06:13:54.516 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.517 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.517 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.517 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.517 INFO:teuthology.orchestra.run.smithi123.stdout: "prefix": { 2024-05-08T06:13:54.517 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "prefix", 2024-05-08T06:13:54.517 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.517 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.517 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.517 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.517 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.517 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.517 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.517 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "name prefix for any created local pool", 2024-05-08T06:13:54.517 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.518 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.518 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.518 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.518 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.518 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.518 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.518 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.518 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.518 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.518 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.518 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.518 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.518 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.518 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.519 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.519 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.519 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.519 INFO:teuthology.orchestra.run.smithi123.stdout: "subtree": { 2024-05-08T06:13:54.519 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "subtree", 2024-05-08T06:13:54.519 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.519 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.519 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.519 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "rack", 2024-05-08T06:13:54.519 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.519 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.519 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.519 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-08T06:13:54.519 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-08T06:13:54.519 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.520 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.520 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.520 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.520 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.520 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.520 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mds_autoscaler", 2024-05-08T06:13:54.520 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.520 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.520 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.520 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.520 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.520 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.520 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.520 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.520 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.520 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.520 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.521 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.521 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.521 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.521 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.521 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.521 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.521 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.521 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.521 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.521 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.521 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.521 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.521 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.521 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.521 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.521 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.522 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.522 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.522 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.522 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.522 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.522 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.522 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.522 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.522 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.522 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.522 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.522 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.522 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.522 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.522 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.523 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.523 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.523 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.523 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.523 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.523 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.523 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.523 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.523 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.523 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.523 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.523 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.523 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.523 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.523 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.523 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.523 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.524 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.524 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.524 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.524 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.524 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.524 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.524 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.524 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.524 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.524 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.524 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.524 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.524 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.524 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.524 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.525 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.525 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.525 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.525 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.525 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.525 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.525 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.525 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.525 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.525 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.525 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.525 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.525 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.525 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.525 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.525 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.526 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mirroring", 2024-05-08T06:13:54.526 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.526 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.526 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.526 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.526 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.526 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.526 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.526 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.526 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.526 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.526 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.526 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.526 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.526 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.527 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.527 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.527 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.527 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.527 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.527 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.527 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.527 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.527 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.527 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.527 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.527 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.527 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.527 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.527 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.527 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.527 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.528 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.528 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.528 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.528 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.528 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.528 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.528 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.528 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.528 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.528 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.528 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.528 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.528 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.528 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.528 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.528 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.529 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.529 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.529 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.529 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.529 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.529 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.529 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.529 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.529 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.529 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.529 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.529 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.529 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.529 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.529 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.529 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.530 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.530 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.530 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.530 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.530 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.530 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.530 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.530 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.530 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.530 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.530 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.530 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.530 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.530 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.530 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.530 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.531 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.531 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.531 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.531 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.531 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.531 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.531 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.531 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.531 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.531 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.531 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.531 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "nfs", 2024-05-08T06:13:54.531 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.531 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.531 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.531 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.532 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.532 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.532 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.532 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.532 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.532 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.532 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.532 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.532 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.532 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.532 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.532 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.532 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.532 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.532 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.532 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.532 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.533 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.533 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.533 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.533 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.533 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.533 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.533 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.533 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.533 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.533 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.533 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.533 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.533 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.533 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.533 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.534 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.534 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.534 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.534 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.534 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.534 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.534 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.534 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.534 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.534 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.534 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.534 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.534 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.534 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.534 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.535 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.535 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.535 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.535 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.535 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.535 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.535 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.535 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.535 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.535 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.535 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.535 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.535 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.535 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.535 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.535 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.536 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.536 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.536 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.536 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.536 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.536 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.536 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.536 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.536 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.536 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.536 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.536 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.536 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.536 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.536 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.536 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.537 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.537 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.537 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "orchestrator", 2024-05-08T06:13:54.537 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.537 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.537 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.537 INFO:teuthology.orchestra.run.smithi123.stdout: "fail_fs": { 2024-05-08T06:13:54.537 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "fail_fs", 2024-05-08T06:13:54.537 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.537 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.537 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.537 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.537 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.537 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.538 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.538 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-08T06:13:54.538 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.538 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.538 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.538 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.538 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.538 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.538 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.538 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.538 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.538 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.538 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.538 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.538 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.539 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.539 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.539 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.539 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.539 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.539 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.539 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.539 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.539 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.539 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.539 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.539 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.539 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.539 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.539 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.539 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.540 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.540 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.540 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.540 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.540 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.540 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.540 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.540 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.540 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.540 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.540 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.540 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.540 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.540 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.540 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.541 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.541 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.541 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.541 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.541 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.541 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.541 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.541 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.541 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.541 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.541 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.541 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.541 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.541 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.541 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.541 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.542 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.542 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.542 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.542 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.542 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.542 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.542 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.542 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.542 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.542 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.542 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.542 INFO:teuthology.orchestra.run.smithi123.stdout: "orchestrator": { 2024-05-08T06:13:54.542 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "orchestrator", 2024-05-08T06:13:54.542 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.543 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.543 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.543 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.543 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.543 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.543 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.543 INFO:teuthology.orchestra.run.smithi123.stdout: "cephadm", 2024-05-08T06:13:54.543 INFO:teuthology.orchestra.run.smithi123.stdout: "rook", 2024-05-08T06:13:54.543 INFO:teuthology.orchestra.run.smithi123.stdout: "test_orchestrator" 2024-05-08T06:13:54.543 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.543 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Orchestrator backend", 2024-05-08T06:13:54.543 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.543 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.543 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.543 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.543 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.544 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.544 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.544 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.544 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.544 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.544 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.544 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.544 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.544 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.544 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.544 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.544 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.544 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.544 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.544 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.544 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "osd_perf_query", 2024-05-08T06:13:54.545 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.545 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.545 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.545 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.545 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.545 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.545 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.545 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.545 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.545 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.545 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.545 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.545 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.545 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.545 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.545 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.546 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.546 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.546 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.546 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.546 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.546 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.546 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.546 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.546 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.546 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.546 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.546 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.546 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.547 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.547 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.547 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.547 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.547 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.547 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.547 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.547 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.547 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.547 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.547 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.547 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.548 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.548 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.548 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.548 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.548 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.548 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.548 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.548 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.548 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.548 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.548 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.548 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.548 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.548 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.549 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.549 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.549 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.549 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.549 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.549 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.549 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.549 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.549 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.549 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.549 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.549 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.549 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.550 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.550 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.550 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.550 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.550 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.550 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.550 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.550 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.550 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.550 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.550 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.550 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.551 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.551 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.551 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.551 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.551 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.551 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.551 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.551 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "osd_support", 2024-05-08T06:13:54.551 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.551 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.551 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.551 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.551 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.551 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.552 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.552 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.552 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.552 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.552 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.552 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.552 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.552 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.552 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.552 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.552 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.552 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.552 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.552 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.552 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.553 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.553 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.553 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.553 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.553 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.553 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.553 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.553 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.553 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.553 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.553 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.553 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.553 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.553 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.553 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.554 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.554 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.554 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.554 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.554 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.554 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.554 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.554 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.554 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.554 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.554 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.554 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.554 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.554 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.554 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.555 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.555 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.555 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.555 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.555 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.555 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.555 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.555 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.555 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.555 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.555 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.555 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.555 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.556 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.556 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.556 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.556 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.556 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.556 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.556 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.556 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.556 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.556 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.556 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.556 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.556 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.556 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.557 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.557 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.557 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.557 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.557 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.557 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.557 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.557 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.557 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "pg_autoscaler", 2024-05-08T06:13:54.557 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.557 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.557 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.557 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.557 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.558 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.558 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.558 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.558 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.558 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.558 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.558 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.558 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.558 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.558 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.558 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.558 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.558 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.558 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.559 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.559 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.559 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.559 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.559 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.559 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.559 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.559 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.559 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.559 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.559 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.559 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.559 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.560 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.560 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.560 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.560 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.560 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.560 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.560 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.560 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.560 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.560 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.560 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.560 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.560 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.560 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.561 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.561 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.561 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.561 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.561 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.561 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.561 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.561 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.561 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.561 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.561 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.561 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.561 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.561 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.562 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.562 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.562 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.562 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.562 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.562 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.562 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.562 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.562 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.562 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.562 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-05-08T06:13:54.562 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-05-08T06:13:54.562 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.562 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.563 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.563 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-05-08T06:13:54.563 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.563 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.563 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.563 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.563 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.563 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.563 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.563 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.563 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.563 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.563 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.564 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.564 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.564 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.564 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.564 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.564 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.564 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.564 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.564 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.564 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.564 INFO:teuthology.orchestra.run.smithi123.stdout: "threshold": { 2024-05-08T06:13:54.564 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "threshold", 2024-05-08T06:13:54.564 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-05-08T06:13:54.564 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.565 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.565 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "3.0", 2024-05-08T06:13:54.565 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1.0", 2024-05-08T06:13:54.565 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.565 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.565 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "scaling threshold", 2024-05-08T06:13:54.565 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-08T06:13:54.565 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.565 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.565 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.565 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.565 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "progress", 2024-05-08T06:13:54.565 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.566 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.566 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.566 INFO:teuthology.orchestra.run.smithi123.stdout: "allow_pg_recovery_event": { 2024-05-08T06:13:54.566 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "allow_pg_recovery_event", 2024-05-08T06:13:54.566 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.566 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.566 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.566 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.566 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.566 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.566 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.566 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-08T06:13:54.566 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.567 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.567 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.567 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.567 INFO:teuthology.orchestra.run.smithi123.stdout: "enabled": { 2024-05-08T06:13:54.567 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "enabled", 2024-05-08T06:13:54.567 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.567 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.567 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.567 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.567 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.567 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.567 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.567 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.567 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.567 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.568 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.568 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.568 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.568 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.568 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.568 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.568 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.568 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.568 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.568 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.568 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.568 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.568 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.568 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.569 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.569 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.569 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.569 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.569 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.569 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.569 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.569 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.569 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.569 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.569 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.569 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.569 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.569 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.570 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.570 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.570 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.570 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.570 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.570 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.570 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.570 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.570 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.570 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.570 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.570 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.570 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.570 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.571 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.571 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.571 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.571 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.571 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.571 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.571 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.571 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.571 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.571 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.571 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.571 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.571 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.571 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.571 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.571 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.571 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.572 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.572 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.572 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.572 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.572 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.572 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.572 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.572 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.572 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.572 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.572 INFO:teuthology.orchestra.run.smithi123.stdout: "max_completed_events": { 2024-05-08T06:13:54.572 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_completed_events", 2024-05-08T06:13:54.572 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.572 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.573 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.573 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "50", 2024-05-08T06:13:54.573 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.573 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.573 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.573 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "number of past completed events to remember", 2024-05-08T06:13:54.573 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.573 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.573 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.573 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-05-08T06:13:54.573 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-05-08T06:13:54.573 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.573 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.573 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.573 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-05-08T06:13:54.574 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.574 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.574 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.574 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how long the module is going to sleep", 2024-05-08T06:13:54.574 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.574 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.574 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.574 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.574 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.574 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.574 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.574 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.574 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.575 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.575 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.575 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.575 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.575 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.575 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.575 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.575 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.575 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.575 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.575 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.575 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "prometheus", 2024-05-08T06:13:54.575 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.575 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.575 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.575 INFO:teuthology.orchestra.run.smithi123.stdout: "cache": { 2024-05-08T06:13:54.575 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cache", 2024-05-08T06:13:54.576 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.576 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.576 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.576 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.576 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.576 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.576 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.576 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.576 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.576 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.576 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.576 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.576 INFO:teuthology.orchestra.run.smithi123.stdout: "exclude_perf_counters": { 2024-05-08T06:13:54.576 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "exclude_perf_counters", 2024-05-08T06:13:54.576 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.577 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.577 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.577 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.577 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.577 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.577 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.577 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-08T06:13:54.577 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-08T06:13:54.577 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.577 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.577 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.577 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.577 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.578 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.578 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.578 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.578 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.578 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.578 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.578 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.578 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.578 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.578 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.578 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.578 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.578 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.578 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.578 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.579 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.579 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.579 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.579 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.579 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.579 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.579 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.579 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.579 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.579 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.579 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.579 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.579 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.579 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.580 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.580 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.580 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.580 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.580 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.580 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.580 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.580 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.580 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.580 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.580 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.580 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.580 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.580 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.580 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.581 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.581 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.581 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.581 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.581 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.581 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.581 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.581 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.581 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.581 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.581 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.581 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.581 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.581 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.582 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.582 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.582 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.582 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.582 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.582 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.582 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.582 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_stats_pools": { 2024-05-08T06:13:54.582 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rbd_stats_pools", 2024-05-08T06:13:54.582 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.582 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.582 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.582 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.582 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.582 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.583 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.583 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.583 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.583 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.583 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.583 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-08T06:13:54.583 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-08T06:13:54.583 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.583 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.583 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.583 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "300", 2024-05-08T06:13:54.583 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.583 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.583 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.584 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.584 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.584 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.584 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.584 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.584 INFO:teuthology.orchestra.run.smithi123.stdout: "scrape_interval": { 2024-05-08T06:13:54.584 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "scrape_interval", 2024-05-08T06:13:54.584 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-05-08T06:13:54.584 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.584 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.584 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "15.0", 2024-05-08T06:13:54.584 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.584 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.584 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.584 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.585 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.585 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.585 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.585 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.585 INFO:teuthology.orchestra.run.smithi123.stdout: "server_addr": { 2024-05-08T06:13:54.585 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_addr", 2024-05-08T06:13:54.585 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.585 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.585 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.585 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "::", 2024-05-08T06:13:54.585 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.585 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.585 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.585 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-08T06:13:54.585 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.586 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.586 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.586 INFO:teuthology.orchestra.run.smithi123.stdout: "server_port": { 2024-05-08T06:13:54.586 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_port", 2024-05-08T06:13:54.586 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.586 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.586 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.586 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "9283", 2024-05-08T06:13:54.586 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.586 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.586 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.586 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-08T06:13:54.586 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.586 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.587 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.587 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.587 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.587 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.587 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.587 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.587 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.587 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.587 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.587 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.587 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.587 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.587 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.588 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.588 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.588 INFO:teuthology.orchestra.run.smithi123.stdout: "stale_cache_strategy": { 2024-05-08T06:13:54.588 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "stale_cache_strategy", 2024-05-08T06:13:54.588 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.588 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.588 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.588 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "log", 2024-05-08T06:13:54.588 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.588 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.589 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:53 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/3394650494' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T06:13:54.589 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:53 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/2782919950' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T06:13:54.590 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:53 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/1172555814' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-08T06:13:54.590 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:53 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/3394650494' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T06:13:54.590 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:53 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/2782919950' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T06:13:54.590 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:53 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/1172555814' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-08T06:13:54.593 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.594 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.594 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.594 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.594 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.594 INFO:teuthology.orchestra.run.smithi123.stdout: "standby_behaviour": { 2024-05-08T06:13:54.594 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "standby_behaviour", 2024-05-08T06:13:54.594 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.594 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.594 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.594 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "default", 2024-05-08T06:13:54.594 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.595 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.595 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.595 INFO:teuthology.orchestra.run.smithi123.stdout: "default", 2024-05-08T06:13:54.595 INFO:teuthology.orchestra.run.smithi123.stdout: "error" 2024-05-08T06:13:54.595 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.595 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.595 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.595 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.595 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.595 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.595 INFO:teuthology.orchestra.run.smithi123.stdout: "standby_error_status_code": { 2024-05-08T06:13:54.595 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "standby_error_status_code", 2024-05-08T06:13:54.595 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.596 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.596 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.596 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "500", 2024-05-08T06:13:54.596 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "400", 2024-05-08T06:13:54.596 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "599", 2024-05-08T06:13:54.596 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.596 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.596 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.596 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.596 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.596 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.596 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.596 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.596 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.597 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rbd_support", 2024-05-08T06:13:54.597 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.597 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.597 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.597 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.597 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.597 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.597 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.597 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.597 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.597 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.597 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.597 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.597 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.597 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.598 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.598 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.598 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.598 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.598 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.598 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.598 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.598 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.598 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.598 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.598 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.598 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.599 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.599 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.599 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.599 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.599 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.599 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.599 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.599 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.599 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.599 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.599 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.599 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.599 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.600 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.600 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.600 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.600 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.600 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.600 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.600 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.600 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.600 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.600 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.600 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.600 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.600 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.600 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.601 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.601 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.601 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.601 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.601 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.601 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.601 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.601 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.601 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.601 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.601 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.601 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.601 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.602 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.602 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.602 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.602 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.602 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.602 INFO:teuthology.orchestra.run.smithi123.stdout: "max_concurrent_snap_create": { 2024-05-08T06:13:54.602 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_concurrent_snap_create", 2024-05-08T06:13:54.602 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.602 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.602 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.602 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-05-08T06:13:54.602 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.602 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.602 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.603 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.603 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.603 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.603 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.603 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.603 INFO:teuthology.orchestra.run.smithi123.stdout: "mirror_snapshot_schedule": { 2024-05-08T06:13:54.603 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mirror_snapshot_schedule", 2024-05-08T06:13:54.603 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.603 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.603 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.603 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.603 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.603 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.603 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.603 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.603 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.603 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.604 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.604 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.604 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.604 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.604 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.604 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.604 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.604 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.604 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.604 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.604 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.604 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.604 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.604 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.604 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.604 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.605 INFO:teuthology.orchestra.run.smithi123.stdout: "trash_purge_schedule": { 2024-05-08T06:13:54.605 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "trash_purge_schedule", 2024-05-08T06:13:54.605 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.605 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.605 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.605 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.605 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.605 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.605 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.605 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.605 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.605 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.605 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.605 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.605 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.606 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.606 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.606 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "restful", 2024-05-08T06:13:54.606 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.606 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.606 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.606 INFO:teuthology.orchestra.run.smithi123.stdout: "enable_auth": { 2024-05-08T06:13:54.606 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "enable_auth", 2024-05-08T06:13:54.606 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.606 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.606 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.606 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.606 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.606 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.606 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.606 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.607 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.607 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.607 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.607 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.607 INFO:teuthology.orchestra.run.smithi123.stdout: "key_file": { 2024-05-08T06:13:54.607 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "key_file", 2024-05-08T06:13:54.607 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.607 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.607 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.607 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.607 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.607 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.607 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.608 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.608 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.608 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.608 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.608 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.608 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.608 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.608 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.608 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.608 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.608 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.608 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.608 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.608 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.608 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.608 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.609 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.609 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.609 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.609 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.609 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.609 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.609 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.609 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.609 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.609 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.609 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.609 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.609 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.609 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.609 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.610 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.610 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.610 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.610 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.610 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.610 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.610 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.610 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.610 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.610 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.610 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.610 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.610 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.610 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.610 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.611 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.611 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.611 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.611 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.611 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.611 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.611 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.611 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.611 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.611 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.611 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.611 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.611 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.611 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.611 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.612 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.612 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.612 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.612 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.612 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.612 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.612 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.612 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.612 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.612 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.612 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.612 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.612 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.613 INFO:teuthology.orchestra.run.smithi123.stdout: "server_addr": { 2024-05-08T06:13:54.613 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_addr", 2024-05-08T06:13:54.613 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.613 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.613 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.613 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.613 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.613 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.613 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.613 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.613 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.613 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.613 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.613 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.613 INFO:teuthology.orchestra.run.smithi123.stdout: "server_port": { 2024-05-08T06:13:54.614 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_port", 2024-05-08T06:13:54.614 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.614 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.614 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.614 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.614 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.614 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.614 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.614 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.614 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.614 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.614 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.614 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.614 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.614 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.614 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.615 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.615 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.615 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.615 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.615 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.615 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.615 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.615 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.615 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.615 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.615 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.615 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.615 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.615 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rgw", 2024-05-08T06:13:54.615 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.616 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.616 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.616 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.616 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.616 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.616 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.616 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.616 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.616 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.616 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.616 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.616 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.616 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.616 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.616 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.616 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.616 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.617 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.617 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.617 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.617 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.617 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.617 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.617 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.617 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.617 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.617 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.617 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.617 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.617 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.617 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.617 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.618 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.618 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.618 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.618 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.618 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.618 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.618 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.618 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.618 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.618 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.618 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.618 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.618 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.618 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.618 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.619 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.619 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.619 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.619 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.619 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.619 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.619 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.619 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.619 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.619 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.619 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.619 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.619 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.619 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.619 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.620 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.620 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.620 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.620 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.620 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.620 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.620 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.620 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.620 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.620 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.620 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.620 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.620 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.620 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.620 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.620 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.621 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.621 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.621 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.621 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.621 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.621 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.621 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.621 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.621 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.621 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.621 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.621 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rook", 2024-05-08T06:13:54.621 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.621 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.621 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.621 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.622 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.622 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.622 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.622 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.622 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.622 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.622 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.622 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.622 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.622 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.622 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.622 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.622 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.623 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.623 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.623 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.623 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.623 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.623 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.623 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.623 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.623 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.623 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.623 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.623 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.623 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.623 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.623 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.624 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.624 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.624 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.624 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.624 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.624 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.624 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.624 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.624 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.624 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.624 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.624 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.624 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.624 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.624 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.624 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.625 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.625 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.625 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.625 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.625 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.625 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.625 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.625 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.625 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.625 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.625 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.625 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.625 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.625 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.626 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.626 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.626 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.626 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.626 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.626 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.626 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.626 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.626 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.626 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.626 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.626 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.626 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.626 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.627 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.627 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.627 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.627 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.627 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.627 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.627 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.627 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.627 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.627 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.627 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.627 INFO:teuthology.orchestra.run.smithi123.stdout: "storage_class": { 2024-05-08T06:13:54.627 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "storage_class", 2024-05-08T06:13:54.627 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.628 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.628 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.628 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "local", 2024-05-08T06:13:54.628 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.628 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.628 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.628 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-08T06:13:54.628 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.628 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.628 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.628 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.628 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.628 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.628 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.628 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "selftest", 2024-05-08T06:13:54.629 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.629 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.629 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.629 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.629 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.629 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.629 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.629 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.629 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.629 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.629 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.629 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.629 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.629 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.629 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.629 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.630 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.630 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.630 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.630 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.630 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.630 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.630 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.630 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.630 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.630 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.630 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.630 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.630 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.630 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.630 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.631 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.631 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.631 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.631 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.631 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.631 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.631 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.631 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.631 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.631 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.631 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.631 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.631 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.631 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.631 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.631 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.632 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.632 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.632 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.632 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.632 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.632 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.632 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.632 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.632 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.632 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.632 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.632 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.632 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.632 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.633 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.633 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.633 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.633 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.633 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.633 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.633 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.633 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.633 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.633 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.633 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.633 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.633 INFO:teuthology.orchestra.run.smithi123.stdout: "roption1": { 2024-05-08T06:13:54.633 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "roption1", 2024-05-08T06:13:54.633 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.634 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.634 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.634 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.634 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.634 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.634 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.634 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.634 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.634 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.634 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.634 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.634 INFO:teuthology.orchestra.run.smithi123.stdout: "roption2": { 2024-05-08T06:13:54.634 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "roption2", 2024-05-08T06:13:54.634 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.634 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.635 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.635 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "xyz", 2024-05-08T06:13:54.635 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.635 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.635 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.635 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.635 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.635 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.635 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.635 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.635 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption1": { 2024-05-08T06:13:54.635 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption1", 2024-05-08T06:13:54.635 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.635 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.636 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.636 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.636 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.636 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.636 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.636 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.636 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.636 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.636 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.636 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.636 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption2": { 2024-05-08T06:13:54.636 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption2", 2024-05-08T06:13:54.636 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.636 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.636 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.637 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.637 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.637 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.637 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.637 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.637 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.637 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.637 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.637 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.637 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption3": { 2024-05-08T06:13:54.637 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption3", 2024-05-08T06:13:54.637 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-05-08T06:13:54.637 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.637 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.637 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.638 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.638 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.638 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.638 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.638 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.638 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.638 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.638 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.638 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption4": { 2024-05-08T06:13:54.638 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption4", 2024-05-08T06:13:54.638 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.638 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.638 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.639 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.639 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.639 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.639 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.639 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.639 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.639 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.639 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.639 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.639 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption5": { 2024-05-08T06:13:54.639 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption5", 2024-05-08T06:13:54.639 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.639 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.639 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.640 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.640 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.640 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.640 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.640 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.640 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.640 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.640 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.640 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.640 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption6": { 2024-05-08T06:13:54.640 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption6", 2024-05-08T06:13:54.640 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.640 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.640 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.640 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.640 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.641 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.641 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.641 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.641 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.641 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.641 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.641 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.641 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption7": { 2024-05-08T06:13:54.641 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption7", 2024-05-08T06:13:54.641 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.641 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.641 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.641 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.641 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1", 2024-05-08T06:13:54.641 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "42", 2024-05-08T06:13:54.642 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.642 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.642 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.642 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.642 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.642 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.642 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.642 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.642 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.642 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.642 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.642 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.642 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.642 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.643 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.643 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.643 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.643 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.643 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.643 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.643 INFO:teuthology.orchestra.run.smithi123.stdout: "testkey": { 2024-05-08T06:13:54.643 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "testkey", 2024-05-08T06:13:54.643 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.643 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.643 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.643 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.643 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.643 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.644 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.644 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.644 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.644 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.644 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.644 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.644 INFO:teuthology.orchestra.run.smithi123.stdout: "testlkey": { 2024-05-08T06:13:54.644 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "testlkey", 2024-05-08T06:13:54.644 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.644 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.644 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.644 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.644 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.644 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.645 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.645 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.645 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.645 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.645 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.645 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.645 INFO:teuthology.orchestra.run.smithi123.stdout: "testnewline": { 2024-05-08T06:13:54.645 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "testnewline", 2024-05-08T06:13:54.645 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.645 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.645 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.645 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.645 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.645 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.645 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.646 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.646 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.646 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.646 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.646 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.646 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.646 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.646 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.646 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smb", 2024-05-08T06:13:54.646 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.646 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.646 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.646 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.646 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.647 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.647 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.647 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.647 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.647 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.647 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.647 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.647 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.647 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.647 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.647 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.647 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.647 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.648 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.648 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.648 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.648 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.648 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.648 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.648 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.648 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.648 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.648 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.648 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.648 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.648 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.649 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.649 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.649 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.649 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.649 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.649 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.649 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.649 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.649 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.649 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.649 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.649 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.649 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.649 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.650 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.650 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.650 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.650 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.650 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.650 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.650 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.650 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.650 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.650 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.650 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.650 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.650 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.650 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.651 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.651 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.651 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.651 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.651 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.651 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.651 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.651 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.651 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.651 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.651 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.651 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.651 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.651 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.652 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.652 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.652 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.652 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.652 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.652 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.652 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.652 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.652 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.652 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.652 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.652 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.652 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.653 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.653 INFO:teuthology.orchestra.run.smithi123.stdout: "update_orchestration": { 2024-05-08T06:13:54.653 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "update_orchestration", 2024-05-08T06:13:54.653 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.653 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.653 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.653 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.653 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.653 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.653 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.653 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-08T06:13:54.653 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.653 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.653 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.653 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.654 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.654 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.654 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.654 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "snap_schedule", 2024-05-08T06:13:54.654 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.654 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.654 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.654 INFO:teuthology.orchestra.run.smithi123.stdout: "allow_m_granularity": { 2024-05-08T06:13:54.654 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "allow_m_granularity", 2024-05-08T06:13:54.654 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.654 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.654 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.654 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.654 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.655 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.655 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.655 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "allow minute scheduled snapshots", 2024-05-08T06:13:54.655 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.655 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.655 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.655 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.655 INFO:teuthology.orchestra.run.smithi123.stdout: "dump_on_update": { 2024-05-08T06:13:54.655 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "dump_on_update", 2024-05-08T06:13:54.655 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.655 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.655 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.655 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.655 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.655 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.655 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.656 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "dump database to debug log on update", 2024-05-08T06:13:54.656 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.656 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.656 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.656 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.656 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.656 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.656 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.656 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.656 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.656 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.656 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.656 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.656 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.656 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.657 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.657 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.657 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.657 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.657 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.657 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.657 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.657 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.657 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.657 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.657 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.657 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.657 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.657 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.658 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.658 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.658 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.658 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.658 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.658 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.658 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.658 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.658 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.658 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.658 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.658 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.658 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.658 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.659 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.659 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.659 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.659 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.659 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.659 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.659 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.659 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.659 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.659 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.659 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.659 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.659 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.659 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.660 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.660 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.660 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.660 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.660 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.660 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.660 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.660 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.660 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.660 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.660 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.660 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.660 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.660 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.661 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.661 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.661 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.661 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.661 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.661 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.661 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.661 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.661 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.661 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.661 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.661 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.661 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.661 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.661 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.662 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.662 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.662 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.662 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.662 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.662 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.662 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "stats", 2024-05-08T06:13:54.662 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.662 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.662 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.662 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.662 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.662 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.662 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.663 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.663 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.663 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.663 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.663 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.663 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.663 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.663 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.663 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.663 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.663 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.663 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.663 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.663 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.664 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.664 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.664 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.664 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.664 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.664 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.664 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.664 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.664 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.664 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.664 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.664 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.664 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.664 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.664 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.665 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.665 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.665 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.665 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.665 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.665 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.665 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.665 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.665 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.665 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.665 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.665 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.665 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.665 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.665 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.665 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.666 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.666 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.666 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.666 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.666 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.666 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.666 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.666 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.666 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.666 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.666 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.666 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.666 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.666 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.666 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.666 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.667 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.667 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.667 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.667 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.667 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.667 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.667 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.667 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.667 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.667 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.667 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.667 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.667 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.667 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.667 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.668 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.668 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.668 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.668 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.668 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.668 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.668 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.668 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "status", 2024-05-08T06:13:54.668 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.668 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.668 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.668 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.668 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.668 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.668 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.668 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.668 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.669 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.669 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.669 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.669 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.669 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.669 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.669 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.669 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.669 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.669 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.669 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.669 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.669 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.669 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.669 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.670 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.670 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.670 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.670 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.670 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.670 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.670 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.670 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.670 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.670 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.670 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.670 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.670 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.670 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.670 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.670 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.671 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.671 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.671 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.671 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.671 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.671 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.671 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.671 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.671 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.671 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.671 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.671 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.671 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.671 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.671 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.672 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.672 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.672 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.672 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.672 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.672 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.672 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.672 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.672 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.672 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.672 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.672 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.672 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.672 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.672 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.672 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.673 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.673 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.673 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.673 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.673 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.673 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.673 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.673 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.673 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.673 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.673 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.673 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.673 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.673 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.674 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.674 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.674 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.674 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.674 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.674 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "telegraf", 2024-05-08T06:13:54.674 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.674 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.674 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.674 INFO:teuthology.orchestra.run.smithi123.stdout: "address": { 2024-05-08T06:13:54.674 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "address", 2024-05-08T06:13:54.674 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.674 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.674 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.675 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-08T06:13:54.675 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.675 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.675 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.675 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.675 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.675 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.675 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.675 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.675 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-05-08T06:13:54.675 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-05-08T06:13:54.675 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.675 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.675 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.675 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "15", 2024-05-08T06:13:54.675 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.676 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.676 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.676 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.676 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.676 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.676 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.676 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.676 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.676 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.676 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.676 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.676 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.676 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.676 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.676 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.676 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.677 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.677 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.677 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.677 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.677 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.677 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.677 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.677 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.677 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.677 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.677 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.677 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.677 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.677 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.677 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.678 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.678 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.678 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.678 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.678 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.678 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.678 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.678 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.678 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.678 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.678 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.678 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.678 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.678 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.678 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.679 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.679 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.679 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.679 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.679 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.679 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.679 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.679 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.679 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.679 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.679 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.679 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.679 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.679 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.680 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.680 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.680 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.680 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.680 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.680 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.680 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.680 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.680 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.680 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.680 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.680 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.680 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.680 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.680 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.680 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.681 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.681 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.681 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.681 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.681 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.681 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.681 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.681 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.681 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.681 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.681 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.681 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.681 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.681 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.681 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.681 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.682 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.682 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.682 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "telemetry", 2024-05-08T06:13:54.682 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.682 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.682 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.682 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_basic": { 2024-05-08T06:13:54.682 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_basic", 2024-05-08T06:13:54.682 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.682 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.682 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.682 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.682 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.682 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.682 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.683 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-08T06:13:54.683 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.683 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.683 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.683 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.683 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_crash": { 2024-05-08T06:13:54.683 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_crash", 2024-05-08T06:13:54.683 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.683 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.683 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.683 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.683 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.683 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.683 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.684 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-08T06:13:54.684 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.684 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.684 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.684 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.684 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_device": { 2024-05-08T06:13:54.684 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_device", 2024-05-08T06:13:54.684 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.684 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.684 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.684 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.684 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.684 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.684 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.685 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-08T06:13:54.685 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.685 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.685 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.685 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.685 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_ident": { 2024-05-08T06:13:54.685 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_ident", 2024-05-08T06:13:54.685 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.685 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.685 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.685 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.685 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.685 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.685 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.686 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-08T06:13:54.686 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.686 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.686 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.686 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.686 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_perf": { 2024-05-08T06:13:54.686 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_perf", 2024-05-08T06:13:54.686 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.686 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.686 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.686 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.686 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.686 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.687 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.687 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-08T06:13:54.687 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.687 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.687 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.687 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.687 INFO:teuthology.orchestra.run.smithi123.stdout: "contact": { 2024-05-08T06:13:54.687 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "contact", 2024-05-08T06:13:54.687 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.687 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.687 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.687 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.687 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.687 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.687 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.687 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.688 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.688 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.688 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.688 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.688 INFO:teuthology.orchestra.run.smithi123.stdout: "description": { 2024-05-08T06:13:54.688 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "description", 2024-05-08T06:13:54.688 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.688 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.688 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.688 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.688 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.688 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.688 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.688 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.689 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.689 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.689 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.689 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.689 INFO:teuthology.orchestra.run.smithi123.stdout: "device_url": { 2024-05-08T06:13:54.689 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "device_url", 2024-05-08T06:13:54.689 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.689 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.689 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.689 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-08T06:13:54.689 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.689 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.689 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.689 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.689 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.690 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.690 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.690 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.690 INFO:teuthology.orchestra.run.smithi123.stdout: "enabled": { 2024-05-08T06:13:54.690 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "enabled", 2024-05-08T06:13:54.690 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.690 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.690 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.690 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.690 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.690 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.690 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.690 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.690 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.690 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.690 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.691 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.691 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-05-08T06:13:54.691 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-05-08T06:13:54.691 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.691 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.691 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.691 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "24", 2024-05-08T06:13:54.691 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "8", 2024-05-08T06:13:54.691 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.691 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.691 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.691 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.691 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.691 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.691 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.691 INFO:teuthology.orchestra.run.smithi123.stdout: "last_opt_revision": { 2024-05-08T06:13:54.692 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "last_opt_revision", 2024-05-08T06:13:54.692 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.692 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.692 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.692 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "1", 2024-05-08T06:13:54.692 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.692 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.692 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.692 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.692 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.692 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.693 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.693 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.693 INFO:teuthology.orchestra.run.smithi123.stdout: "leaderboard": { 2024-05-08T06:13:54.693 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "leaderboard", 2024-05-08T06:13:54.693 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.693 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.693 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.693 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.694 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.694 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.694 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.694 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.694 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.694 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.694 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.694 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.695 INFO:teuthology.orchestra.run.smithi123.stdout: "leaderboard_description": { 2024-05-08T06:13:54.695 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "leaderboard_description", 2024-05-08T06:13:54.695 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.695 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.695 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.695 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.695 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.695 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.696 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.696 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.696 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.696 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.696 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.696 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.696 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.696 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.696 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.696 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.696 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.696 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.696 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.696 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.696 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.696 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.697 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.697 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.697 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.697 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.697 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.697 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.697 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.697 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.697 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.697 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.697 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.697 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.697 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.697 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.698 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.698 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.698 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.698 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.698 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.698 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.698 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.698 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.698 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.698 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.698 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.698 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.698 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.698 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.698 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.699 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.699 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.699 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.699 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.699 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.699 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.699 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.699 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.699 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.699 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.699 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.699 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.699 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.699 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.700 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.700 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.700 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.700 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.700 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.700 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.700 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.700 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.700 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.700 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.700 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.700 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.700 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.700 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.700 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.701 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.701 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.701 INFO:teuthology.orchestra.run.smithi123.stdout: "organization": { 2024-05-08T06:13:54.701 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "organization", 2024-05-08T06:13:54.701 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.701 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.701 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.701 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.701 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.701 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.701 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.701 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.701 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.701 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.702 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.702 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.702 INFO:teuthology.orchestra.run.smithi123.stdout: "proxy": { 2024-05-08T06:13:54.702 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "proxy", 2024-05-08T06:13:54.702 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.702 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.702 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.702 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.702 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.702 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.702 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.702 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.702 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.703 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.703 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.703 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.703 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.703 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.703 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.703 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.703 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.703 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.703 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.703 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.703 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.703 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.703 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.703 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.704 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.704 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.704 INFO:teuthology.orchestra.run.smithi123.stdout: "url": { 2024-05-08T06:13:54.704 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "url", 2024-05-08T06:13:54.704 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.704 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.704 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.704 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-08T06:13:54.704 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.704 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.704 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.704 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.704 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.704 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.704 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.705 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.705 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.705 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.705 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.705 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "test_orchestrator", 2024-05-08T06:13:54.705 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.705 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.705 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.705 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.705 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.705 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.705 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.705 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.706 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.706 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.706 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.706 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.706 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.706 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.706 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.706 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.706 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.706 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.706 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.706 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.706 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.707 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.707 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.707 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.707 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.707 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.707 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.707 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.707 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.707 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.707 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.707 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.707 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.707 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.708 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.708 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.708 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.708 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.708 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.708 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.708 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.708 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.708 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.708 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.708 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.708 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.708 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.708 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.708 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.709 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.709 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.709 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.709 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.709 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.709 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.709 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.709 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.709 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.709 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.709 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.709 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.709 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.709 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.709 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.709 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.710 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.710 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.710 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.710 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.710 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.710 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.710 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.710 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.710 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.710 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.710 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.710 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.710 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.711 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.711 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.711 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.711 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.711 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.711 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.711 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.711 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.711 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.711 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.711 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.711 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.711 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "volumes", 2024-05-08T06:13:54.711 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.711 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.711 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.712 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.712 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.712 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.712 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.712 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.712 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.712 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.712 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.712 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.712 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.712 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.712 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.712 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.712 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.713 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.713 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.713 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.713 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.713 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.713 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.713 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.713 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.713 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.713 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.713 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.713 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.713 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.713 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.713 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.713 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.714 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.714 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.714 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.714 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.714 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.714 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.714 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.714 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.714 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.714 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.714 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.714 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.714 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.715 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.715 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.715 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.715 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.715 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.715 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.715 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.715 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.715 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.715 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.715 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.715 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.715 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.715 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.716 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.716 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.716 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.716 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.716 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.716 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.716 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.716 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.716 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.716 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.716 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.716 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.716 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.716 INFO:teuthology.orchestra.run.smithi123.stdout: "max_concurrent_clones": { 2024-05-08T06:13:54.716 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_concurrent_clones", 2024-05-08T06:13:54.716 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.717 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.717 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.717 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "4", 2024-05-08T06:13:54.717 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.717 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.717 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.717 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-08T06:13:54.717 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.717 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.717 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.717 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.717 INFO:teuthology.orchestra.run.smithi123.stdout: "periodic_async_work": { 2024-05-08T06:13:54.717 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "periodic_async_work", 2024-05-08T06:13:54.717 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.717 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.718 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.718 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.718 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.718 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.718 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.718 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Periodically check for async work", 2024-05-08T06:13:54.718 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.718 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.718 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.718 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.718 INFO:teuthology.orchestra.run.smithi123.stdout: "snapshot_clone_delay": { 2024-05-08T06:13:54.718 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "snapshot_clone_delay", 2024-05-08T06:13:54.718 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.718 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.718 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.719 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.719 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.719 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.719 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.719 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-08T06:13:54.719 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.719 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.719 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.719 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.719 INFO:teuthology.orchestra.run.smithi123.stdout: "snapshot_clone_no_wait": { 2024-05-08T06:13:54.719 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "snapshot_clone_no_wait", 2024-05-08T06:13:54.719 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.719 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.719 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.719 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.719 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.720 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.720 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.720 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-08T06:13:54.720 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.720 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.720 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.720 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.720 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.720 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.720 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.720 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.720 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.720 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.720 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.720 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.720 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.720 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.721 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.721 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.721 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.721 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.721 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.721 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.721 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.721 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "zabbix", 2024-05-08T06:13:54.721 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.721 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.721 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.721 INFO:teuthology.orchestra.run.smithi123.stdout: "discovery_interval": { 2024-05-08T06:13:54.721 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "discovery_interval", 2024-05-08T06:13:54.721 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-05-08T06:13:54.721 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.722 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.722 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "100", 2024-05-08T06:13:54.722 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.722 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.722 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.722 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.722 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.722 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.722 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.722 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.722 INFO:teuthology.orchestra.run.smithi123.stdout: "identifier": { 2024-05-08T06:13:54.722 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "identifier", 2024-05-08T06:13:54.722 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.722 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.723 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.723 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.723 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.723 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.723 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.723 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.723 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.723 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.723 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.723 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.723 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-05-08T06:13:54.723 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-05-08T06:13:54.723 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.723 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.723 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.723 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-05-08T06:13:54.723 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.724 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.724 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.724 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.724 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.724 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.724 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.724 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.724 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.724 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.724 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.724 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.724 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.724 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.724 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.724 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.724 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.725 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.725 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.725 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.725 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.725 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.725 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.725 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.725 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.725 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.725 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.725 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.725 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.725 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.725 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.725 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.725 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.726 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.726 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.726 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.726 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.726 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.726 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.726 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.726 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.726 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.726 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.726 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.726 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.726 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.726 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.727 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.727 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.727 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.727 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.727 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.727 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.727 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.727 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.727 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.727 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.727 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.727 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.727 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.727 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.727 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.727 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.728 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.728 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.728 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.728 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.728 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.728 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.728 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.728 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.728 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.728 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.728 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.728 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.728 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.728 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.728 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.728 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.729 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.729 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.729 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.729 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.729 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.729 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.729 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.729 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.729 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.729 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.729 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.729 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.729 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.729 INFO:teuthology.orchestra.run.smithi123.stdout: "zabbix_host": { 2024-05-08T06:13:54.729 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "zabbix_host", 2024-05-08T06:13:54.729 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.729 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.730 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.730 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.730 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.730 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.730 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.730 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.730 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.730 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.730 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.730 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.730 INFO:teuthology.orchestra.run.smithi123.stdout: "zabbix_port": { 2024-05-08T06:13:54.730 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "zabbix_port", 2024-05-08T06:13:54.730 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.730 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.731 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.731 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10051", 2024-05-08T06:13:54.731 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.731 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.731 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.731 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.731 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.731 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.731 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.731 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.731 INFO:teuthology.orchestra.run.smithi123.stdout: "zabbix_sender": { 2024-05-08T06:13:54.731 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "zabbix_sender", 2024-05-08T06:13:54.731 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.731 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.731 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.732 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-08T06:13:54.732 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.732 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.732 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.732 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.732 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.732 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.732 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.732 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.732 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.732 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.732 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-05-08T06:13:54.732 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.732 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.732 INFO:teuthology.orchestra.run.smithi123.stdout: "modules": [ 2024-05-08T06:13:54.732 INFO:teuthology.orchestra.run.smithi123.stdout: "cephadm", 2024-05-08T06:13:54.732 INFO:teuthology.orchestra.run.smithi123.stdout: "dashboard", 2024-05-08T06:13:54.733 INFO:teuthology.orchestra.run.smithi123.stdout: "iostat", 2024-05-08T06:13:54.733 INFO:teuthology.orchestra.run.smithi123.stdout: "nfs", 2024-05-08T06:13:54.733 INFO:teuthology.orchestra.run.smithi123.stdout: "restful" 2024-05-08T06:13:54.733 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.733 INFO:teuthology.orchestra.run.smithi123.stdout: "available_modules": [ 2024-05-08T06:13:54.733 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.733 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "alerts", 2024-05-08T06:13:54.733 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.733 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.733 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.733 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-05-08T06:13:54.733 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-05-08T06:13:54.733 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.733 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.733 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.733 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-05-08T06:13:54.734 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.734 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.734 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.734 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "How frequently to reexamine health status", 2024-05-08T06:13:54.734 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.734 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.734 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.734 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.734 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.734 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.734 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.734 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.734 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.734 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.734 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.735 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.735 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.735 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.735 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.735 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.735 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.735 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.735 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.735 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.735 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.735 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.735 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.735 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.735 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.735 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.735 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.736 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.736 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.736 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.736 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.736 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.736 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.736 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.736 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.736 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.736 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.736 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.736 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.736 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.736 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.736 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.737 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.737 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.737 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.737 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.737 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.737 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.737 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.737 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.737 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.737 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.737 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.737 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.737 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.737 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.737 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.737 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.738 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.738 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.738 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.738 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.738 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.738 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.738 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.738 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.738 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.738 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.738 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.738 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.738 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.738 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.738 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.739 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.739 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_destination": { 2024-05-08T06:13:54.739 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_destination", 2024-05-08T06:13:54.739 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.739 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.739 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.739 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.739 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.739 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.739 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.739 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Email address to send alerts to", 2024-05-08T06:13:54.739 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.739 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.739 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.739 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.740 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_from_name": { 2024-05-08T06:13:54.740 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_from_name", 2024-05-08T06:13:54.740 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.740 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.740 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.740 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "Ceph", 2024-05-08T06:13:54.740 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.740 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.740 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.740 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Email From: name", 2024-05-08T06:13:54.740 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.740 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.740 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.740 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.740 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_host": { 2024-05-08T06:13:54.740 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_host", 2024-05-08T06:13:54.741 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.741 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.741 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.741 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.741 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.741 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.741 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.741 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "SMTP server", 2024-05-08T06:13:54.741 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.741 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.741 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.741 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.741 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_password": { 2024-05-08T06:13:54.741 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_password", 2024-05-08T06:13:54.741 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.741 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.742 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.742 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.742 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.742 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.742 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.742 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Password to authenticate with", 2024-05-08T06:13:54.742 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.742 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.742 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.742 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.742 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_port": { 2024-05-08T06:13:54.742 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_port", 2024-05-08T06:13:54.742 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.742 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.743 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.743 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "465", 2024-05-08T06:13:54.743 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.743 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.743 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.743 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "SMTP port", 2024-05-08T06:13:54.743 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.743 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.743 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.743 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.743 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_sender": { 2024-05-08T06:13:54.743 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_sender", 2024-05-08T06:13:54.743 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.743 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.743 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.743 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.744 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.744 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.744 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.744 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "SMTP envelope sender", 2024-05-08T06:13:54.744 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.744 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.744 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.744 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.744 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_ssl": { 2024-05-08T06:13:54.744 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_ssl", 2024-05-08T06:13:54.744 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.744 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.744 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.744 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.744 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.744 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.745 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.745 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-08T06:13:54.745 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.745 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.745 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.745 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.745 INFO:teuthology.orchestra.run.smithi123.stdout: "smtp_user": { 2024-05-08T06:13:54.745 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smtp_user", 2024-05-08T06:13:54.745 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.745 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.745 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.745 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.745 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.745 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.745 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.745 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "User to authenticate as", 2024-05-08T06:13:54.746 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.746 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.746 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.746 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.746 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.746 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.746 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.746 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.746 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.746 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.746 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.746 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.746 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.746 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.747 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.747 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.747 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.747 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.747 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.747 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.747 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "balancer", 2024-05-08T06:13:54.747 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.747 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.747 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.747 INFO:teuthology.orchestra.run.smithi123.stdout: "active": { 2024-05-08T06:13:54.747 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "active", 2024-05-08T06:13:54.747 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.747 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.747 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.748 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.748 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.748 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.748 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.748 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "automatically balance PGs across cluster", 2024-05-08T06:13:54.748 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.748 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.748 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.748 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.748 INFO:teuthology.orchestra.run.smithi123.stdout: "begin_time": { 2024-05-08T06:13:54.748 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "begin_time", 2024-05-08T06:13:54.748 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.748 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.748 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.748 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0000", 2024-05-08T06:13:54.748 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.748 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.749 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.749 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "beginning time of day to automatically balance", 2024-05-08T06:13:54.749 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T06:13:54.749 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.749 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.749 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.749 INFO:teuthology.orchestra.run.smithi123.stdout: "begin_weekday": { 2024-05-08T06:13:54.749 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "begin_weekday", 2024-05-08T06:13:54.749 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-05-08T06:13:54.749 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.749 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.749 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.749 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "0", 2024-05-08T06:13:54.749 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "6", 2024-05-08T06:13:54.749 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.749 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-08T06:13:54.750 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T06:13:54.750 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.750 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.750 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.750 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_compat_max_iterations": { 2024-05-08T06:13:54.750 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crush_compat_max_iterations", 2024-05-08T06:13:54.750 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-05-08T06:13:54.750 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.750 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.750 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "25", 2024-05-08T06:13:54.750 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1", 2024-05-08T06:13:54.750 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "250", 2024-05-08T06:13:54.750 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.750 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-08T06:13:54.751 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.751 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.751 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.751 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.751 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_compat_metrics": { 2024-05-08T06:13:54.751 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crush_compat_metrics", 2024-05-08T06:13:54.751 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.751 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.751 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.751 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "pgs,objects,bytes", 2024-05-08T06:13:54.751 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.751 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.751 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.751 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-08T06:13:54.751 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-08T06:13:54.751 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.752 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.752 INFO:teuthology.orchestra.run.smithi123.stdout: "crush_compat_step": { 2024-05-08T06:13:54.752 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crush_compat_step", 2024-05-08T06:13:54.752 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-05-08T06:13:54.752 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.752 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.752 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0.5", 2024-05-08T06:13:54.752 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "0.001", 2024-05-08T06:13:54.752 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "0.999", 2024-05-08T06:13:54.752 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.752 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "aggressiveness of optimization", 2024-05-08T06:13:54.752 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-08T06:13:54.752 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.752 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.753 INFO:teuthology.orchestra.run.smithi123.stdout: "end_time": { 2024-05-08T06:13:54.753 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "end_time", 2024-05-08T06:13:54.753 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.753 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.753 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.753 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "2359", 2024-05-08T06:13:54.753 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.753 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.753 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.753 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "ending time of day to automatically balance", 2024-05-08T06:13:54.753 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T06:13:54.753 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.753 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.753 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.753 INFO:teuthology.orchestra.run.smithi123.stdout: "end_weekday": { 2024-05-08T06:13:54.753 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "end_weekday", 2024-05-08T06:13:54.754 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-05-08T06:13:54.754 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.754 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.754 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.754 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "0", 2024-05-08T06:13:54.754 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "6", 2024-05-08T06:13:54.754 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.754 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-08T06:13:54.754 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T06:13:54.754 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.754 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.754 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.754 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.754 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.754 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.755 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.755 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.755 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.755 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.755 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.755 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.755 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.755 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.755 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.755 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.755 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.755 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.755 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.755 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.755 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.756 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.756 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.756 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.756 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.756 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.756 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.756 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.756 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.756 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.756 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.756 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.756 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.756 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.756 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.756 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.757 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.757 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.757 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.757 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.757 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.757 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.757 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.757 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.757 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.757 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.757 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.757 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.757 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.757 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.758 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.758 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.758 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.758 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.758 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.758 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.758 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.758 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.758 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.758 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.758 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.758 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.758 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.758 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.758 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.759 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.759 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.759 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.759 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.759 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.759 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.759 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.759 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.759 INFO:teuthology.orchestra.run.smithi123.stdout: "min_score": { 2024-05-08T06:13:54.759 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "min_score", 2024-05-08T06:13:54.759 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-05-08T06:13:54.759 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.759 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.759 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.759 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.759 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.760 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.760 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-08T06:13:54.760 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.760 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.760 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.760 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.760 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": { 2024-05-08T06:13:54.760 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mode", 2024-05-08T06:13:54.760 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.760 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.760 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.760 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "upmap", 2024-05-08T06:13:54.760 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.760 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.760 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.760 INFO:teuthology.orchestra.run.smithi123.stdout: "crush-compat", 2024-05-08T06:13:54.761 INFO:teuthology.orchestra.run.smithi123.stdout: "none", 2024-05-08T06:13:54.761 INFO:teuthology.orchestra.run.smithi123.stdout: "read", 2024-05-08T06:13:54.761 INFO:teuthology.orchestra.run.smithi123.stdout: "upmap", 2024-05-08T06:13:54.761 INFO:teuthology.orchestra.run.smithi123.stdout: "upmap-read" 2024-05-08T06:13:54.761 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.761 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Balancer mode", 2024-05-08T06:13:54.761 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.761 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.761 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.761 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.761 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_ids": { 2024-05-08T06:13:54.761 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "pool_ids", 2024-05-08T06:13:54.761 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.761 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.761 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.762 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.762 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.762 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.762 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.762 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-08T06:13:54.762 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.762 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.762 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.762 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.762 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-05-08T06:13:54.762 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-05-08T06:13:54.762 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.762 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.763 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.763 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-05-08T06:13:54.763 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.763 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.763 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.763 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-08T06:13:54.763 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.763 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.763 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.763 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.763 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.763 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.763 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.763 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.763 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.763 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.764 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.764 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.764 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.764 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.764 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.764 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.764 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.764 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.764 INFO:teuthology.orchestra.run.smithi123.stdout: "upmap_max_deviation": { 2024-05-08T06:13:54.764 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "upmap_max_deviation", 2024-05-08T06:13:54.764 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.764 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.764 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.764 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-05-08T06:13:54.764 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1", 2024-05-08T06:13:54.765 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.765 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.765 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-08T06:13:54.765 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-08T06:13:54.765 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.765 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.765 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.765 INFO:teuthology.orchestra.run.smithi123.stdout: "upmap_max_optimizations": { 2024-05-08T06:13:54.765 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "upmap_max_optimizations", 2024-05-08T06:13:54.765 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-05-08T06:13:54.765 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.765 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.765 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-05-08T06:13:54.765 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.765 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.765 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.766 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-08T06:13:54.766 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.766 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.766 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.766 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.766 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.766 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.766 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.766 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cephadm", 2024-05-08T06:13:54.766 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.766 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.766 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.766 INFO:teuthology.orchestra.run.smithi123.stdout: "agent_down_multiplier": { 2024-05-08T06:13:54.766 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "agent_down_multiplier", 2024-05-08T06:13:54.766 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-05-08T06:13:54.766 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.767 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.767 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "3.0", 2024-05-08T06:13:54.767 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.767 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.767 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.767 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-08T06:13:54.767 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.767 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.767 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.767 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.767 INFO:teuthology.orchestra.run.smithi123.stdout: "agent_refresh_rate": { 2024-05-08T06:13:54.767 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "agent_refresh_rate", 2024-05-08T06:13:54.767 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.767 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.767 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.768 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "20", 2024-05-08T06:13:54.768 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.768 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.768 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.768 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-08T06:13:54.768 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.768 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.768 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.768 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.768 INFO:teuthology.orchestra.run.smithi123.stdout: "agent_starting_port": { 2024-05-08T06:13:54.768 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "agent_starting_port", 2024-05-08T06:13:54.768 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.768 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.768 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.768 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "4721", 2024-05-08T06:13:54.768 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.769 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.769 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.769 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-08T06:13:54.769 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.769 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.769 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.769 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.769 INFO:teuthology.orchestra.run.smithi123.stdout: "allow_ptrace": { 2024-05-08T06:13:54.769 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "allow_ptrace", 2024-05-08T06:13:54.769 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.769 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.769 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.769 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.769 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.769 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.770 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.770 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-08T06:13:54.770 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-08T06:13:54.770 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.770 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.770 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.770 INFO:teuthology.orchestra.run.smithi123.stdout: "autotune_interval": { 2024-05-08T06:13:54.770 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "autotune_interval", 2024-05-08T06:13:54.770 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.770 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.770 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.770 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-05-08T06:13:54.770 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.770 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.771 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.771 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-08T06:13:54.771 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.771 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.771 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.771 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.771 INFO:teuthology.orchestra.run.smithi123.stdout: "autotune_memory_target_ratio": { 2024-05-08T06:13:54.771 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "autotune_memory_target_ratio", 2024-05-08T06:13:54.771 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-05-08T06:13:54.771 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.771 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.771 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0.7", 2024-05-08T06:13:54.771 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.771 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.771 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.771 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-08T06:13:54.772 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.772 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.772 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.772 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.772 INFO:teuthology.orchestra.run.smithi123.stdout: "cephadm_log_destination": { 2024-05-08T06:13:54.772 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cephadm_log_destination", 2024-05-08T06:13:54.772 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.772 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.772 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.772 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.772 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.772 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.772 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.772 INFO:teuthology.orchestra.run.smithi123.stdout: "file", 2024-05-08T06:13:54.772 INFO:teuthology.orchestra.run.smithi123.stdout: "file,syslog", 2024-05-08T06:13:54.772 INFO:teuthology.orchestra.run.smithi123.stdout: "syslog" 2024-05-08T06:13:54.773 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.773 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-08T06:13:54.773 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.773 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.773 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.773 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.773 INFO:teuthology.orchestra.run.smithi123.stdout: "cgroups_split": { 2024-05-08T06:13:54.773 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cgroups_split", 2024-05-08T06:13:54.773 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.773 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.773 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.773 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.773 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.773 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.774 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.774 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-08T06:13:54.774 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.774 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.774 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.774 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.774 INFO:teuthology.orchestra.run.smithi123.stdout: "config_checks_enabled": { 2024-05-08T06:13:54.774 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "config_checks_enabled", 2024-05-08T06:13:54.774 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.774 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.774 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.774 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.774 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.774 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.774 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.775 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-08T06:13:54.775 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.775 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.775 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.775 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.775 INFO:teuthology.orchestra.run.smithi123.stdout: "config_dashboard": { 2024-05-08T06:13:54.775 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "config_dashboard", 2024-05-08T06:13:54.775 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.775 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.775 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.775 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.775 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.775 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.775 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.775 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-08T06:13:54.776 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.776 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.776 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.776 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.776 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_alertmanager": { 2024-05-08T06:13:54.776 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_alertmanager", 2024-05-08T06:13:54.776 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.776 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.776 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.776 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-08T06:13:54.776 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.776 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.776 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.776 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Prometheus container image", 2024-05-08T06:13:54.776 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.776 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.777 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.777 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.777 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_base": { 2024-05-08T06:13:54.777 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_base", 2024-05-08T06:13:54.777 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.777 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.777 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.777 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-08T06:13:54.777 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.777 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.777 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.777 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Container image name, without the tag", 2024-05-08T06:13:54.777 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.777 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.778 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.778 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.778 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_elasticsearch": { 2024-05-08T06:13:54.778 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_elasticsearch", 2024-05-08T06:13:54.778 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.778 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.778 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.778 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-08T06:13:54.778 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.778 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.778 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.778 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "elasticsearch container image", 2024-05-08T06:13:54.778 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.779 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.779 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.779 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.779 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_grafana": { 2024-05-08T06:13:54.779 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_grafana", 2024-05-08T06:13:54.779 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.779 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.779 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.779 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-08T06:13:54.779 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.779 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.779 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.779 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Prometheus container image", 2024-05-08T06:13:54.779 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.780 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.780 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.780 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.780 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_haproxy": { 2024-05-08T06:13:54.780 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_haproxy", 2024-05-08T06:13:54.780 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.780 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.780 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.780 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-08T06:13:54.780 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.780 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.780 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.780 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "HAproxy container image", 2024-05-08T06:13:54.780 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.780 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.781 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.781 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.781 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_jaeger_agent": { 2024-05-08T06:13:54.781 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_jaeger_agent", 2024-05-08T06:13:54.781 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.781 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.781 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.781 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-08T06:13:54.781 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.781 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.781 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.781 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Jaeger agent container image", 2024-05-08T06:13:54.781 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.781 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.781 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.781 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.782 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_jaeger_collector": { 2024-05-08T06:13:54.782 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_jaeger_collector", 2024-05-08T06:13:54.782 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.782 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.782 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.782 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-08T06:13:54.782 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.782 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.782 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.782 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Jaeger collector container image", 2024-05-08T06:13:54.782 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.782 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.782 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.783 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.783 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_jaeger_query": { 2024-05-08T06:13:54.783 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_jaeger_query", 2024-05-08T06:13:54.783 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.783 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.783 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.783 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-08T06:13:54.783 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.783 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.783 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.783 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Jaeger query container image", 2024-05-08T06:13:54.783 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.783 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.783 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.784 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.784 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_keepalived": { 2024-05-08T06:13:54.784 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_keepalived", 2024-05-08T06:13:54.784 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.784 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.784 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.784 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-08T06:13:54.784 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.784 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.784 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.784 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Keepalived container image", 2024-05-08T06:13:54.784 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.784 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.784 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.784 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.784 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_loki": { 2024-05-08T06:13:54.785 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_loki", 2024-05-08T06:13:54.785 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.785 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.785 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.785 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-08T06:13:54.785 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.785 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.785 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.785 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Loki container image", 2024-05-08T06:13:54.785 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.785 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.785 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.785 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.785 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_node_exporter": { 2024-05-08T06:13:54.785 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_node_exporter", 2024-05-08T06:13:54.785 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.786 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.786 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.786 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-08T06:13:54.786 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.786 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.786 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.786 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Prometheus container image", 2024-05-08T06:13:54.786 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.786 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.786 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.786 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.786 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_nvmeof": { 2024-05-08T06:13:54.786 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_nvmeof", 2024-05-08T06:13:54.786 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.787 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.787 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.787 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-08T06:13:54.787 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.787 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.787 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.787 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Nvme-of container image", 2024-05-08T06:13:54.787 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.787 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.787 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.787 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.787 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_prometheus": { 2024-05-08T06:13:54.787 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_prometheus", 2024-05-08T06:13:54.787 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.787 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.787 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.788 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-08T06:13:54.788 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.788 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.788 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.788 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Prometheus container image", 2024-05-08T06:13:54.788 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.788 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.788 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.788 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.788 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_promtail": { 2024-05-08T06:13:54.788 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_promtail", 2024-05-08T06:13:54.788 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.788 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.788 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.788 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-08T06:13:54.788 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.789 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.789 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.789 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Promtail container image", 2024-05-08T06:13:54.789 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.789 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.789 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.789 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.789 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_samba": { 2024-05-08T06:13:54.789 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_samba", 2024-05-08T06:13:54.789 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.789 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.789 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.789 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-08T06:13:54.789 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.789 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.790 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.790 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Samba/SMB container image", 2024-05-08T06:13:54.790 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.790 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.790 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.790 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.790 INFO:teuthology.orchestra.run.smithi123.stdout: "container_image_snmp_gateway": { 2024-05-08T06:13:54.790 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_image_snmp_gateway", 2024-05-08T06:13:54.790 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.790 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.790 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.790 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-08T06:13:54.790 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.790 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.791 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.791 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "SNMP Gateway container image", 2024-05-08T06:13:54.791 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.791 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.791 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.791 INFO:teuthology.orchestra.run.smithi123.stdout: "container_init": { 2024-05-08T06:13:54.791 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "container_init", 2024-05-08T06:13:54.791 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.791 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.791 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.791 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.791 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.791 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.791 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.791 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Run podman/docker with `--init`", 2024-05-08T06:13:54.792 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.792 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.792 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.792 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.792 INFO:teuthology.orchestra.run.smithi123.stdout: "daemon_cache_timeout": { 2024-05-08T06:13:54.792 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "daemon_cache_timeout", 2024-05-08T06:13:54.792 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.792 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.792 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.792 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-05-08T06:13:54.792 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.792 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.792 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.792 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-08T06:13:54.792 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.792 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.792 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.793 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.793 INFO:teuthology.orchestra.run.smithi123.stdout: "default_cephadm_command_timeout": { 2024-05-08T06:13:54.793 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "default_cephadm_command_timeout", 2024-05-08T06:13:54.793 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.793 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.793 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.793 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "900", 2024-05-08T06:13:54.793 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.793 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.793 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.793 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-08T06:13:54.793 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.793 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.793 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.793 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.793 INFO:teuthology.orchestra.run.smithi123.stdout: "default_registry": { 2024-05-08T06:13:54.794 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "default_registry", 2024-05-08T06:13:54.794 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.794 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.794 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.794 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "docker.io", 2024-05-08T06:13:54.794 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.794 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.794 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.794 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-08T06:13:54.794 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.794 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.794 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.794 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.794 INFO:teuthology.orchestra.run.smithi123.stdout: "device_cache_timeout": { 2024-05-08T06:13:54.794 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "device_cache_timeout", 2024-05-08T06:13:54.794 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.794 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.795 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.795 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "1800", 2024-05-08T06:13:54.795 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.795 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.795 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.795 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "seconds to cache device inventory", 2024-05-08T06:13:54.795 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.795 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.795 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.795 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.795 INFO:teuthology.orchestra.run.smithi123.stdout: "device_enhanced_scan": { 2024-05-08T06:13:54.795 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "device_enhanced_scan", 2024-05-08T06:13:54.795 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.795 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.795 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.796 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.796 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.796 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.796 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.796 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-08T06:13:54.796 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.796 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.796 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.796 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.796 INFO:teuthology.orchestra.run.smithi123.stdout: "facts_cache_timeout": { 2024-05-08T06:13:54.796 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "facts_cache_timeout", 2024-05-08T06:13:54.796 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.796 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.796 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.796 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-05-08T06:13:54.796 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.796 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.797 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.797 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "seconds to cache host facts data", 2024-05-08T06:13:54.797 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.797 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.797 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.797 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.797 INFO:teuthology.orchestra.run.smithi123.stdout: "grafana_dashboards_path": { 2024-05-08T06:13:54.797 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "grafana_dashboards_path", 2024-05-08T06:13:54.797 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.797 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.797 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.797 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-08T06:13:54.797 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.797 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.797 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.797 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-08T06:13:54.798 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.798 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.798 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.798 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.798 INFO:teuthology.orchestra.run.smithi123.stdout: "host_check_interval": { 2024-05-08T06:13:54.798 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "host_check_interval", 2024-05-08T06:13:54.798 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.798 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.798 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.798 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-05-08T06:13:54.798 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.798 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.798 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.799 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to perform a host check", 2024-05-08T06:13:54.799 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.799 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.799 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.799 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.799 INFO:teuthology.orchestra.run.smithi123.stdout: "hw_monitoring": { 2024-05-08T06:13:54.799 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "hw_monitoring", 2024-05-08T06:13:54.799 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.799 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.799 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.799 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.799 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.799 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.800 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.800 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-08T06:13:54.800 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.800 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.800 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.800 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.800 INFO:teuthology.orchestra.run.smithi123.stdout: "inventory_list_all": { 2024-05-08T06:13:54.800 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "inventory_list_all", 2024-05-08T06:13:54.800 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.800 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.800 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.800 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.800 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.801 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.801 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.801 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-08T06:13:54.801 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.801 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.801 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.801 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.801 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.801 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.801 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.801 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.801 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.801 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.801 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.801 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.802 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.802 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.802 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.802 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.802 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.802 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.802 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.802 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.802 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.802 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.802 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.802 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.802 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.802 INFO:teuthology.orchestra.run.smithi123.stdout: "log_refresh_metadata": { 2024-05-08T06:13:54.803 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_refresh_metadata", 2024-05-08T06:13:54.803 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.803 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.803 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.803 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.803 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.803 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.803 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.803 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-08T06:13:54.803 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.803 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.803 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.803 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.803 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.803 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.804 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.804 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.804 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.804 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.804 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.804 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.804 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.804 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-08T06:13:54.804 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.804 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.804 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.804 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.804 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.804 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.804 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.805 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.805 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.805 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.805 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.805 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.805 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.805 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.805 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.805 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.805 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.805 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.805 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.805 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.805 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.805 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.805 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.806 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.806 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.806 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.806 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.806 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.806 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.806 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.806 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.806 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.806 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.806 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.806 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.806 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.806 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.806 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.806 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.807 INFO:teuthology.orchestra.run.smithi123.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-08T06:13:54.807 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-08T06:13:54.807 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.807 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.807 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.807 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.807 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.807 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.807 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.807 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-08T06:13:54.807 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.807 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.807 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.807 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.807 INFO:teuthology.orchestra.run.smithi123.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-08T06:13:54.807 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-08T06:13:54.808 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.808 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.808 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.808 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "*", 2024-05-08T06:13:54.808 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.808 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.808 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.808 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-08T06:13:54.808 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.808 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.808 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.808 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.808 INFO:teuthology.orchestra.run.smithi123.stdout: "max_count_per_host": { 2024-05-08T06:13:54.808 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_count_per_host", 2024-05-08T06:13:54.808 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.809 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.809 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.809 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-05-08T06:13:54.809 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.809 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.809 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.809 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "max number of daemons per service per host", 2024-05-08T06:13:54.809 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.809 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.809 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.809 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.809 INFO:teuthology.orchestra.run.smithi123.stdout: "max_osd_draining_count": { 2024-05-08T06:13:54.809 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_osd_draining_count", 2024-05-08T06:13:54.810 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.810 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.810 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.810 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-05-08T06:13:54.810 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.810 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.810 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.810 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-08T06:13:54.810 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.810 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.810 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.810 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.810 INFO:teuthology.orchestra.run.smithi123.stdout: "migration_current": { 2024-05-08T06:13:54.810 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "migration_current", 2024-05-08T06:13:54.811 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.811 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.811 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.811 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.811 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.811 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.811 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.811 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "internal - do not modify", 2024-05-08T06:13:54.811 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.811 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.811 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.811 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.811 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": { 2024-05-08T06:13:54.811 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mode", 2024-05-08T06:13:54.811 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.811 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.812 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.812 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "root", 2024-05-08T06:13:54.812 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.812 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.812 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.812 INFO:teuthology.orchestra.run.smithi123.stdout: "cephadm-package", 2024-05-08T06:13:54.812 INFO:teuthology.orchestra.run.smithi123.stdout: "root" 2024-05-08T06:13:54.812 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.812 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "mode for remote execution of cephadm", 2024-05-08T06:13:54.812 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.812 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.812 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.812 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.812 INFO:teuthology.orchestra.run.smithi123.stdout: "oob_default_addr": { 2024-05-08T06:13:54.812 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "oob_default_addr", 2024-05-08T06:13:54.812 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.813 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.813 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.813 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "169.254.1.1", 2024-05-08T06:13:54.813 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.813 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.813 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.813 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-08T06:13:54.813 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.813 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.813 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.813 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.813 INFO:teuthology.orchestra.run.smithi123.stdout: "prometheus_alerts_path": { 2024-05-08T06:13:54.813 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "prometheus_alerts_path", 2024-05-08T06:13:54.813 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.813 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.813 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.814 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-08T06:13:54.814 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.814 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.814 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.814 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-08T06:13:54.814 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.814 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.814 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.814 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.814 INFO:teuthology.orchestra.run.smithi123.stdout: "registry_insecure": { 2024-05-08T06:13:54.814 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "registry_insecure", 2024-05-08T06:13:54.814 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.814 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.814 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.815 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.815 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.815 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.815 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.815 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-08T06:13:54.815 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.815 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.815 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.815 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.815 INFO:teuthology.orchestra.run.smithi123.stdout: "registry_password": { 2024-05-08T06:13:54.815 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "registry_password", 2024-05-08T06:13:54.815 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.815 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.815 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.815 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.815 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.816 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.816 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.816 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-08T06:13:54.816 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.816 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.816 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.816 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.816 INFO:teuthology.orchestra.run.smithi123.stdout: "registry_url": { 2024-05-08T06:13:54.816 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "registry_url", 2024-05-08T06:13:54.816 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.816 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.816 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.816 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.816 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.816 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.816 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.817 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-08T06:13:54.817 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.817 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.817 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.817 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.817 INFO:teuthology.orchestra.run.smithi123.stdout: "registry_username": { 2024-05-08T06:13:54.817 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "registry_username", 2024-05-08T06:13:54.817 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.817 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.817 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.817 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.817 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.817 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.817 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.817 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-08T06:13:54.817 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.817 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.818 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.818 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.818 INFO:teuthology.orchestra.run.smithi123.stdout: "secure_monitoring_stack": { 2024-05-08T06:13:54.818 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "secure_monitoring_stack", 2024-05-08T06:13:54.818 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.818 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.818 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.818 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.818 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.818 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.818 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.818 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-08T06:13:54.818 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.818 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.818 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.819 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.819 INFO:teuthology.orchestra.run.smithi123.stdout: "service_discovery_port": { 2024-05-08T06:13:54.819 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "service_discovery_port", 2024-05-08T06:13:54.819 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.819 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.819 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.819 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8765", 2024-05-08T06:13:54.819 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.819 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.819 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.819 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "cephadm service discovery port", 2024-05-08T06:13:54.819 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.819 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.819 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.819 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.820 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.820 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.820 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.820 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.820 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.820 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.820 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.820 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.820 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.820 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.820 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.820 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.820 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.820 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.820 INFO:teuthology.orchestra.run.smithi123.stdout: "ssh_config_file": { 2024-05-08T06:13:54.820 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ssh_config_file", 2024-05-08T06:13:54.820 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.821 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.821 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.821 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.821 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.821 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.821 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.821 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-08T06:13:54.821 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.821 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.821 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.821 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.821 INFO:teuthology.orchestra.run.smithi123.stdout: "use_agent": { 2024-05-08T06:13:54.821 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "use_agent", 2024-05-08T06:13:54.821 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.821 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.821 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.821 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.822 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.822 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.822 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.822 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-08T06:13:54.822 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.822 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.822 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.822 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.822 INFO:teuthology.orchestra.run.smithi123.stdout: "use_repo_digest": { 2024-05-08T06:13:54.822 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "use_repo_digest", 2024-05-08T06:13:54.822 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.822 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.822 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.822 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.822 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.823 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.823 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.823 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-08T06:13:54.823 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.823 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.823 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.823 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.823 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_on_failed_host_check": { 2024-05-08T06:13:54.823 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_on_failed_host_check", 2024-05-08T06:13:54.823 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.823 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.823 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.823 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.823 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.823 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.824 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.824 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "raise a health warning if the host check fails", 2024-05-08T06:13:54.824 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.824 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.824 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.824 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.824 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_on_stray_daemons": { 2024-05-08T06:13:54.824 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_on_stray_daemons", 2024-05-08T06:13:54.824 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.824 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.824 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.824 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.824 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.824 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.824 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.824 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-08T06:13:54.825 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.825 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.825 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.825 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.825 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_on_stray_hosts": { 2024-05-08T06:13:54.825 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_on_stray_hosts", 2024-05-08T06:13:54.825 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.825 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.825 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.825 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.825 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.825 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.825 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.825 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-08T06:13:54.825 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.825 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.825 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.826 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.826 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.826 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.826 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.826 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crash", 2024-05-08T06:13:54.826 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.826 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.826 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.826 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.826 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.826 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.826 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.826 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.826 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.826 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.826 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.827 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.827 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.827 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.827 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.827 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.827 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.827 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.827 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.827 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.827 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.827 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.827 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.827 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.827 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.827 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.827 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.828 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.828 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.828 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.828 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.828 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.828 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.828 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.828 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.828 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.828 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.828 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.828 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.828 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.828 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.828 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.828 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.829 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.829 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.829 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.829 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.829 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.829 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.829 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.829 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.829 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.829 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.829 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.829 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.829 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.829 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.829 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.830 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.830 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.830 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.830 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.830 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.830 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.830 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.830 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.830 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.830 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.830 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.830 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.830 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.830 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.830 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.830 INFO:teuthology.orchestra.run.smithi123.stdout: "retain_interval": { 2024-05-08T06:13:54.831 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "retain_interval", 2024-05-08T06:13:54.831 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.831 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.831 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.831 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "31536000", 2024-05-08T06:13:54.831 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.831 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.831 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.831 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-08T06:13:54.831 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.831 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.831 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.831 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.831 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.831 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.831 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.832 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.832 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.832 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.832 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.832 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.832 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.832 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.832 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.832 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.832 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.832 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.832 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_recent_interval": { 2024-05-08T06:13:54.832 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_recent_interval", 2024-05-08T06:13:54.832 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.832 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.832 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.832 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "1209600", 2024-05-08T06:13:54.833 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.833 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.833 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.833 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-08T06:13:54.833 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.833 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.833 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.833 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.833 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.833 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.833 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.833 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "dashboard", 2024-05-08T06:13:54.833 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.833 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.833 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.833 INFO:teuthology.orchestra.run.smithi123.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-08T06:13:54.834 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-08T06:13:54.834 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.834 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.834 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.834 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-05-08T06:13:54.834 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.834 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.834 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.834 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.834 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.834 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.834 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.834 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.834 INFO:teuthology.orchestra.run.smithi123.stdout: "ALERTMANAGER_API_HOST": { 2024-05-08T06:13:54.834 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-08T06:13:54.835 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.835 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.835 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.835 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.835 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.835 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.835 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.835 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.835 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.835 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.835 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.835 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.835 INFO:teuthology.orchestra.run.smithi123.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-08T06:13:54.835 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-08T06:13:54.835 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.836 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.836 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.836 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.836 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.836 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.836 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.836 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.836 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.836 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.836 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.836 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.836 INFO:teuthology.orchestra.run.smithi123.stdout: "AUDIT_API_ENABLED": { 2024-05-08T06:13:54.836 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "AUDIT_API_ENABLED", 2024-05-08T06:13:54.836 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.836 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.836 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.837 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.837 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.837 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.837 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.837 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.837 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.837 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.837 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.837 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.837 INFO:teuthology.orchestra.run.smithi123.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-08T06:13:54.837 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-08T06:13:54.837 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.837 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.837 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.837 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.837 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.838 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.838 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.838 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.838 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.838 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.838 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.838 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.838 INFO:teuthology.orchestra.run.smithi123.stdout: "ENABLE_BROWSABLE_API": { 2024-05-08T06:13:54.838 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-08T06:13:54.838 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.838 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.838 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.838 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.838 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.838 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.839 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.839 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.839 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.839 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.839 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.839 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.839 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-08T06:13:54.839 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-08T06:13:54.839 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.839 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.839 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.839 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.839 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.839 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.840 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.840 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.840 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.840 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.840 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.840 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.840 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-08T06:13:54.840 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-08T06:13:54.840 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.840 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.840 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.840 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.840 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.840 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.840 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.840 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.841 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.841 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.841 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.841 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.841 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-08T06:13:54.841 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-08T06:13:54.841 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.841 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.841 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.841 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.841 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.841 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.841 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.841 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.841 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.841 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.842 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.842 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.842 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-08T06:13:54.842 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-08T06:13:54.842 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.842 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.842 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.842 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.842 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.842 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.842 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.842 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.842 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.842 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.842 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.842 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.843 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-08T06:13:54.843 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-08T06:13:54.843 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.843 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.843 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.843 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.843 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.843 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.843 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.843 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.843 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.843 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.843 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.843 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.843 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-08T06:13:54.843 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-08T06:13:54.844 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.844 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.844 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.844 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.844 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.844 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.844 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.844 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.844 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.844 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.844 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.844 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.844 INFO:teuthology.orchestra.run.smithi123.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-08T06:13:54.844 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-08T06:13:54.844 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.844 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.844 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.845 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.845 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.845 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.845 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.845 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.845 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.845 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.845 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.845 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.845 INFO:teuthology.orchestra.run.smithi123.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-08T06:13:54.845 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-08T06:13:54.845 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.845 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.845 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.845 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.845 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.846 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.846 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.846 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.846 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.846 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.846 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.846 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.846 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_API_PASSWORD": { 2024-05-08T06:13:54.846 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-08T06:13:54.846 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.846 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.846 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.846 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "admin", 2024-05-08T06:13:54.846 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.846 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.847 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.847 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.847 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.847 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.847 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.847 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.847 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-08T06:13:54.847 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-08T06:13:54.847 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.847 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.847 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.847 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.847 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.847 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.847 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.847 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.848 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.848 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.848 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.848 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.848 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_API_URL": { 2024-05-08T06:13:54.848 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_API_URL", 2024-05-08T06:13:54.848 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.848 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.848 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.848 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.848 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.848 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.848 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.848 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.848 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.848 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.849 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.849 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.849 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_API_USERNAME": { 2024-05-08T06:13:54.849 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-08T06:13:54.849 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.849 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.849 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.849 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "admin", 2024-05-08T06:13:54.849 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.849 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.849 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.849 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.849 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.849 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.849 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.849 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.850 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-08T06:13:54.850 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-08T06:13:54.850 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.850 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.850 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.850 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.850 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.850 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.850 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.850 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.850 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.850 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.850 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.850 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.850 INFO:teuthology.orchestra.run.smithi123.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-08T06:13:54.851 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-08T06:13:54.851 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.851 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.851 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.851 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.851 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.851 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.851 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.851 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.851 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.851 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.851 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.851 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.851 INFO:teuthology.orchestra.run.smithi123.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-08T06:13:54.851 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-08T06:13:54.851 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.852 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.852 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.852 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.852 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.852 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.852 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.852 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.852 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.852 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.852 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.852 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.852 INFO:teuthology.orchestra.run.smithi123.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-08T06:13:54.852 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-08T06:13:54.852 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.852 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.853 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.853 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.853 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.853 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.853 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.853 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.853 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.853 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.853 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.853 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.853 INFO:teuthology.orchestra.run.smithi123.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-08T06:13:54.853 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-08T06:13:54.853 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.853 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.853 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.853 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "[]", 2024-05-08T06:13:54.854 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.854 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.854 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.854 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.854 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.854 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.854 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.854 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.854 INFO:teuthology.orchestra.run.smithi123.stdout: "MULTICLUSTER_CONFIG": { 2024-05-08T06:13:54.854 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-08T06:13:54.854 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.854 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.854 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.854 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "{}", 2024-05-08T06:13:54.855 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.855 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.855 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.855 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.855 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.855 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.855 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.855 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.855 INFO:teuthology.orchestra.run.smithi123.stdout: "PROMETHEUS_API_HOST": { 2024-05-08T06:13:54.855 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-08T06:13:54.855 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.855 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.855 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.855 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.856 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.856 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.856 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.856 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.856 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.856 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.856 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.856 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.856 INFO:teuthology.orchestra.run.smithi123.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-08T06:13:54.856 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-08T06:13:54.856 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.856 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.856 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.856 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.856 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.856 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.857 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.857 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.857 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.857 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.857 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.857 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.857 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-08T06:13:54.857 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-08T06:13:54.857 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.857 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.857 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.857 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.857 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.857 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.857 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.857 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.858 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.858 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.858 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.858 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.858 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-08T06:13:54.858 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-08T06:13:54.858 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.858 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.858 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.858 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.858 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.858 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.858 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.858 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.858 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.858 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.859 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.859 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.859 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-08T06:13:54.859 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-08T06:13:54.859 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.859 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.859 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.859 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.859 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.859 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.859 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.859 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.859 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.859 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.859 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.859 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.860 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-08T06:13:54.860 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-08T06:13:54.860 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.860 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.860 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.860 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.860 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.860 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.860 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.860 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.860 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.860 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.860 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.860 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.861 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-08T06:13:54.861 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-08T06:13:54.861 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.861 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.861 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.861 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.861 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.861 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.861 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.861 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.861 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.861 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.861 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.861 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-08T06:13:54.861 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-08T06:13:54.861 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.862 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.862 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.862 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.862 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.862 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.862 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.862 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.862 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.862 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.862 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.862 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.862 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-08T06:13:54.862 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-08T06:13:54.862 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.862 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.862 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.863 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.863 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.863 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.863 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.863 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.863 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.863 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.863 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.863 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.863 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_ENABLED": { 2024-05-08T06:13:54.863 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-08T06:13:54.863 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.863 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.863 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.863 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.863 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.864 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.864 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.864 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.864 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.864 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.864 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.864 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.864 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-08T06:13:54.864 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-08T06:13:54.864 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.864 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.864 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.864 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-08T06:13:54.864 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.864 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.864 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.865 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.865 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.865 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.865 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.865 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.865 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-08T06:13:54.865 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-08T06:13:54.865 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.865 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.865 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.865 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-05-08T06:13:54.865 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.865 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.865 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.865 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.865 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.865 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.866 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.866 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.866 INFO:teuthology.orchestra.run.smithi123.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-08T06:13:54.866 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-08T06:13:54.866 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.866 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.866 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.866 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8", 2024-05-08T06:13:54.866 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.866 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.866 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.866 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.866 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.866 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.866 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.867 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.867 INFO:teuthology.orchestra.run.smithi123.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-08T06:13:54.867 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-08T06:13:54.867 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.867 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.867 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.867 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "45", 2024-05-08T06:13:54.867 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.867 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.867 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.867 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.867 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.867 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.867 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.867 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.867 INFO:teuthology.orchestra.run.smithi123.stdout: "RGW_API_ACCESS_KEY": { 2024-05-08T06:13:54.868 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-08T06:13:54.868 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.868 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.868 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.868 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.868 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.868 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.868 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.868 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.868 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.868 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.868 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.868 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.868 INFO:teuthology.orchestra.run.smithi123.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-08T06:13:54.868 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-08T06:13:54.868 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.869 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.869 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.869 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "admin", 2024-05-08T06:13:54.869 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.869 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.869 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.869 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.869 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.869 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.869 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.869 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.869 INFO:teuthology.orchestra.run.smithi123.stdout: "RGW_API_SECRET_KEY": { 2024-05-08T06:13:54.869 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-08T06:13:54.869 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.869 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.869 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.870 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.870 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.870 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.870 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.870 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.870 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.870 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.870 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.870 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.870 INFO:teuthology.orchestra.run.smithi123.stdout: "RGW_API_SSL_VERIFY": { 2024-05-08T06:13:54.870 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-08T06:13:54.870 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.870 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.870 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.871 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.871 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.871 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.871 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.871 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.871 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.871 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.871 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.871 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.871 INFO:teuthology.orchestra.run.smithi123.stdout: "UNSAFE_TLS_v1_2": { 2024-05-08T06:13:54.872 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-08T06:13:54.872 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.872 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.872 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.872 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.872 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.872 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.872 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.873 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.873 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.873 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.873 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.873 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.873 INFO:teuthology.orchestra.run.smithi123.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-08T06:13:54.873 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-08T06:13:54.873 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.873 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.874 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.874 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.874 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.874 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.874 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.874 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.874 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.875 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.875 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.875 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.875 INFO:teuthology.orchestra.run.smithi123.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-08T06:13:54.875 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-08T06:13:54.875 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.875 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.875 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.875 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-05-08T06:13:54.875 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.875 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.875 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.875 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.876 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.876 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.876 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.876 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.876 INFO:teuthology.orchestra.run.smithi123.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-08T06:13:54.876 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-08T06:13:54.876 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.876 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.876 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.876 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-05-08T06:13:54.876 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.876 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.876 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.876 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.876 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.876 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.877 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.877 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.877 INFO:teuthology.orchestra.run.smithi123.stdout: "cross_origin_url": { 2024-05-08T06:13:54.877 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cross_origin_url", 2024-05-08T06:13:54.877 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.877 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.877 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.877 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.877 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.877 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.877 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.877 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.877 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.877 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.877 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.877 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.877 INFO:teuthology.orchestra.run.smithi123.stdout: "crt_file": { 2024-05-08T06:13:54.878 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "crt_file", 2024-05-08T06:13:54.878 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.878 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.878 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.878 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.878 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.878 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.878 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.878 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.878 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.878 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.878 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.878 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.878 INFO:teuthology.orchestra.run.smithi123.stdout: "debug": { 2024-05-08T06:13:54.878 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "debug", 2024-05-08T06:13:54.879 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.879 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.879 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.879 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.879 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.879 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.879 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.879 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Enable/disable debug options", 2024-05-08T06:13:54.879 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.879 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.879 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.879 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.879 INFO:teuthology.orchestra.run.smithi123.stdout: "jwt_token_ttl": { 2024-05-08T06:13:54.879 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "jwt_token_ttl", 2024-05-08T06:13:54.879 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.879 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.880 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.880 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "28800", 2024-05-08T06:13:54.880 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.880 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.880 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.880 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.880 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.880 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.880 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.880 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.880 INFO:teuthology.orchestra.run.smithi123.stdout: "key_file": { 2024-05-08T06:13:54.880 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "key_file", 2024-05-08T06:13:54.880 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.880 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.880 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.881 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.881 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.881 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.881 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.881 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.881 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.881 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.881 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.881 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.881 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.881 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.881 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.881 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.881 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.881 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.881 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.882 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.882 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.882 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.882 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.882 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.882 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.882 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.882 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.882 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.882 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.882 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.882 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.882 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.882 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.882 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.883 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.883 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.883 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.883 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.883 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.883 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.883 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.883 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.883 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.883 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.883 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.883 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.883 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.883 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.883 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.884 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.884 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.884 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.884 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.884 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.884 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.884 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.884 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.884 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.884 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.884 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.884 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.884 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.884 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.884 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.885 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.885 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.885 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.885 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.885 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.885 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.885 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.885 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.885 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.885 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.885 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.885 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.885 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.885 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.885 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.886 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.886 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.886 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.886 INFO:teuthology.orchestra.run.smithi123.stdout: "motd": { 2024-05-08T06:13:54.886 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "motd", 2024-05-08T06:13:54.886 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.886 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.886 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.886 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.886 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.886 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.886 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.886 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "The message of the day", 2024-05-08T06:13:54.886 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.886 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.886 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.887 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.887 INFO:teuthology.orchestra.run.smithi123.stdout: "redirect_resolve_ip_addr": { 2024-05-08T06:13:54.887 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "redirect_resolve_ip_addr", 2024-05-08T06:13:54.887 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.887 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.887 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.887 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.887 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.887 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.887 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.887 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.887 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.887 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.887 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.887 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.887 INFO:teuthology.orchestra.run.smithi123.stdout: "server_addr": { 2024-05-08T06:13:54.888 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_addr", 2024-05-08T06:13:54.888 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.888 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.888 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.888 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "::", 2024-05-08T06:13:54.888 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.888 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.888 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.888 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.888 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.888 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.888 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.888 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.888 INFO:teuthology.orchestra.run.smithi123.stdout: "server_port": { 2024-05-08T06:13:54.888 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_port", 2024-05-08T06:13:54.889 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.889 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.889 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.889 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8080", 2024-05-08T06:13:54.889 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.889 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.889 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.889 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.889 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.889 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.889 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.889 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.889 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.889 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.889 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.890 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.890 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.890 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.890 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.890 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.890 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.890 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.890 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.890 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.890 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.890 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.890 INFO:teuthology.orchestra.run.smithi123.stdout: "ssl": { 2024-05-08T06:13:54.890 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ssl", 2024-05-08T06:13:54.890 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.890 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.891 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.891 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.891 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.891 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.891 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.891 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.891 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.891 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.891 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.891 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.891 INFO:teuthology.orchestra.run.smithi123.stdout: "ssl_server_port": { 2024-05-08T06:13:54.891 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ssl_server_port", 2024-05-08T06:13:54.891 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.891 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.892 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.892 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8443", 2024-05-08T06:13:54.892 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.892 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.892 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.892 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.892 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.892 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.892 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.892 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.892 INFO:teuthology.orchestra.run.smithi123.stdout: "standby_behaviour": { 2024-05-08T06:13:54.892 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "standby_behaviour", 2024-05-08T06:13:54.892 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.892 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.892 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.892 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "redirect", 2024-05-08T06:13:54.893 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.893 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.893 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.893 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.893 INFO:teuthology.orchestra.run.smithi123.stdout: "redirect" 2024-05-08T06:13:54.893 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.893 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.893 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.893 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.893 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.893 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.893 INFO:teuthology.orchestra.run.smithi123.stdout: "standby_error_status_code": { 2024-05-08T06:13:54.893 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "standby_error_status_code", 2024-05-08T06:13:54.893 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.893 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.894 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.894 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "500", 2024-05-08T06:13:54.894 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "400", 2024-05-08T06:13:54.894 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "599", 2024-05-08T06:13:54.894 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.894 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.894 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.894 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.894 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.894 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.894 INFO:teuthology.orchestra.run.smithi123.stdout: "url_prefix": { 2024-05-08T06:13:54.894 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "url_prefix", 2024-05-08T06:13:54.894 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.894 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.895 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.895 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.895 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.895 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.895 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.895 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.895 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.895 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.895 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.895 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.895 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.895 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.895 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.895 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "devicehealth", 2024-05-08T06:13:54.895 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.896 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.896 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.896 INFO:teuthology.orchestra.run.smithi123.stdout: "enable_monitoring": { 2024-05-08T06:13:54.896 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "enable_monitoring", 2024-05-08T06:13:54.896 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.896 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.896 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.896 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.896 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.896 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.896 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.896 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "monitor device health metrics", 2024-05-08T06:13:54.896 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.896 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.896 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.897 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.897 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.897 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.897 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.897 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.897 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.897 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.897 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.897 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.897 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.897 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.897 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.897 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.897 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.897 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.898 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.898 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.898 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.898 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.898 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.898 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.898 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.898 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.898 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.898 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.898 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.898 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.898 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.898 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.899 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.899 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.899 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.899 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.899 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.899 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.899 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.899 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.899 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.899 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.899 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.899 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.899 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.899 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.899 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.900 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.900 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.900 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.900 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.900 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.900 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.900 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.900 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.900 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.900 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.900 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.900 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.900 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.900 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.900 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.901 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.901 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.901 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.901 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.901 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.901 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.901 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.901 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.901 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.901 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.901 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.901 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.901 INFO:teuthology.orchestra.run.smithi123.stdout: "mark_out_threshold": { 2024-05-08T06:13:54.901 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mark_out_threshold", 2024-05-08T06:13:54.901 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.902 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.902 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.902 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "2419200", 2024-05-08T06:13:54.902 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.902 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.902 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.902 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-08T06:13:54.902 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.902 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.902 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.902 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.902 INFO:teuthology.orchestra.run.smithi123.stdout: "pool_name": { 2024-05-08T06:13:54.902 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "pool_name", 2024-05-08T06:13:54.903 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.903 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.903 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.903 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "device_health_metrics", 2024-05-08T06:13:54.903 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.903 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.903 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.903 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-08T06:13:54.903 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.903 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.903 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.903 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.903 INFO:teuthology.orchestra.run.smithi123.stdout: "retention_period": { 2024-05-08T06:13:54.903 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "retention_period", 2024-05-08T06:13:54.903 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.904 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.904 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.904 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "15552000", 2024-05-08T06:13:54.904 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.904 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.904 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.904 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how long to retain device health metrics", 2024-05-08T06:13:54.904 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.904 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.904 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.904 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.904 INFO:teuthology.orchestra.run.smithi123.stdout: "scrape_frequency": { 2024-05-08T06:13:54.904 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "scrape_frequency", 2024-05-08T06:13:54.904 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.904 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.904 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.905 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "86400", 2024-05-08T06:13:54.905 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.905 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.905 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.905 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-08T06:13:54.905 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.905 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.905 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.905 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.905 INFO:teuthology.orchestra.run.smithi123.stdout: "self_heal": { 2024-05-08T06:13:54.905 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "self_heal", 2024-05-08T06:13:54.905 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.905 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.905 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.905 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:54.906 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.906 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.906 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.906 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-08T06:13:54.906 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.906 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.906 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.906 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.906 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-05-08T06:13:54.906 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-05-08T06:13:54.906 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.906 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.906 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.906 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-05-08T06:13:54.907 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.907 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.907 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.907 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how frequently to wake up and check device health", 2024-05-08T06:13:54.907 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.907 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.907 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.907 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.907 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.907 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.907 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.907 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.907 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.907 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.907 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.908 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.908 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.908 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.908 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.908 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.908 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.908 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.908 INFO:teuthology.orchestra.run.smithi123.stdout: "warn_threshold": { 2024-05-08T06:13:54.908 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "warn_threshold", 2024-05-08T06:13:54.908 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.908 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.908 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.908 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "7257600", 2024-05-08T06:13:54.908 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.908 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.909 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.909 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-08T06:13:54.909 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.909 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.909 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.909 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.909 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.909 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.909 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.909 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "diskprediction_local", 2024-05-08T06:13:54.909 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.909 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.909 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.909 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.909 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.910 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.910 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.910 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.910 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.910 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.910 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.910 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.910 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.910 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.910 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.910 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.910 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.910 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.910 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.911 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.911 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.911 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.911 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.911 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.911 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.911 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.911 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.911 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.911 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.911 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.911 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.911 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.911 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.911 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.911 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.912 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.912 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.912 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.912 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.912 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.912 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.912 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.912 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.912 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.912 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.912 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.912 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.912 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.912 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.912 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.912 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.913 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.913 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.913 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.913 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.913 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.913 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.913 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.913 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.913 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.913 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.913 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.913 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.913 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.913 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.913 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.914 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.914 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.914 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.914 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.914 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.914 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.914 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.914 INFO:teuthology.orchestra.run.smithi123.stdout: "predict_interval": { 2024-05-08T06:13:54.914 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "predict_interval", 2024-05-08T06:13:54.914 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.914 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.914 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.914 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "86400", 2024-05-08T06:13:54.914 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.915 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.915 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.915 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.915 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.915 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.915 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.915 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.915 INFO:teuthology.orchestra.run.smithi123.stdout: "predictor_model": { 2024-05-08T06:13:54.915 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "predictor_model", 2024-05-08T06:13:54.915 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.915 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.915 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.915 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "prophetstor", 2024-05-08T06:13:54.915 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.915 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.916 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.916 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.916 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.916 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.916 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.916 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.916 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-05-08T06:13:54.916 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-05-08T06:13:54.916 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.916 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.916 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.916 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "600", 2024-05-08T06:13:54.916 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.916 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.916 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.917 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.917 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.917 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.917 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.917 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.917 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.917 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.917 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.917 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.917 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.917 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.917 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.917 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.917 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.917 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.918 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.918 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.918 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.918 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.918 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.918 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.918 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.918 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "influx", 2024-05-08T06:13:54.918 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": false, 2024-05-08T06:13:54.918 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "influxdb python module not found", 2024-05-08T06:13:54.918 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.918 INFO:teuthology.orchestra.run.smithi123.stdout: "batch_size": { 2024-05-08T06:13:54.918 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "batch_size", 2024-05-08T06:13:54.918 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.918 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.919 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.919 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5000", 2024-05-08T06:13:54.919 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.919 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.919 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.919 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-08T06:13:54.919 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.919 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.919 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.919 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.919 INFO:teuthology.orchestra.run.smithi123.stdout: "database": { 2024-05-08T06:13:54.919 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "database", 2024-05-08T06:13:54.919 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.919 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.919 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.919 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "ceph", 2024-05-08T06:13:54.920 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.920 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.920 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.920 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-08T06:13:54.920 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.920 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.920 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.920 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.920 INFO:teuthology.orchestra.run.smithi123.stdout: "hostname": { 2024-05-08T06:13:54.920 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "hostname", 2024-05-08T06:13:54.920 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.920 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.920 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.920 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.920 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.921 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.921 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.921 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "InfluxDB server hostname", 2024-05-08T06:13:54.921 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.921 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.921 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.921 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.921 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-05-08T06:13:54.921 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-05-08T06:13:54.921 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:54.921 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.921 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.921 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "30", 2024-05-08T06:13:54.921 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "5", 2024-05-08T06:13:54.921 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.921 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.922 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-08T06:13:54.922 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.922 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.922 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.922 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.922 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.922 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.922 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.922 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.922 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.922 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.922 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.922 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.922 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.923 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.923 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.923 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.923 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.923 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.923 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.923 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.923 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.923 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.923 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.923 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.923 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.923 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.923 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.923 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.924 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.924 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.924 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.924 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.924 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.924 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.924 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.924 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.924 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.924 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.924 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.924 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.924 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.924 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.924 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.924 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.924 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.925 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.925 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.925 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.925 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.925 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.925 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.925 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.925 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.925 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.925 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.925 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.925 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.925 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.925 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.925 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.925 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.926 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.926 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.926 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.926 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.926 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.926 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.926 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.926 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.926 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.926 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.926 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.926 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.926 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.926 INFO:teuthology.orchestra.run.smithi123.stdout: "password": { 2024-05-08T06:13:54.927 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "password", 2024-05-08T06:13:54.927 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.927 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.927 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.927 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.927 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.927 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.927 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.927 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "password of InfluxDB server user", 2024-05-08T06:13:54.927 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.927 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.927 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.927 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.927 INFO:teuthology.orchestra.run.smithi123.stdout: "port": { 2024-05-08T06:13:54.927 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "port", 2024-05-08T06:13:54.927 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.928 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.928 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.928 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "8086", 2024-05-08T06:13:54.928 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.928 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.928 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.928 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "InfluxDB server port", 2024-05-08T06:13:54.928 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.928 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.928 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.928 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.928 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.928 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.928 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.928 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.928 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.929 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.929 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.929 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.929 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.929 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.929 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.929 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.929 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.929 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.929 INFO:teuthology.orchestra.run.smithi123.stdout: "ssl": { 2024-05-08T06:13:54.929 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ssl", 2024-05-08T06:13:54.929 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.929 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.929 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.929 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "false", 2024-05-08T06:13:54.929 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.930 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.930 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.930 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T06:13:54.930 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.930 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.930 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.930 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.930 INFO:teuthology.orchestra.run.smithi123.stdout: "threads": { 2024-05-08T06:13:54.930 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "threads", 2024-05-08T06:13:54.930 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.930 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.930 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.930 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-05-08T06:13:54.930 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1", 2024-05-08T06:13:54.930 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "32", 2024-05-08T06:13:54.931 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.931 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-08T06:13:54.931 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.931 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.931 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.931 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.931 INFO:teuthology.orchestra.run.smithi123.stdout: "username": { 2024-05-08T06:13:54.931 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "username", 2024-05-08T06:13:54.931 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.931 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.931 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.931 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.931 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.931 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.931 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.931 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "username of InfluxDB server user", 2024-05-08T06:13:54.932 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.932 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.932 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.932 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.932 INFO:teuthology.orchestra.run.smithi123.stdout: "verify_ssl": { 2024-05-08T06:13:54.932 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "verify_ssl", 2024-05-08T06:13:54.932 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.932 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.932 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.932 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "true", 2024-05-08T06:13:54.932 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.932 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.932 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.932 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T06:13:54.932 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.932 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.933 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.933 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.933 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.933 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.933 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.933 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "insights", 2024-05-08T06:13:54.933 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.933 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.933 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.933 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.933 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.933 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.933 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.933 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.933 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.933 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.934 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.934 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.934 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.934 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.934 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.934 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.934 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.934 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.934 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.934 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.934 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.934 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.934 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.934 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.935 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.935 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.935 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.935 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.935 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.935 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.935 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.935 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.935 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.935 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.935 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.935 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.935 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.935 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.935 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.935 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.936 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.936 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.936 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.936 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.936 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.936 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.936 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.936 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.936 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.936 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.936 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.936 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.936 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.936 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.936 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.936 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.936 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.937 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.937 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.937 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.937 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.937 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.937 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.937 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.937 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.937 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.937 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.937 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.937 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.937 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.937 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.937 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.937 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.937 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.938 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.938 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.938 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.938 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.938 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.938 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.938 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.938 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.938 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.938 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.938 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.938 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.938 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.938 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.938 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.939 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.939 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "iostat", 2024-05-08T06:13:54.939 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.939 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.939 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.939 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.939 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.939 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.939 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.939 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.939 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.939 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.939 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.939 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.939 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.939 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.940 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.940 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.940 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.940 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.940 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.940 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.940 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.940 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.940 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.940 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.940 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.940 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.940 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.940 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.940 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.941 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.941 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.941 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.941 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.941 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.941 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.941 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.941 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.941 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.941 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.941 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.941 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.941 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.941 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.941 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.941 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.942 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.942 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.942 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.942 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.942 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.942 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.942 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.942 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.942 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.942 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.942 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.942 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.942 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.942 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.943 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.943 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.943 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.943 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.943 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.943 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.943 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.943 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.943 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.943 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.943 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.943 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.943 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.943 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.944 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.944 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.944 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.944 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.944 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.944 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.944 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.944 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.944 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.944 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.944 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.944 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.944 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.944 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.944 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.945 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.945 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.945 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "k8sevents", 2024-05-08T06:13:54.945 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.945 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.945 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.945 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_event_retention_days": { 2024-05-08T06:13:54.945 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "ceph_event_retention_days", 2024-05-08T06:13:54.945 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.945 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.945 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.945 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "7", 2024-05-08T06:13:54.945 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.945 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.945 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.945 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-08T06:13:54.946 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.946 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.946 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.946 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.946 INFO:teuthology.orchestra.run.smithi123.stdout: "config_check_secs": { 2024-05-08T06:13:54.946 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "config_check_secs", 2024-05-08T06:13:54.946 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.946 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.946 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.946 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-05-08T06:13:54.946 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "10", 2024-05-08T06:13:54.946 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.946 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.946 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-08T06:13:54.947 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.947 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.947 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.947 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.947 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.947 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.947 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.947 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.947 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.947 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.947 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.947 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.947 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.947 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.947 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.948 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.948 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.948 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.948 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.948 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.948 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.948 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.948 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.948 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.948 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.948 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.948 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.948 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.948 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.948 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.948 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.949 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.949 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.949 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.949 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.949 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.949 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.949 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.949 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.949 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.949 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.949 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.949 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.949 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.949 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.949 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.950 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.950 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.950 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.950 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.950 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.950 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.950 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.950 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.950 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.950 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.950 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.950 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.950 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.950 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.951 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.951 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.951 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.951 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.951 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.951 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.951 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.951 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.951 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.951 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.951 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.951 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.951 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.951 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.951 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.951 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.952 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.952 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.952 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.952 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.952 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.952 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.952 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.952 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.952 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.952 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.952 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.952 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.952 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.952 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.952 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.952 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "localpool", 2024-05-08T06:13:54.952 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.953 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.953 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.953 INFO:teuthology.orchestra.run.smithi123.stdout: "failure_domain": { 2024-05-08T06:13:54.953 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "failure_domain", 2024-05-08T06:13:54.953 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.953 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.953 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.953 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "host", 2024-05-08T06:13:54.953 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.953 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.953 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.953 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "failure domain for any created local pool", 2024-05-08T06:13:54.953 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-08T06:13:54.953 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.953 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.954 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.954 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.954 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.954 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.954 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.954 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.954 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.954 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.954 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.954 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.954 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.954 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.954 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.954 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.955 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.955 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.955 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.955 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.955 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.955 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.955 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.955 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.955 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.955 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.955 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.955 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.955 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.955 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.955 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.955 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.956 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.956 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.956 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.956 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.956 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.956 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.956 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.956 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.956 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.956 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.956 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.956 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.956 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.956 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.956 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.956 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.956 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.957 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.957 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.957 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.957 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.957 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.957 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.957 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.957 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.957 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.957 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.957 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.957 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.957 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.957 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.957 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.957 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.957 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.958 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.958 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.958 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.958 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.958 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.958 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.958 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.958 INFO:teuthology.orchestra.run.smithi123.stdout: "min_size": { 2024-05-08T06:13:54.958 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "min_size", 2024-05-08T06:13:54.958 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.958 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.958 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.958 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.958 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.958 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.959 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.959 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "default min_size for any created local pool", 2024-05-08T06:13:54.959 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-08T06:13:54.959 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.959 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.959 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.959 INFO:teuthology.orchestra.run.smithi123.stdout: "num_rep": { 2024-05-08T06:13:54.959 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "num_rep", 2024-05-08T06:13:54.959 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.959 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.959 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.959 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "3", 2024-05-08T06:13:54.959 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.959 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.959 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.960 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "default replica count for any created local pool", 2024-05-08T06:13:54.960 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.960 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.960 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.960 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.960 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_num": { 2024-05-08T06:13:54.960 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "pg_num", 2024-05-08T06:13:54.960 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.960 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.960 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.960 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "128", 2024-05-08T06:13:54.960 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.960 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.960 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.960 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "default pg_num for any created local pool", 2024-05-08T06:13:54.960 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.960 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.961 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.961 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.961 INFO:teuthology.orchestra.run.smithi123.stdout: "prefix": { 2024-05-08T06:13:54.961 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "prefix", 2024-05-08T06:13:54.961 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.961 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.961 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.961 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.961 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.961 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.961 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.961 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "name prefix for any created local pool", 2024-05-08T06:13:54.961 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.961 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.961 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.961 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.962 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.962 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.962 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.962 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.962 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.962 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.962 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.962 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.962 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.962 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.962 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.962 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.962 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.962 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.963 INFO:teuthology.orchestra.run.smithi123.stdout: "subtree": { 2024-05-08T06:13:54.963 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "subtree", 2024-05-08T06:13:54.963 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.963 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.963 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.963 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "rack", 2024-05-08T06:13:54.963 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.963 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.963 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.963 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-08T06:13:54.963 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-08T06:13:54.963 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.963 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.963 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.963 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.964 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.964 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.964 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mds_autoscaler", 2024-05-08T06:13:54.964 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.964 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.964 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.964 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.964 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.964 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.964 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.964 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.964 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.964 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.964 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.964 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.965 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.965 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.965 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.965 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.965 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.965 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.965 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.965 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.965 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.965 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.965 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.965 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.965 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.965 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.965 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.965 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.966 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.966 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.966 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.966 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.966 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.966 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.966 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.966 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.966 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.966 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.966 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.966 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.966 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.966 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.966 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.967 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.967 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.967 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.967 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.967 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.967 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.967 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.967 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.967 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.967 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.967 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.967 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.967 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.967 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.967 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.968 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.968 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.968 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.968 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.968 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.968 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.968 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.968 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.968 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.968 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.968 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.968 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.968 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.968 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.968 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.968 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.968 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.969 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.969 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.969 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.969 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.969 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.969 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.969 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.969 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.969 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.969 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.969 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.969 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.969 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.969 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.969 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.970 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mirroring", 2024-05-08T06:13:54.970 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.970 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.970 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.970 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.970 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.970 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.970 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.970 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.970 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.970 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.970 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.970 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.970 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.971 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.971 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.971 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.971 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.971 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.971 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.971 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.971 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.971 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.971 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.971 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.971 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.971 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.972 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.972 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.972 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.972 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.972 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.972 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.972 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.972 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.972 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.972 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.972 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.972 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.972 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.972 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.973 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.973 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.973 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.973 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.973 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.973 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.973 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.973 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.973 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.973 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.973 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.973 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.973 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.973 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.973 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.973 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.974 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.974 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.974 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.974 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.974 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.974 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.974 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.974 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.974 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.974 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.974 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.974 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.974 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.975 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.975 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.975 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.975 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.975 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.975 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.975 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.975 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.975 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.975 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.975 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.975 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.975 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.975 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.975 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.976 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.976 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.976 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.976 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.976 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.976 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.976 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "nfs", 2024-05-08T06:13:54.976 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.976 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.976 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.976 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.976 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.976 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.976 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.976 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.977 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.977 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.977 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.977 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.977 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.977 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.977 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.977 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.977 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.977 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.977 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.977 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.977 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.977 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.977 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.977 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.978 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.978 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.978 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.978 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.978 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.978 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.978 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.978 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.978 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.978 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.978 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.978 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.978 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.978 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.979 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.979 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.979 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.979 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.979 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.979 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.979 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.979 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.979 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.979 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.979 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.979 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.979 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.980 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.980 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.980 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.980 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.980 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.980 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.980 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.980 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.980 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.980 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.980 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.980 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.980 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.980 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.980 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.981 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.981 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.981 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.981 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.981 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.981 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.981 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.981 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.981 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.981 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.981 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.981 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.981 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.981 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.981 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.982 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.982 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.982 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.982 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.982 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.982 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.982 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.982 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.982 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.982 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "orchestrator", 2024-05-08T06:13:54.982 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.982 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.982 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.982 INFO:teuthology.orchestra.run.smithi123.stdout: "fail_fs": { 2024-05-08T06:13:54.983 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "fail_fs", 2024-05-08T06:13:54.983 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.983 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.983 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:54.983 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.983 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.983 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.983 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.983 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-08T06:13:54.983 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.983 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.983 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.983 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.983 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.983 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.984 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.984 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.984 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.984 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.984 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.984 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.984 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.984 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.984 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.984 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.984 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.984 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.984 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.984 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.985 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.985 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.985 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.985 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.985 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.985 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.985 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.985 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.985 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.985 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.985 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.985 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.985 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.985 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.985 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.985 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.986 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.986 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.986 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.986 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.986 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.986 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.986 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.986 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.986 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.986 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.986 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.986 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.986 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.986 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.987 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.987 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.987 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.987 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.987 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.987 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.987 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.987 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.987 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.987 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.987 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.987 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.987 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.987 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.988 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.988 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.988 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.988 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.988 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.988 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.988 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.988 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.988 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.988 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.988 INFO:teuthology.orchestra.run.smithi123.stdout: "orchestrator": { 2024-05-08T06:13:54.988 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "orchestrator", 2024-05-08T06:13:54.988 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.988 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.988 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.989 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.989 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.989 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.989 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.989 INFO:teuthology.orchestra.run.smithi123.stdout: "cephadm", 2024-05-08T06:13:54.989 INFO:teuthology.orchestra.run.smithi123.stdout: "rook", 2024-05-08T06:13:54.989 INFO:teuthology.orchestra.run.smithi123.stdout: "test_orchestrator" 2024-05-08T06:13:54.989 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.989 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Orchestrator backend", 2024-05-08T06:13:54.989 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.989 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.989 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.989 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.989 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.989 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.989 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.990 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.990 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.990 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.990 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.990 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.990 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.990 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.990 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.990 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.990 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.990 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.990 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.990 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.990 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.990 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "osd_perf_query", 2024-05-08T06:13:54.990 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.991 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.991 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.991 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.991 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.991 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.991 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.991 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.991 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.991 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.991 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.991 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.991 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.991 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.991 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.991 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.991 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.992 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.992 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.992 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.992 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.992 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.992 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.992 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.992 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.992 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.992 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.992 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.992 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.992 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.992 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.992 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.993 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.993 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.993 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.993 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.993 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.993 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.993 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.993 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.993 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.993 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.993 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.993 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.993 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.993 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.993 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.993 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.994 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.994 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.994 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.994 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.994 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.994 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.994 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.994 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.994 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.994 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.994 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.994 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:54.994 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:54.994 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.994 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.994 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.995 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.995 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.995 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.995 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.995 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.995 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.995 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.995 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.995 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.995 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:54.995 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:54.995 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:54.995 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:54.995 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.995 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:54.996 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.996 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.996 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.996 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.996 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.996 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.996 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.996 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.996 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:54.996 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.996 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:54.996 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "osd_support", 2024-05-08T06:13:54.996 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:54.996 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:54.996 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:54.997 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:54.997 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:54.997 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.997 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.997 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.997 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:54.997 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.997 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.997 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.997 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.997 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.997 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:54.997 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:54.997 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:54.997 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:54.997 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:54.998 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.998 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.998 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.998 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.998 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.998 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:54.998 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:54.998 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:54.998 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.998 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.998 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:54.998 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.998 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.998 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:54.998 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:54.999 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:54.999 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:54.999 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:54.999 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:54.999 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:54.999 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:54.999 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:54.999 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:54.999 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:54.999 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:54.999 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:54.999 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:54.999 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:54.999 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:54.999 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:54.999 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:55.000 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:55.000 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:55.000 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:55.000 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.000 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.000 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.000 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.000 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.000 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.000 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:55.000 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:55.000 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.000 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.000 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.000 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.000 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.001 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.001 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.001 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.001 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.001 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.001 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.001 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.001 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:55.001 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:55.001 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:55.001 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:55.001 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.001 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:55.001 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.001 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.001 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.001 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.002 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.002 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.002 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.002 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.002 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.002 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.002 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:55.002 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "pg_autoscaler", 2024-05-08T06:13:55.002 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:55.002 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:55.002 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:55.002 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:55.002 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:55.002 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.002 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.003 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.003 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.003 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.003 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.003 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:55.003 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:55.003 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:55.003 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:55.003 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:55.003 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:55.003 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:55.003 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.003 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.003 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.003 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.004 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.004 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.004 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:55.004 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:55.004 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.004 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.004 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.004 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.004 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.004 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.004 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.004 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.004 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.004 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.004 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.004 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.005 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:55.005 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:55.005 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.005 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.005 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.005 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:55.005 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.005 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.005 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:55.005 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:55.005 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:55.005 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:55.005 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:55.005 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:55.005 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:55.006 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.006 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.006 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.006 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.006 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.006 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.006 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:55.006 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:55.006 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.006 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.006 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.006 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.006 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.006 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.006 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.007 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.007 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.007 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.007 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.007 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.007 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-05-08T06:13:55.007 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-05-08T06:13:55.007 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:55.007 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.007 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.007 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-05-08T06:13:55.007 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.007 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.007 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.007 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.007 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.008 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.008 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.008 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.008 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:55.008 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:55.008 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:55.008 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:55.008 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.008 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:55.008 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.008 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.008 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.008 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.008 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.008 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.008 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.009 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.009 INFO:teuthology.orchestra.run.smithi123.stdout: "threshold": { 2024-05-08T06:13:55.009 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "threshold", 2024-05-08T06:13:55.009 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-05-08T06:13:55.009 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.009 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.009 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "3.0", 2024-05-08T06:13:55.009 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1.0", 2024-05-08T06:13:55.009 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.009 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.009 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "scaling threshold", 2024-05-08T06:13:55.009 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-08T06:13:55.009 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.009 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.009 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.009 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.010 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.010 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:55.010 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "progress", 2024-05-08T06:13:55.010 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:55.010 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:55.010 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:55.010 INFO:teuthology.orchestra.run.smithi123.stdout: "allow_pg_recovery_event": { 2024-05-08T06:13:55.010 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "allow_pg_recovery_event", 2024-05-08T06:13:55.010 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.010 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.010 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.010 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.010 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.010 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.010 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.010 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-08T06:13:55.011 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.011 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.011 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.011 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.011 INFO:teuthology.orchestra.run.smithi123.stdout: "enabled": { 2024-05-08T06:13:55.011 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "enabled", 2024-05-08T06:13:55.011 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.011 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.011 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.011 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:55.011 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.011 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.011 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.011 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.011 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.012 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.012 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.012 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.012 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:55.012 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:55.012 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.012 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.012 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.012 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.012 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.012 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.012 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:55.012 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:55.012 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:55.012 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:55.013 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:55.013 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:55.013 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:55.013 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.013 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.013 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.013 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.013 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.013 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.013 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:55.013 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:55.013 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.013 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.013 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.013 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.013 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.014 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.014 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.014 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.014 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.014 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.014 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.014 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.014 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:55.014 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:55.014 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.014 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.014 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.014 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:55.014 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.014 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.015 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:55.015 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:55.015 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:55.015 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:55.015 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:55.015 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:55.015 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:55.015 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.015 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.015 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.015 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.015 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.015 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.015 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:55.015 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:55.016 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.016 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.016 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.016 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.016 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.016 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.016 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.016 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.016 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.016 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.016 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.016 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.016 INFO:teuthology.orchestra.run.smithi123.stdout: "max_completed_events": { 2024-05-08T06:13:55.016 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_completed_events", 2024-05-08T06:13:55.016 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:55.017 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.017 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.017 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "50", 2024-05-08T06:13:55.017 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.017 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.017 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.017 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "number of past completed events to remember", 2024-05-08T06:13:55.017 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.017 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.017 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.017 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.017 INFO:teuthology.orchestra.run.smithi123.stdout: "sleep_interval": { 2024-05-08T06:13:55.017 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sleep_interval", 2024-05-08T06:13:55.017 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:55.017 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.017 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.018 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "5", 2024-05-08T06:13:55.018 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.018 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.018 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.018 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "how long the module is going to sleep", 2024-05-08T06:13:55.018 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.018 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.018 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.018 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.018 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:55.018 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:55.018 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:55.018 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:55.018 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.018 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:55.018 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.019 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.019 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.019 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.019 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.019 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.019 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.019 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.019 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.019 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.019 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:55.019 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "prometheus", 2024-05-08T06:13:55.019 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:55.019 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:55.019 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:55.019 INFO:teuthology.orchestra.run.smithi123.stdout: "cache": { 2024-05-08T06:13:55.020 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "cache", 2024-05-08T06:13:55.020 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.020 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.020 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.020 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:55.020 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.020 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.020 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.020 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.020 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.020 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.020 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.020 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.020 INFO:teuthology.orchestra.run.smithi123.stdout: "exclude_perf_counters": { 2024-05-08T06:13:55.020 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "exclude_perf_counters", 2024-05-08T06:13:55.021 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.021 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.021 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.021 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:55.021 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.021 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.021 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.021 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-08T06:13:55.021 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-08T06:13:55.021 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.021 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.021 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.021 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:55.021 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:55.021 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.022 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.022 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.022 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.022 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.022 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.022 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:55.022 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:55.022 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:55.022 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:55.022 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:55.022 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:55.022 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:55.022 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.022 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.023 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.023 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.023 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.023 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.023 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:55.023 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:55.023 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.023 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.023 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.023 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.023 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.023 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.023 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.023 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.023 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.024 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.024 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.024 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.024 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:55.024 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:55.024 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.024 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.024 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.024 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:55.024 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.024 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.024 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:55.024 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:55.024 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:55.024 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:55.024 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:55.025 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:55.025 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:55.025 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.025 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.025 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.025 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.025 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.025 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.025 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:55.025 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:55.025 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.025 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.025 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.025 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.025 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.026 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.026 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.026 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.026 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.026 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.026 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.026 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.026 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_stats_pools": { 2024-05-08T06:13:55.026 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rbd_stats_pools", 2024-05-08T06:13:55.026 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.026 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.026 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.026 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.026 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.026 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.027 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.027 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.027 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.027 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.027 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.027 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.027 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-08T06:13:55.027 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-08T06:13:55.027 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:55.027 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.027 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.027 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "300", 2024-05-08T06:13:55.027 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.027 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.027 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.027 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.028 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.028 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.028 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.028 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.028 INFO:teuthology.orchestra.run.smithi123.stdout: "scrape_interval": { 2024-05-08T06:13:55.028 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "scrape_interval", 2024-05-08T06:13:55.028 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-05-08T06:13:55.028 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.028 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.028 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "15.0", 2024-05-08T06:13:55.028 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.028 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.028 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.028 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.028 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.028 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.029 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.029 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.029 INFO:teuthology.orchestra.run.smithi123.stdout: "server_addr": { 2024-05-08T06:13:55.029 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_addr", 2024-05-08T06:13:55.029 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.029 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.029 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.029 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "::", 2024-05-08T06:13:55.029 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.029 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.029 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.029 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-08T06:13:55.029 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.029 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.029 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.029 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.030 INFO:teuthology.orchestra.run.smithi123.stdout: "server_port": { 2024-05-08T06:13:55.030 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_port", 2024-05-08T06:13:55.030 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:55.030 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.030 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.030 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "9283", 2024-05-08T06:13:55.030 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.030 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.030 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.030 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-08T06:13:55.030 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.030 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.030 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.030 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.030 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:55.030 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:55.031 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:55.031 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:55.031 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.031 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:55.031 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.031 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.031 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.031 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.031 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.031 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.031 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.031 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.031 INFO:teuthology.orchestra.run.smithi123.stdout: "stale_cache_strategy": { 2024-05-08T06:13:55.031 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "stale_cache_strategy", 2024-05-08T06:13:55.031 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.031 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.031 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.032 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "log", 2024-05-08T06:13:55.032 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.032 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.032 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.032 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.032 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.032 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.032 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.032 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.032 INFO:teuthology.orchestra.run.smithi123.stdout: "standby_behaviour": { 2024-05-08T06:13:55.032 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "standby_behaviour", 2024-05-08T06:13:55.032 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.032 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.032 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.032 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "default", 2024-05-08T06:13:55.032 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.033 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.033 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:55.033 INFO:teuthology.orchestra.run.smithi123.stdout: "default", 2024-05-08T06:13:55.033 INFO:teuthology.orchestra.run.smithi123.stdout: "error" 2024-05-08T06:13:55.033 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.033 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.033 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.033 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.033 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.033 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.033 INFO:teuthology.orchestra.run.smithi123.stdout: "standby_error_status_code": { 2024-05-08T06:13:55.033 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "standby_error_status_code", 2024-05-08T06:13:55.033 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:55.033 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.033 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.033 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "500", 2024-05-08T06:13:55.034 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "400", 2024-05-08T06:13:55.034 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "599", 2024-05-08T06:13:55.034 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.034 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.034 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.034 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.034 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.034 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.034 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.034 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.034 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:55.034 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rbd_support", 2024-05-08T06:13:55.034 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:55.034 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:55.034 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:55.035 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:55.035 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:55.035 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.035 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.035 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.035 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.035 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.035 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.035 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:55.035 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:55.035 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:55.035 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:55.035 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:55.035 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:55.035 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:55.035 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.035 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.036 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.036 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.036 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.036 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.036 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:55.036 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:55.036 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.036 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.036 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.036 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.036 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.036 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.036 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.036 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.036 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.037 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.037 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.037 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.037 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:55.037 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:55.037 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.037 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.037 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.037 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:55.037 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.037 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.037 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:55.037 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:55.037 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:55.037 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:55.037 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:55.038 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:55.038 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:55.038 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.038 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.038 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.038 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.038 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.038 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.038 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:55.038 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:55.038 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.038 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.038 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.038 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.038 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.039 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.039 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.039 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.039 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.039 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.039 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.039 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.039 INFO:teuthology.orchestra.run.smithi123.stdout: "max_concurrent_snap_create": { 2024-05-08T06:13:55.039 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_concurrent_snap_create", 2024-05-08T06:13:55.039 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:55.039 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.039 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.039 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10", 2024-05-08T06:13:55.039 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.039 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.039 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.039 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.040 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.040 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.040 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.040 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.040 INFO:teuthology.orchestra.run.smithi123.stdout: "mirror_snapshot_schedule": { 2024-05-08T06:13:55.040 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "mirror_snapshot_schedule", 2024-05-08T06:13:55.040 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.040 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.040 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.040 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.040 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.040 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.040 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.040 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.040 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.040 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.041 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.041 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.041 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:55.041 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:55.041 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:55.041 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:55.041 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.041 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:55.041 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.041 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.041 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.041 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.041 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.041 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.041 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.041 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.042 INFO:teuthology.orchestra.run.smithi123.stdout: "trash_purge_schedule": { 2024-05-08T06:13:55.042 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "trash_purge_schedule", 2024-05-08T06:13:55.042 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.042 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.042 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.042 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.042 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.042 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.042 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.042 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.042 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.042 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.042 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.042 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.042 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.042 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.043 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:55.043 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "restful", 2024-05-08T06:13:55.043 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:55.043 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:55.043 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:55.043 INFO:teuthology.orchestra.run.smithi123.stdout: "enable_auth": { 2024-05-08T06:13:55.043 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "enable_auth", 2024-05-08T06:13:55.043 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.043 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.043 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.043 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:55.043 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.043 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.043 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.043 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.043 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.043 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.044 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.044 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.044 INFO:teuthology.orchestra.run.smithi123.stdout: "key_file": { 2024-05-08T06:13:55.044 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "key_file", 2024-05-08T06:13:55.044 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.044 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.044 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.044 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.044 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.044 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.044 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.044 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.044 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.044 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.044 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.044 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.045 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:55.045 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:55.045 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.045 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.045 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.045 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.045 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.045 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.045 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:55.045 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:55.045 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:55.045 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:55.045 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:55.045 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:55.045 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:55.045 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.045 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.046 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.046 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.046 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.046 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.046 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:55.046 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:55.046 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.046 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.046 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.046 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.046 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.046 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.046 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.046 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.047 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.047 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.047 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.047 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.047 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:55.047 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:55.047 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.047 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.047 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.047 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:55.047 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.047 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.047 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:55.047 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:55.047 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:55.047 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:55.048 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:55.048 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:55.048 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:55.048 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.048 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.048 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.048 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.048 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.048 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.048 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:55.048 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:55.048 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.048 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.048 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.048 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.048 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.048 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.049 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.049 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.049 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.049 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.049 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.049 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.049 INFO:teuthology.orchestra.run.smithi123.stdout: "server_addr": { 2024-05-08T06:13:55.049 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_addr", 2024-05-08T06:13:55.049 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.049 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.049 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.049 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.049 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.049 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.049 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.049 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.050 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.050 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.050 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.050 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.050 INFO:teuthology.orchestra.run.smithi123.stdout: "server_port": { 2024-05-08T06:13:55.050 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "server_port", 2024-05-08T06:13:55.050 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.050 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.050 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.050 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.050 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.050 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.050 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.050 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.050 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.050 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.051 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.051 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.051 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:55.051 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:55.051 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:55.051 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:55.051 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.051 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:55.051 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.051 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.051 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.051 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.051 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.051 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.051 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.052 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.052 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.052 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.052 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:55.052 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rgw", 2024-05-08T06:13:55.052 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:55.052 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:55.052 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:55.052 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:55.052 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:55.052 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.052 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.052 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.052 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.052 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.052 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.053 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:55.053 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:55.053 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:55.053 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:55.053 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:55.053 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:55.053 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:55.053 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.053 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.053 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.053 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.053 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.053 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.053 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:55.053 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:55.053 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.054 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.054 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.054 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.054 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.054 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.054 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.054 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.054 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.054 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.054 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.054 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.054 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:55.054 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:55.054 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.054 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.054 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.055 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:55.055 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.055 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.055 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:55.055 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:55.055 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:55.055 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:55.055 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:55.055 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:55.055 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:55.055 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.055 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.055 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.055 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.055 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.055 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.056 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:55.056 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:55.056 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.056 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.056 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.056 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.056 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.056 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.056 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.056 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.056 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.056 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.056 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.056 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.056 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:55.056 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:55.057 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:55.057 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:55.057 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.057 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:55.057 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.057 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.057 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.057 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.057 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.057 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.057 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.057 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.057 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.057 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.057 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:55.057 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rook", 2024-05-08T06:13:55.058 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:55.058 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:55.058 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:55.058 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:55.058 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:55.058 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.058 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.058 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.058 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.058 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.058 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.058 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:55.058 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:55.058 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:55.058 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:55.058 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:55.058 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:55.059 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:55.059 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.059 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.059 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.059 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.059 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.059 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.059 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:55.059 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:55.059 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.059 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.059 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.059 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.059 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.059 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.060 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.060 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.060 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.060 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.060 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.060 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.060 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:55.060 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:55.060 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.060 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.060 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.060 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:55.060 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.060 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.060 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:55.060 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:55.061 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:55.061 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:55.061 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:55.061 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:55.061 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:55.061 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.061 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.061 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.061 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.061 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.061 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.061 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:55.061 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:55.061 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.061 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.061 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.061 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.062 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.062 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.062 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.062 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.062 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.062 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.062 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.062 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.062 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:55.062 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:55.062 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:55.062 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:55.062 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.062 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:55.062 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.063 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.063 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.063 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.063 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.063 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.063 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.063 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.063 INFO:teuthology.orchestra.run.smithi123.stdout: "storage_class": { 2024-05-08T06:13:55.063 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "storage_class", 2024-05-08T06:13:55.063 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.063 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.063 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.063 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "local", 2024-05-08T06:13:55.063 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.063 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.063 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.064 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-08T06:13:55.064 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.064 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.064 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.064 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.064 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.064 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.064 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:55.064 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "selftest", 2024-05-08T06:13:55.064 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:55.064 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:55.064 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:55.064 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:55.064 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:55.064 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.065 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.065 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.065 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.065 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.065 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.065 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:55.065 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:55.065 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:55.065 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:55.065 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:55.065 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:55.065 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:55.065 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.065 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.065 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.065 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.066 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.066 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.066 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:55.066 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:55.066 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.066 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.066 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.066 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.066 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.066 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.066 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.066 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.066 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.066 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.066 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.066 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.067 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:55.067 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:55.067 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.067 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.067 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.067 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:55.067 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.067 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.067 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:55.067 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:55.067 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:55.067 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:55.067 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:55.067 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:55.067 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:55.067 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.068 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.068 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.068 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.068 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.068 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.068 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:55.068 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:55.068 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.068 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.068 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.068 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.068 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.068 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.068 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.068 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.068 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.069 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.069 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.069 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.069 INFO:teuthology.orchestra.run.smithi123.stdout: "roption1": { 2024-05-08T06:13:55.069 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "roption1", 2024-05-08T06:13:55.069 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.069 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.069 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.069 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.069 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.069 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.069 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.069 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.069 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.069 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.069 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.070 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.070 INFO:teuthology.orchestra.run.smithi123.stdout: "roption2": { 2024-05-08T06:13:55.070 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "roption2", 2024-05-08T06:13:55.070 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.070 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.070 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.070 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "xyz", 2024-05-08T06:13:55.070 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.070 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.070 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.070 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.070 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.070 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.070 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.070 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.071 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption1": { 2024-05-08T06:13:55.071 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption1", 2024-05-08T06:13:55.071 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.071 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.071 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.071 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.071 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.071 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.071 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.071 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.071 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.071 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.071 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.071 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.071 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption2": { 2024-05-08T06:13:55.071 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption2", 2024-05-08T06:13:55.071 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:55.072 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.072 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.072 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.072 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.072 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.072 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.072 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.072 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.072 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.072 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.072 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.072 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption3": { 2024-05-08T06:13:55.072 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption3", 2024-05-08T06:13:55.072 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "float", 2024-05-08T06:13:55.072 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.072 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.073 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.073 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.073 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.073 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.073 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.073 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.073 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.073 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.073 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption4": { 2024-05-08T06:13:55.073 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption4", 2024-05-08T06:13:55.073 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.073 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.073 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.073 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.073 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.074 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.074 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.074 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.074 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.074 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.074 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.074 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.074 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption5": { 2024-05-08T06:13:55.074 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption5", 2024-05-08T06:13:55.074 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.074 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.074 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.074 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.074 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.074 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.074 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.075 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.075 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.075 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.075 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.075 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.075 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption6": { 2024-05-08T06:13:55.075 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption6", 2024-05-08T06:13:55.075 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.075 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.075 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.075 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:55.075 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.075 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.075 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.075 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.076 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.076 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.076 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.076 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.076 INFO:teuthology.orchestra.run.smithi123.stdout: "rwoption7": { 2024-05-08T06:13:55.076 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rwoption7", 2024-05-08T06:13:55.076 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:55.076 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.076 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.076 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.076 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "1", 2024-05-08T06:13:55.076 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "42", 2024-05-08T06:13:55.076 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.076 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.076 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.076 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.077 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.077 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.077 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:55.077 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:55.077 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:55.077 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:55.077 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.077 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:55.077 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.077 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.077 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.077 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.077 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.077 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.077 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.077 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.078 INFO:teuthology.orchestra.run.smithi123.stdout: "testkey": { 2024-05-08T06:13:55.078 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "testkey", 2024-05-08T06:13:55.078 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.078 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.078 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.078 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.078 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.078 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.078 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.078 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.078 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.078 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.078 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.078 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.078 INFO:teuthology.orchestra.run.smithi123.stdout: "testlkey": { 2024-05-08T06:13:55.078 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "testlkey", 2024-05-08T06:13:55.079 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.079 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.079 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.079 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.079 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.079 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.079 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.079 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.079 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.079 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.079 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.079 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.079 INFO:teuthology.orchestra.run.smithi123.stdout: "testnewline": { 2024-05-08T06:13:55.079 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "testnewline", 2024-05-08T06:13:55.079 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.080 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.080 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.080 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.080 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.080 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.080 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.080 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.080 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.080 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.080 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.080 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.080 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.080 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.080 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:55.080 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "smb", 2024-05-08T06:13:55.080 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:55.081 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:55.081 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:55.081 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:55.081 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:55.081 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.081 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.081 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.081 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.081 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.081 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.081 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:55.081 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:55.081 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:55.081 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:55.081 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:55.082 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:55.082 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:55.082 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.082 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.082 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.082 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.082 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.082 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.082 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:55.082 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:55.082 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.082 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.082 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.082 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.082 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.083 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.083 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.083 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.083 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.083 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.083 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.083 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.083 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:55.083 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:55.083 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.083 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.083 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.083 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:55.083 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.083 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.083 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:55.084 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:55.084 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:55.084 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:55.084 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:55.084 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:55.084 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:55.084 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.084 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.084 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.084 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.084 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.084 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.084 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:55.084 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:55.084 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.085 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.085 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.085 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.085 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.085 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.085 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.085 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.085 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.085 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.085 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.085 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.085 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:55.085 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:55.085 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:55.085 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:55.085 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.086 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:55.086 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.086 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.086 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.086 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.086 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.086 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.086 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.086 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.086 INFO:teuthology.orchestra.run.smithi123.stdout: "update_orchestration": { 2024-05-08T06:13:55.086 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "update_orchestration", 2024-05-08T06:13:55.086 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.086 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.086 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.086 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:55.087 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.087 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.087 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.087 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-08T06:13:55.087 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.087 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.087 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.087 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.087 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.087 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.087 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:55.087 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "snap_schedule", 2024-05-08T06:13:55.087 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:55.087 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:55.087 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:55.088 INFO:teuthology.orchestra.run.smithi123.stdout: "allow_m_granularity": { 2024-05-08T06:13:55.088 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "allow_m_granularity", 2024-05-08T06:13:55.088 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.088 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.088 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.088 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.088 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.088 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.088 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.088 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "allow minute scheduled snapshots", 2024-05-08T06:13:55.088 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.088 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.088 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.088 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.088 INFO:teuthology.orchestra.run.smithi123.stdout: "dump_on_update": { 2024-05-08T06:13:55.088 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "dump_on_update", 2024-05-08T06:13:55.089 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.089 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.089 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.089 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.089 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.089 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.089 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.089 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "dump database to debug log on update", 2024-05-08T06:13:55.089 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.089 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.089 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.089 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.089 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:55.089 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:55.089 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.090 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.090 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.090 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.090 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.090 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.090 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:55.090 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:55.090 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:55.090 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:55.090 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:55.090 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:55.090 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:55.090 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.090 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.090 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.091 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.091 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.091 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.091 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:55.091 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:55.091 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.091 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.091 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.091 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.091 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.091 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.091 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.091 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.091 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.091 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.091 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.092 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.092 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:55.092 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:55.092 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.092 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.092 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.092 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:55.092 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.092 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.092 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:55.092 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:55.092 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:55.092 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:55.092 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:55.092 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:55.092 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:55.093 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.093 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.093 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.093 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.093 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.093 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.093 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:55.093 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:55.093 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.093 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.093 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.093 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.093 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.093 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.093 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.093 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.093 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.094 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.094 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.094 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.094 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:55.094 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:55.094 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:55.094 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:55.094 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.094 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:55.094 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.094 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.094 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.094 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.094 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.094 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.094 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.095 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.095 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.095 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.095 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:55.095 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "stats", 2024-05-08T06:13:55.095 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:55.095 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:55.095 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:55.095 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:55.095 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:55.095 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.095 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.095 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.095 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.095 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.095 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.096 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:55.096 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:55.096 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:55.096 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:55.096 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:55.096 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:55.096 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:55.096 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.096 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.096 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.096 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.096 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.096 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.096 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:55.097 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:55.097 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.097 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.097 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.097 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.097 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.097 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.097 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.097 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.097 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.097 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.097 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.097 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.097 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:55.097 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:55.097 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.098 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.098 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.098 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:55.098 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.098 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.098 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:55.098 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:55.098 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:55.098 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:55.098 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:55.098 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:55.098 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:55.098 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.098 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.098 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.099 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.099 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.099 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.099 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:55.099 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:55.099 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.099 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.099 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.099 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.099 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.099 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.099 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.099 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.099 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.099 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.099 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.100 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.100 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:55.100 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:55.100 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:55.100 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:55.100 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.100 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:55.100 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.100 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.100 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.100 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.100 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.100 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.100 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.100 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.100 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.101 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.101 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:55.101 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "status", 2024-05-08T06:13:55.101 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:55.101 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:55.101 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:55.101 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:55.101 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:55.101 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.101 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.101 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.101 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.101 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.101 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.101 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:55.101 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:55.101 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:55.102 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:55.102 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:55.102 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:55.102 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:55.102 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.102 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.102 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.102 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.102 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.102 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.102 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:55.102 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:55.102 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.102 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.103 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.103 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.103 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.103 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.103 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.103 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.103 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.103 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.103 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.103 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.103 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:55.103 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:55.103 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.103 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.103 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.104 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:55.104 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.104 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.104 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:55.104 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:55.104 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:55.104 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:55.104 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:55.104 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:55.104 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:55.104 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.104 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.104 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.104 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.104 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.104 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.104 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:55.105 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:55.105 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.105 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.105 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.105 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.105 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.105 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.105 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.105 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.105 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.105 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.105 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.105 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.105 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:55.105 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:55.105 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:55.106 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:55.106 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.106 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:55.106 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.106 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.106 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.106 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.106 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.106 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.106 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.106 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.106 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.106 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.106 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:55.106 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "telegraf", 2024-05-08T06:13:55.107 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:55.107 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:55.107 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:55.107 INFO:teuthology.orchestra.run.smithi123.stdout: "address": { 2024-05-08T06:13:55.107 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "address", 2024-05-08T06:13:55.107 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.107 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.107 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.107 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-08T06:13:55.107 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.107 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.107 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.107 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.107 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.107 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.107 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.107 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.108 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-05-08T06:13:55.108 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-05-08T06:13:55.108 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:55.108 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.108 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.108 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "15", 2024-05-08T06:13:55.108 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.108 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.108 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.108 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.108 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.108 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.108 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.108 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.108 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:55.108 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:55.109 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.109 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.109 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.109 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.109 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.109 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.109 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:55.109 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:55.109 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:55.109 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:55.109 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:55.109 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:55.109 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:55.109 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.109 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.109 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.110 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.110 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.110 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.110 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:55.110 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:55.110 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.110 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.110 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.110 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.110 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.110 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.110 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.110 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.110 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.110 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.111 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.111 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.111 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:55.111 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:55.111 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.111 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.111 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.111 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:55.111 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.111 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.111 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:55.111 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:55.111 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:55.111 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:55.111 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:55.112 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:55.112 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:55.112 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.112 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.112 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.112 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.112 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.112 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.112 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:55.112 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:55.112 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.112 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.112 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.112 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.112 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.113 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.113 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.113 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.113 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.113 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.113 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.113 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.113 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:55.113 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:55.113 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:55.113 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:55.113 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.113 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:55.113 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.113 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.113 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.114 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.114 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.114 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.114 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.114 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.114 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.114 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.114 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:55.114 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "telemetry", 2024-05-08T06:13:55.114 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:55.114 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:55.114 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:55.114 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_basic": { 2024-05-08T06:13:55.114 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_basic", 2024-05-08T06:13:55.114 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.115 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.115 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.115 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:55.115 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.115 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.115 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.115 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-08T06:13:55.115 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.115 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.115 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.115 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.115 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_crash": { 2024-05-08T06:13:55.115 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_crash", 2024-05-08T06:13:55.115 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.115 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.115 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.116 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:55.116 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.116 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.116 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.116 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-08T06:13:55.116 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.116 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.116 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.116 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.116 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_device": { 2024-05-08T06:13:55.116 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_device", 2024-05-08T06:13:55.116 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.116 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.116 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.117 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:55.117 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.117 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.117 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.117 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-08T06:13:55.117 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.117 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.117 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.117 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.117 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_ident": { 2024-05-08T06:13:55.117 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_ident", 2024-05-08T06:13:55.117 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.117 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.117 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.117 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.118 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.118 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.118 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.118 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-08T06:13:55.118 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.118 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.118 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.118 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.118 INFO:teuthology.orchestra.run.smithi123.stdout: "channel_perf": { 2024-05-08T06:13:55.118 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "channel_perf", 2024-05-08T06:13:55.118 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.118 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.118 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.118 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.119 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.119 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.119 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.119 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-08T06:13:55.119 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.119 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.119 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.119 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.119 INFO:teuthology.orchestra.run.smithi123.stdout: "contact": { 2024-05-08T06:13:55.119 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "contact", 2024-05-08T06:13:55.119 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.119 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.119 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.119 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.119 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.119 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.119 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.120 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.120 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.120 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.120 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.120 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.120 INFO:teuthology.orchestra.run.smithi123.stdout: "description": { 2024-05-08T06:13:55.120 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "description", 2024-05-08T06:13:55.120 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.120 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.120 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.120 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.120 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.120 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.120 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.120 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.120 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.121 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.121 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.121 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.121 INFO:teuthology.orchestra.run.smithi123.stdout: "device_url": { 2024-05-08T06:13:55.121 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "device_url", 2024-05-08T06:13:55.121 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.121 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.121 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.121 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-08T06:13:55.121 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.121 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.121 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.121 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.121 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.121 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.121 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.122 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.122 INFO:teuthology.orchestra.run.smithi123.stdout: "enabled": { 2024-05-08T06:13:55.122 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "enabled", 2024-05-08T06:13:55.122 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.122 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.122 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.122 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.122 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.122 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.122 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.122 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.122 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.122 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.122 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.122 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.122 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-05-08T06:13:55.123 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-05-08T06:13:55.123 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:55.123 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.123 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.123 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "24", 2024-05-08T06:13:55.123 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "8", 2024-05-08T06:13:55.123 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.123 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.123 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.123 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.123 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.123 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.123 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.123 INFO:teuthology.orchestra.run.smithi123.stdout: "last_opt_revision": { 2024-05-08T06:13:55.123 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "last_opt_revision", 2024-05-08T06:13:55.124 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:55.124 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.124 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.124 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "1", 2024-05-08T06:13:55.124 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.124 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.124 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.124 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.124 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.124 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.124 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.124 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.124 INFO:teuthology.orchestra.run.smithi123.stdout: "leaderboard": { 2024-05-08T06:13:55.124 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "leaderboard", 2024-05-08T06:13:55.124 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.124 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.124 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.125 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.125 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.125 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.125 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.125 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.125 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.125 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.125 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.125 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.125 INFO:teuthology.orchestra.run.smithi123.stdout: "leaderboard_description": { 2024-05-08T06:13:55.125 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "leaderboard_description", 2024-05-08T06:13:55.125 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.125 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.125 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.125 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.125 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.125 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.126 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.126 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.126 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.126 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.126 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.126 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.126 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:55.126 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:55.126 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.126 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.126 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.126 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.126 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.126 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.126 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:55.127 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:55.127 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:55.127 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:55.127 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:55.127 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:55.127 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:55.127 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.127 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.127 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.127 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.127 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.127 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.127 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:55.127 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:55.127 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.127 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.128 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.128 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.128 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.128 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.128 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.128 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.128 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.128 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.128 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.128 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.128 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:55.128 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:55.128 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.128 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.128 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.129 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:55.129 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.129 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.129 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:55.129 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:55.129 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:55.129 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:55.129 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:55.129 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:55.129 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:55.129 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.129 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.129 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.129 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.129 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.129 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.130 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:55.130 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:55.130 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.130 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.130 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.130 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.130 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.130 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.130 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.130 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.130 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.130 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.130 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.130 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.130 INFO:teuthology.orchestra.run.smithi123.stdout: "organization": { 2024-05-08T06:13:55.131 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "organization", 2024-05-08T06:13:55.131 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.131 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.131 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.131 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.131 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.131 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.131 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.131 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.131 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.131 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.131 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.131 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.131 INFO:teuthology.orchestra.run.smithi123.stdout: "proxy": { 2024-05-08T06:13:55.131 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "proxy", 2024-05-08T06:13:55.131 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.132 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.132 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.132 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.132 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.132 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.132 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.132 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.132 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.132 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.132 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.132 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.132 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:55.132 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:55.132 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:55.132 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:55.132 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.133 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:55.133 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.133 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.133 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.133 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.133 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.133 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.133 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.133 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.133 INFO:teuthology.orchestra.run.smithi123.stdout: "url": { 2024-05-08T06:13:55.133 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "url", 2024-05-08T06:13:55.133 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.133 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.133 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.133 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-08T06:13:55.133 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.134 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.134 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.134 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.134 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.134 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.134 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.134 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.134 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.134 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.134 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:55.134 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "test_orchestrator", 2024-05-08T06:13:55.134 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:55.134 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:55.134 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:55.134 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:55.135 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:55.135 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.135 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.135 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.135 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.135 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.135 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.135 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:55.135 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:55.135 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:55.135 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:55.135 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:55.135 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:55.135 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:55.135 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.135 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.135 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.136 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.136 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.136 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.136 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:55.136 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:55.136 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.136 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.136 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.136 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.136 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.136 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.136 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.136 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.136 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.136 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.136 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.137 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.137 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:55.137 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:55.137 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.137 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.137 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.137 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:55.137 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.137 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.137 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:55.137 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:55.137 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:55.137 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:55.137 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:55.137 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:55.138 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:55.138 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.138 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.138 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.138 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.138 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.138 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.138 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:55.138 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:55.138 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.138 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.138 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.138 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.138 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.138 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.138 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.139 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.139 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.139 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.139 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.139 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.139 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:55.139 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:55.139 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:55.139 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:55.139 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.139 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:55.139 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.139 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.139 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.139 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.139 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.140 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.140 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.140 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.140 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.140 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.140 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:55.140 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "volumes", 2024-05-08T06:13:55.140 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:55.140 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:55.140 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:55.140 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:55.140 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:55.140 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.140 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.140 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.140 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.141 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.141 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.141 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:55.141 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:55.141 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:55.141 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:55.141 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:55.141 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:55.141 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:55.141 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.141 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.141 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.141 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.141 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.141 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.141 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:55.141 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:55.142 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.142 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.142 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.142 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.142 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.142 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.142 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.142 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.142 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.142 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.142 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.142 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.142 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:55.142 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:55.142 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.142 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.143 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.143 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:55.143 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.143 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.143 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:55.143 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:55.143 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:55.143 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:55.143 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:55.143 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:55.143 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:55.143 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.143 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.143 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.143 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.143 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.144 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.144 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:55.144 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:55.144 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.144 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.144 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.144 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.144 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.144 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.144 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.144 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.144 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.144 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.144 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.144 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.144 INFO:teuthology.orchestra.run.smithi123.stdout: "max_concurrent_clones": { 2024-05-08T06:13:55.145 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "max_concurrent_clones", 2024-05-08T06:13:55.145 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:55.145 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.145 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.145 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "4", 2024-05-08T06:13:55.145 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.145 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.145 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.145 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-08T06:13:55.145 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.145 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.145 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.145 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.145 INFO:teuthology.orchestra.run.smithi123.stdout: "periodic_async_work": { 2024-05-08T06:13:55.145 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "periodic_async_work", 2024-05-08T06:13:55.145 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.145 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.146 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.146 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.146 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.146 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.146 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.146 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Periodically check for async work", 2024-05-08T06:13:55.146 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.146 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.146 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.146 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.146 INFO:teuthology.orchestra.run.smithi123.stdout: "snapshot_clone_delay": { 2024-05-08T06:13:55.146 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "snapshot_clone_delay", 2024-05-08T06:13:55.146 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:55.146 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.146 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.146 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:55.147 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.147 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.147 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.147 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-08T06:13:55.147 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.147 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.147 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.147 INFO:teuthology.orchestra.run.smithi123.stdout: "snapshot_clone_no_wait": { 2024-05-08T06:13:55.147 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "snapshot_clone_no_wait", 2024-05-08T06:13:55.147 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.147 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.147 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.147 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "True", 2024-05-08T06:13:55.147 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.148 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.148 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.148 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-08T06:13:55.148 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.148 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.148 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.148 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:55.148 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:55.148 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:55.148 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:55.148 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.148 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:55.148 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.148 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.148 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.148 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.149 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.149 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.149 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.149 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.149 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.149 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.149 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:55.149 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "zabbix", 2024-05-08T06:13:55.149 INFO:teuthology.orchestra.run.smithi123.stdout: "can_run": true, 2024-05-08T06:13:55.149 INFO:teuthology.orchestra.run.smithi123.stdout: "error_string": "", 2024-05-08T06:13:55.149 INFO:teuthology.orchestra.run.smithi123.stdout: "module_options": { 2024-05-08T06:13:55.149 INFO:teuthology.orchestra.run.smithi123.stdout: "discovery_interval": { 2024-05-08T06:13:55.149 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "discovery_interval", 2024-05-08T06:13:55.149 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "uint", 2024-05-08T06:13:55.149 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.150 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.150 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "100", 2024-05-08T06:13:55.150 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.150 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.150 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.150 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.150 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.150 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.150 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.150 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.150 INFO:teuthology.orchestra.run.smithi123.stdout: "identifier": { 2024-05-08T06:13:55.150 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "identifier", 2024-05-08T06:13:55.150 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.150 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.150 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.150 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.151 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.151 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.151 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.151 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.151 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.151 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.151 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.151 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.151 INFO:teuthology.orchestra.run.smithi123.stdout: "interval": { 2024-05-08T06:13:55.151 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "interval", 2024-05-08T06:13:55.151 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "secs", 2024-05-08T06:13:55.151 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.151 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.151 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "60", 2024-05-08T06:13:55.152 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.152 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.152 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.152 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.152 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.152 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.152 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.152 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.152 INFO:teuthology.orchestra.run.smithi123.stdout: "log_level": { 2024-05-08T06:13:55.152 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_level", 2024-05-08T06:13:55.152 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.152 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.152 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.152 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.152 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.152 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.153 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:55.153 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:55.153 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:55.153 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:55.153 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:55.153 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:55.153 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:55.153 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.153 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.153 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.153 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.153 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.153 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.153 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster": { 2024-05-08T06:13:55.154 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster", 2024-05-08T06:13:55.154 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.154 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.154 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.154 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.154 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.154 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.154 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.154 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.154 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.154 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.154 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.154 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.154 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_cluster_level": { 2024-05-08T06:13:55.154 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_cluster_level", 2024-05-08T06:13:55.155 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.155 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.155 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.155 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "info", 2024-05-08T06:13:55.155 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.155 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.155 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [ 2024-05-08T06:13:55.155 INFO:teuthology.orchestra.run.smithi123.stdout: "", 2024-05-08T06:13:55.155 INFO:teuthology.orchestra.run.smithi123.stdout: "critical", 2024-05-08T06:13:55.155 INFO:teuthology.orchestra.run.smithi123.stdout: "debug", 2024-05-08T06:13:55.155 INFO:teuthology.orchestra.run.smithi123.stdout: "error", 2024-05-08T06:13:55.155 INFO:teuthology.orchestra.run.smithi123.stdout: "info", 2024-05-08T06:13:55.155 INFO:teuthology.orchestra.run.smithi123.stdout: "warning" 2024-05-08T06:13:55.155 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.156 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.156 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.156 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.156 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.156 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.156 INFO:teuthology.orchestra.run.smithi123.stdout: "log_to_file": { 2024-05-08T06:13:55.156 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "log_to_file", 2024-05-08T06:13:55.156 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "bool", 2024-05-08T06:13:55.156 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.156 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.156 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "False", 2024-05-08T06:13:55.156 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.156 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.156 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.156 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.156 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.157 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.157 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.157 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.157 INFO:teuthology.orchestra.run.smithi123.stdout: "sqlite3_killpoint": { 2024-05-08T06:13:55.157 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:13:55.157 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:55.157 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "dev", 2024-05-08T06:13:55.157 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 1, 2024-05-08T06:13:55.157 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "0", 2024-05-08T06:13:55.157 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.157 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.157 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.157 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.157 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.158 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.158 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.158 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.158 INFO:teuthology.orchestra.run.smithi123.stdout: "zabbix_host": { 2024-05-08T06:13:55.158 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "zabbix_host", 2024-05-08T06:13:55.158 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.158 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.158 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.158 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "", 2024-05-08T06:13:55.158 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.158 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.158 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.158 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.158 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.158 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.158 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.159 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.159 INFO:teuthology.orchestra.run.smithi123.stdout: "zabbix_port": { 2024-05-08T06:13:55.159 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "zabbix_port", 2024-05-08T06:13:55.159 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "int", 2024-05-08T06:13:55.159 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.159 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.159 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "10051", 2024-05-08T06:13:55.159 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.159 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.159 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.159 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.159 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.159 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.159 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.160 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.160 INFO:teuthology.orchestra.run.smithi123.stdout: "zabbix_sender": { 2024-05-08T06:13:55.160 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "zabbix_sender", 2024-05-08T06:13:55.160 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "str", 2024-05-08T06:13:55.160 INFO:teuthology.orchestra.run.smithi123.stdout: "level": "advanced", 2024-05-08T06:13:55.160 INFO:teuthology.orchestra.run.smithi123.stdout: "flags": 0, 2024-05-08T06:13:55.160 INFO:teuthology.orchestra.run.smithi123.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-08T06:13:55.160 INFO:teuthology.orchestra.run.smithi123.stdout: "min": "", 2024-05-08T06:13:55.160 INFO:teuthology.orchestra.run.smithi123.stdout: "max": "", 2024-05-08T06:13:55.160 INFO:teuthology.orchestra.run.smithi123.stdout: "enum_allowed": [], 2024-05-08T06:13:55.160 INFO:teuthology.orchestra.run.smithi123.stdout: "desc": "", 2024-05-08T06:13:55.160 INFO:teuthology.orchestra.run.smithi123.stdout: "long_desc": "", 2024-05-08T06:13:55.160 INFO:teuthology.orchestra.run.smithi123.stdout: "tags": [], 2024-05-08T06:13:55.160 INFO:teuthology.orchestra.run.smithi123.stdout: "see_also": [] 2024-05-08T06:13:55.161 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.161 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.161 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.161 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.161 INFO:teuthology.orchestra.run.smithi123.stdout: "services": { 2024-05-08T06:13:55.161 INFO:teuthology.orchestra.run.smithi123.stdout: "dashboard": "https://172.21.15.123:8443/" 2024-05-08T06:13:55.161 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.161 INFO:teuthology.orchestra.run.smithi123.stdout: "always_on_modules": { 2024-05-08T06:13:55.161 INFO:teuthology.orchestra.run.smithi123.stdout: "octopus": [ 2024-05-08T06:13:55.161 INFO:teuthology.orchestra.run.smithi123.stdout: "balancer", 2024-05-08T06:13:55.161 INFO:teuthology.orchestra.run.smithi123.stdout: "crash", 2024-05-08T06:13:55.161 INFO:teuthology.orchestra.run.smithi123.stdout: "devicehealth", 2024-05-08T06:13:55.161 INFO:teuthology.orchestra.run.smithi123.stdout: "orchestrator", 2024-05-08T06:13:55.161 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_autoscaler", 2024-05-08T06:13:55.161 INFO:teuthology.orchestra.run.smithi123.stdout: "progress", 2024-05-08T06:13:55.162 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_support", 2024-05-08T06:13:55.162 INFO:teuthology.orchestra.run.smithi123.stdout: "status", 2024-05-08T06:13:55.162 INFO:teuthology.orchestra.run.smithi123.stdout: "telemetry", 2024-05-08T06:13:55.162 INFO:teuthology.orchestra.run.smithi123.stdout: "volumes" 2024-05-08T06:13:55.162 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.162 INFO:teuthology.orchestra.run.smithi123.stdout: "pacific": [ 2024-05-08T06:13:55.162 INFO:teuthology.orchestra.run.smithi123.stdout: "balancer", 2024-05-08T06:13:55.162 INFO:teuthology.orchestra.run.smithi123.stdout: "crash", 2024-05-08T06:13:55.162 INFO:teuthology.orchestra.run.smithi123.stdout: "devicehealth", 2024-05-08T06:13:55.162 INFO:teuthology.orchestra.run.smithi123.stdout: "orchestrator", 2024-05-08T06:13:55.162 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_autoscaler", 2024-05-08T06:13:55.162 INFO:teuthology.orchestra.run.smithi123.stdout: "progress", 2024-05-08T06:13:55.162 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_support", 2024-05-08T06:13:55.162 INFO:teuthology.orchestra.run.smithi123.stdout: "status", 2024-05-08T06:13:55.162 INFO:teuthology.orchestra.run.smithi123.stdout: "telemetry", 2024-05-08T06:13:55.162 INFO:teuthology.orchestra.run.smithi123.stdout: "volumes" 2024-05-08T06:13:55.163 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.163 INFO:teuthology.orchestra.run.smithi123.stdout: "quincy": [ 2024-05-08T06:13:55.163 INFO:teuthology.orchestra.run.smithi123.stdout: "balancer", 2024-05-08T06:13:55.163 INFO:teuthology.orchestra.run.smithi123.stdout: "crash", 2024-05-08T06:13:55.163 INFO:teuthology.orchestra.run.smithi123.stdout: "devicehealth", 2024-05-08T06:13:55.163 INFO:teuthology.orchestra.run.smithi123.stdout: "orchestrator", 2024-05-08T06:13:55.163 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_autoscaler", 2024-05-08T06:13:55.163 INFO:teuthology.orchestra.run.smithi123.stdout: "progress", 2024-05-08T06:13:55.163 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_support", 2024-05-08T06:13:55.163 INFO:teuthology.orchestra.run.smithi123.stdout: "status", 2024-05-08T06:13:55.163 INFO:teuthology.orchestra.run.smithi123.stdout: "telemetry", 2024-05-08T06:13:55.163 INFO:teuthology.orchestra.run.smithi123.stdout: "volumes" 2024-05-08T06:13:55.163 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.163 INFO:teuthology.orchestra.run.smithi123.stdout: "reef": [ 2024-05-08T06:13:55.164 INFO:teuthology.orchestra.run.smithi123.stdout: "balancer", 2024-05-08T06:13:55.164 INFO:teuthology.orchestra.run.smithi123.stdout: "crash", 2024-05-08T06:13:55.164 INFO:teuthology.orchestra.run.smithi123.stdout: "devicehealth", 2024-05-08T06:13:55.164 INFO:teuthology.orchestra.run.smithi123.stdout: "orchestrator", 2024-05-08T06:13:55.164 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_autoscaler", 2024-05-08T06:13:55.164 INFO:teuthology.orchestra.run.smithi123.stdout: "progress", 2024-05-08T06:13:55.164 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_support", 2024-05-08T06:13:55.164 INFO:teuthology.orchestra.run.smithi123.stdout: "status", 2024-05-08T06:13:55.164 INFO:teuthology.orchestra.run.smithi123.stdout: "telemetry", 2024-05-08T06:13:55.164 INFO:teuthology.orchestra.run.smithi123.stdout: "volumes" 2024-05-08T06:13:55.164 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-05-08T06:13:55.164 INFO:teuthology.orchestra.run.smithi123.stdout: "squid": [ 2024-05-08T06:13:55.164 INFO:teuthology.orchestra.run.smithi123.stdout: "balancer", 2024-05-08T06:13:55.164 INFO:teuthology.orchestra.run.smithi123.stdout: "crash", 2024-05-08T06:13:55.164 INFO:teuthology.orchestra.run.smithi123.stdout: "devicehealth", 2024-05-08T06:13:55.165 INFO:teuthology.orchestra.run.smithi123.stdout: "orchestrator", 2024-05-08T06:13:55.165 INFO:teuthology.orchestra.run.smithi123.stdout: "pg_autoscaler", 2024-05-08T06:13:55.165 INFO:teuthology.orchestra.run.smithi123.stdout: "progress", 2024-05-08T06:13:55.165 INFO:teuthology.orchestra.run.smithi123.stdout: "rbd_support", 2024-05-08T06:13:55.165 INFO:teuthology.orchestra.run.smithi123.stdout: "status", 2024-05-08T06:13:55.165 INFO:teuthology.orchestra.run.smithi123.stdout: "telemetry", 2024-05-08T06:13:55.165 INFO:teuthology.orchestra.run.smithi123.stdout: "volumes" 2024-05-08T06:13:55.165 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-05-08T06:13:55.165 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.165 INFO:teuthology.orchestra.run.smithi123.stdout: "last_failure_osd_epoch": 3, 2024-05-08T06:13:55.165 INFO:teuthology.orchestra.run.smithi123.stdout: "active_clients": [ 2024-05-08T06:13:55.165 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:55.165 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "devicehealth", 2024-05-08T06:13:55.165 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-05-08T06:13:55.165 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:55.166 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-05-08T06:13:55.166 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:0", 2024-05-08T06:13:55.166 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1310876594 2024-05-08T06:13:55.166 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.166 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-05-08T06:13:55.166 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.166 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:55.166 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "libcephsqlite", 2024-05-08T06:13:55.166 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-05-08T06:13:55.166 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:55.166 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-05-08T06:13:55.166 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:0", 2024-05-08T06:13:55.166 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3375959212 2024-05-08T06:13:55.166 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.166 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-05-08T06:13:55.167 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.167 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:55.167 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "rbd_support", 2024-05-08T06:13:55.167 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-05-08T06:13:55.167 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:55.167 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-05-08T06:13:55.167 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:0", 2024-05-08T06:13:55.167 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2973550986 2024-05-08T06:13:55.167 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.167 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-05-08T06:13:55.167 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:55.167 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:55.167 INFO:teuthology.orchestra.run.smithi123.stdout: "name": "volumes", 2024-05-08T06:13:55.167 INFO:teuthology.orchestra.run.smithi123.stdout: "addrvec": [ 2024-05-08T06:13:55.167 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-05-08T06:13:55.167 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v2", 2024-05-08T06:13:55.168 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "172.21.15.123:0", 2024-05-08T06:13:55.168 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2128370455 2024-05-08T06:13:55.168 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.168 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-05-08T06:13:55.168 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:55.168 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-05-08T06:13:55.168 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-05-08T06:13:55.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.850+0000 7fb7c5400640 1 -- 172.21.15.123:0/1172555814 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fb7a40b27f0 msgr2=0x7fb7a40b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:55.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.850+0000 7fb7c5400640 1 --2- 172.21.15.123:0/1172555814 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fb7a40b27f0 0x7fb7a40b4cb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8000c00 tx=0x7fb7a80314a0 comp rx=0 tx=0).stop 2024-05-08T06:13:55.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.850+0000 7fb7c5400640 1 -- 172.21.15.123:0/1172555814 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7c0109010 msgr2=0x7fb7c019f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:13:55.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.850+0000 7fb7c5400640 1 --2- 172.21.15.123:0/1172555814 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7c0109010 0x7fb7c019f250 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000e9c0 tx=0x7fb7b000edc0 comp rx=0 tx=0).stop 2024-05-08T06:13:55.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.850+0000 7fb7c5400640 1 -- 172.21.15.123:0/1172555814 shutdown_connections 2024-05-08T06:13:55.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.850+0000 7fb7c5400640 1 --2- 172.21.15.123:0/1172555814 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fb7a40b27f0 0x7fb7a40b4cb0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:55.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.851+0000 7fb7c5400640 1 --2- 172.21.15.123:0/1172555814 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7c010abc0 0x7fb7c01a6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:55.169 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.851+0000 7fb7c5400640 1 --2- 172.21.15.123:0/1172555814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7c010a200 0x7fb7c019f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:55.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.851+0000 7fb7c5400640 1 --2- 172.21.15.123:0/1172555814 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7c0109010 0x7fb7c019f250 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:13:55.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.851+0000 7fb7c5400640 1 -- 172.21.15.123:0/1172555814 >> 172.21.15.123:0/1172555814 conn(0x7fb7c0077fd0 msgr2=0x7fb7c010d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:13:55.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.851+0000 7fb7c5400640 1 -- 172.21.15.123:0/1172555814 shutdown_connections 2024-05-08T06:13:55.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:13:53.851+0000 7fb7c5400640 1 -- 172.21.15.123:0/1172555814 wait complete. 2024-05-08T06:13:55.175 DEBUG:tasks.check_counter:Getting stats from a 2024-05-08T06:13:55.176 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-08T06:13:55.176 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:53 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/3394650494' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T06:13:55.176 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:53 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/2782919950' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T06:13:55.176 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:53 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/1172555814' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-08T06:13:55.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:54 smithi123 ceph-mon[29787]: pgmap v563: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:13:55.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:54 smithi163 ceph-mon[34520]: pgmap v563: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:13:55.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:54 smithi192 ceph-mon[34664]: pgmap v563: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:13:55.617 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:13:57.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:56 smithi123 ceph-mon[29787]: pgmap v564: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T06:13:57.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:56 smithi163 ceph-mon[34520]: pgmap v564: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T06:13:57.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:56 smithi192 ceph-mon[34664]: pgmap v564: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T06:13:58.423 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-05-08T06:13:58.423 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T06:13:58.423 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 612, 2024-05-08T06:13:58.423 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 572, 2024-05-08T06:13:58.423 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 952060, 2024-05-08T06:13:58.423 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 236930, 2024-05-08T06:13:58.423 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 9, 2024-05-08T06:13:58.423 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 3, 2024-05-08T06:13:58.423 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.150365689, 2024-05-08T06:13:58.423 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.055430541, 2024-05-08T06:13:58.423 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.099964785, 2024-05-08T06:13:58.423 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.010811838, 2024-05-08T06:13:58.424 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:13:58.424 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 571, 2024-05-08T06:13:58.424 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.063716673, 2024-05-08T06:13:58.424 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000111587 2024-05-08T06:13:58.424 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.424 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:13:58.424 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 781, 2024-05-08T06:13:58.424 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000274500, 2024-05-08T06:13:58.424 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000351 2024-05-08T06:13:58.424 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.424 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 827280, 2024-05-08T06:13:58.424 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 90656 2024-05-08T06:13:58.424 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.424 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T06:13:58.424 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 20, 2024-05-08T06:13:58.424 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 125, 2024-05-08T06:13:58.425 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 5304, 2024-05-08T06:13:58.425 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 311498, 2024-05-08T06:13:58.425 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 12, 2024-05-08T06:13:58.425 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 4, 2024-05-08T06:13:58.425 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.034085173, 2024-05-08T06:13:58.425 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.020074891, 2024-05-08T06:13:58.425 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.010801879, 2024-05-08T06:13:58.425 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000363814, 2024-05-08T06:13:58.425 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:13:58.425 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 123, 2024-05-08T06:13:58.425 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.024283962, 2024-05-08T06:13:58.425 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000197430 2024-05-08T06:13:58.425 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.425 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:13:58.425 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 13, 2024-05-08T06:13:58.425 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000003656, 2024-05-08T06:13:58.426 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000281 2024-05-08T06:13:58.426 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.426 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 1936, 2024-05-08T06:13:58.426 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 258144 2024-05-08T06:13:58.426 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.426 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T06:13:58.426 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 38, 2024-05-08T06:13:58.426 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 38, 2024-05-08T06:13:58.426 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 11365, 2024-05-08T06:13:58.426 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 113123, 2024-05-08T06:13:58.426 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 15, 2024-05-08T06:13:58.426 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 7, 2024-05-08T06:13:58.426 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.014534336, 2024-05-08T06:13:58.427 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.002120498, 2024-05-08T06:13:58.427 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.011030177, 2024-05-08T06:13:58.427 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000555630, 2024-05-08T06:13:58.427 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:13:58.427 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 37, 2024-05-08T06:13:58.427 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.040445488, 2024-05-08T06:13:58.427 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.001093121 2024-05-08T06:13:58.427 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.427 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:13:58.427 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 23, 2024-05-08T06:13:58.427 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000003309, 2024-05-08T06:13:58.427 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000143 2024-05-08T06:13:58.427 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.427 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 5216, 2024-05-08T06:13:58.427 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 3776 2024-05-08T06:13:58.427 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.428 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-05-08T06:13:58.428 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 1, 2024-05-08T06:13:58.428 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-05-08T06:13:58.428 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.428 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-MDSRank": { 2024-05-08T06:13:58.428 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-05-08T06:13:58.428 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-05-08T06:13:58.428 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 37, 2024-05-08T06:13:58.428 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.002749037, 2024-05-08T06:13:58.428 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000074298 2024-05-08T06:13:58.428 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:58.428 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.428 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-PurgeQueue": { 2024-05-08T06:13:58.428 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-05-08T06:13:58.428 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-05-08T06:13:58.428 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2, 2024-05-08T06:13:58.429 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000007575, 2024-05-08T06:13:58.429 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000003787 2024-05-08T06:13:58.429 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:58.429 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.429 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": { 2024-05-08T06:13:58.429 INFO:teuthology.orchestra.run.smithi123.stdout: "request": 6, 2024-05-08T06:13:58.429 INFO:teuthology.orchestra.run.smithi123.stdout: "reply": 4, 2024-05-08T06:13:58.429 INFO:teuthology.orchestra.run.smithi123.stdout: "reply_latency": { 2024-05-08T06:13:58.429 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 4, 2024-05-08T06:13:58.429 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.002938778, 2024-05-08T06:13:58.429 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000734694 2024-05-08T06:13:58.429 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.429 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_reply": 0, 2024-05-08T06:13:58.429 INFO:teuthology.orchestra.run.smithi123.stdout: "forward": 2, 2024-05-08T06:13:58.429 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_complete": 0, 2024-05-08T06:13:58.429 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_keys": 0, 2024-05-08T06:13:58.430 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_commit": 11, 2024-05-08T06:13:58.430 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_split": 0, 2024-05-08T06:13:58.430 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_merge": 0, 2024-05-08T06:13:58.430 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes": 17, 2024-05-08T06:13:58.430 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_top": 1, 2024-05-08T06:13:58.430 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_bottom": 16, 2024-05-08T06:13:58.430 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pin_tail": 0, 2024-05-08T06:13:58.430 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pinned": 13, 2024-05-08T06:13:58.430 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_expired": 8, 2024-05-08T06:13:58.430 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_with_caps": 1, 2024-05-08T06:13:58.430 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": 1, 2024-05-08T06:13:58.430 INFO:teuthology.orchestra.run.smithi123.stdout: "subtrees": 3, 2024-05-08T06:13:58.430 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse": 16, 2024-05-08T06:13:58.430 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_hit": 12, 2024-05-08T06:13:58.430 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_forward": 1, 2024-05-08T06:13:58.430 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_discover": 3, 2024-05-08T06:13:58.431 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_dir_fetch": 0, 2024-05-08T06:13:58.431 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_remote_ino": 0, 2024-05-08T06:13:58.431 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_lock": 0, 2024-05-08T06:13:58.431 INFO:teuthology.orchestra.run.smithi123.stdout: "load_cent": 0, 2024-05-08T06:13:58.431 INFO:teuthology.orchestra.run.smithi123.stdout: "q": 0, 2024-05-08T06:13:58.431 INFO:teuthology.orchestra.run.smithi123.stdout: "exported": 0, 2024-05-08T06:13:58.431 INFO:teuthology.orchestra.run.smithi123.stdout: "exported_inodes": 0, 2024-05-08T06:13:58.431 INFO:teuthology.orchestra.run.smithi123.stdout: "imported": 0, 2024-05-08T06:13:58.431 INFO:teuthology.orchestra.run.smithi123.stdout: "imported_inodes": 0, 2024-05-08T06:13:58.431 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_dir_fetch": 0, 2024-05-08T06:13:58.431 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_backtrace_fetch": 0, 2024-05-08T06:13:58.431 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_peer_discover": 0, 2024-05-08T06:13:58.431 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rfiles": 4, 2024-05-08T06:13:58.431 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rbytes": 234, 2024-05-08T06:13:58.431 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rsnaps": 0, 2024-05-08T06:13:58.431 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T06:13:58.431 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_set_tag": 0, 2024-05-08T06:13:58.432 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T06:13:58.432 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_inotable_repaired": 0, 2024-05-08T06:13:58.432 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_inodes": 0, 2024-05-08T06:13:58.432 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T06:13:58.432 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T06:13:58.432 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_file_inodes": 0, 2024-05-08T06:13:58.432 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_inode_file_caps": 0, 2024-05-08T06:13:58.432 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T06:13:58.432 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_grant": 0, 2024-05-08T06:13:58.432 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T06:13:58.432 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T06:13:58.432 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T06:13:58.432 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps": 0, 2024-05-08T06:13:58.432 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps_dirty": 0, 2024-05-08T06:13:58.432 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_cap_release": 0, 2024-05-08T06:13:58.432 INFO:teuthology.orchestra.run.smithi123.stdout: "process_request_cap_release": 0 2024-05-08T06:13:58.433 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.433 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache": { 2024-05-08T06:13:58.433 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update": 0, 2024-05-08T06:13:58.433 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update_receipt": 15, 2024-05-08T06:13:58.433 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_try_discover": 9, 2024-05-08T06:13:58.433 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_send_discover": 10, 2024-05-08T06:13:58.433 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_handle_discover": 0, 2024-05-08T06:13:58.433 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays": 0, 2024-05-08T06:13:58.433 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_delayed": 0, 2024-05-08T06:13:58.433 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_enqueuing": 0, 2024-05-08T06:13:58.433 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_created": 0, 2024-05-08T06:13:58.433 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_enqueued": 0, 2024-05-08T06:13:58.433 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_reintegrated": 0, 2024-05-08T06:13:58.433 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_migrated": 0, 2024-05-08T06:13:58.433 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_processing": 0, 2024-05-08T06:13:58.433 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_enqueued": 0, 2024-05-08T06:13:58.433 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_prioritized": 0, 2024-05-08T06:13:58.434 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_started": 0, 2024-05-08T06:13:58.434 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_completed": 0, 2024-05-08T06:13:58.434 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_path": 0, 2024-05-08T06:13:58.434 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_inode": 0, 2024-05-08T06:13:58.434 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T06:13:58.434 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_exportdir": 0, 2024-05-08T06:13:58.434 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_flush": 0, 2024-05-08T06:13:58.434 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragmentdir": 0, 2024-05-08T06:13:58.434 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragstats": 0, 2024-05-08T06:13:58.434 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_inodestats": 0 2024-05-08T06:13:58.434 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.434 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log": { 2024-05-08T06:13:58.434 INFO:teuthology.orchestra.run.smithi123.stdout: "evlrg": 0, 2024-05-08T06:13:58.434 INFO:teuthology.orchestra.run.smithi123.stdout: "evadd": 5, 2024-05-08T06:13:58.434 INFO:teuthology.orchestra.run.smithi123.stdout: "evex": 0, 2024-05-08T06:13:58.434 INFO:teuthology.orchestra.run.smithi123.stdout: "evtrm": 0, 2024-05-08T06:13:58.435 INFO:teuthology.orchestra.run.smithi123.stdout: "ev": 5, 2024-05-08T06:13:58.435 INFO:teuthology.orchestra.run.smithi123.stdout: "evexg": 0, 2024-05-08T06:13:58.435 INFO:teuthology.orchestra.run.smithi123.stdout: "evexd": 0, 2024-05-08T06:13:58.435 INFO:teuthology.orchestra.run.smithi123.stdout: "segadd": 2, 2024-05-08T06:13:58.435 INFO:teuthology.orchestra.run.smithi123.stdout: "segex": 0, 2024-05-08T06:13:58.435 INFO:teuthology.orchestra.run.smithi123.stdout: "segtrm": 0, 2024-05-08T06:13:58.435 INFO:teuthology.orchestra.run.smithi123.stdout: "seg": 2, 2024-05-08T06:13:58.435 INFO:teuthology.orchestra.run.smithi123.stdout: "segmjr": 2, 2024-05-08T06:13:58.435 INFO:teuthology.orchestra.run.smithi123.stdout: "segexg": 0, 2024-05-08T06:13:58.435 INFO:teuthology.orchestra.run.smithi123.stdout: "segexd": 0, 2024-05-08T06:13:58.435 INFO:teuthology.orchestra.run.smithi123.stdout: "expos": 4194304, 2024-05-08T06:13:58.435 INFO:teuthology.orchestra.run.smithi123.stdout: "wrpos": 4195734, 2024-05-08T06:13:58.435 INFO:teuthology.orchestra.run.smithi123.stdout: "rdpos": 0, 2024-05-08T06:13:58.435 INFO:teuthology.orchestra.run.smithi123.stdout: "jlat": { 2024-05-08T06:13:58.435 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 4, 2024-05-08T06:13:58.435 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.008545761, 2024-05-08T06:13:58.435 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.002136440 2024-05-08T06:13:58.436 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.436 INFO:teuthology.orchestra.run.smithi123.stdout: "replayed": 0 2024-05-08T06:13:58.436 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.436 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_mem": { 2024-05-08T06:13:58.436 INFO:teuthology.orchestra.run.smithi123.stdout: "ino": 21, 2024-05-08T06:13:58.436 INFO:teuthology.orchestra.run.smithi123.stdout: "ino+": 27, 2024-05-08T06:13:58.436 INFO:teuthology.orchestra.run.smithi123.stdout: "ino-": 6, 2024-05-08T06:13:58.436 INFO:teuthology.orchestra.run.smithi123.stdout: "dir": 20, 2024-05-08T06:13:58.436 INFO:teuthology.orchestra.run.smithi123.stdout: "dir+": 26, 2024-05-08T06:13:58.436 INFO:teuthology.orchestra.run.smithi123.stdout: "dir-": 6, 2024-05-08T06:13:58.436 INFO:teuthology.orchestra.run.smithi123.stdout: "dn": 17, 2024-05-08T06:13:58.436 INFO:teuthology.orchestra.run.smithi123.stdout: "dn+": 25, 2024-05-08T06:13:58.436 INFO:teuthology.orchestra.run.smithi123.stdout: "dn-": 8, 2024-05-08T06:13:58.436 INFO:teuthology.orchestra.run.smithi123.stdout: "cap": 1, 2024-05-08T06:13:58.436 INFO:teuthology.orchestra.run.smithi123.stdout: "cap+": 4, 2024-05-08T06:13:58.436 INFO:teuthology.orchestra.run.smithi123.stdout: "cap-": 3, 2024-05-08T06:13:58.436 INFO:teuthology.orchestra.run.smithi123.stdout: "rss": 50668, 2024-05-08T06:13:58.437 INFO:teuthology.orchestra.run.smithi123.stdout: "heap": 223608 2024-05-08T06:13:58.437 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.437 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_server": { 2024-05-08T06:13:58.437 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_client_request": 9, 2024-05-08T06:13:58.437 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_server_request": 0, 2024-05-08T06:13:58.437 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_request": 6, 2024-05-08T06:13:58.437 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_session": 7, 2024-05-08T06:13:58.437 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_peer_request": 1, 2024-05-08T06:13:58.437 INFO:teuthology.orchestra.run.smithi123.stdout: "req_create_latency": { 2024-05-08T06:13:58.437 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:13:58.437 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:13:58.437 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:13:58.437 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.437 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getattr_latency": { 2024-05-08T06:13:58.437 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1, 2024-05-08T06:13:58.438 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.002029921, 2024-05-08T06:13:58.438 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.002029921 2024-05-08T06:13:58.438 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.438 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getfilelock_latency": { 2024-05-08T06:13:58.438 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:13:58.438 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:13:58.438 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:13:58.438 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.438 INFO:teuthology.orchestra.run.smithi123.stdout: "req_link_latency": { 2024-05-08T06:13:58.438 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:13:58.438 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:13:58.438 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:13:58.438 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.438 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookup_latency": { 2024-05-08T06:13:58.438 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2, 2024-05-08T06:13:58.439 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000600263, 2024-05-08T06:13:58.439 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000300131 2024-05-08T06:13:58.439 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.439 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookuphash_latency": { 2024-05-08T06:13:58.439 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:13:58.439 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:13:58.439 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:13:58.439 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.439 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupino_latency": { 2024-05-08T06:13:58.439 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1, 2024-05-08T06:13:58.439 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000308594, 2024-05-08T06:13:58.439 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000308594 2024-05-08T06:13:58.439 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.439 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupname_latency": { 2024-05-08T06:13:58.439 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:13:58.439 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:13:58.440 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:13:58.440 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.440 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupparent_latency": { 2024-05-08T06:13:58.440 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:13:58.440 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:13:58.440 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:13:58.440 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.440 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupsnap_latency": { 2024-05-08T06:13:58.440 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:13:58.440 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:13:58.440 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:13:58.440 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.440 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lssnap_latency": { 2024-05-08T06:13:58.440 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:13:58.440 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:13:58.440 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:13:58.440 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.441 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mkdir_latency": { 2024-05-08T06:13:58.441 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:13:58.441 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:13:58.441 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:13:58.441 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.441 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mknod_latency": { 2024-05-08T06:13:58.441 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:13:58.441 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:13:58.441 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:13:58.442 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.442 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mksnap_latency": { 2024-05-08T06:13:58.442 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:13:58.442 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:13:58.442 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:13:58.442 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.442 INFO:teuthology.orchestra.run.smithi123.stdout: "req_open_latency": { 2024-05-08T06:13:58.442 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:13:58.442 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:13:58.442 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:13:58.442 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.442 INFO:teuthology.orchestra.run.smithi123.stdout: "req_readdir_latency": { 2024-05-08T06:13:58.442 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:13:58.442 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:13:58.442 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:13:58.442 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.443 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rename_latency": { 2024-05-08T06:13:58.443 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:13:58.443 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:13:58.443 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:13:58.443 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.443 INFO:teuthology.orchestra.run.smithi123.stdout: "req_renamesnap_latency": { 2024-05-08T06:13:58.443 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:13:58.443 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:13:58.443 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:13:58.443 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.443 INFO:teuthology.orchestra.run.smithi123.stdout: "req_snapdiff_latency": { 2024-05-08T06:13:58.443 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:13:58.443 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:13:58.443 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:13:58.443 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.443 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmdir_latency": { 2024-05-08T06:13:58.444 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:13:58.444 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:13:58.444 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:13:58.444 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.444 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmsnap_latency": { 2024-05-08T06:13:58.444 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:13:58.444 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:13:58.444 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:13:58.444 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.444 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmxattr_latency": { 2024-05-08T06:13:58.444 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:13:58.444 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:13:58.444 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:13:58.444 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.444 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setattr_latency": { 2024-05-08T06:13:58.444 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:13:58.444 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:13:58.445 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:13:58.445 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.445 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setdirlayout_latency": { 2024-05-08T06:13:58.445 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:13:58.445 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:13:58.445 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:13:58.445 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.445 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setfilelock_latency": { 2024-05-08T06:13:58.445 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:13:58.445 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:13:58.445 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:13:58.445 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.445 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setlayout_latency": { 2024-05-08T06:13:58.445 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:13:58.445 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:13:58.445 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:13:58.446 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.446 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setxattr_latency": { 2024-05-08T06:13:58.446 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:13:58.446 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:13:58.446 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:13:58.446 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.446 INFO:teuthology.orchestra.run.smithi123.stdout: "req_symlink_latency": { 2024-05-08T06:13:58.446 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:13:58.446 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:13:58.446 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:13:58.446 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.446 INFO:teuthology.orchestra.run.smithi123.stdout: "req_unlink_latency": { 2024-05-08T06:13:58.446 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:13:58.446 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:13:58.446 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:13:58.446 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.447 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_revoke_eviction": 0, 2024-05-08T06:13:58.447 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_acquisition_throttle": 0, 2024-05-08T06:13:58.447 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getvxattr_latency": { 2024-05-08T06:13:58.447 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:13:58.447 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:13:58.447 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:13:58.447 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:58.447 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.447 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_sessions": { 2024-05-08T06:13:58.447 INFO:teuthology.orchestra.run.smithi123.stdout: "session_count": 1, 2024-05-08T06:13:58.447 INFO:teuthology.orchestra.run.smithi123.stdout: "session_add": 2, 2024-05-08T06:13:58.447 INFO:teuthology.orchestra.run.smithi123.stdout: "session_remove": 1, 2024-05-08T06:13:58.447 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_open": 1, 2024-05-08T06:13:58.447 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_stale": 0, 2024-05-08T06:13:58.447 INFO:teuthology.orchestra.run.smithi123.stdout: "total_load": 3, 2024-05-08T06:13:58.447 INFO:teuthology.orchestra.run.smithi123.stdout: "average_load": 1, 2024-05-08T06:13:58.447 INFO:teuthology.orchestra.run.smithi123.stdout: "avg_session_uptime": 225, 2024-05-08T06:13:58.448 INFO:teuthology.orchestra.run.smithi123.stdout: "mdthresh_evicted": 0 2024-05-08T06:13:58.448 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.448 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-05-08T06:13:58.448 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-05-08T06:13:58.448 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-05-08T06:13:58.448 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T06:13:58.448 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-05-08T06:13:58.448 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T06:13:58.448 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-05-08T06:13:58.448 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T06:13:58.448 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T06:13:58.448 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T06:13:58.448 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T06:13:58.448 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T06:13:58.448 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-05-08T06:13:58.448 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T06:13:58.449 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T06:13:58.449 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-05-08T06:13:58.449 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-05-08T06:13:58.449 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-05-08T06:13:58.449 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-05-08T06:13:58.449 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T06:13:58.449 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T06:13:58.449 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T06:13:58.449 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T06:13:58.449 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T06:13:58.449 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-05-08T06:13:58.449 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-05-08T06:13:58.449 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-05-08T06:13:58.449 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T06:13:58.449 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T06:13:58.449 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-05-08T06:13:58.450 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-05-08T06:13:58.450 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-05-08T06:13:58.450 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-05-08T06:13:58.450 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T06:13:58.450 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-05-08T06:13:58.450 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T06:13:58.450 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-05-08T06:13:58.450 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 255799, 2024-05-08T06:13:58.450 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 93, 2024-05-08T06:13:58.450 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 88, 2024-05-08T06:13:58.450 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 1, 2024-05-08T06:13:58.450 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-05-08T06:13:58.450 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-05-08T06:13:58.450 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-05-08T06:13:58.451 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-05-08T06:13:58.451 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-05-08T06:13:58.451 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-05-08T06:13:58.451 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 7528, 2024-05-08T06:13:58.451 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 158, 2024-05-08T06:13:58.451 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T06:13:58.451 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-05-08T06:13:58.451 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-05-08T06:13:58.451 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-05-08T06:13:58.451 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 85581, 2024-05-08T06:13:58.451 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 349, 2024-05-08T06:13:58.451 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-05-08T06:13:58.451 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-05-08T06:13:58.451 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-05-08T06:13:58.451 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-05-08T06:13:58.451 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.451 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter": { 2024-05-08T06:13:58.452 INFO:teuthology.orchestra.run.smithi123.stdout: "op_active": 0, 2024-05-08T06:13:58.452 INFO:teuthology.orchestra.run.smithi123.stdout: "op_laggy": 0, 2024-05-08T06:13:58.452 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send": 43, 2024-05-08T06:13:58.452 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send_bytes": 12148, 2024-05-08T06:13:58.452 INFO:teuthology.orchestra.run.smithi123.stdout: "op_resend": 28, 2024-05-08T06:13:58.452 INFO:teuthology.orchestra.run.smithi123.stdout: "op_reply": 43, 2024-05-08T06:13:58.452 INFO:teuthology.orchestra.run.smithi123.stdout: "op_latency": { 2024-05-08T06:13:58.452 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 43, 2024-05-08T06:13:58.452 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.282993426, 2024-05-08T06:13:58.452 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.006581242 2024-05-08T06:13:58.452 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.452 INFO:teuthology.orchestra.run.smithi123.stdout: "op_inflight": 0, 2024-05-08T06:13:58.452 INFO:teuthology.orchestra.run.smithi123.stdout: "oplen_avg": { 2024-05-08T06:13:58.452 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 43, 2024-05-08T06:13:58.452 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 68 2024-05-08T06:13:58.452 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.452 INFO:teuthology.orchestra.run.smithi123.stdout: "op": 43, 2024-05-08T06:13:58.453 INFO:teuthology.orchestra.run.smithi123.stdout: "op_r": 0, 2024-05-08T06:13:58.453 INFO:teuthology.orchestra.run.smithi123.stdout: "op_w": 43, 2024-05-08T06:13:58.453 INFO:teuthology.orchestra.run.smithi123.stdout: "op_rmw": 0, 2024-05-08T06:13:58.453 INFO:teuthology.orchestra.run.smithi123.stdout: "op_pg": 0, 2024-05-08T06:13:58.453 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_stat": 0, 2024-05-08T06:13:58.453 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_create": 10, 2024-05-08T06:13:58.453 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_read": 0, 2024-05-08T06:13:58.453 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_write": 4, 2024-05-08T06:13:58.453 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writefull": 7, 2024-05-08T06:13:58.453 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writesame": 0, 2024-05-08T06:13:58.453 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_append": 0, 2024-05-08T06:13:58.453 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_zero": 0, 2024-05-08T06:13:58.453 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_truncate": 0, 2024-05-08T06:13:58.453 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_delete": 6, 2024-05-08T06:13:58.453 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_mapext": 0, 2024-05-08T06:13:58.453 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_sparse_read": 0, 2024-05-08T06:13:58.453 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_clonerange": 0, 2024-05-08T06:13:58.454 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_getxattr": 0, 2024-05-08T06:13:58.454 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_setxattr": 20, 2024-05-08T06:13:58.454 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_cmpxattr": 0, 2024-05-08T06:13:58.454 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_rmxattr": 0, 2024-05-08T06:13:58.454 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_resetxattrs": 0, 2024-05-08T06:13:58.454 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_call": 0, 2024-05-08T06:13:58.454 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_watch": 0, 2024-05-08T06:13:58.454 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_notify": 0, 2024-05-08T06:13:58.454 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T06:13:58.454 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls": 0, 2024-05-08T06:13:58.454 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls_filter": 0, 2024-05-08T06:13:58.454 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_other": 0, 2024-05-08T06:13:58.454 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_active": 0, 2024-05-08T06:13:58.454 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_send": 0, 2024-05-08T06:13:58.454 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_resend": 0, 2024-05-08T06:13:58.455 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_ping": 0, 2024-05-08T06:13:58.455 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_active": 0, 2024-05-08T06:13:58.455 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_send": 0, 2024-05-08T06:13:58.455 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_resend": 0, 2024-05-08T06:13:58.455 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_active": 0, 2024-05-08T06:13:58.455 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_send": 0, 2024-05-08T06:13:58.455 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_resend": 0, 2024-05-08T06:13:58.455 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_active": 0, 2024-05-08T06:13:58.455 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_send": 0, 2024-05-08T06:13:58.455 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_resend": 0, 2024-05-08T06:13:58.455 INFO:teuthology.orchestra.run.smithi123.stdout: "command_active": 0, 2024-05-08T06:13:58.455 INFO:teuthology.orchestra.run.smithi123.stdout: "command_send": 0, 2024-05-08T06:13:58.455 INFO:teuthology.orchestra.run.smithi123.stdout: "command_resend": 0, 2024-05-08T06:13:58.455 INFO:teuthology.orchestra.run.smithi123.stdout: "map_epoch": 82, 2024-05-08T06:13:58.455 INFO:teuthology.orchestra.run.smithi123.stdout: "map_full": 0, 2024-05-08T06:13:58.455 INFO:teuthology.orchestra.run.smithi123.stdout: "map_inc": 2, 2024-05-08T06:13:58.455 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_sessions": 9, 2024-05-08T06:13:58.456 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_open": 9, 2024-05-08T06:13:58.456 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_close": 0, 2024-05-08T06:13:58.456 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_laggy": 0, 2024-05-08T06:13:58.456 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_wr": 20, 2024-05-08T06:13:58.456 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_rd": 0, 2024-05-08T06:13:58.456 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_del": 1 2024-05-08T06:13:58.456 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.456 INFO:teuthology.orchestra.run.smithi123.stdout: "oft": { 2024-05-08T06:13:58.456 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_objs": 1, 2024-05-08T06:13:58.456 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_kv_pairs": 0, 2024-05-08T06:13:58.456 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_updates": 0, 2024-05-08T06:13:58.456 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_removes": 0 2024-05-08T06:13:58.456 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.456 INFO:teuthology.orchestra.run.smithi123.stdout: "osd-slow-ops": { 2024-05-08T06:13:58.456 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_ops_count": 0 2024-05-08T06:13:58.456 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.456 INFO:teuthology.orchestra.run.smithi123.stdout: "purge_queue": { 2024-05-08T06:13:58.457 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops": 0, 2024-05-08T06:13:58.457 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T06:13:58.457 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing": 0, 2024-05-08T06:13:58.457 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_high_water": 0, 2024-05-08T06:13:58.457 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed_ops": 0, 2024-05-08T06:13:58.457 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed": 0, 2024-05-08T06:13:58.457 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_item_in_journal": 0 2024-05-08T06:13:58.457 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.457 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T06:13:58.457 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-05-08T06:13:58.457 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-05-08T06:13:58.457 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-05-08T06:13:58.457 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 666, 2024-05-08T06:13:58.457 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 895905, 2024-05-08T06:13:58.457 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-05-08T06:13:58.457 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 666, 2024-05-08T06:13:58.458 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-05-08T06:13:58.458 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-05-08T06:13:58.458 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 666, 2024-05-08T06:13:58.458 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 895905, 2024-05-08T06:13:58.458 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-05-08T06:13:58.458 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:13:58.458 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:13:58.458 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:13:58.458 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:58.458 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.458 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_bytes": { 2024-05-08T06:13:58.458 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-05-08T06:13:58.458 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-05-08T06:13:58.458 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-05-08T06:13:58.458 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-05-08T06:13:58.458 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-05-08T06:13:58.459 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-05-08T06:13:58.459 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-05-08T06:13:58.459 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 43, 2024-05-08T06:13:58.459 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 12148, 2024-05-08T06:13:58.459 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 43, 2024-05-08T06:13:58.459 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 12148, 2024-05-08T06:13:58.459 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-05-08T06:13:58.459 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:13:58.459 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:13:58.459 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:13:58.459 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:58.459 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.459 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_ops": { 2024-05-08T06:13:58.459 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-05-08T06:13:58.459 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 1024, 2024-05-08T06:13:58.459 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-05-08T06:13:58.459 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-05-08T06:13:58.460 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-05-08T06:13:58.460 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-05-08T06:13:58.460 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-05-08T06:13:58.460 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 43, 2024-05-08T06:13:58.460 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 43, 2024-05-08T06:13:58.460 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 43, 2024-05-08T06:13:58.460 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 43, 2024-05-08T06:13:58.460 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-05-08T06:13:58.460 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:13:58.460 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:13:58.460 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:13:58.460 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:58.460 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.460 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle": { 2024-05-08T06:13:58.460 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-05-08T06:13:58.460 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-05-08T06:13:58.460 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-05-08T06:13:58.461 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-05-08T06:13:58.461 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-05-08T06:13:58.461 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-05-08T06:13:58.461 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-05-08T06:13:58.461 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-05-08T06:13:58.461 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-05-08T06:13:58.461 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 0, 2024-05-08T06:13:58.461 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 0, 2024-05-08T06:13:58.461 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-05-08T06:13:58.461 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:13:58.461 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:13:58.461 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:13:58.461 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:58.461 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:13:58.461 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle-0x55d0efc74500": { 2024-05-08T06:13:58.461 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-05-08T06:13:58.461 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-05-08T06:13:58.462 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-05-08T06:13:58.462 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 5, 2024-05-08T06:13:58.462 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 1430, 2024-05-08T06:13:58.462 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-05-08T06:13:58.462 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 5, 2024-05-08T06:13:58.462 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-05-08T06:13:58.462 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-05-08T06:13:58.462 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 4, 2024-05-08T06:13:58.462 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 1430, 2024-05-08T06:13:58.462 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-05-08T06:13:58.462 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:13:58.462 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:13:58.462 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:13:58.462 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:58.463 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:13:58.463 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-05-08T06:13:59.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:13:58 smithi123 ceph-mon[29787]: pgmap v565: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:13:59.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:13:58 smithi163 ceph-mon[34520]: pgmap v565: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:13:59.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:13:58 smithi192 ceph-mon[34664]: pgmap v565: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:13:59.610 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-05-08T06:13:59.610 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-05-08T06:13:59.610 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-05-08T06:13:59.610 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=15 2024-05-08T06:13:59.610 DEBUG:tasks.check_counter:Getting stats from d 2024-05-08T06:13:59.611 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-08T06:14:00.109 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:14:00.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:00 smithi123 ceph-mon[29787]: pgmap v566: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-05-08T06:14:01.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:00 smithi163 ceph-mon[34520]: pgmap v566: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-05-08T06:14:01.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:00 smithi192 ceph-mon[34664]: pgmap v566: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-05-08T06:14:02.684 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-05-08T06:14:02.684 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T06:14:02.684 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 4, 2024-05-08T06:14:02.684 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 4, 2024-05-08T06:14:02.684 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 3040, 2024-05-08T06:14:02.684 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 448, 2024-05-08T06:14:02.684 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 4, 2024-05-08T06:14:02.684 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 0, 2024-05-08T06:14:02.684 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.004049355, 2024-05-08T06:14:02.684 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.000051265, 2024-05-08T06:14:02.684 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.001098046, 2024-05-08T06:14:02.684 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:14:02.685 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:14:02.685 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3, 2024-05-08T06:14:02.685 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.001655103, 2024-05-08T06:14:02.685 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000551701 2024-05-08T06:14:02.685 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:14:02.685 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:14:02.685 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:14:02.685 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:14:02.685 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:14:02.685 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:14:02.685 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-05-08T06:14:02.685 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 448 2024-05-08T06:14:02.685 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:14:02.685 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T06:14:02.685 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 153, 2024-05-08T06:14:02.686 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 148, 2024-05-08T06:14:02.686 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 794144, 2024-05-08T06:14:02.686 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 90288, 2024-05-08T06:14:02.686 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 2, 2024-05-08T06:14:02.686 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 1, 2024-05-08T06:14:02.686 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.061049105, 2024-05-08T06:14:02.686 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.022077351, 2024-05-08T06:14:02.686 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.043971541, 2024-05-08T06:14:02.686 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.006241608, 2024-05-08T06:14:02.686 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:14:02.686 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 147, 2024-05-08T06:14:02.686 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.013646256, 2024-05-08T06:14:02.686 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000092831 2024-05-08T06:14:02.686 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:14:02.687 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:14:02.687 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:14:02.687 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:14:02.687 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:14:02.687 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:14:02.687 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 794144, 2024-05-08T06:14:02.687 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 90288 2024-05-08T06:14:02.687 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:14:02.687 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T06:14:02.687 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 1, 2024-05-08T06:14:02.687 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 112, 2024-05-08T06:14:02.687 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 160, 2024-05-08T06:14:02.687 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 98320, 2024-05-08T06:14:02.687 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 3, 2024-05-08T06:14:02.687 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 2, 2024-05-08T06:14:02.688 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.021964736, 2024-05-08T06:14:02.688 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.015716059, 2024-05-08T06:14:02.688 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.003517819, 2024-05-08T06:14:02.688 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:14:02.688 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:14:02.688 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 110, 2024-05-08T06:14:02.688 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.010614690, 2024-05-08T06:14:02.688 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000096497 2024-05-08T06:14:02.688 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:14:02.688 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:14:02.688 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:14:02.688 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:14:02.689 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:14:02.689 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:14:02.689 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-08T06:14:02.689 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 98320 2024-05-08T06:14:02.689 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:14:02.689 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-05-08T06:14:02.689 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 0, 2024-05-08T06:14:02.689 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-05-08T06:14:02.689 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:14:02.689 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-05-08T06:14:02.689 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-05-08T06:14:02.689 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-05-08T06:14:02.689 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T06:14:02.689 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-05-08T06:14:02.689 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T06:14:02.689 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-05-08T06:14:02.690 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T06:14:02.690 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T06:14:02.690 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T06:14:02.690 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T06:14:02.690 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T06:14:02.690 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-05-08T06:14:02.690 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T06:14:02.690 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T06:14:02.690 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-05-08T06:14:02.690 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-05-08T06:14:02.690 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-05-08T06:14:02.690 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-05-08T06:14:02.690 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T06:14:02.690 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T06:14:02.690 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T06:14:02.691 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T06:14:02.691 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T06:14:02.691 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-05-08T06:14:02.691 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-05-08T06:14:02.691 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-05-08T06:14:02.691 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T06:14:02.691 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T06:14:02.691 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-05-08T06:14:02.691 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-05-08T06:14:02.691 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-05-08T06:14:02.691 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-05-08T06:14:02.691 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T06:14:02.692 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-05-08T06:14:02.692 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T06:14:02.692 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-05-08T06:14:02.692 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 148984, 2024-05-08T06:14:02.692 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 30, 2024-05-08T06:14:02.692 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2024-05-08T06:14:02.692 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2024-05-08T06:14:02.692 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-05-08T06:14:02.692 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-05-08T06:14:02.692 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-05-08T06:14:02.692 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-05-08T06:14:02.692 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-05-08T06:14:02.692 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-05-08T06:14:02.692 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 0, 2024-05-08T06:14:02.693 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 0, 2024-05-08T06:14:02.693 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T06:14:02.693 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-05-08T06:14:02.693 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-05-08T06:14:02.693 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-05-08T06:14:02.693 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 928, 2024-05-08T06:14:02.693 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 2, 2024-05-08T06:14:02.693 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-05-08T06:14:02.693 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-05-08T06:14:02.693 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-05-08T06:14:02.693 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-05-08T06:14:02.693 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:14:02.693 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T06:14:02.693 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-05-08T06:14:02.693 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-05-08T06:14:02.693 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-05-08T06:14:02.694 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 154, 2024-05-08T06:14:02.694 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 772551, 2024-05-08T06:14:02.694 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-05-08T06:14:02.694 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 154, 2024-05-08T06:14:02.694 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-05-08T06:14:02.694 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-05-08T06:14:02.694 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 154, 2024-05-08T06:14:02.694 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 772551, 2024-05-08T06:14:02.694 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-05-08T06:14:02.694 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:14:02.694 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:14:02.694 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:14:02.694 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:14:02.694 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:14:02.694 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-05-08T06:14:03.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:02 smithi123 ceph-mon[29787]: pgmap v567: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:14:03.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:02 smithi163 ceph-mon[34520]: pgmap v567: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:14:03.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:02 smithi192 ceph-mon[34664]: pgmap v567: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:14:03.837 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-08T06:14:03.838 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-08T06:14:03.838 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-05-08T06:14:03.838 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-05-08T06:14:03.838 DEBUG:tasks.check_counter:Getting stats from g 2024-05-08T06:14:03.838 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-08T06:14:04.280 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:14:04.948 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:04 smithi123 ceph-mon[29787]: pgmap v568: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:14:05.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:04 smithi163 ceph-mon[34520]: pgmap v568: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:14:05.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:04 smithi192 ceph-mon[34664]: pgmap v568: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:14:06.786 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-05-08T06:14:06.786 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T06:14:06.786 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 0, 2024-05-08T06:14:06.786 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 2, 2024-05-08T06:14:06.787 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 0, 2024-05-08T06:14:06.787 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 0, 2024-05-08T06:14:06.787 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 4, 2024-05-08T06:14:06.787 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 0, 2024-05-08T06:14:06.787 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.004406400, 2024-05-08T06:14:06.787 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.000000000, 2024-05-08T06:14:06.787 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.000851473, 2024-05-08T06:14:06.787 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:14:06.787 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:14:06.787 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:14:06.787 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:14:06.787 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:14:06.787 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:14:06.787 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:14:06.787 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:14:06.787 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:14:06.788 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:14:06.788 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:14:06.788 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T06:14:06.788 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T06:14:06.788 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:14:06.788 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T06:14:06.788 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 1, 2024-05-08T06:14:06.788 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 108, 2024-05-08T06:14:06.788 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 160, 2024-05-08T06:14:06.788 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 97616, 2024-05-08T06:14:06.788 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 3, 2024-05-08T06:14:06.788 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 2, 2024-05-08T06:14:06.788 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.020489605, 2024-05-08T06:14:06.788 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.015405938, 2024-05-08T06:14:06.788 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.002447722, 2024-05-08T06:14:06.789 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:14:06.789 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:14:06.789 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 106, 2024-05-08T06:14:06.789 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.010573032, 2024-05-08T06:14:06.789 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000099745 2024-05-08T06:14:06.789 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:14:06.789 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:14:06.789 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:14:06.789 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:14:06.789 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:14:06.789 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:14:06.789 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-08T06:14:06.789 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 97616 2024-05-08T06:14:06.790 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:14:06.790 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T06:14:06.790 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 152, 2024-05-08T06:14:06.790 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 145, 2024-05-08T06:14:06.790 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 795200, 2024-05-08T06:14:06.790 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 87840, 2024-05-08T06:14:06.790 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 2, 2024-05-08T06:14:06.790 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 1, 2024-05-08T06:14:06.790 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.057611262, 2024-05-08T06:14:06.790 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.022036757, 2024-05-08T06:14:06.790 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.038114706, 2024-05-08T06:14:06.790 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.005961466, 2024-05-08T06:14:06.790 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:14:06.790 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 145, 2024-05-08T06:14:06.791 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.014745480, 2024-05-08T06:14:06.791 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000101692 2024-05-08T06:14:06.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:14:06.791 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:14:06.791 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:14:06.791 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:14:06.791 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:14:06.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:14:06.791 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 795200, 2024-05-08T06:14:06.791 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 87840 2024-05-08T06:14:06.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:14:06.791 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-05-08T06:14:06.791 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 0, 2024-05-08T06:14:06.791 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-05-08T06:14:06.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:14:06.791 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-05-08T06:14:06.791 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-05-08T06:14:06.792 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-05-08T06:14:06.792 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T06:14:06.792 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-05-08T06:14:06.792 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T06:14:06.792 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-05-08T06:14:06.792 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T06:14:06.792 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T06:14:06.792 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T06:14:06.792 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T06:14:06.792 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T06:14:06.792 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-05-08T06:14:06.792 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T06:14:06.792 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T06:14:06.792 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-05-08T06:14:06.792 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-05-08T06:14:06.792 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-05-08T06:14:06.793 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-05-08T06:14:06.793 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T06:14:06.793 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T06:14:06.793 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T06:14:06.793 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T06:14:06.793 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T06:14:06.793 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-05-08T06:14:06.793 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-05-08T06:14:06.793 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-05-08T06:14:06.793 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T06:14:06.793 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T06:14:06.793 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-05-08T06:14:06.793 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-05-08T06:14:06.793 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-05-08T06:14:06.793 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-05-08T06:14:06.793 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T06:14:06.794 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-05-08T06:14:06.794 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T06:14:06.794 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-05-08T06:14:06.794 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 149998, 2024-05-08T06:14:06.794 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 30, 2024-05-08T06:14:06.794 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2024-05-08T06:14:06.794 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2024-05-08T06:14:06.794 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-05-08T06:14:06.794 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-05-08T06:14:06.794 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-05-08T06:14:06.794 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-05-08T06:14:06.795 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-05-08T06:14:06.795 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-05-08T06:14:06.795 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 0, 2024-05-08T06:14:06.795 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 0, 2024-05-08T06:14:06.795 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T06:14:06.795 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-05-08T06:14:06.795 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-05-08T06:14:06.795 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-05-08T06:14:06.795 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 928, 2024-05-08T06:14:06.795 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 2, 2024-05-08T06:14:06.795 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-05-08T06:14:06.795 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-05-08T06:14:06.795 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-05-08T06:14:06.795 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-05-08T06:14:06.795 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:14:06.795 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T06:14:06.796 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-05-08T06:14:06.796 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-05-08T06:14:06.796 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-05-08T06:14:06.796 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 149, 2024-05-08T06:14:06.796 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 771264, 2024-05-08T06:14:06.796 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-05-08T06:14:06.796 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 149, 2024-05-08T06:14:06.796 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-05-08T06:14:06.796 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-05-08T06:14:06.796 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 149, 2024-05-08T06:14:06.796 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 771264, 2024-05-08T06:14:06.796 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-05-08T06:14:06.796 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:14:06.796 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:14:06.796 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:14:06.797 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:14:06.797 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:14:06.797 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-05-08T06:14:07.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:06 smithi123 ceph-mon[29787]: pgmap v569: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T06:14:07.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:06 smithi163 ceph-mon[34520]: pgmap v569: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T06:14:07.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:06 smithi192 ceph-mon[34664]: pgmap v569: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T06:14:07.797 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-05-08T06:14:07.797 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-05-08T06:14:07.797 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-05-08T06:14:07.797 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-05-08T06:14:07.797 DEBUG:tasks.check_counter:Getting stats from j 2024-05-08T06:14:07.798 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-08T06:14:08.243 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:14:08.926 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:08 smithi123 ceph-mon[29787]: pgmap v570: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:14:09.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:08 smithi163 ceph-mon[34520]: pgmap v570: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:14:09.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:08 smithi192 ceph-mon[34664]: pgmap v570: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:14:10.840 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-05-08T06:14:10.840 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T06:14:10.841 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 160, 2024-05-08T06:14:10.841 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 153, 2024-05-08T06:14:10.841 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 798000, 2024-05-08T06:14:10.841 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 92656, 2024-05-08T06:14:10.841 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 4, 2024-05-08T06:14:10.841 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 0, 2024-05-08T06:14:10.841 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.060750806, 2024-05-08T06:14:10.841 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.021832196, 2024-05-08T06:14:10.841 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.038098483, 2024-05-08T06:14:10.841 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.006521328, 2024-05-08T06:14:10.841 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:14:10.841 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 153, 2024-05-08T06:14:10.841 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.016276440, 2024-05-08T06:14:10.841 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000106381 2024-05-08T06:14:10.841 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:14:10.842 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:14:10.842 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:14:10.842 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:14:10.842 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:14:10.842 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:14:10.842 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 798000, 2024-05-08T06:14:10.842 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 92656 2024-05-08T06:14:10.842 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:14:10.842 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T06:14:10.842 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 1, 2024-05-08T06:14:10.842 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 114, 2024-05-08T06:14:10.842 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 160, 2024-05-08T06:14:10.842 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 98672, 2024-05-08T06:14:10.843 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 3, 2024-05-08T06:14:10.843 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 2, 2024-05-08T06:14:10.843 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.020942583, 2024-05-08T06:14:10.843 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.016116889, 2024-05-08T06:14:10.843 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.002283729, 2024-05-08T06:14:10.843 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:14:10.843 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:14:10.843 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 112, 2024-05-08T06:14:10.843 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.010067935, 2024-05-08T06:14:10.843 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000089892 2024-05-08T06:14:10.843 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:14:10.843 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:14:10.843 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:14:10.844 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:14:10.844 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:14:10.844 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:14:10.844 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-08T06:14:10.844 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 98672 2024-05-08T06:14:10.844 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:14:10.844 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T06:14:10.844 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 0, 2024-05-08T06:14:10.844 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 2, 2024-05-08T06:14:10.844 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 0, 2024-05-08T06:14:10.844 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 0, 2024-05-08T06:14:10.844 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 2, 2024-05-08T06:14:10.844 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 1, 2024-05-08T06:14:10.844 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.001086792, 2024-05-08T06:14:10.844 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.000000000, 2024-05-08T06:14:10.844 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.000865738, 2024-05-08T06:14:10.845 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:14:10.845 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:14:10.845 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:14:10.845 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:14:10.845 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:14:10.845 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:14:10.845 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:14:10.845 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:14:10.845 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:14:10.845 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:14:10.845 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:14:10.845 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T06:14:10.845 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T06:14:10.845 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:14:10.846 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-05-08T06:14:10.846 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 0, 2024-05-08T06:14:10.846 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-05-08T06:14:10.846 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:14:10.846 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-05-08T06:14:10.846 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-05-08T06:14:10.846 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-05-08T06:14:10.846 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T06:14:10.846 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-05-08T06:14:10.846 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T06:14:10.846 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-05-08T06:14:10.846 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T06:14:10.847 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T06:14:10.847 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T06:14:10.847 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T06:14:10.847 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T06:14:10.847 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-05-08T06:14:10.847 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T06:14:10.847 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T06:14:10.847 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-05-08T06:14:10.847 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-05-08T06:14:10.847 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-05-08T06:14:10.847 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-05-08T06:14:10.847 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T06:14:10.847 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T06:14:10.847 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T06:14:10.848 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T06:14:10.848 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T06:14:10.848 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-05-08T06:14:10.848 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-05-08T06:14:10.848 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-05-08T06:14:10.848 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T06:14:10.848 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T06:14:10.848 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-05-08T06:14:10.848 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-05-08T06:14:10.848 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-05-08T06:14:10.848 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-05-08T06:14:10.848 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T06:14:10.848 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-05-08T06:14:10.849 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T06:14:10.849 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-05-08T06:14:10.849 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 151029, 2024-05-08T06:14:10.849 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 31, 2024-05-08T06:14:10.849 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2024-05-08T06:14:10.849 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2024-05-08T06:14:10.849 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-05-08T06:14:10.849 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-05-08T06:14:10.849 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-05-08T06:14:10.849 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-05-08T06:14:10.849 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-05-08T06:14:10.849 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-05-08T06:14:10.849 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 0, 2024-05-08T06:14:10.850 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 0, 2024-05-08T06:14:10.850 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T06:14:10.850 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-05-08T06:14:10.850 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-05-08T06:14:10.850 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-05-08T06:14:10.850 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 928, 2024-05-08T06:14:10.850 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 2, 2024-05-08T06:14:10.850 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-05-08T06:14:10.850 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-05-08T06:14:10.850 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-05-08T06:14:10.850 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-05-08T06:14:10.851 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-08T06:14:10.851 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T06:14:10.851 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-05-08T06:14:10.851 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-05-08T06:14:10.851 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-05-08T06:14:10.851 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 157, 2024-05-08T06:14:10.851 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 772941, 2024-05-08T06:14:10.851 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-05-08T06:14:10.851 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 157, 2024-05-08T06:14:10.851 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-05-08T06:14:10.851 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-05-08T06:14:10.851 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 157, 2024-05-08T06:14:10.851 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 772941, 2024-05-08T06:14:10.851 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-05-08T06:14:10.852 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-08T06:14:10.852 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-08T06:14:10.852 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-08T06:14:10.852 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:14:10.852 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-08T06:14:10.852 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-05-08T06:14:11.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:10 smithi123 ceph-mon[29787]: pgmap v571: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-05-08T06:14:11.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:10 smithi163 ceph-mon[34520]: pgmap v571: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-05-08T06:14:11.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:10 smithi192 ceph-mon[34664]: pgmap v571: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-05-08T06:14:11.919 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-08T06:14:11.919 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-08T06:14:11.920 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-05-08T06:14:11.920 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-05-08T06:14:11.920 DEBUG:tasks.check_counter:Getting stats from b 2024-05-08T06:14:11.920 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-08T06:14:12.171 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T06:14:13.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:12 smithi123 ceph-mon[29787]: pgmap v572: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:14:13.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:12 smithi163 ceph-mon[34520]: pgmap v572: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:14:13.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:12 smithi192 ceph-mon[34664]: pgmap v572: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:14:13.944 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2024-05-08T06:14:13.944 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T06:14:13.944 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 3563, 2024-05-08T06:14:13.945 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 3510, 2024-05-08T06:14:13.945 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 934145, 2024-05-08T06:14:13.945 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 10106102, 2024-05-08T06:14:13.945 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 50, 2024-05-08T06:14:13.945 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 4, 2024-05-08T06:14:13.945 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.286553290, 2024-05-08T06:14:13.945 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.104604221, 2024-05-08T06:14:13.945 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.078579979, 2024-05-08T06:14:13.945 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.080119048, 2024-05-08T06:14:13.945 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:14:13.945 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 3511, 2024-05-08T06:14:13.945 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.341845922, 2024-05-08T06:14:13.945 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000097364 2024-05-08T06:14:13.945 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.946 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:14:13.946 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 868, 2024-05-08T06:14:13.946 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000181628, 2024-05-08T06:14:13.946 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000209 2024-05-08T06:14:13.946 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.946 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 36992, 2024-05-08T06:14:13.946 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 1456 2024-05-08T06:14:13.946 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.946 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T06:14:13.946 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 7851, 2024-05-08T06:14:13.946 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 12476, 2024-05-08T06:14:13.946 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 2931063, 2024-05-08T06:14:13.946 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 10183338, 2024-05-08T06:14:13.947 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 31, 2024-05-08T06:14:13.947 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 5, 2024-05-08T06:14:13.947 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.599027504, 2024-05-08T06:14:13.947 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.240057251, 2024-05-08T06:14:13.947 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.351945492, 2024-05-08T06:14:13.947 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.078116337, 2024-05-08T06:14:13.947 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:14:13.947 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 12474, 2024-05-08T06:14:13.947 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.479544018, 2024-05-08T06:14:13.947 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000038443 2024-05-08T06:14:13.947 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.947 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:14:13.947 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 11342, 2024-05-08T06:14:13.947 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.002021403, 2024-05-08T06:14:13.948 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000178 2024-05-08T06:14:13.948 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.948 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 2152768, 2024-05-08T06:14:13.948 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 4758768 2024-05-08T06:14:13.948 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.948 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T06:14:13.948 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 2028, 2024-05-08T06:14:13.948 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 2052, 2024-05-08T06:14:13.948 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 1306030, 2024-05-08T06:14:13.948 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 988580, 2024-05-08T06:14:13.948 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 8, 2024-05-08T06:14:13.948 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 7, 2024-05-08T06:14:13.948 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.172435715, 2024-05-08T06:14:13.948 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.059496483, 2024-05-08T06:14:13.948 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.061804641, 2024-05-08T06:14:13.949 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.047694364, 2024-05-08T06:14:13.949 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:14:13.949 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 2048, 2024-05-08T06:14:13.949 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.228538854, 2024-05-08T06:14:13.949 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000111591 2024-05-08T06:14:13.949 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.949 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:14:13.949 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 766, 2024-05-08T06:14:13.949 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000183024, 2024-05-08T06:14:13.949 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000238 2024-05-08T06:14:13.949 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.949 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 834448, 2024-05-08T06:14:13.949 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 98976 2024-05-08T06:14:13.950 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.950 INFO:teuthology.orchestra.run.smithi163.stdout: "cct": { 2024-05-08T06:14:13.950 INFO:teuthology.orchestra.run.smithi163.stdout: "total_workers": 1, 2024-05-08T06:14:13.950 INFO:teuthology.orchestra.run.smithi163.stdout: "unhealthy_workers": 0 2024-05-08T06:14:13.950 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.950 INFO:teuthology.orchestra.run.smithi163.stdout: "finisher-MDSRank": { 2024-05-08T06:14:13.950 INFO:teuthology.orchestra.run.smithi163.stdout: "queue_len": 0, 2024-05-08T06:14:13.950 INFO:teuthology.orchestra.run.smithi163.stdout: "complete_latency": { 2024-05-08T06:14:13.950 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 5093, 2024-05-08T06:14:13.950 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 2.490863244, 2024-05-08T06:14:13.950 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000489075 2024-05-08T06:14:13.950 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:14:13.950 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.951 INFO:teuthology.orchestra.run.smithi163.stdout: "finisher-PurgeQueue": { 2024-05-08T06:14:13.951 INFO:teuthology.orchestra.run.smithi163.stdout: "queue_len": 0, 2024-05-08T06:14:13.951 INFO:teuthology.orchestra.run.smithi163.stdout: "complete_latency": { 2024-05-08T06:14:13.951 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 5541, 2024-05-08T06:14:13.951 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.159443392, 2024-05-08T06:14:13.951 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000028775 2024-05-08T06:14:13.951 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:14:13.951 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.951 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": { 2024-05-08T06:14:13.951 INFO:teuthology.orchestra.run.smithi163.stdout: "request": 2469, 2024-05-08T06:14:13.951 INFO:teuthology.orchestra.run.smithi163.stdout: "reply": 2469, 2024-05-08T06:14:13.951 INFO:teuthology.orchestra.run.smithi163.stdout: "reply_latency": { 2024-05-08T06:14:13.951 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 2469, 2024-05-08T06:14:13.951 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 2.952995376, 2024-05-08T06:14:13.951 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.001196028 2024-05-08T06:14:13.952 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.952 INFO:teuthology.orchestra.run.smithi163.stdout: "slow_reply": 0, 2024-05-08T06:14:13.952 INFO:teuthology.orchestra.run.smithi163.stdout: "forward": 0, 2024-05-08T06:14:13.952 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_fetch_complete": 0, 2024-05-08T06:14:13.952 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_fetch_keys": 0, 2024-05-08T06:14:13.952 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_commit": 69, 2024-05-08T06:14:13.952 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_split": 0, 2024-05-08T06:14:13.952 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_merge": 0, 2024-05-08T06:14:13.952 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes": 172, 2024-05-08T06:14:13.952 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_top": 5, 2024-05-08T06:14:13.952 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_bottom": 167, 2024-05-08T06:14:13.952 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_pin_tail": 0, 2024-05-08T06:14:13.952 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_pinned": 164, 2024-05-08T06:14:13.952 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_expired": 0, 2024-05-08T06:14:13.953 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_with_caps": 2, 2024-05-08T06:14:13.953 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": 2, 2024-05-08T06:14:13.953 INFO:teuthology.orchestra.run.smithi163.stdout: "subtrees": 2, 2024-05-08T06:14:13.953 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse": 2494, 2024-05-08T06:14:13.953 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_hit": 2481, 2024-05-08T06:14:13.953 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_forward": 0, 2024-05-08T06:14:13.953 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_discover": 0, 2024-05-08T06:14:13.953 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_dir_fetch": 0, 2024-05-08T06:14:13.953 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_remote_ino": 0, 2024-05-08T06:14:13.953 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_lock": 0, 2024-05-08T06:14:13.953 INFO:teuthology.orchestra.run.smithi163.stdout: "load_cent": 0, 2024-05-08T06:14:13.953 INFO:teuthology.orchestra.run.smithi163.stdout: "q": 0, 2024-05-08T06:14:13.953 INFO:teuthology.orchestra.run.smithi163.stdout: "exported": 0, 2024-05-08T06:14:13.953 INFO:teuthology.orchestra.run.smithi163.stdout: "exported_inodes": 0, 2024-05-08T06:14:13.953 INFO:teuthology.orchestra.run.smithi163.stdout: "imported": 0, 2024-05-08T06:14:13.954 INFO:teuthology.orchestra.run.smithi163.stdout: "imported_inodes": 0, 2024-05-08T06:14:13.954 INFO:teuthology.orchestra.run.smithi163.stdout: "openino_dir_fetch": 0, 2024-05-08T06:14:13.954 INFO:teuthology.orchestra.run.smithi163.stdout: "openino_backtrace_fetch": 64, 2024-05-08T06:14:13.954 INFO:teuthology.orchestra.run.smithi163.stdout: "openino_peer_discover": 0, 2024-05-08T06:14:13.954 INFO:teuthology.orchestra.run.smithi163.stdout: "root_rfiles": 4, 2024-05-08T06:14:13.954 INFO:teuthology.orchestra.run.smithi163.stdout: "root_rbytes": 234, 2024-05-08T06:14:13.954 INFO:teuthology.orchestra.run.smithi163.stdout: "root_rsnaps": 0, 2024-05-08T06:14:13.954 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_backtrace_fetch": 64, 2024-05-08T06:14:13.954 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_set_tag": 0, 2024-05-08T06:14:13.954 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T06:14:13.954 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_inotable_repaired": 0, 2024-05-08T06:14:13.954 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_dir_inodes": 42, 2024-05-08T06:14:13.954 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_dir_base_inodes": 5, 2024-05-08T06:14:13.954 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_dirfrag_rstats": 37, 2024-05-08T06:14:13.955 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_file_inodes": 22, 2024-05-08T06:14:13.955 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_inode_file_caps": 0, 2024-05-08T06:14:13.955 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_revoke": 21, 2024-05-08T06:14:13.955 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_grant": 3640, 2024-05-08T06:14:13.955 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T06:14:13.955 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T06:14:13.955 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T06:14:13.955 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_caps": 2438, 2024-05-08T06:14:13.955 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_caps_dirty": 1212, 2024-05-08T06:14:13.955 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_cap_release": 15, 2024-05-08T06:14:13.955 INFO:teuthology.orchestra.run.smithi163.stdout: "process_request_cap_release": 1240 2024-05-08T06:14:13.955 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.955 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_cache": { 2024-05-08T06:14:13.955 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_update": 11, 2024-05-08T06:14:13.956 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_update_receipt": 0, 2024-05-08T06:14:13.956 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_try_discover": 0, 2024-05-08T06:14:13.956 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_send_discover": 0, 2024-05-08T06:14:13.956 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_handle_discover": 20, 2024-05-08T06:14:13.956 INFO:teuthology.orchestra.run.smithi163.stdout: "num_strays": 0, 2024-05-08T06:14:13.956 INFO:teuthology.orchestra.run.smithi163.stdout: "num_strays_delayed": 0, 2024-05-08T06:14:13.956 INFO:teuthology.orchestra.run.smithi163.stdout: "num_strays_enqueuing": 0, 2024-05-08T06:14:13.956 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_created": 1212, 2024-05-08T06:14:13.956 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_enqueued": 1212, 2024-05-08T06:14:13.956 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_reintegrated": 0, 2024-05-08T06:14:13.956 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_migrated": 0, 2024-05-08T06:14:13.956 INFO:teuthology.orchestra.run.smithi163.stdout: "num_recovering_processing": 0, 2024-05-08T06:14:13.956 INFO:teuthology.orchestra.run.smithi163.stdout: "num_recovering_enqueued": 0, 2024-05-08T06:14:13.956 INFO:teuthology.orchestra.run.smithi163.stdout: "num_recovering_prioritized": 0, 2024-05-08T06:14:13.956 INFO:teuthology.orchestra.run.smithi163.stdout: "recovery_started": 0, 2024-05-08T06:14:13.957 INFO:teuthology.orchestra.run.smithi163.stdout: "recovery_completed": 0, 2024-05-08T06:14:13.957 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_quiesce_path": 0, 2024-05-08T06:14:13.957 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_quiesce_inode": 0, 2024-05-08T06:14:13.957 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_enqueue_scrub": 5, 2024-05-08T06:14:13.957 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_exportdir": 0, 2024-05-08T06:14:13.957 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_flush": 0, 2024-05-08T06:14:13.957 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_fragmentdir": 0, 2024-05-08T06:14:13.957 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_fragstats": 0, 2024-05-08T06:14:13.957 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_inodestats": 0 2024-05-08T06:14:13.957 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.957 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_log": { 2024-05-08T06:14:13.957 INFO:teuthology.orchestra.run.smithi163.stdout: "evlrg": 0, 2024-05-08T06:14:13.957 INFO:teuthology.orchestra.run.smithi163.stdout: "evadd": 6142, 2024-05-08T06:14:13.957 INFO:teuthology.orchestra.run.smithi163.stdout: "evex": 4738, 2024-05-08T06:14:13.958 INFO:teuthology.orchestra.run.smithi163.stdout: "evtrm": 4738, 2024-05-08T06:14:13.958 INFO:teuthology.orchestra.run.smithi163.stdout: "ev": 1404, 2024-05-08T06:14:13.958 INFO:teuthology.orchestra.run.smithi163.stdout: "evexg": 0, 2024-05-08T06:14:13.958 INFO:teuthology.orchestra.run.smithi163.stdout: "evexd": 2339, 2024-05-08T06:14:13.958 INFO:teuthology.orchestra.run.smithi163.stdout: "segadd": 19, 2024-05-08T06:14:13.958 INFO:teuthology.orchestra.run.smithi163.stdout: "segex": 16, 2024-05-08T06:14:13.958 INFO:teuthology.orchestra.run.smithi163.stdout: "segtrm": 16, 2024-05-08T06:14:13.958 INFO:teuthology.orchestra.run.smithi163.stdout: "seg": 3, 2024-05-08T06:14:13.958 INFO:teuthology.orchestra.run.smithi163.stdout: "segmjr": 1, 2024-05-08T06:14:13.958 INFO:teuthology.orchestra.run.smithi163.stdout: "segexg": 0, 2024-05-08T06:14:13.958 INFO:teuthology.orchestra.run.smithi163.stdout: "segexd": 3, 2024-05-08T06:14:13.958 INFO:teuthology.orchestra.run.smithi163.stdout: "expos": 14232263, 2024-05-08T06:14:13.958 INFO:teuthology.orchestra.run.smithi163.stdout: "wrpos": 16911917, 2024-05-08T06:14:13.958 INFO:teuthology.orchestra.run.smithi163.stdout: "rdpos": 0, 2024-05-08T06:14:13.959 INFO:teuthology.orchestra.run.smithi163.stdout: "jlat": { 2024-05-08T06:14:13.959 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 2468, 2024-05-08T06:14:13.959 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 4.039212570, 2024-05-08T06:14:13.959 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.001636633 2024-05-08T06:14:13.959 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.959 INFO:teuthology.orchestra.run.smithi163.stdout: "replayed": 0 2024-05-08T06:14:13.960 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.960 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_mem": { 2024-05-08T06:14:13.960 INFO:teuthology.orchestra.run.smithi163.stdout: "ino": 23, 2024-05-08T06:14:13.960 INFO:teuthology.orchestra.run.smithi163.stdout: "ino+": 1240, 2024-05-08T06:14:13.960 INFO:teuthology.orchestra.run.smithi163.stdout: "ino-": 1217, 2024-05-08T06:14:13.960 INFO:teuthology.orchestra.run.smithi163.stdout: "dir": 18, 2024-05-08T06:14:13.960 INFO:teuthology.orchestra.run.smithi163.stdout: "dir+": 24, 2024-05-08T06:14:13.960 INFO:teuthology.orchestra.run.smithi163.stdout: "dir-": 6, 2024-05-08T06:14:13.960 INFO:teuthology.orchestra.run.smithi163.stdout: "dn": 172, 2024-05-08T06:14:13.960 INFO:teuthology.orchestra.run.smithi163.stdout: "dn+": 1243, 2024-05-08T06:14:13.960 INFO:teuthology.orchestra.run.smithi163.stdout: "dn-": 1071, 2024-05-08T06:14:13.961 INFO:teuthology.orchestra.run.smithi163.stdout: "cap": 2, 2024-05-08T06:14:13.961 INFO:teuthology.orchestra.run.smithi163.stdout: "cap+": 1226, 2024-05-08T06:14:13.961 INFO:teuthology.orchestra.run.smithi163.stdout: "cap-": 1224, 2024-05-08T06:14:13.961 INFO:teuthology.orchestra.run.smithi163.stdout: "rss": 60012, 2024-05-08T06:14:13.961 INFO:teuthology.orchestra.run.smithi163.stdout: "heap": 231800 2024-05-08T06:14:13.961 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.961 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_server": { 2024-05-08T06:14:13.961 INFO:teuthology.orchestra.run.smithi163.stdout: "dispatch_client_request": 2524, 2024-05-08T06:14:13.961 INFO:teuthology.orchestra.run.smithi163.stdout: "dispatch_server_request": 1, 2024-05-08T06:14:13.961 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_request": 2469, 2024-05-08T06:14:13.961 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_session": 15, 2024-05-08T06:14:13.961 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_peer_request": 2, 2024-05-08T06:14:13.961 INFO:teuthology.orchestra.run.smithi163.stdout: "req_create_latency": { 2024-05-08T06:14:13.961 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 1210, 2024-05-08T06:14:13.961 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 1.113650086, 2024-05-08T06:14:13.962 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000920371 2024-05-08T06:14:13.962 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.962 INFO:teuthology.orchestra.run.smithi163.stdout: "req_getattr_latency": { 2024-05-08T06:14:13.962 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 16, 2024-05-08T06:14:13.962 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.004284252, 2024-05-08T06:14:13.962 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000267765 2024-05-08T06:14:13.962 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.962 INFO:teuthology.orchestra.run.smithi163.stdout: "req_getfilelock_latency": { 2024-05-08T06:14:13.962 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:13.962 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:13.962 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:13.963 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.963 INFO:teuthology.orchestra.run.smithi163.stdout: "req_link_latency": { 2024-05-08T06:14:13.963 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:13.963 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:13.963 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:13.963 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.964 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookup_latency": { 2024-05-08T06:14:13.964 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:13.964 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:13.964 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:13.964 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.964 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookuphash_latency": { 2024-05-08T06:14:13.964 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:13.964 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:13.964 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:13.964 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.964 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupino_latency": { 2024-05-08T06:14:13.964 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 1, 2024-05-08T06:14:13.965 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000269825, 2024-05-08T06:14:13.965 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000269825 2024-05-08T06:14:13.965 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.965 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupname_latency": { 2024-05-08T06:14:13.965 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:13.965 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:13.965 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:13.965 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.965 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupparent_latency": { 2024-05-08T06:14:13.965 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:13.965 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:13.965 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:13.965 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.965 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupsnap_latency": { 2024-05-08T06:14:13.966 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:13.966 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:13.966 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:13.966 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.966 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lssnap_latency": { 2024-05-08T06:14:13.966 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:13.967 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:13.967 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:13.967 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.967 INFO:teuthology.orchestra.run.smithi163.stdout: "req_mkdir_latency": { 2024-05-08T06:14:13.967 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 12, 2024-05-08T06:14:13.967 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.018510952, 2024-05-08T06:14:13.967 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.001542579 2024-05-08T06:14:13.967 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.967 INFO:teuthology.orchestra.run.smithi163.stdout: "req_mknod_latency": { 2024-05-08T06:14:13.967 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:13.967 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:13.967 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:13.967 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.968 INFO:teuthology.orchestra.run.smithi163.stdout: "req_mksnap_latency": { 2024-05-08T06:14:13.968 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:13.968 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:13.968 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:13.968 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.968 INFO:teuthology.orchestra.run.smithi163.stdout: "req_open_latency": { 2024-05-08T06:14:13.968 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 1, 2024-05-08T06:14:13.968 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000242809, 2024-05-08T06:14:13.968 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000242809 2024-05-08T06:14:13.968 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.968 INFO:teuthology.orchestra.run.smithi163.stdout: "req_readdir_latency": { 2024-05-08T06:14:13.968 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 6, 2024-05-08T06:14:13.968 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.009412438, 2024-05-08T06:14:13.968 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.001568739 2024-05-08T06:14:13.968 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.969 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rename_latency": { 2024-05-08T06:14:13.969 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 2, 2024-05-08T06:14:13.969 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.002332149, 2024-05-08T06:14:13.969 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.001166074 2024-05-08T06:14:13.969 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.969 INFO:teuthology.orchestra.run.smithi163.stdout: "req_renamesnap_latency": { 2024-05-08T06:14:13.969 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:13.969 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:13.969 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:13.969 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.969 INFO:teuthology.orchestra.run.smithi163.stdout: "req_snapdiff_latency": { 2024-05-08T06:14:13.969 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:13.969 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:13.970 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:13.970 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.970 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rmdir_latency": { 2024-05-08T06:14:13.970 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 6, 2024-05-08T06:14:13.970 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.038468266, 2024-05-08T06:14:13.970 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.006411377 2024-05-08T06:14:13.970 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.970 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rmsnap_latency": { 2024-05-08T06:14:13.970 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:13.970 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:13.970 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:13.970 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.971 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rmxattr_latency": { 2024-05-08T06:14:13.971 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:13.971 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:13.971 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:13.971 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.971 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setattr_latency": { 2024-05-08T06:14:13.971 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 1, 2024-05-08T06:14:13.971 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.001076245, 2024-05-08T06:14:13.971 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.001076245 2024-05-08T06:14:13.971 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.971 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setdirlayout_latency": { 2024-05-08T06:14:13.971 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:13.972 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:13.972 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:13.972 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.972 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setfilelock_latency": { 2024-05-08T06:14:13.972 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:13.972 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:13.972 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:13.972 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.972 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setlayout_latency": { 2024-05-08T06:14:13.973 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:13.973 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:13.973 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:13.973 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.973 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setxattr_latency": { 2024-05-08T06:14:13.973 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 8, 2024-05-08T06:14:13.973 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.020094729, 2024-05-08T06:14:13.973 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.002511841 2024-05-08T06:14:13.973 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.973 INFO:teuthology.orchestra.run.smithi163.stdout: "req_symlink_latency": { 2024-05-08T06:14:13.973 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:13.973 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:13.973 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:13.973 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.974 INFO:teuthology.orchestra.run.smithi163.stdout: "req_unlink_latency": { 2024-05-08T06:14:13.974 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 1206, 2024-05-08T06:14:13.974 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 1.744653625, 2024-05-08T06:14:13.974 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.001446644 2024-05-08T06:14:13.974 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.974 INFO:teuthology.orchestra.run.smithi163.stdout: "cap_revoke_eviction": 0, 2024-05-08T06:14:13.974 INFO:teuthology.orchestra.run.smithi163.stdout: "cap_acquisition_throttle": 0, 2024-05-08T06:14:13.974 INFO:teuthology.orchestra.run.smithi163.stdout: "req_getvxattr_latency": { 2024-05-08T06:14:13.975 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:13.975 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:13.975 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:13.975 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:14:13.975 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.975 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_sessions": { 2024-05-08T06:14:13.976 INFO:teuthology.orchestra.run.smithi163.stdout: "session_count": 1, 2024-05-08T06:14:13.976 INFO:teuthology.orchestra.run.smithi163.stdout: "session_add": 3, 2024-05-08T06:14:13.976 INFO:teuthology.orchestra.run.smithi163.stdout: "session_remove": 2, 2024-05-08T06:14:13.976 INFO:teuthology.orchestra.run.smithi163.stdout: "sessions_open": 1, 2024-05-08T06:14:13.976 INFO:teuthology.orchestra.run.smithi163.stdout: "sessions_stale": 0, 2024-05-08T06:14:13.976 INFO:teuthology.orchestra.run.smithi163.stdout: "total_load": 2020, 2024-05-08T06:14:13.977 INFO:teuthology.orchestra.run.smithi163.stdout: "average_load": 2020, 2024-05-08T06:14:13.977 INFO:teuthology.orchestra.run.smithi163.stdout: "avg_session_uptime": 238, 2024-05-08T06:14:13.977 INFO:teuthology.orchestra.run.smithi163.stdout: "mdthresh_evicted": 0 2024-05-08T06:14:13.977 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.977 INFO:teuthology.orchestra.run.smithi163.stdout: "mempool": { 2024-05-08T06:14:13.978 INFO:teuthology.orchestra.run.smithi163.stdout: "bloom_filter_bytes": 0, 2024-05-08T06:14:13.978 INFO:teuthology.orchestra.run.smithi163.stdout: "bloom_filter_items": 0, 2024-05-08T06:14:13.978 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T06:14:13.978 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_alloc_items": 0, 2024-05-08T06:14:13.978 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T06:14:13.978 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_data_items": 0, 2024-05-08T06:14:13.979 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T06:14:13.979 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T06:14:13.979 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T06:14:13.979 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T06:14:13.979 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T06:14:13.980 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_other_items": 0, 2024-05-08T06:14:13.980 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T06:14:13.980 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T06:14:13.980 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_extent_bytes": 0, 2024-05-08T06:14:13.980 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_extent_items": 0, 2024-05-08T06:14:13.980 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_blob_bytes": 0, 2024-05-08T06:14:13.981 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_blob_items": 0, 2024-05-08T06:14:13.981 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T06:14:13.981 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T06:14:13.981 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T06:14:13.981 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T06:14:13.981 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T06:14:13.982 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_fsck_items": 0, 2024-05-08T06:14:13.982 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_txc_bytes": 0, 2024-05-08T06:14:13.982 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_txc_items": 0, 2024-05-08T06:14:13.982 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T06:14:13.982 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T06:14:13.982 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_bytes": 0, 2024-05-08T06:14:13.982 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_items": 0, 2024-05-08T06:14:13.983 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_bytes": 0, 2024-05-08T06:14:13.983 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_items": 0, 2024-05-08T06:14:13.983 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T06:14:13.983 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_reader_items": 0, 2024-05-08T06:14:13.983 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T06:14:13.983 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_writer_items": 0, 2024-05-08T06:14:13.983 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_anon_bytes": 300215, 2024-05-08T06:14:13.984 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_anon_items": 129, 2024-05-08T06:14:13.984 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_meta_bytes": 704, 2024-05-08T06:14:13.984 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_meta_items": 8, 2024-05-08T06:14:13.984 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_bytes": 0, 2024-05-08T06:14:13.984 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_items": 0, 2024-05-08T06:14:13.984 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_mapbl_bytes": 0, 2024-05-08T06:14:13.984 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_mapbl_items": 0, 2024-05-08T06:14:13.985 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_pglog_bytes": 0, 2024-05-08T06:14:13.985 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_pglog_items": 0, 2024-05-08T06:14:13.985 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_bytes": 7528, 2024-05-08T06:14:13.985 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_items": 158, 2024-05-08T06:14:13.985 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T06:14:13.985 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_mapping_items": 0, 2024-05-08T06:14:13.985 INFO:teuthology.orchestra.run.smithi163.stdout: "pgmap_bytes": 0, 2024-05-08T06:14:13.985 INFO:teuthology.orchestra.run.smithi163.stdout: "pgmap_items": 0, 2024-05-08T06:14:13.985 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_co_bytes": 205154, 2024-05-08T06:14:13.986 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_co_items": 1886, 2024-05-08T06:14:13.986 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_1_bytes": 0, 2024-05-08T06:14:13.986 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_1_items": 0, 2024-05-08T06:14:13.986 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_2_bytes": 0, 2024-05-08T06:14:13.986 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_2_items": 0 2024-05-08T06:14:13.986 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.986 INFO:teuthology.orchestra.run.smithi163.stdout: "objecter": { 2024-05-08T06:14:13.987 INFO:teuthology.orchestra.run.smithi163.stdout: "op_active": 0, 2024-05-08T06:14:13.987 INFO:teuthology.orchestra.run.smithi163.stdout: "op_laggy": 0, 2024-05-08T06:14:13.987 INFO:teuthology.orchestra.run.smithi163.stdout: "op_send": 7051, 2024-05-08T06:14:13.987 INFO:teuthology.orchestra.run.smithi163.stdout: "op_send_bytes": 13177468, 2024-05-08T06:14:13.987 INFO:teuthology.orchestra.run.smithi163.stdout: "op_resend": 32, 2024-05-08T06:14:13.987 INFO:teuthology.orchestra.run.smithi163.stdout: "op_reply": 7051, 2024-05-08T06:14:13.987 INFO:teuthology.orchestra.run.smithi163.stdout: "op_latency": { 2024-05-08T06:14:13.988 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 7051, 2024-05-08T06:14:13.988 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 40.005095481, 2024-05-08T06:14:13.988 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.005673676 2024-05-08T06:14:13.988 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.988 INFO:teuthology.orchestra.run.smithi163.stdout: "op_inflight": 0, 2024-05-08T06:14:13.988 INFO:teuthology.orchestra.run.smithi163.stdout: "oplen_avg": { 2024-05-08T06:14:13.989 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 7051, 2024-05-08T06:14:13.989 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 7674 2024-05-08T06:14:13.989 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.989 INFO:teuthology.orchestra.run.smithi163.stdout: "op": 7051, 2024-05-08T06:14:13.989 INFO:teuthology.orchestra.run.smithi163.stdout: "op_r": 104, 2024-05-08T06:14:13.989 INFO:teuthology.orchestra.run.smithi163.stdout: "op_w": 6947, 2024-05-08T06:14:13.990 INFO:teuthology.orchestra.run.smithi163.stdout: "op_rmw": 0, 2024-05-08T06:14:13.990 INFO:teuthology.orchestra.run.smithi163.stdout: "op_pg": 0, 2024-05-08T06:14:13.990 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_stat": 48, 2024-05-08T06:14:13.990 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_create": 220, 2024-05-08T06:14:13.990 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_read": 35, 2024-05-08T06:14:13.991 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_write": 2498, 2024-05-08T06:14:13.991 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_writefull": 66, 2024-05-08T06:14:13.991 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_writesame": 0, 2024-05-08T06:14:13.991 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_append": 0, 2024-05-08T06:14:13.991 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_zero": 0, 2024-05-08T06:14:13.991 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_truncate": 0, 2024-05-08T06:14:13.991 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_delete": 4015, 2024-05-08T06:14:13.991 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_mapext": 0, 2024-05-08T06:14:13.992 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_sparse_read": 0, 2024-05-08T06:14:13.992 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_clonerange": 0, 2024-05-08T06:14:13.992 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_getxattr": 69, 2024-05-08T06:14:13.992 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_setxattr": 440, 2024-05-08T06:14:13.992 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_cmpxattr": 0, 2024-05-08T06:14:13.992 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_rmxattr": 0, 2024-05-08T06:14:13.992 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_resetxattrs": 0, 2024-05-08T06:14:13.992 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_call": 0, 2024-05-08T06:14:13.992 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_watch": 0, 2024-05-08T06:14:13.993 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_notify": 0, 2024-05-08T06:14:13.993 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T06:14:13.993 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_pgls": 0, 2024-05-08T06:14:13.993 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_pgls_filter": 0, 2024-05-08T06:14:13.993 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_other": 0, 2024-05-08T06:14:13.993 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_active": 0, 2024-05-08T06:14:13.993 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_send": 0, 2024-05-08T06:14:13.993 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_resend": 0, 2024-05-08T06:14:13.994 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_ping": 0, 2024-05-08T06:14:13.994 INFO:teuthology.orchestra.run.smithi163.stdout: "poolop_active": 0, 2024-05-08T06:14:13.994 INFO:teuthology.orchestra.run.smithi163.stdout: "poolop_send": 0, 2024-05-08T06:14:13.994 INFO:teuthology.orchestra.run.smithi163.stdout: "poolop_resend": 0, 2024-05-08T06:14:13.994 INFO:teuthology.orchestra.run.smithi163.stdout: "poolstat_active": 0, 2024-05-08T06:14:13.994 INFO:teuthology.orchestra.run.smithi163.stdout: "poolstat_send": 0, 2024-05-08T06:14:13.994 INFO:teuthology.orchestra.run.smithi163.stdout: "poolstat_resend": 0, 2024-05-08T06:14:13.995 INFO:teuthology.orchestra.run.smithi163.stdout: "statfs_active": 0, 2024-05-08T06:14:13.995 INFO:teuthology.orchestra.run.smithi163.stdout: "statfs_send": 0, 2024-05-08T06:14:13.995 INFO:teuthology.orchestra.run.smithi163.stdout: "statfs_resend": 0, 2024-05-08T06:14:13.995 INFO:teuthology.orchestra.run.smithi163.stdout: "command_active": 0, 2024-05-08T06:14:13.995 INFO:teuthology.orchestra.run.smithi163.stdout: "command_send": 0, 2024-05-08T06:14:13.995 INFO:teuthology.orchestra.run.smithi163.stdout: "command_resend": 0, 2024-05-08T06:14:13.995 INFO:teuthology.orchestra.run.smithi163.stdout: "map_epoch": 82, 2024-05-08T06:14:13.996 INFO:teuthology.orchestra.run.smithi163.stdout: "map_full": 0, 2024-05-08T06:14:13.996 INFO:teuthology.orchestra.run.smithi163.stdout: "map_inc": 2, 2024-05-08T06:14:13.996 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_sessions": 12, 2024-05-08T06:14:13.996 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_session_open": 12, 2024-05-08T06:14:13.996 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_session_close": 0, 2024-05-08T06:14:13.996 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_laggy": 0, 2024-05-08T06:14:13.997 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_wr": 255, 2024-05-08T06:14:13.997 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_rd": 0, 2024-05-08T06:14:13.997 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_del": 28 2024-05-08T06:14:13.997 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.997 INFO:teuthology.orchestra.run.smithi163.stdout: "oft": { 2024-05-08T06:14:13.997 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_objs": 1, 2024-05-08T06:14:13.997 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_kv_pairs": 0, 2024-05-08T06:14:13.997 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_updates": 0, 2024-05-08T06:14:13.997 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_removes": 0 2024-05-08T06:14:13.997 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.997 INFO:teuthology.orchestra.run.smithi163.stdout: "osd-slow-ops": { 2024-05-08T06:14:13.998 INFO:teuthology.orchestra.run.smithi163.stdout: "slow_ops_count": 0 2024-05-08T06:14:13.998 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.998 INFO:teuthology.orchestra.run.smithi163.stdout: "purge_queue": { 2024-05-08T06:14:13.998 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing_ops": 0, 2024-05-08T06:14:13.998 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing_ops_high_water": 12, 2024-05-08T06:14:13.998 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing": 0, 2024-05-08T06:14:13.998 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing_high_water": 6, 2024-05-08T06:14:13.998 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executed_ops": 3498, 2024-05-08T06:14:13.998 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executed": 1212, 2024-05-08T06:14:13.998 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_item_in_journal": 0 2024-05-08T06:14:13.998 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:13.998 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T06:14:13.998 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-05-08T06:14:13.999 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 104857600, 2024-05-08T06:14:13.999 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-05-08T06:14:13.999 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 13438, 2024-05-08T06:14:13.999 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 3696509, 2024-05-08T06:14:13.999 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-05-08T06:14:13.999 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 13438, 2024-05-08T06:14:13.999 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2024-05-08T06:14:13.999 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2024-05-08T06:14:13.999 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 13438, 2024-05-08T06:14:13.999 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 3696509, 2024-05-08T06:14:13.999 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-05-08T06:14:13.999 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:13.999 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:13.999 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:13.999 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:14:14.000 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:14.000 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-objecter_bytes": { 2024-05-08T06:14:14.000 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-05-08T06:14:14.000 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 104857600, 2024-05-08T06:14:14.000 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-05-08T06:14:14.000 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2024-05-08T06:14:14.000 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2024-05-08T06:14:14.000 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-05-08T06:14:14.000 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2024-05-08T06:14:14.000 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 7051, 2024-05-08T06:14:14.000 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 13299880, 2024-05-08T06:14:14.000 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 7051, 2024-05-08T06:14:14.000 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 13299880, 2024-05-08T06:14:14.000 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-05-08T06:14:14.001 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:14.001 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:14.001 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:14.001 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:14:14.001 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:14.001 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-objecter_ops": { 2024-05-08T06:14:14.001 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-05-08T06:14:14.001 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 1024, 2024-05-08T06:14:14.001 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-05-08T06:14:14.001 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2024-05-08T06:14:14.001 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2024-05-08T06:14:14.001 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-05-08T06:14:14.001 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2024-05-08T06:14:14.001 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 7051, 2024-05-08T06:14:14.001 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 7051, 2024-05-08T06:14:14.002 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 7051, 2024-05-08T06:14:14.002 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 7051, 2024-05-08T06:14:14.002 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-05-08T06:14:14.002 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:14.002 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:14.002 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:14.002 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:14:14.002 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:14.002 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-write_buf_throttle": { 2024-05-08T06:14:14.002 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-05-08T06:14:14.002 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 3758096384, 2024-05-08T06:14:14.002 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-05-08T06:14:14.002 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 1212, 2024-05-08T06:14:14.002 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 122412, 2024-05-08T06:14:14.003 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-05-08T06:14:14.003 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 1212, 2024-05-08T06:14:14.003 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2024-05-08T06:14:14.003 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2024-05-08T06:14:14.003 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 30, 2024-05-08T06:14:14.003 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 122412, 2024-05-08T06:14:14.003 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-05-08T06:14:14.003 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:14.003 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:14.003 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:14.003 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:14:14.003 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:14.003 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-write_buf_throttle-0x563f07bd0500": { 2024-05-08T06:14:14.003 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-05-08T06:14:14.003 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 3758096384, 2024-05-08T06:14:14.003 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-05-08T06:14:14.004 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 6142, 2024-05-08T06:14:14.004 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 12717613, 2024-05-08T06:14:14.004 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-05-08T06:14:14.004 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 6142, 2024-05-08T06:14:14.004 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2024-05-08T06:14:14.004 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2024-05-08T06:14:14.004 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 2468, 2024-05-08T06:14:14.004 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 12717613, 2024-05-08T06:14:14.004 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-05-08T06:14:14.004 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:14.004 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:14.004 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:14.004 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:14:14.004 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:14:14.004 INFO:teuthology.orchestra.run.smithi163.stdout:} 2024-05-08T06:14:14.833 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-05-08T06:14:14.833 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-05-08T06:14:14.833 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=11 2024-05-08T06:14:14.833 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-05-08T06:14:14.833 DEBUG:tasks.check_counter:Getting stats from e 2024-05-08T06:14:14.833 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-08T06:14:15.082 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T06:14:15.117 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:14 smithi163 ceph-mon[34520]: pgmap v573: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:14:15.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:14 smithi123 ceph-mon[29787]: pgmap v573: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:14:15.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:14 smithi192 ceph-mon[34664]: pgmap v573: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:14:16.736 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2024-05-08T06:14:16.737 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T06:14:16.737 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 0, 2024-05-08T06:14:16.737 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 2, 2024-05-08T06:14:16.737 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 0, 2024-05-08T06:14:16.737 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 0, 2024-05-08T06:14:16.737 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 4, 2024-05-08T06:14:16.737 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 0, 2024-05-08T06:14:16.737 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.000991189, 2024-05-08T06:14:16.737 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.000000000, 2024-05-08T06:14:16.737 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.000155119, 2024-05-08T06:14:16.737 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:14:16.737 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:14:16.738 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:16.738 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:16.738 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:16.738 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:16.738 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:14:16.738 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:16.738 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:16.738 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:16.738 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:16.738 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T06:14:16.738 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T06:14:16.738 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:16.738 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T06:14:16.738 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 5, 2024-05-08T06:14:16.739 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 117, 2024-05-08T06:14:16.739 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 3200, 2024-05-08T06:14:16.739 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 178256, 2024-05-08T06:14:16.739 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 4, 2024-05-08T06:14:16.739 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 2, 2024-05-08T06:14:16.739 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.008205800, 2024-05-08T06:14:16.739 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.005352258, 2024-05-08T06:14:16.739 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.001449743, 2024-05-08T06:14:16.739 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:14:16.739 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:14:16.739 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 116, 2024-05-08T06:14:16.739 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.010149222, 2024-05-08T06:14:16.739 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000087493 2024-05-08T06:14:16.739 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:16.739 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:14:16.740 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:16.740 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:16.740 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:16.740 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:16.740 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-05-08T06:14:16.740 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 178256 2024-05-08T06:14:16.740 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:16.740 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T06:14:16.740 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 155, 2024-05-08T06:14:16.740 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 150, 2024-05-08T06:14:16.740 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 794688, 2024-05-08T06:14:16.740 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 91568, 2024-05-08T06:14:16.740 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 2, 2024-05-08T06:14:16.740 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 1, 2024-05-08T06:14:16.740 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.030043238, 2024-05-08T06:14:16.741 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.010220441, 2024-05-08T06:14:16.741 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.019289397, 2024-05-08T06:14:16.741 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.004132063, 2024-05-08T06:14:16.741 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:14:16.741 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 149, 2024-05-08T06:14:16.741 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.008922672, 2024-05-08T06:14:16.741 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000059883 2024-05-08T06:14:16.741 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:16.741 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:14:16.741 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:16.741 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:16.741 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:16.741 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:16.741 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 794688, 2024-05-08T06:14:16.741 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 91568 2024-05-08T06:14:16.742 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:16.742 INFO:teuthology.orchestra.run.smithi163.stdout: "cct": { 2024-05-08T06:14:16.742 INFO:teuthology.orchestra.run.smithi163.stdout: "total_workers": 0, 2024-05-08T06:14:16.742 INFO:teuthology.orchestra.run.smithi163.stdout: "unhealthy_workers": 0 2024-05-08T06:14:16.742 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:16.742 INFO:teuthology.orchestra.run.smithi163.stdout: "mempool": { 2024-05-08T06:14:16.742 INFO:teuthology.orchestra.run.smithi163.stdout: "bloom_filter_bytes": 0, 2024-05-08T06:14:16.742 INFO:teuthology.orchestra.run.smithi163.stdout: "bloom_filter_items": 0, 2024-05-08T06:14:16.742 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T06:14:16.742 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_alloc_items": 0, 2024-05-08T06:14:16.742 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T06:14:16.742 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_data_items": 0, 2024-05-08T06:14:16.742 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T06:14:16.742 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T06:14:16.743 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T06:14:16.743 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T06:14:16.743 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T06:14:16.743 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_other_items": 0, 2024-05-08T06:14:16.743 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T06:14:16.743 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T06:14:16.743 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_extent_bytes": 0, 2024-05-08T06:14:16.743 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_extent_items": 0, 2024-05-08T06:14:16.743 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_blob_bytes": 0, 2024-05-08T06:14:16.743 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_blob_items": 0, 2024-05-08T06:14:16.743 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T06:14:16.743 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T06:14:16.743 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T06:14:16.743 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T06:14:16.743 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T06:14:16.744 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_fsck_items": 0, 2024-05-08T06:14:16.744 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_txc_bytes": 0, 2024-05-08T06:14:16.744 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_txc_items": 0, 2024-05-08T06:14:16.744 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T06:14:16.744 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T06:14:16.744 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_bytes": 0, 2024-05-08T06:14:16.744 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_items": 0, 2024-05-08T06:14:16.744 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_bytes": 0, 2024-05-08T06:14:16.744 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_items": 0, 2024-05-08T06:14:16.744 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T06:14:16.744 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_reader_items": 0, 2024-05-08T06:14:16.744 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T06:14:16.744 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_writer_items": 0, 2024-05-08T06:14:16.744 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_anon_bytes": 148945, 2024-05-08T06:14:16.744 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_anon_items": 29, 2024-05-08T06:14:16.745 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_meta_bytes": 0, 2024-05-08T06:14:16.745 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_meta_items": 0, 2024-05-08T06:14:16.745 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_bytes": 0, 2024-05-08T06:14:16.745 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_items": 0, 2024-05-08T06:14:16.745 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_mapbl_bytes": 0, 2024-05-08T06:14:16.745 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_mapbl_items": 0, 2024-05-08T06:14:16.745 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_pglog_bytes": 0, 2024-05-08T06:14:16.745 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_pglog_items": 0, 2024-05-08T06:14:16.745 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_bytes": 0, 2024-05-08T06:14:16.745 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_items": 0, 2024-05-08T06:14:16.745 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T06:14:16.745 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_mapping_items": 0, 2024-05-08T06:14:16.745 INFO:teuthology.orchestra.run.smithi163.stdout: "pgmap_bytes": 0, 2024-05-08T06:14:16.745 INFO:teuthology.orchestra.run.smithi163.stdout: "pgmap_items": 0, 2024-05-08T06:14:16.745 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_co_bytes": 928, 2024-05-08T06:14:16.745 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_co_items": 2, 2024-05-08T06:14:16.746 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_1_bytes": 0, 2024-05-08T06:14:16.746 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_1_items": 0, 2024-05-08T06:14:16.746 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_2_bytes": 0, 2024-05-08T06:14:16.746 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_2_items": 0 2024-05-08T06:14:16.746 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:16.746 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T06:14:16.746 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-05-08T06:14:16.746 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 104857600, 2024-05-08T06:14:16.746 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-05-08T06:14:16.746 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 156, 2024-05-08T06:14:16.746 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 772811, 2024-05-08T06:14:16.746 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-05-08T06:14:16.746 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 156, 2024-05-08T06:14:16.746 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2024-05-08T06:14:16.746 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2024-05-08T06:14:16.747 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 156, 2024-05-08T06:14:16.747 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 772811, 2024-05-08T06:14:16.747 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-05-08T06:14:16.747 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:16.747 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:16.747 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:16.747 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:14:16.747 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:14:16.747 INFO:teuthology.orchestra.run.smithi163.stdout:} 2024-05-08T06:14:17.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:16 smithi163 ceph-mon[34520]: pgmap v574: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T06:14:17.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:16 smithi123 ceph-mon[29787]: pgmap v574: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T06:14:17.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:16 smithi192 ceph-mon[34664]: pgmap v574: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T06:14:17.390 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2024-05-08T06:14:17.390 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2024-05-08T06:14:17.390 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.e 2024-05-08T06:14:17.390 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.e 2024-05-08T06:14:17.390 DEBUG:tasks.check_counter:Getting stats from h 2024-05-08T06:14:17.390 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-08T06:14:17.628 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T06:14:19.175 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:18 smithi163 ceph-mon[34520]: pgmap v575: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:14:19.201 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2024-05-08T06:14:19.201 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T06:14:19.201 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 1136, 2024-05-08T06:14:19.201 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 1129, 2024-05-08T06:14:19.201 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 5300674, 2024-05-08T06:14:19.201 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 401372, 2024-05-08T06:14:19.201 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 5, 2024-05-08T06:14:19.201 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 1, 2024-05-08T06:14:19.202 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.115897577, 2024-05-08T06:14:19.202 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.043242729, 2024-05-08T06:14:19.202 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.038668721, 2024-05-08T06:14:19.202 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.028990059, 2024-05-08T06:14:19.202 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:14:19.202 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 1128, 2024-05-08T06:14:19.202 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.044774084, 2024-05-08T06:14:19.202 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000039693 2024-05-08T06:14:19.202 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.202 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:14:19.202 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.202 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.202 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.202 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.203 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 817344, 2024-05-08T06:14:19.203 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 92272 2024-05-08T06:14:19.203 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.203 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T06:14:19.203 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 143, 2024-05-08T06:14:19.203 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 252, 2024-05-08T06:14:19.203 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 4368473, 2024-05-08T06:14:19.203 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 307134, 2024-05-08T06:14:19.203 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 5, 2024-05-08T06:14:19.203 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 4, 2024-05-08T06:14:19.203 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.019245525, 2024-05-08T06:14:19.203 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.007292548, 2024-05-08T06:14:19.203 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.006592019, 2024-05-08T06:14:19.203 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.003426346, 2024-05-08T06:14:19.203 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:14:19.204 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 251, 2024-05-08T06:14:19.204 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.016377295, 2024-05-08T06:14:19.204 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000065248 2024-05-08T06:14:19.204 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.204 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:14:19.204 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.204 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.204 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.204 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.204 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-05-08T06:14:19.204 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 263184 2024-05-08T06:14:19.204 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.204 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T06:14:19.204 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 400, 2024-05-08T06:14:19.204 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 402, 2024-05-08T06:14:19.205 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 4293034, 2024-05-08T06:14:19.205 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 127207, 2024-05-08T06:14:19.205 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 4, 2024-05-08T06:14:19.205 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 3, 2024-05-08T06:14:19.205 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.035430927, 2024-05-08T06:14:19.205 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.009639188, 2024-05-08T06:14:19.205 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.010089607, 2024-05-08T06:14:19.205 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.012762339, 2024-05-08T06:14:19.205 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:14:19.205 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 400, 2024-05-08T06:14:19.205 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.011770172, 2024-05-08T06:14:19.205 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000029425 2024-05-08T06:14:19.205 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.205 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:14:19.206 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.206 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.206 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.206 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.206 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T06:14:19.206 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T06:14:19.206 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.206 INFO:teuthology.orchestra.run.smithi163.stdout: "cct": { 2024-05-08T06:14:19.206 INFO:teuthology.orchestra.run.smithi163.stdout: "total_workers": 1, 2024-05-08T06:14:19.206 INFO:teuthology.orchestra.run.smithi163.stdout: "unhealthy_workers": 0 2024-05-08T06:14:19.206 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.206 INFO:teuthology.orchestra.run.smithi163.stdout: "finisher-MDSRank": { 2024-05-08T06:14:19.206 INFO:teuthology.orchestra.run.smithi163.stdout: "queue_len": 0, 2024-05-08T06:14:19.206 INFO:teuthology.orchestra.run.smithi163.stdout: "complete_latency": { 2024-05-08T06:14:19.207 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 2534, 2024-05-08T06:14:19.207 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.064899508, 2024-05-08T06:14:19.207 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000025611 2024-05-08T06:14:19.207 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:14:19.207 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.207 INFO:teuthology.orchestra.run.smithi163.stdout: "finisher-PurgeQueue": { 2024-05-08T06:14:19.207 INFO:teuthology.orchestra.run.smithi163.stdout: "queue_len": 0, 2024-05-08T06:14:19.207 INFO:teuthology.orchestra.run.smithi163.stdout: "complete_latency": { 2024-05-08T06:14:19.207 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.207 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.207 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.207 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:14:19.207 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.207 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": { 2024-05-08T06:14:19.208 INFO:teuthology.orchestra.run.smithi163.stdout: "request": 0, 2024-05-08T06:14:19.208 INFO:teuthology.orchestra.run.smithi163.stdout: "reply": 0, 2024-05-08T06:14:19.208 INFO:teuthology.orchestra.run.smithi163.stdout: "reply_latency": { 2024-05-08T06:14:19.208 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.208 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.208 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.208 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.208 INFO:teuthology.orchestra.run.smithi163.stdout: "slow_reply": 0, 2024-05-08T06:14:19.208 INFO:teuthology.orchestra.run.smithi163.stdout: "forward": 0, 2024-05-08T06:14:19.208 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_fetch_complete": 0, 2024-05-08T06:14:19.208 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_fetch_keys": 0, 2024-05-08T06:14:19.208 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_commit": 0, 2024-05-08T06:14:19.208 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_split": 0, 2024-05-08T06:14:19.208 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_merge": 0, 2024-05-08T06:14:19.208 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes": 1235, 2024-05-08T06:14:19.209 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_top": 575, 2024-05-08T06:14:19.209 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_bottom": 660, 2024-05-08T06:14:19.209 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_pin_tail": 0, 2024-05-08T06:14:19.209 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_pinned": 413, 2024-05-08T06:14:19.209 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_expired": 0, 2024-05-08T06:14:19.209 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_with_caps": 0, 2024-05-08T06:14:19.209 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": 0, 2024-05-08T06:14:19.209 INFO:teuthology.orchestra.run.smithi163.stdout: "subtrees": 2, 2024-05-08T06:14:19.209 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse": 0, 2024-05-08T06:14:19.209 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_hit": 0, 2024-05-08T06:14:19.209 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_forward": 0, 2024-05-08T06:14:19.209 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_discover": 0, 2024-05-08T06:14:19.209 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_dir_fetch": 0, 2024-05-08T06:14:19.209 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_remote_ino": 0, 2024-05-08T06:14:19.209 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_lock": 0, 2024-05-08T06:14:19.210 INFO:teuthology.orchestra.run.smithi163.stdout: "load_cent": 0, 2024-05-08T06:14:19.210 INFO:teuthology.orchestra.run.smithi163.stdout: "q": 0, 2024-05-08T06:14:19.210 INFO:teuthology.orchestra.run.smithi163.stdout: "exported": 0, 2024-05-08T06:14:19.210 INFO:teuthology.orchestra.run.smithi163.stdout: "exported_inodes": 0, 2024-05-08T06:14:19.210 INFO:teuthology.orchestra.run.smithi163.stdout: "imported": 0, 2024-05-08T06:14:19.210 INFO:teuthology.orchestra.run.smithi163.stdout: "imported_inodes": 0, 2024-05-08T06:14:19.210 INFO:teuthology.orchestra.run.smithi163.stdout: "openino_dir_fetch": 0, 2024-05-08T06:14:19.210 INFO:teuthology.orchestra.run.smithi163.stdout: "openino_backtrace_fetch": 0, 2024-05-08T06:14:19.210 INFO:teuthology.orchestra.run.smithi163.stdout: "openino_peer_discover": 0, 2024-05-08T06:14:19.210 INFO:teuthology.orchestra.run.smithi163.stdout: "root_rfiles": 4, 2024-05-08T06:14:19.210 INFO:teuthology.orchestra.run.smithi163.stdout: "root_rbytes": 234, 2024-05-08T06:14:19.210 INFO:teuthology.orchestra.run.smithi163.stdout: "root_rsnaps": 0, 2024-05-08T06:14:19.210 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T06:14:19.210 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_set_tag": 0, 2024-05-08T06:14:19.211 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T06:14:19.211 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_inotable_repaired": 0, 2024-05-08T06:14:19.211 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_dir_inodes": 0, 2024-05-08T06:14:19.211 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T06:14:19.211 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T06:14:19.211 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_file_inodes": 0, 2024-05-08T06:14:19.211 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_inode_file_caps": 0, 2024-05-08T06:14:19.211 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T06:14:19.211 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_grant": 0, 2024-05-08T06:14:19.211 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T06:14:19.211 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T06:14:19.211 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T06:14:19.211 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_caps": 0, 2024-05-08T06:14:19.211 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_caps_dirty": 0, 2024-05-08T06:14:19.211 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_cap_release": 0, 2024-05-08T06:14:19.212 INFO:teuthology.orchestra.run.smithi163.stdout: "process_request_cap_release": 0 2024-05-08T06:14:19.212 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.212 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_cache": { 2024-05-08T06:14:19.212 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_update": 0, 2024-05-08T06:14:19.212 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_update_receipt": 0, 2024-05-08T06:14:19.212 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_try_discover": 0, 2024-05-08T06:14:19.212 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_send_discover": 0, 2024-05-08T06:14:19.212 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_handle_discover": 0, 2024-05-08T06:14:19.212 INFO:teuthology.orchestra.run.smithi163.stdout: "num_strays": 0, 2024-05-08T06:14:19.212 INFO:teuthology.orchestra.run.smithi163.stdout: "num_strays_delayed": 0, 2024-05-08T06:14:19.212 INFO:teuthology.orchestra.run.smithi163.stdout: "num_strays_enqueuing": 0, 2024-05-08T06:14:19.212 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_created": 0, 2024-05-08T06:14:19.212 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_enqueued": 0, 2024-05-08T06:14:19.212 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_reintegrated": 0, 2024-05-08T06:14:19.212 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_migrated": 0, 2024-05-08T06:14:19.212 INFO:teuthology.orchestra.run.smithi163.stdout: "num_recovering_processing": 0, 2024-05-08T06:14:19.212 INFO:teuthology.orchestra.run.smithi163.stdout: "num_recovering_enqueued": 0, 2024-05-08T06:14:19.213 INFO:teuthology.orchestra.run.smithi163.stdout: "num_recovering_prioritized": 0, 2024-05-08T06:14:19.213 INFO:teuthology.orchestra.run.smithi163.stdout: "recovery_started": 0, 2024-05-08T06:14:19.213 INFO:teuthology.orchestra.run.smithi163.stdout: "recovery_completed": 0, 2024-05-08T06:14:19.213 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_quiesce_path": 0, 2024-05-08T06:14:19.213 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_quiesce_inode": 0, 2024-05-08T06:14:19.213 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T06:14:19.213 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_exportdir": 0, 2024-05-08T06:14:19.213 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_flush": 0, 2024-05-08T06:14:19.213 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_fragmentdir": 0, 2024-05-08T06:14:19.213 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_fragstats": 0, 2024-05-08T06:14:19.213 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_inodestats": 0 2024-05-08T06:14:19.213 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.213 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_log": { 2024-05-08T06:14:19.214 INFO:teuthology.orchestra.run.smithi163.stdout: "evlrg": 0, 2024-05-08T06:14:19.214 INFO:teuthology.orchestra.run.smithi163.stdout: "evadd": 0, 2024-05-08T06:14:19.214 INFO:teuthology.orchestra.run.smithi163.stdout: "evex": 0, 2024-05-08T06:14:19.214 INFO:teuthology.orchestra.run.smithi163.stdout: "evtrm": 0, 2024-05-08T06:14:19.214 INFO:teuthology.orchestra.run.smithi163.stdout: "ev": 6142, 2024-05-08T06:14:19.214 INFO:teuthology.orchestra.run.smithi163.stdout: "evexg": 0, 2024-05-08T06:14:19.214 INFO:teuthology.orchestra.run.smithi163.stdout: "evexd": 0, 2024-05-08T06:14:19.214 INFO:teuthology.orchestra.run.smithi163.stdout: "segadd": 0, 2024-05-08T06:14:19.214 INFO:teuthology.orchestra.run.smithi163.stdout: "segex": 0, 2024-05-08T06:14:19.214 INFO:teuthology.orchestra.run.smithi163.stdout: "segtrm": 0, 2024-05-08T06:14:19.214 INFO:teuthology.orchestra.run.smithi163.stdout: "seg": 3, 2024-05-08T06:14:19.214 INFO:teuthology.orchestra.run.smithi163.stdout: "segmjr": 13, 2024-05-08T06:14:19.214 INFO:teuthology.orchestra.run.smithi163.stdout: "segexg": 0, 2024-05-08T06:14:19.215 INFO:teuthology.orchestra.run.smithi163.stdout: "segexd": 0, 2024-05-08T06:14:19.215 INFO:teuthology.orchestra.run.smithi163.stdout: "expos": 14232263, 2024-05-08T06:14:19.215 INFO:teuthology.orchestra.run.smithi163.stdout: "wrpos": 16911917, 2024-05-08T06:14:19.215 INFO:teuthology.orchestra.run.smithi163.stdout: "rdpos": 16909420, 2024-05-08T06:14:19.215 INFO:teuthology.orchestra.run.smithi163.stdout: "jlat": { 2024-05-08T06:14:19.215 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.215 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.215 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.215 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.215 INFO:teuthology.orchestra.run.smithi163.stdout: "replayed": 6142 2024-05-08T06:14:19.215 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.215 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_mem": { 2024-05-08T06:14:19.215 INFO:teuthology.orchestra.run.smithi163.stdout: "ino": 23, 2024-05-08T06:14:19.215 INFO:teuthology.orchestra.run.smithi163.stdout: "ino+": 1235, 2024-05-08T06:14:19.216 INFO:teuthology.orchestra.run.smithi163.stdout: "ino-": 1212, 2024-05-08T06:14:19.216 INFO:teuthology.orchestra.run.smithi163.stdout: "dir": 18, 2024-05-08T06:14:19.216 INFO:teuthology.orchestra.run.smithi163.stdout: "dir+": 24, 2024-05-08T06:14:19.216 INFO:teuthology.orchestra.run.smithi163.stdout: "dir-": 6, 2024-05-08T06:14:19.216 INFO:teuthology.orchestra.run.smithi163.stdout: "dn": 1235, 2024-05-08T06:14:19.216 INFO:teuthology.orchestra.run.smithi163.stdout: "dn+": 1242, 2024-05-08T06:14:19.216 INFO:teuthology.orchestra.run.smithi163.stdout: "dn-": 7, 2024-05-08T06:14:19.216 INFO:teuthology.orchestra.run.smithi163.stdout: "cap": 0, 2024-05-08T06:14:19.216 INFO:teuthology.orchestra.run.smithi163.stdout: "cap+": 0, 2024-05-08T06:14:19.216 INFO:teuthology.orchestra.run.smithi163.stdout: "cap-": 0, 2024-05-08T06:14:19.216 INFO:teuthology.orchestra.run.smithi163.stdout: "rss": 54872, 2024-05-08T06:14:19.216 INFO:teuthology.orchestra.run.smithi163.stdout: "heap": 215416 2024-05-08T06:14:19.216 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.216 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_server": { 2024-05-08T06:14:19.216 INFO:teuthology.orchestra.run.smithi163.stdout: "dispatch_client_request": 0, 2024-05-08T06:14:19.216 INFO:teuthology.orchestra.run.smithi163.stdout: "dispatch_server_request": 0, 2024-05-08T06:14:19.216 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_request": 0, 2024-05-08T06:14:19.217 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_session": 0, 2024-05-08T06:14:19.217 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_peer_request": 0, 2024-05-08T06:14:19.217 INFO:teuthology.orchestra.run.smithi163.stdout: "req_create_latency": { 2024-05-08T06:14:19.217 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.217 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.217 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.217 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.217 INFO:teuthology.orchestra.run.smithi163.stdout: "req_getattr_latency": { 2024-05-08T06:14:19.217 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.217 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.217 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.217 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.217 INFO:teuthology.orchestra.run.smithi163.stdout: "req_getfilelock_latency": { 2024-05-08T06:14:19.217 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.217 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.217 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.217 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.218 INFO:teuthology.orchestra.run.smithi163.stdout: "req_link_latency": { 2024-05-08T06:14:19.218 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.218 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.218 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.218 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.218 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookup_latency": { 2024-05-08T06:14:19.218 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.218 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.218 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.218 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.218 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookuphash_latency": { 2024-05-08T06:14:19.218 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.218 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.218 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.219 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.219 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupino_latency": { 2024-05-08T06:14:19.219 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.219 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.219 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.219 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.219 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupname_latency": { 2024-05-08T06:14:19.219 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.219 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.219 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.219 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.219 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupparent_latency": { 2024-05-08T06:14:19.219 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.219 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.219 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.220 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.220 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupsnap_latency": { 2024-05-08T06:14:19.220 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.220 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.220 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.220 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.220 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lssnap_latency": { 2024-05-08T06:14:19.220 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.220 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.220 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.220 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.220 INFO:teuthology.orchestra.run.smithi163.stdout: "req_mkdir_latency": { 2024-05-08T06:14:19.221 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:18 smithi123 ceph-mon[29787]: pgmap v575: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:14:19.221 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.221 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.221 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.221 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.222 INFO:teuthology.orchestra.run.smithi163.stdout: "req_mknod_latency": { 2024-05-08T06:14:19.222 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.222 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.222 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.222 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.222 INFO:teuthology.orchestra.run.smithi163.stdout: "req_mksnap_latency": { 2024-05-08T06:14:19.222 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.222 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.222 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.222 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.222 INFO:teuthology.orchestra.run.smithi163.stdout: "req_open_latency": { 2024-05-08T06:14:19.222 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.222 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.222 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.223 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.223 INFO:teuthology.orchestra.run.smithi163.stdout: "req_readdir_latency": { 2024-05-08T06:14:19.223 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.223 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.223 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.223 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.223 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rename_latency": { 2024-05-08T06:14:19.223 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.223 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.223 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.223 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.223 INFO:teuthology.orchestra.run.smithi163.stdout: "req_renamesnap_latency": { 2024-05-08T06:14:19.223 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.225 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.225 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.225 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.225 INFO:teuthology.orchestra.run.smithi163.stdout: "req_snapdiff_latency": { 2024-05-08T06:14:19.225 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.225 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.225 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.226 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.226 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rmdir_latency": { 2024-05-08T06:14:19.226 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.226 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.226 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.226 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.226 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rmsnap_latency": { 2024-05-08T06:14:19.226 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.226 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.226 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.226 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.226 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rmxattr_latency": { 2024-05-08T06:14:19.226 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.226 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.226 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.227 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.227 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setattr_latency": { 2024-05-08T06:14:19.227 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.227 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.227 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.227 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.227 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setdirlayout_latency": { 2024-05-08T06:14:19.227 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.227 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.227 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.227 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.227 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setfilelock_latency": { 2024-05-08T06:14:19.227 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.227 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.227 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.227 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.228 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setlayout_latency": { 2024-05-08T06:14:19.228 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.228 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.228 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.228 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.228 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setxattr_latency": { 2024-05-08T06:14:19.228 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.228 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.228 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.228 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.228 INFO:teuthology.orchestra.run.smithi163.stdout: "req_symlink_latency": { 2024-05-08T06:14:19.228 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.228 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.229 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.229 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.229 INFO:teuthology.orchestra.run.smithi163.stdout: "req_unlink_latency": { 2024-05-08T06:14:19.229 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.229 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.229 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.229 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.229 INFO:teuthology.orchestra.run.smithi163.stdout: "cap_revoke_eviction": 0, 2024-05-08T06:14:19.229 INFO:teuthology.orchestra.run.smithi163.stdout: "cap_acquisition_throttle": 0, 2024-05-08T06:14:19.229 INFO:teuthology.orchestra.run.smithi163.stdout: "req_getvxattr_latency": { 2024-05-08T06:14:19.229 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.229 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.229 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.229 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:14:19.230 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.230 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_sessions": { 2024-05-08T06:14:19.230 INFO:teuthology.orchestra.run.smithi163.stdout: "session_count": 1, 2024-05-08T06:14:19.230 INFO:teuthology.orchestra.run.smithi163.stdout: "session_add": 3, 2024-05-08T06:14:19.230 INFO:teuthology.orchestra.run.smithi163.stdout: "session_remove": 2, 2024-05-08T06:14:19.230 INFO:teuthology.orchestra.run.smithi163.stdout: "sessions_open": 2, 2024-05-08T06:14:19.230 INFO:teuthology.orchestra.run.smithi163.stdout: "sessions_stale": 0, 2024-05-08T06:14:19.230 INFO:teuthology.orchestra.run.smithi163.stdout: "total_load": 0, 2024-05-08T06:14:19.230 INFO:teuthology.orchestra.run.smithi163.stdout: "average_load": 0, 2024-05-08T06:14:19.230 INFO:teuthology.orchestra.run.smithi163.stdout: "avg_session_uptime": 3508, 2024-05-08T06:14:19.230 INFO:teuthology.orchestra.run.smithi163.stdout: "mdthresh_evicted": 0 2024-05-08T06:14:19.230 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.230 INFO:teuthology.orchestra.run.smithi163.stdout: "mempool": { 2024-05-08T06:14:19.230 INFO:teuthology.orchestra.run.smithi163.stdout: "bloom_filter_bytes": 0, 2024-05-08T06:14:19.230 INFO:teuthology.orchestra.run.smithi163.stdout: "bloom_filter_items": 0, 2024-05-08T06:14:19.231 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T06:14:19.231 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_alloc_items": 0, 2024-05-08T06:14:19.231 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T06:14:19.231 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_data_items": 0, 2024-05-08T06:14:19.231 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T06:14:19.231 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T06:14:19.231 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T06:14:19.231 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T06:14:19.231 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T06:14:19.231 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_other_items": 0, 2024-05-08T06:14:19.231 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T06:14:19.231 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T06:14:19.231 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_extent_bytes": 0, 2024-05-08T06:14:19.231 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_extent_items": 0, 2024-05-08T06:14:19.232 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_blob_bytes": 0, 2024-05-08T06:14:19.232 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_blob_items": 0, 2024-05-08T06:14:19.232 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T06:14:19.232 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T06:14:19.232 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T06:14:19.232 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T06:14:19.232 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T06:14:19.232 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_fsck_items": 0, 2024-05-08T06:14:19.232 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_txc_bytes": 0, 2024-05-08T06:14:19.232 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_txc_items": 0, 2024-05-08T06:14:19.232 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T06:14:19.232 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T06:14:19.232 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_bytes": 0, 2024-05-08T06:14:19.232 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_items": 0, 2024-05-08T06:14:19.232 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_bytes": 0, 2024-05-08T06:14:19.233 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_items": 0, 2024-05-08T06:14:19.233 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T06:14:19.233 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_reader_items": 0, 2024-05-08T06:14:19.233 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T06:14:19.233 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_writer_items": 0, 2024-05-08T06:14:19.233 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_anon_bytes": 189361, 2024-05-08T06:14:19.233 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_anon_items": 50, 2024-05-08T06:14:19.233 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_meta_bytes": 0, 2024-05-08T06:14:19.233 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_meta_items": 0, 2024-05-08T06:14:19.233 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_bytes": 0, 2024-05-08T06:14:19.233 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_items": 0, 2024-05-08T06:14:19.233 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_mapbl_bytes": 0, 2024-05-08T06:14:19.233 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_mapbl_items": 0, 2024-05-08T06:14:19.233 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_pglog_bytes": 0, 2024-05-08T06:14:19.234 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_pglog_items": 0, 2024-05-08T06:14:19.234 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_bytes": 7528, 2024-05-08T06:14:19.234 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_items": 158, 2024-05-08T06:14:19.234 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T06:14:19.234 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_mapping_items": 0, 2024-05-08T06:14:19.234 INFO:teuthology.orchestra.run.smithi163.stdout: "pgmap_bytes": 0, 2024-05-08T06:14:19.234 INFO:teuthology.orchestra.run.smithi163.stdout: "pgmap_items": 0, 2024-05-08T06:14:19.234 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_co_bytes": 943834, 2024-05-08T06:14:19.234 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_co_items": 5229, 2024-05-08T06:14:19.234 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_1_bytes": 0, 2024-05-08T06:14:19.234 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_1_items": 0, 2024-05-08T06:14:19.234 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_2_bytes": 0, 2024-05-08T06:14:19.234 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_2_items": 0 2024-05-08T06:14:19.234 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.235 INFO:teuthology.orchestra.run.smithi163.stdout: "objecter": { 2024-05-08T06:14:19.235 INFO:teuthology.orchestra.run.smithi163.stdout: "op_active": 0, 2024-05-08T06:14:19.235 INFO:teuthology.orchestra.run.smithi163.stdout: "op_laggy": 0, 2024-05-08T06:14:19.235 INFO:teuthology.orchestra.run.smithi163.stdout: "op_send": 1510, 2024-05-08T06:14:19.235 INFO:teuthology.orchestra.run.smithi163.stdout: "op_send_bytes": 26, 2024-05-08T06:14:19.235 INFO:teuthology.orchestra.run.smithi163.stdout: "op_resend": 4, 2024-05-08T06:14:19.235 INFO:teuthology.orchestra.run.smithi163.stdout: "op_reply": 1510, 2024-05-08T06:14:19.235 INFO:teuthology.orchestra.run.smithi163.stdout: "op_latency": { 2024-05-08T06:14:19.235 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 1510, 2024-05-08T06:14:19.235 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.700984135, 2024-05-08T06:14:19.235 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000464227 2024-05-08T06:14:19.235 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.235 INFO:teuthology.orchestra.run.smithi163.stdout: "op_inflight": 0, 2024-05-08T06:14:19.235 INFO:teuthology.orchestra.run.smithi163.stdout: "oplen_avg": { 2024-05-08T06:14:19.235 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 1510, 2024-05-08T06:14:19.236 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 1511 2024-05-08T06:14:19.236 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.236 INFO:teuthology.orchestra.run.smithi163.stdout: "op": 1510, 2024-05-08T06:14:19.236 INFO:teuthology.orchestra.run.smithi163.stdout: "op_r": 1510, 2024-05-08T06:14:19.236 INFO:teuthology.orchestra.run.smithi163.stdout: "op_w": 0, 2024-05-08T06:14:19.236 INFO:teuthology.orchestra.run.smithi163.stdout: "op_rmw": 0, 2024-05-08T06:14:19.236 INFO:teuthology.orchestra.run.smithi163.stdout: "op_pg": 0, 2024-05-08T06:14:19.236 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_stat": 950, 2024-05-08T06:14:19.236 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_create": 0, 2024-05-08T06:14:19.236 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_read": 557, 2024-05-08T06:14:19.236 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_write": 0, 2024-05-08T06:14:19.236 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_writefull": 0, 2024-05-08T06:14:19.236 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_writesame": 0, 2024-05-08T06:14:19.236 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_append": 0, 2024-05-08T06:14:19.236 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_zero": 0, 2024-05-08T06:14:19.236 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_truncate": 0, 2024-05-08T06:14:19.237 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_delete": 0, 2024-05-08T06:14:19.237 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_mapext": 0, 2024-05-08T06:14:19.237 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_sparse_read": 0, 2024-05-08T06:14:19.237 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_clonerange": 0, 2024-05-08T06:14:19.237 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_getxattr": 2, 2024-05-08T06:14:19.237 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_setxattr": 0, 2024-05-08T06:14:19.237 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_cmpxattr": 0, 2024-05-08T06:14:19.237 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_rmxattr": 0, 2024-05-08T06:14:19.237 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_resetxattrs": 0, 2024-05-08T06:14:19.237 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_call": 0, 2024-05-08T06:14:19.237 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_watch": 0, 2024-05-08T06:14:19.237 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_notify": 0, 2024-05-08T06:14:19.237 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T06:14:19.237 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_pgls": 0, 2024-05-08T06:14:19.238 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_pgls_filter": 0, 2024-05-08T06:14:19.238 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_other": 0, 2024-05-08T06:14:19.238 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_active": 0, 2024-05-08T06:14:19.238 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_send": 0, 2024-05-08T06:14:19.238 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_resend": 0, 2024-05-08T06:14:19.238 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_ping": 0, 2024-05-08T06:14:19.238 INFO:teuthology.orchestra.run.smithi163.stdout: "poolop_active": 0, 2024-05-08T06:14:19.238 INFO:teuthology.orchestra.run.smithi163.stdout: "poolop_send": 0, 2024-05-08T06:14:19.238 INFO:teuthology.orchestra.run.smithi163.stdout: "poolop_resend": 0, 2024-05-08T06:14:19.238 INFO:teuthology.orchestra.run.smithi163.stdout: "poolstat_active": 0, 2024-05-08T06:14:19.238 INFO:teuthology.orchestra.run.smithi163.stdout: "poolstat_send": 0, 2024-05-08T06:14:19.238 INFO:teuthology.orchestra.run.smithi163.stdout: "poolstat_resend": 0, 2024-05-08T06:14:19.238 INFO:teuthology.orchestra.run.smithi163.stdout: "statfs_active": 0, 2024-05-08T06:14:19.238 INFO:teuthology.orchestra.run.smithi163.stdout: "statfs_send": 0, 2024-05-08T06:14:19.239 INFO:teuthology.orchestra.run.smithi163.stdout: "statfs_resend": 0, 2024-05-08T06:14:19.239 INFO:teuthology.orchestra.run.smithi163.stdout: "command_active": 0, 2024-05-08T06:14:19.239 INFO:teuthology.orchestra.run.smithi163.stdout: "command_send": 0, 2024-05-08T06:14:19.239 INFO:teuthology.orchestra.run.smithi163.stdout: "command_resend": 0, 2024-05-08T06:14:19.239 INFO:teuthology.orchestra.run.smithi163.stdout: "map_epoch": 82, 2024-05-08T06:14:19.239 INFO:teuthology.orchestra.run.smithi163.stdout: "map_full": 0, 2024-05-08T06:14:19.239 INFO:teuthology.orchestra.run.smithi163.stdout: "map_inc": 2, 2024-05-08T06:14:19.239 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_sessions": 5, 2024-05-08T06:14:19.239 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_session_open": 5, 2024-05-08T06:14:19.239 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_session_close": 0, 2024-05-08T06:14:19.239 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_laggy": 0, 2024-05-08T06:14:19.239 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_wr": 0, 2024-05-08T06:14:19.239 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_rd": 2, 2024-05-08T06:14:19.239 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_del": 0 2024-05-08T06:14:19.240 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.240 INFO:teuthology.orchestra.run.smithi163.stdout: "oft": { 2024-05-08T06:14:19.240 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_objs": 0, 2024-05-08T06:14:19.240 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_kv_pairs": 0, 2024-05-08T06:14:19.240 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_updates": 0, 2024-05-08T06:14:19.240 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_removes": 0 2024-05-08T06:14:19.240 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.240 INFO:teuthology.orchestra.run.smithi163.stdout: "osd-slow-ops": { 2024-05-08T06:14:19.240 INFO:teuthology.orchestra.run.smithi163.stdout: "slow_ops_count": 0 2024-05-08T06:14:19.240 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.240 INFO:teuthology.orchestra.run.smithi163.stdout: "purge_queue": { 2024-05-08T06:14:19.240 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing_ops": 0, 2024-05-08T06:14:19.240 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T06:14:19.240 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing": 0, 2024-05-08T06:14:19.241 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing_high_water": 0, 2024-05-08T06:14:19.241 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executed_ops": 0, 2024-05-08T06:14:19.241 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executed": 0, 2024-05-08T06:14:19.241 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_item_in_journal": 0 2024-05-08T06:14:19.241 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.241 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T06:14:19.241 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-05-08T06:14:19.241 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 104857600, 2024-05-08T06:14:19.241 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-05-08T06:14:19.241 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 1675, 2024-05-08T06:14:19.241 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 13799968, 2024-05-08T06:14:19.241 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-05-08T06:14:19.241 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 1675, 2024-05-08T06:14:19.241 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2024-05-08T06:14:19.242 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2024-05-08T06:14:19.242 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 1675, 2024-05-08T06:14:19.242 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 13799968, 2024-05-08T06:14:19.242 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-05-08T06:14:19.242 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.242 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.242 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.242 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:14:19.242 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.242 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-objecter_bytes": { 2024-05-08T06:14:19.242 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-05-08T06:14:19.242 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 104857600, 2024-05-08T06:14:19.242 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-05-08T06:14:19.242 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2024-05-08T06:14:19.243 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2024-05-08T06:14:19.243 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-05-08T06:14:19.243 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2024-05-08T06:14:19.243 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 1510, 2024-05-08T06:14:19.243 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 12717623, 2024-05-08T06:14:19.243 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 1510, 2024-05-08T06:14:19.243 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 12717623, 2024-05-08T06:14:19.243 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-05-08T06:14:19.243 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.243 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.243 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.243 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:14:19.243 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.243 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-objecter_ops": { 2024-05-08T06:14:19.243 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-05-08T06:14:19.243 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 1024, 2024-05-08T06:14:19.243 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-05-08T06:14:19.244 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2024-05-08T06:14:19.244 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2024-05-08T06:14:19.244 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-05-08T06:14:19.244 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2024-05-08T06:14:19.244 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 1510, 2024-05-08T06:14:19.244 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 1510, 2024-05-08T06:14:19.244 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 1510, 2024-05-08T06:14:19.244 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 1510, 2024-05-08T06:14:19.244 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-05-08T06:14:19.244 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.244 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.244 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.244 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:14:19.244 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.244 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-write_buf_throttle": { 2024-05-08T06:14:19.244 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-05-08T06:14:19.244 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 3758096384, 2024-05-08T06:14:19.245 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-05-08T06:14:19.245 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2024-05-08T06:14:19.245 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2024-05-08T06:14:19.245 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-05-08T06:14:19.245 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2024-05-08T06:14:19.245 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2024-05-08T06:14:19.245 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2024-05-08T06:14:19.245 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 0, 2024-05-08T06:14:19.245 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 0, 2024-05-08T06:14:19.245 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-05-08T06:14:19.245 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.245 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.245 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.245 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:14:19.245 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:19.245 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-write_buf_throttle-0x5631cfd9c6e0": { 2024-05-08T06:14:19.246 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-05-08T06:14:19.246 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 3758096384, 2024-05-08T06:14:19.246 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-05-08T06:14:19.246 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2024-05-08T06:14:19.246 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2024-05-08T06:14:19.246 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-05-08T06:14:19.246 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2024-05-08T06:14:19.246 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2024-05-08T06:14:19.246 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2024-05-08T06:14:19.246 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 0, 2024-05-08T06:14:19.246 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 0, 2024-05-08T06:14:19.246 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-05-08T06:14:19.246 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:19.247 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:19.247 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:19.247 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:14:19.247 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:14:19.247 INFO:teuthology.orchestra.run.smithi163.stdout:} 2024-05-08T06:14:19.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:18 smithi192 ceph-mon[34664]: pgmap v575: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:14:20.075 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-05-08T06:14:20.075 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-05-08T06:14:20.075 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-05-08T06:14:20.075 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-05-08T06:14:20.075 DEBUG:tasks.check_counter:Getting stats from k 2024-05-08T06:14:20.075 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-08T06:14:20.310 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T06:14:21.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:20 smithi163 ceph-mon[34520]: pgmap v576: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-05-08T06:14:21.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:20 smithi123 ceph-mon[29787]: pgmap v576: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-05-08T06:14:21.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:20 smithi192 ceph-mon[34664]: pgmap v576: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-05-08T06:14:21.974 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2024-05-08T06:14:21.974 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T06:14:21.974 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 160, 2024-05-08T06:14:21.974 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 155, 2024-05-08T06:14:21.975 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 796048, 2024-05-08T06:14:21.975 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 94768, 2024-05-08T06:14:21.975 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 4, 2024-05-08T06:14:21.975 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 0, 2024-05-08T06:14:21.975 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.028629202, 2024-05-08T06:14:21.975 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.009318476, 2024-05-08T06:14:21.975 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.017607411, 2024-05-08T06:14:21.975 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.003809022, 2024-05-08T06:14:21.975 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:14:21.975 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 154, 2024-05-08T06:14:21.975 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.008254402, 2024-05-08T06:14:21.975 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000053600 2024-05-08T06:14:21.975 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:21.975 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:14:21.975 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:21.976 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:21.976 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:21.976 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:21.976 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 796048, 2024-05-08T06:14:21.976 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 94768 2024-05-08T06:14:21.976 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:21.976 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T06:14:21.976 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 5, 2024-05-08T06:14:21.976 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 120, 2024-05-08T06:14:21.976 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 3200, 2024-05-08T06:14:21.976 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 99824, 2024-05-08T06:14:21.976 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 3, 2024-05-08T06:14:21.976 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 2, 2024-05-08T06:14:21.976 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.007917191, 2024-05-08T06:14:21.976 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.005784848, 2024-05-08T06:14:21.977 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.000782566, 2024-05-08T06:14:21.977 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:14:21.977 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:14:21.977 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 119, 2024-05-08T06:14:21.977 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.007812123, 2024-05-08T06:14:21.977 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000065648 2024-05-08T06:14:21.977 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:21.977 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:14:21.977 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:21.977 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:21.977 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:21.977 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:21.977 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-05-08T06:14:21.977 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 99824 2024-05-08T06:14:21.978 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:21.978 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T06:14:21.978 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 0, 2024-05-08T06:14:21.978 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 2, 2024-05-08T06:14:21.978 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 0, 2024-05-08T06:14:21.978 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 0, 2024-05-08T06:14:21.978 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 2, 2024-05-08T06:14:21.978 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 1, 2024-05-08T06:14:21.978 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.000268782, 2024-05-08T06:14:21.978 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.000000000, 2024-05-08T06:14:21.978 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.000207929, 2024-05-08T06:14:21.978 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:14:21.978 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:14:21.978 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:21.978 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:21.979 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:21.979 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:21.979 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:14:21.979 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:21.979 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:21.979 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:21.979 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:21.979 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T06:14:21.979 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T06:14:21.979 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:21.979 INFO:teuthology.orchestra.run.smithi163.stdout: "cct": { 2024-05-08T06:14:21.979 INFO:teuthology.orchestra.run.smithi163.stdout: "total_workers": 0, 2024-05-08T06:14:21.979 INFO:teuthology.orchestra.run.smithi163.stdout: "unhealthy_workers": 0 2024-05-08T06:14:21.979 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:21.979 INFO:teuthology.orchestra.run.smithi163.stdout: "mempool": { 2024-05-08T06:14:21.980 INFO:teuthology.orchestra.run.smithi163.stdout: "bloom_filter_bytes": 0, 2024-05-08T06:14:21.980 INFO:teuthology.orchestra.run.smithi163.stdout: "bloom_filter_items": 0, 2024-05-08T06:14:21.980 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T06:14:21.980 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_alloc_items": 0, 2024-05-08T06:14:21.980 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T06:14:21.980 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_data_items": 0, 2024-05-08T06:14:21.980 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T06:14:21.980 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T06:14:21.980 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T06:14:21.980 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T06:14:21.980 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T06:14:21.980 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_other_items": 0, 2024-05-08T06:14:21.980 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T06:14:21.980 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T06:14:21.980 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_extent_bytes": 0, 2024-05-08T06:14:21.980 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_extent_items": 0, 2024-05-08T06:14:21.980 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_blob_bytes": 0, 2024-05-08T06:14:21.981 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_blob_items": 0, 2024-05-08T06:14:21.981 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T06:14:21.981 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T06:14:21.981 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T06:14:21.981 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T06:14:21.981 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T06:14:21.981 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_fsck_items": 0, 2024-05-08T06:14:21.981 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_txc_bytes": 0, 2024-05-08T06:14:21.981 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_txc_items": 0, 2024-05-08T06:14:21.982 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T06:14:21.982 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T06:14:21.982 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_bytes": 0, 2024-05-08T06:14:21.982 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_items": 0, 2024-05-08T06:14:21.982 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_bytes": 0, 2024-05-08T06:14:21.982 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_items": 0, 2024-05-08T06:14:21.982 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T06:14:21.982 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_reader_items": 0, 2024-05-08T06:14:21.982 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T06:14:21.982 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_writer_items": 0, 2024-05-08T06:14:21.982 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_anon_bytes": 148945, 2024-05-08T06:14:21.982 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_anon_items": 29, 2024-05-08T06:14:21.982 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_meta_bytes": 0, 2024-05-08T06:14:21.982 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_meta_items": 0, 2024-05-08T06:14:21.983 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_bytes": 0, 2024-05-08T06:14:21.983 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_items": 0, 2024-05-08T06:14:21.983 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_mapbl_bytes": 0, 2024-05-08T06:14:21.983 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_mapbl_items": 0, 2024-05-08T06:14:21.983 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_pglog_bytes": 0, 2024-05-08T06:14:21.983 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_pglog_items": 0, 2024-05-08T06:14:21.983 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_bytes": 0, 2024-05-08T06:14:21.983 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_items": 0, 2024-05-08T06:14:21.983 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T06:14:21.983 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_mapping_items": 0, 2024-05-08T06:14:21.983 INFO:teuthology.orchestra.run.smithi163.stdout: "pgmap_bytes": 0, 2024-05-08T06:14:21.983 INFO:teuthology.orchestra.run.smithi163.stdout: "pgmap_items": 0, 2024-05-08T06:14:21.984 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_co_bytes": 928, 2024-05-08T06:14:21.984 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_co_items": 2, 2024-05-08T06:14:21.984 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_1_bytes": 0, 2024-05-08T06:14:21.984 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_1_items": 0, 2024-05-08T06:14:21.984 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_2_bytes": 0, 2024-05-08T06:14:21.984 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_2_items": 0 2024-05-08T06:14:21.984 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-05-08T06:14:21.984 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T06:14:21.984 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-05-08T06:14:21.984 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 104857600, 2024-05-08T06:14:21.984 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-05-08T06:14:21.984 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 161, 2024-05-08T06:14:21.984 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 773461, 2024-05-08T06:14:21.984 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-05-08T06:14:21.984 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 161, 2024-05-08T06:14:21.985 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2024-05-08T06:14:21.985 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2024-05-08T06:14:21.985 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 161, 2024-05-08T06:14:21.985 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 773461, 2024-05-08T06:14:21.985 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-05-08T06:14:21.985 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-05-08T06:14:21.985 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-05-08T06:14:21.985 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-05-08T06:14:21.985 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:14:21.985 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-05-08T06:14:21.985 INFO:teuthology.orchestra.run.smithi163.stdout:} 2024-05-08T06:14:22.761 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-05-08T06:14:22.761 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-05-08T06:14:22.761 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-05-08T06:14:22.761 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-05-08T06:14:22.761 DEBUG:tasks.check_counter:Getting stats from c 2024-05-08T06:14:22.762 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-08T06:14:23.010 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T06:14:23.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:22 smithi163 ceph-mon[34520]: pgmap v577: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:14:23.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:22 smithi192 ceph-mon[34664]: pgmap v577: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:14:23.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:22 smithi123 ceph-mon[29787]: pgmap v577: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:14:24.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:23 smithi123 ceph-mon[29787]: pgmap v578: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:14:24.237 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:23 smithi192 ceph-mon[34664]: pgmap v578: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:14:24.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:23 smithi163 ceph-mon[34520]: pgmap v578: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:14:24.805 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T06:14:24.805 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T06:14:24.805 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 1235, 2024-05-08T06:14:24.805 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 1235, 2024-05-08T06:14:24.805 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 359435, 2024-05-08T06:14:24.805 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 391913, 2024-05-08T06:14:24.805 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 6, 2024-05-08T06:14:24.805 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 2, 2024-05-08T06:14:24.805 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.112977797, 2024-05-08T06:14:24.805 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.038870547, 2024-05-08T06:14:24.806 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.027916055, 2024-05-08T06:14:24.806 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.036308950, 2024-05-08T06:14:24.806 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:14:24.806 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 1234, 2024-05-08T06:14:24.806 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.049098079, 2024-05-08T06:14:24.806 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000039787 2024-05-08T06:14:24.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.806 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:14:24.806 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.806 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.806 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.807 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-05-08T06:14:24.807 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 448 2024-05-08T06:14:24.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.807 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T06:14:24.807 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 4, 2024-05-08T06:14:24.807 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 118, 2024-05-08T06:14:24.807 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 876, 2024-05-08T06:14:24.808 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 344714, 2024-05-08T06:14:24.808 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 5, 2024-05-08T06:14:24.808 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 3, 2024-05-08T06:14:24.808 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.007960445, 2024-05-08T06:14:24.808 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.005340292, 2024-05-08T06:14:24.808 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.001254758, 2024-05-08T06:14:24.808 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.000076719, 2024-05-08T06:14:24.808 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:14:24.808 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 116, 2024-05-08T06:14:24.808 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.010154345, 2024-05-08T06:14:24.808 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000087537 2024-05-08T06:14:24.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.808 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:14:24.808 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.809 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.809 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.809 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-08T06:14:24.809 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 344016 2024-05-08T06:14:24.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.809 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T06:14:24.809 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 166, 2024-05-08T06:14:24.809 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 159, 2024-05-08T06:14:24.809 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 817996, 2024-05-08T06:14:24.809 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 93710, 2024-05-08T06:14:24.809 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 3, 2024-05-08T06:14:24.809 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 2, 2024-05-08T06:14:24.809 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.029600430, 2024-05-08T06:14:24.809 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.009857972, 2024-05-08T06:14:24.810 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.019785680, 2024-05-08T06:14:24.810 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.003661640, 2024-05-08T06:14:24.810 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:14:24.810 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 158, 2024-05-08T06:14:24.810 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.009516150, 2024-05-08T06:14:24.810 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000060228 2024-05-08T06:14:24.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.810 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:14:24.810 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.810 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.810 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.810 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 817728, 2024-05-08T06:14:24.810 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 93392 2024-05-08T06:14:24.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.811 INFO:teuthology.orchestra.run.smithi192.stdout: "cct": { 2024-05-08T06:14:24.811 INFO:teuthology.orchestra.run.smithi192.stdout: "total_workers": 1, 2024-05-08T06:14:24.811 INFO:teuthology.orchestra.run.smithi192.stdout: "unhealthy_workers": 0 2024-05-08T06:14:24.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.811 INFO:teuthology.orchestra.run.smithi192.stdout: "finisher-MDSRank": { 2024-05-08T06:14:24.811 INFO:teuthology.orchestra.run.smithi192.stdout: "queue_len": 0, 2024-05-08T06:14:24.811 INFO:teuthology.orchestra.run.smithi192.stdout: "complete_latency": { 2024-05-08T06:14:24.811 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 2208, 2024-05-08T06:14:24.811 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.063081394, 2024-05-08T06:14:24.811 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000028569 2024-05-08T06:14:24.811 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T06:14:24.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.811 INFO:teuthology.orchestra.run.smithi192.stdout: "finisher-PurgeQueue": { 2024-05-08T06:14:24.812 INFO:teuthology.orchestra.run.smithi192.stdout: "queue_len": 0, 2024-05-08T06:14:24.812 INFO:teuthology.orchestra.run.smithi192.stdout: "complete_latency": { 2024-05-08T06:14:24.812 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.812 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.812 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.812 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T06:14:24.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.812 INFO:teuthology.orchestra.run.smithi192.stdout: "mds": { 2024-05-08T06:14:24.812 INFO:teuthology.orchestra.run.smithi192.stdout: "request": 0, 2024-05-08T06:14:24.812 INFO:teuthology.orchestra.run.smithi192.stdout: "reply": 0, 2024-05-08T06:14:24.812 INFO:teuthology.orchestra.run.smithi192.stdout: "reply_latency": { 2024-05-08T06:14:24.812 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.812 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.812 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.813 INFO:teuthology.orchestra.run.smithi192.stdout: "slow_reply": 0, 2024-05-08T06:14:24.813 INFO:teuthology.orchestra.run.smithi192.stdout: "forward": 0, 2024-05-08T06:14:24.813 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_fetch_complete": 0, 2024-05-08T06:14:24.813 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_fetch_keys": 0, 2024-05-08T06:14:24.813 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_commit": 0, 2024-05-08T06:14:24.813 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_split": 0, 2024-05-08T06:14:24.813 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_merge": 0, 2024-05-08T06:14:24.813 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes": 0, 2024-05-08T06:14:24.813 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_top": 0, 2024-05-08T06:14:24.813 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_bottom": 0, 2024-05-08T06:14:24.813 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_pin_tail": 0, 2024-05-08T06:14:24.813 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_pinned": 0, 2024-05-08T06:14:24.813 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_expired": 0, 2024-05-08T06:14:24.813 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_with_caps": 0, 2024-05-08T06:14:24.814 INFO:teuthology.orchestra.run.smithi192.stdout: "caps": 0, 2024-05-08T06:14:24.814 INFO:teuthology.orchestra.run.smithi192.stdout: "subtrees": 1, 2024-05-08T06:14:24.814 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse": 0, 2024-05-08T06:14:24.814 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_hit": 0, 2024-05-08T06:14:24.814 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_forward": 0, 2024-05-08T06:14:24.814 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_discover": 0, 2024-05-08T06:14:24.814 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_dir_fetch": 0, 2024-05-08T06:14:24.814 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_remote_ino": 0, 2024-05-08T06:14:24.814 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_lock": 0, 2024-05-08T06:14:24.814 INFO:teuthology.orchestra.run.smithi192.stdout: "load_cent": 0, 2024-05-08T06:14:24.814 INFO:teuthology.orchestra.run.smithi192.stdout: "q": 0, 2024-05-08T06:14:24.814 INFO:teuthology.orchestra.run.smithi192.stdout: "exported": 0, 2024-05-08T06:14:24.814 INFO:teuthology.orchestra.run.smithi192.stdout: "exported_inodes": 0, 2024-05-08T06:14:24.815 INFO:teuthology.orchestra.run.smithi192.stdout: "imported": 0, 2024-05-08T06:14:24.815 INFO:teuthology.orchestra.run.smithi192.stdout: "imported_inodes": 0, 2024-05-08T06:14:24.815 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_dir_fetch": 0, 2024-05-08T06:14:24.815 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_backtrace_fetch": 0, 2024-05-08T06:14:24.815 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_peer_discover": 0, 2024-05-08T06:14:24.815 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rfiles": 0, 2024-05-08T06:14:24.815 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rbytes": 0, 2024-05-08T06:14:24.815 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rsnaps": 0, 2024-05-08T06:14:24.815 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T06:14:24.815 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_set_tag": 0, 2024-05-08T06:14:24.815 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T06:14:24.815 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_inotable_repaired": 0, 2024-05-08T06:14:24.815 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dir_inodes": 0, 2024-05-08T06:14:24.815 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T06:14:24.815 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T06:14:24.815 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_file_inodes": 0, 2024-05-08T06:14:24.816 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_inode_file_caps": 0, 2024-05-08T06:14:24.816 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T06:14:24.816 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_grant": 0, 2024-05-08T06:14:24.816 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T06:14:24.816 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T06:14:24.816 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T06:14:24.816 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_caps": 0, 2024-05-08T06:14:24.816 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_caps_dirty": 0, 2024-05-08T06:14:24.816 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_cap_release": 0, 2024-05-08T06:14:24.816 INFO:teuthology.orchestra.run.smithi192.stdout: "process_request_cap_release": 0 2024-05-08T06:14:24.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.816 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_cache": { 2024-05-08T06:14:24.816 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_update": 0, 2024-05-08T06:14:24.816 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_update_receipt": 0, 2024-05-08T06:14:24.817 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_try_discover": 0, 2024-05-08T06:14:24.817 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_send_discover": 0, 2024-05-08T06:14:24.817 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_handle_discover": 0, 2024-05-08T06:14:24.817 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays": 0, 2024-05-08T06:14:24.817 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays_delayed": 0, 2024-05-08T06:14:24.817 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays_enqueuing": 0, 2024-05-08T06:14:24.817 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_created": 0, 2024-05-08T06:14:24.817 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_enqueued": 0, 2024-05-08T06:14:24.817 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_reintegrated": 0, 2024-05-08T06:14:24.817 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_migrated": 0, 2024-05-08T06:14:24.817 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_processing": 0, 2024-05-08T06:14:24.817 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_enqueued": 0, 2024-05-08T06:14:24.817 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_prioritized": 0, 2024-05-08T06:14:24.817 INFO:teuthology.orchestra.run.smithi192.stdout: "recovery_started": 0, 2024-05-08T06:14:24.817 INFO:teuthology.orchestra.run.smithi192.stdout: "recovery_completed": 0, 2024-05-08T06:14:24.817 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_quiesce_path": 0, 2024-05-08T06:14:24.818 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_quiesce_inode": 0, 2024-05-08T06:14:24.818 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T06:14:24.818 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_exportdir": 0, 2024-05-08T06:14:24.818 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_flush": 0, 2024-05-08T06:14:24.818 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_fragmentdir": 0, 2024-05-08T06:14:24.818 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_fragstats": 0, 2024-05-08T06:14:24.818 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_inodestats": 0 2024-05-08T06:14:24.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.818 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_log": { 2024-05-08T06:14:24.818 INFO:teuthology.orchestra.run.smithi192.stdout: "evlrg": 0, 2024-05-08T06:14:24.818 INFO:teuthology.orchestra.run.smithi192.stdout: "evadd": 0, 2024-05-08T06:14:24.818 INFO:teuthology.orchestra.run.smithi192.stdout: "evex": 0, 2024-05-08T06:14:24.818 INFO:teuthology.orchestra.run.smithi192.stdout: "evtrm": 0, 2024-05-08T06:14:24.818 INFO:teuthology.orchestra.run.smithi192.stdout: "ev": 5, 2024-05-08T06:14:24.819 INFO:teuthology.orchestra.run.smithi192.stdout: "evexg": 0, 2024-05-08T06:14:24.819 INFO:teuthology.orchestra.run.smithi192.stdout: "evexd": 0, 2024-05-08T06:14:24.819 INFO:teuthology.orchestra.run.smithi192.stdout: "segadd": 0, 2024-05-08T06:14:24.819 INFO:teuthology.orchestra.run.smithi192.stdout: "segex": 0, 2024-05-08T06:14:24.819 INFO:teuthology.orchestra.run.smithi192.stdout: "segtrm": 0, 2024-05-08T06:14:24.819 INFO:teuthology.orchestra.run.smithi192.stdout: "seg": 2, 2024-05-08T06:14:24.819 INFO:teuthology.orchestra.run.smithi192.stdout: "segmjr": 2, 2024-05-08T06:14:24.819 INFO:teuthology.orchestra.run.smithi192.stdout: "segexg": 0, 2024-05-08T06:14:24.819 INFO:teuthology.orchestra.run.smithi192.stdout: "segexd": 0, 2024-05-08T06:14:24.819 INFO:teuthology.orchestra.run.smithi192.stdout: "expos": 4194304, 2024-05-08T06:14:24.819 INFO:teuthology.orchestra.run.smithi192.stdout: "wrpos": 4195734, 2024-05-08T06:14:24.819 INFO:teuthology.orchestra.run.smithi192.stdout: "rdpos": 4195593, 2024-05-08T06:14:24.819 INFO:teuthology.orchestra.run.smithi192.stdout: "jlat": { 2024-05-08T06:14:24.819 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.819 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.820 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.820 INFO:teuthology.orchestra.run.smithi192.stdout: "replayed": 5 2024-05-08T06:14:24.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.820 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_mem": { 2024-05-08T06:14:24.820 INFO:teuthology.orchestra.run.smithi192.stdout: "ino": 3, 2024-05-08T06:14:24.821 INFO:teuthology.orchestra.run.smithi192.stdout: "ino+": 3, 2024-05-08T06:14:24.821 INFO:teuthology.orchestra.run.smithi192.stdout: "ino-": 0, 2024-05-08T06:14:24.821 INFO:teuthology.orchestra.run.smithi192.stdout: "dir": 1, 2024-05-08T06:14:24.821 INFO:teuthology.orchestra.run.smithi192.stdout: "dir+": 1, 2024-05-08T06:14:24.821 INFO:teuthology.orchestra.run.smithi192.stdout: "dir-": 0, 2024-05-08T06:14:24.821 INFO:teuthology.orchestra.run.smithi192.stdout: "dn": 0, 2024-05-08T06:14:24.821 INFO:teuthology.orchestra.run.smithi192.stdout: "dn+": 0, 2024-05-08T06:14:24.822 INFO:teuthology.orchestra.run.smithi192.stdout: "dn-": 0, 2024-05-08T06:14:24.822 INFO:teuthology.orchestra.run.smithi192.stdout: "cap": 0, 2024-05-08T06:14:24.822 INFO:teuthology.orchestra.run.smithi192.stdout: "cap+": 0, 2024-05-08T06:14:24.822 INFO:teuthology.orchestra.run.smithi192.stdout: "cap-": 0, 2024-05-08T06:14:24.822 INFO:teuthology.orchestra.run.smithi192.stdout: "rss": 52244, 2024-05-08T06:14:24.822 INFO:teuthology.orchestra.run.smithi192.stdout: "heap": 215416 2024-05-08T06:14:24.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.822 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_server": { 2024-05-08T06:14:24.822 INFO:teuthology.orchestra.run.smithi192.stdout: "dispatch_client_request": 0, 2024-05-08T06:14:24.822 INFO:teuthology.orchestra.run.smithi192.stdout: "dispatch_server_request": 0, 2024-05-08T06:14:24.822 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_request": 0, 2024-05-08T06:14:24.822 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_session": 0, 2024-05-08T06:14:24.823 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_peer_request": 0, 2024-05-08T06:14:24.823 INFO:teuthology.orchestra.run.smithi192.stdout: "req_create_latency": { 2024-05-08T06:14:24.823 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.823 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.823 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.823 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getattr_latency": { 2024-05-08T06:14:24.823 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.823 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.823 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.823 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getfilelock_latency": { 2024-05-08T06:14:24.823 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.824 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.824 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.824 INFO:teuthology.orchestra.run.smithi192.stdout: "req_link_latency": { 2024-05-08T06:14:24.824 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.824 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.824 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.824 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookup_latency": { 2024-05-08T06:14:24.824 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.824 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.824 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.824 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookuphash_latency": { 2024-05-08T06:14:24.825 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.825 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.825 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.825 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupino_latency": { 2024-05-08T06:14:24.825 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.825 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.825 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.825 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupname_latency": { 2024-05-08T06:14:24.825 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.825 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.825 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.825 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupparent_latency": { 2024-05-08T06:14:24.826 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.826 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.826 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.826 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupsnap_latency": { 2024-05-08T06:14:24.826 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.826 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.826 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.826 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lssnap_latency": { 2024-05-08T06:14:24.826 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.826 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.826 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.827 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mkdir_latency": { 2024-05-08T06:14:24.827 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.827 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.827 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.828 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mknod_latency": { 2024-05-08T06:14:24.828 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.828 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.828 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.828 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mksnap_latency": { 2024-05-08T06:14:24.828 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.828 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.829 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.829 INFO:teuthology.orchestra.run.smithi192.stdout: "req_open_latency": { 2024-05-08T06:14:24.829 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.829 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.829 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.829 INFO:teuthology.orchestra.run.smithi192.stdout: "req_readdir_latency": { 2024-05-08T06:14:24.829 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.829 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.829 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.829 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rename_latency": { 2024-05-08T06:14:24.829 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.829 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.830 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.830 INFO:teuthology.orchestra.run.smithi192.stdout: "req_renamesnap_latency": { 2024-05-08T06:14:24.830 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.830 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.830 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.830 INFO:teuthology.orchestra.run.smithi192.stdout: "req_snapdiff_latency": { 2024-05-08T06:14:24.830 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.830 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.830 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.830 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmdir_latency": { 2024-05-08T06:14:24.831 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.831 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.831 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.831 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmsnap_latency": { 2024-05-08T06:14:24.831 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.831 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.831 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.831 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmxattr_latency": { 2024-05-08T06:14:24.831 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.831 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.831 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.832 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setattr_latency": { 2024-05-08T06:14:24.832 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.832 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.832 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.832 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setdirlayout_latency": { 2024-05-08T06:14:24.832 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.832 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.832 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.832 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setfilelock_latency": { 2024-05-08T06:14:24.832 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.832 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.832 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.833 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setlayout_latency": { 2024-05-08T06:14:24.833 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.833 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.833 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.833 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setxattr_latency": { 2024-05-08T06:14:24.833 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.833 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.833 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.833 INFO:teuthology.orchestra.run.smithi192.stdout: "req_symlink_latency": { 2024-05-08T06:14:24.833 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.833 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.833 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.833 INFO:teuthology.orchestra.run.smithi192.stdout: "req_unlink_latency": { 2024-05-08T06:14:24.834 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.834 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.834 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.834 INFO:teuthology.orchestra.run.smithi192.stdout: "cap_revoke_eviction": 0, 2024-05-08T06:14:24.834 INFO:teuthology.orchestra.run.smithi192.stdout: "cap_acquisition_throttle": 0, 2024-05-08T06:14:24.834 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getvxattr_latency": { 2024-05-08T06:14:24.834 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.834 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.834 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.834 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T06:14:24.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.834 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_sessions": { 2024-05-08T06:14:24.835 INFO:teuthology.orchestra.run.smithi192.stdout: "session_count": 1, 2024-05-08T06:14:24.835 INFO:teuthology.orchestra.run.smithi192.stdout: "session_add": 2, 2024-05-08T06:14:24.835 INFO:teuthology.orchestra.run.smithi192.stdout: "session_remove": 1, 2024-05-08T06:14:24.835 INFO:teuthology.orchestra.run.smithi192.stdout: "sessions_open": 2, 2024-05-08T06:14:24.835 INFO:teuthology.orchestra.run.smithi192.stdout: "sessions_stale": 0, 2024-05-08T06:14:24.835 INFO:teuthology.orchestra.run.smithi192.stdout: "total_load": 0, 2024-05-08T06:14:24.835 INFO:teuthology.orchestra.run.smithi192.stdout: "average_load": 0, 2024-05-08T06:14:24.835 INFO:teuthology.orchestra.run.smithi192.stdout: "avg_session_uptime": 3613, 2024-05-08T06:14:24.835 INFO:teuthology.orchestra.run.smithi192.stdout: "mdthresh_evicted": 0 2024-05-08T06:14:24.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.836 INFO:teuthology.orchestra.run.smithi192.stdout: "mempool": { 2024-05-08T06:14:24.836 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_bytes": 0, 2024-05-08T06:14:24.836 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_items": 0, 2024-05-08T06:14:24.836 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T06:14:24.836 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_items": 0, 2024-05-08T06:14:24.836 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T06:14:24.836 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_items": 0, 2024-05-08T06:14:24.836 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T06:14:24.836 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T06:14:24.836 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T06:14:24.836 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T06:14:24.837 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T06:14:24.837 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_items": 0, 2024-05-08T06:14:24.837 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T06:14:24.837 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T06:14:24.837 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_bytes": 0, 2024-05-08T06:14:24.837 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_items": 0, 2024-05-08T06:14:24.837 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_bytes": 0, 2024-05-08T06:14:24.837 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_items": 0, 2024-05-08T06:14:24.837 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T06:14:24.837 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T06:14:24.837 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T06:14:24.837 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T06:14:24.837 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T06:14:24.837 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_items": 0, 2024-05-08T06:14:24.837 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_bytes": 0, 2024-05-08T06:14:24.838 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_items": 0, 2024-05-08T06:14:24.838 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T06:14:24.838 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T06:14:24.838 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_bytes": 0, 2024-05-08T06:14:24.838 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_items": 0, 2024-05-08T06:14:24.838 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_bytes": 0, 2024-05-08T06:14:24.838 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_items": 0, 2024-05-08T06:14:24.838 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T06:14:24.838 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_items": 0, 2024-05-08T06:14:24.838 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T06:14:24.838 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_items": 0, 2024-05-08T06:14:24.838 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_bytes": 181321, 2024-05-08T06:14:24.838 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_items": 47, 2024-05-08T06:14:24.838 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_bytes": 0, 2024-05-08T06:14:24.839 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_items": 0, 2024-05-08T06:14:24.839 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_bytes": 0, 2024-05-08T06:14:24.839 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_items": 0, 2024-05-08T06:14:24.839 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_bytes": 0, 2024-05-08T06:14:24.839 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_items": 0, 2024-05-08T06:14:24.839 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_bytes": 0, 2024-05-08T06:14:24.839 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_items": 0, 2024-05-08T06:14:24.839 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_bytes": 7528, 2024-05-08T06:14:24.839 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_items": 158, 2024-05-08T06:14:24.839 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T06:14:24.839 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_items": 0, 2024-05-08T06:14:24.839 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_bytes": 0, 2024-05-08T06:14:24.839 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_items": 0, 2024-05-08T06:14:24.840 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_bytes": 8512, 2024-05-08T06:14:24.840 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_items": 13, 2024-05-08T06:14:24.840 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_bytes": 0, 2024-05-08T06:14:24.840 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_items": 0, 2024-05-08T06:14:24.840 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_bytes": 0, 2024-05-08T06:14:24.840 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_items": 0 2024-05-08T06:14:24.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.840 INFO:teuthology.orchestra.run.smithi192.stdout: "objecter": { 2024-05-08T06:14:24.840 INFO:teuthology.orchestra.run.smithi192.stdout: "op_active": 0, 2024-05-08T06:14:24.840 INFO:teuthology.orchestra.run.smithi192.stdout: "op_laggy": 0, 2024-05-08T06:14:24.840 INFO:teuthology.orchestra.run.smithi192.stdout: "op_send": 1234, 2024-05-08T06:14:24.840 INFO:teuthology.orchestra.run.smithi192.stdout: "op_send_bytes": 21, 2024-05-08T06:14:24.841 INFO:teuthology.orchestra.run.smithi192.stdout: "op_resend": 3, 2024-05-08T06:14:24.841 INFO:teuthology.orchestra.run.smithi192.stdout: "op_reply": 1234, 2024-05-08T06:14:24.841 INFO:teuthology.orchestra.run.smithi192.stdout: "op_latency": { 2024-05-08T06:14:24.841 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 1234, 2024-05-08T06:14:24.841 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.563988833, 2024-05-08T06:14:24.841 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000457041 2024-05-08T06:14:24.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.841 INFO:teuthology.orchestra.run.smithi192.stdout: "op_inflight": 0, 2024-05-08T06:14:24.841 INFO:teuthology.orchestra.run.smithi192.stdout: "oplen_avg": { 2024-05-08T06:14:24.841 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 1234, 2024-05-08T06:14:24.841 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 1235 2024-05-08T06:14:24.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.841 INFO:teuthology.orchestra.run.smithi192.stdout: "op": 1234, 2024-05-08T06:14:24.841 INFO:teuthology.orchestra.run.smithi192.stdout: "op_r": 1234, 2024-05-08T06:14:24.841 INFO:teuthology.orchestra.run.smithi192.stdout: "op_w": 0, 2024-05-08T06:14:24.842 INFO:teuthology.orchestra.run.smithi192.stdout: "op_rmw": 0, 2024-05-08T06:14:24.842 INFO:teuthology.orchestra.run.smithi192.stdout: "op_pg": 0, 2024-05-08T06:14:24.842 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_stat": 739, 2024-05-08T06:14:24.842 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_create": 0, 2024-05-08T06:14:24.842 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_read": 493, 2024-05-08T06:14:24.842 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_write": 0, 2024-05-08T06:14:24.842 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_writefull": 0, 2024-05-08T06:14:24.842 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_writesame": 0, 2024-05-08T06:14:24.842 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_append": 0, 2024-05-08T06:14:24.842 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_zero": 0, 2024-05-08T06:14:24.842 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_truncate": 0, 2024-05-08T06:14:24.842 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_delete": 0, 2024-05-08T06:14:24.842 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_mapext": 0, 2024-05-08T06:14:24.843 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_sparse_read": 0, 2024-05-08T06:14:24.843 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_clonerange": 0, 2024-05-08T06:14:24.843 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_getxattr": 1, 2024-05-08T06:14:24.843 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_setxattr": 0, 2024-05-08T06:14:24.843 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_cmpxattr": 0, 2024-05-08T06:14:24.843 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_rmxattr": 0, 2024-05-08T06:14:24.843 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_resetxattrs": 0, 2024-05-08T06:14:24.843 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_call": 0, 2024-05-08T06:14:24.843 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_watch": 0, 2024-05-08T06:14:24.843 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_notify": 0, 2024-05-08T06:14:24.843 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T06:14:24.843 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_pgls": 0, 2024-05-08T06:14:24.843 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_pgls_filter": 0, 2024-05-08T06:14:24.843 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_other": 0, 2024-05-08T06:14:24.843 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_active": 0, 2024-05-08T06:14:24.844 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_send": 0, 2024-05-08T06:14:24.844 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_resend": 0, 2024-05-08T06:14:24.844 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_ping": 0, 2024-05-08T06:14:24.844 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_active": 0, 2024-05-08T06:14:24.844 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_send": 0, 2024-05-08T06:14:24.844 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_resend": 0, 2024-05-08T06:14:24.844 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_active": 0, 2024-05-08T06:14:24.844 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_send": 0, 2024-05-08T06:14:24.844 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_resend": 0, 2024-05-08T06:14:24.844 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_active": 0, 2024-05-08T06:14:24.844 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_send": 0, 2024-05-08T06:14:24.844 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_resend": 0, 2024-05-08T06:14:24.844 INFO:teuthology.orchestra.run.smithi192.stdout: "command_active": 0, 2024-05-08T06:14:24.844 INFO:teuthology.orchestra.run.smithi192.stdout: "command_send": 0, 2024-05-08T06:14:24.844 INFO:teuthology.orchestra.run.smithi192.stdout: "command_resend": 0, 2024-05-08T06:14:24.845 INFO:teuthology.orchestra.run.smithi192.stdout: "map_epoch": 82, 2024-05-08T06:14:24.845 INFO:teuthology.orchestra.run.smithi192.stdout: "map_full": 0, 2024-05-08T06:14:24.845 INFO:teuthology.orchestra.run.smithi192.stdout: "map_inc": 2, 2024-05-08T06:14:24.845 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_sessions": 4, 2024-05-08T06:14:24.845 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_session_open": 4, 2024-05-08T06:14:24.845 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_session_close": 0, 2024-05-08T06:14:24.845 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_laggy": 0, 2024-05-08T06:14:24.845 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_wr": 0, 2024-05-08T06:14:24.845 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_rd": 2, 2024-05-08T06:14:24.845 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_del": 0 2024-05-08T06:14:24.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.845 INFO:teuthology.orchestra.run.smithi192.stdout: "oft": { 2024-05-08T06:14:24.845 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_objs": 0, 2024-05-08T06:14:24.845 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_kv_pairs": 0, 2024-05-08T06:14:24.845 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_updates": 0, 2024-05-08T06:14:24.846 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_removes": 0 2024-05-08T06:14:24.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.846 INFO:teuthology.orchestra.run.smithi192.stdout: "osd-slow-ops": { 2024-05-08T06:14:24.846 INFO:teuthology.orchestra.run.smithi192.stdout: "slow_ops_count": 0 2024-05-08T06:14:24.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.846 INFO:teuthology.orchestra.run.smithi192.stdout: "purge_queue": { 2024-05-08T06:14:24.846 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_ops": 0, 2024-05-08T06:14:24.846 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T06:14:24.846 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing": 0, 2024-05-08T06:14:24.846 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_high_water": 0, 2024-05-08T06:14:24.846 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executed_ops": 0, 2024-05-08T06:14:24.846 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executed": 0, 2024-05-08T06:14:24.846 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_item_in_journal": 0 2024-05-08T06:14:24.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.847 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T06:14:24.847 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-05-08T06:14:24.847 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2024-05-08T06:14:24.847 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-05-08T06:14:24.847 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 1401, 2024-05-08T06:14:24.847 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 1040653, 2024-05-08T06:14:24.847 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-05-08T06:14:24.847 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 1401, 2024-05-08T06:14:24.847 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-05-08T06:14:24.847 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-05-08T06:14:24.847 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 1401, 2024-05-08T06:14:24.847 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 1040653, 2024-05-08T06:14:24.847 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-05-08T06:14:24.847 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.848 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.848 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.848 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T06:14:24.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.848 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-objecter_bytes": { 2024-05-08T06:14:24.848 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-05-08T06:14:24.848 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2024-05-08T06:14:24.848 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-05-08T06:14:24.848 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-05-08T06:14:24.848 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-05-08T06:14:24.848 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-05-08T06:14:24.848 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-05-08T06:14:24.849 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 1234, 2024-05-08T06:14:24.849 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 1435, 2024-05-08T06:14:24.849 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 1234, 2024-05-08T06:14:24.849 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 1435, 2024-05-08T06:14:24.849 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-05-08T06:14:24.849 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.849 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.849 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.849 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T06:14:24.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.849 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-objecter_ops": { 2024-05-08T06:14:24.849 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-05-08T06:14:24.849 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 1024, 2024-05-08T06:14:24.849 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-05-08T06:14:24.849 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-05-08T06:14:24.850 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-05-08T06:14:24.850 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-05-08T06:14:24.850 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-05-08T06:14:24.850 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 1234, 2024-05-08T06:14:24.850 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 1234, 2024-05-08T06:14:24.850 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 1234, 2024-05-08T06:14:24.850 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 1234, 2024-05-08T06:14:24.850 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-05-08T06:14:24.850 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.850 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.850 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.850 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T06:14:24.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.851 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-write_buf_throttle": { 2024-05-08T06:14:24.851 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-05-08T06:14:24.851 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 3758096384, 2024-05-08T06:14:24.851 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-05-08T06:14:24.851 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-05-08T06:14:24.851 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-05-08T06:14:24.851 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-05-08T06:14:24.851 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-05-08T06:14:24.851 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-05-08T06:14:24.851 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-05-08T06:14:24.851 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 0, 2024-05-08T06:14:24.851 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 0, 2024-05-08T06:14:24.851 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-05-08T06:14:24.851 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.852 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.852 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.852 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T06:14:24.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:24.852 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-write_buf_throttle-0x556eb9d746e0": { 2024-05-08T06:14:24.852 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-05-08T06:14:24.852 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 3758096384, 2024-05-08T06:14:24.852 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-05-08T06:14:24.852 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-05-08T06:14:24.852 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-05-08T06:14:24.852 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-05-08T06:14:24.852 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-05-08T06:14:24.853 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-05-08T06:14:24.853 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-05-08T06:14:24.853 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 0, 2024-05-08T06:14:24.853 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 0, 2024-05-08T06:14:24.853 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-05-08T06:14:24.853 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:24.853 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:24.853 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:24.853 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T06:14:24.853 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T06:14:24.853 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T06:14:25.561 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-05-08T06:14:25.562 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-05-08T06:14:25.562 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-05-08T06:14:25.562 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-05-08T06:14:25.562 DEBUG:tasks.check_counter:Getting stats from f 2024-05-08T06:14:25.562 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-08T06:14:25.823 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T06:14:26.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:26 smithi192 ceph-mon[34664]: pgmap v579: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T06:14:26.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:26 smithi123 ceph-mon[29787]: pgmap v579: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T06:14:26.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:26 smithi163 ceph-mon[34520]: pgmap v579: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T06:14:27.548 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T06:14:27.549 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T06:14:27.549 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 497, 2024-05-08T06:14:27.549 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 499, 2024-05-08T06:14:27.549 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 132425, 2024-05-08T06:14:27.549 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 158110, 2024-05-08T06:14:27.549 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 6, 2024-05-08T06:14:27.549 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 2, 2024-05-08T06:14:27.549 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.054050070, 2024-05-08T06:14:27.549 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.015336906, 2024-05-08T06:14:27.549 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.015462578, 2024-05-08T06:14:27.549 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.017356992, 2024-05-08T06:14:27.549 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:14:27.549 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 497, 2024-05-08T06:14:27.549 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.024542590, 2024-05-08T06:14:27.550 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000049381 2024-05-08T06:14:27.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.550 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:14:27.550 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.550 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.550 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.550 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T06:14:27.550 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T06:14:27.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.550 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T06:14:27.550 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 558, 2024-05-08T06:14:27.550 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 549, 2024-05-08T06:14:27.550 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 916510, 2024-05-08T06:14:27.551 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 217869, 2024-05-08T06:14:27.551 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 4, 2024-05-08T06:14:27.551 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 3, 2024-05-08T06:14:27.551 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.073748227, 2024-05-08T06:14:27.551 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.021352105, 2024-05-08T06:14:27.551 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.032986773, 2024-05-08T06:14:27.551 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.021030219, 2024-05-08T06:14:27.551 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:14:27.551 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 549, 2024-05-08T06:14:27.551 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.025329791, 2024-05-08T06:14:27.551 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000046138 2024-05-08T06:14:27.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.551 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:14:27.551 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.551 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.552 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.552 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 821040, 2024-05-08T06:14:27.552 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 94480 2024-05-08T06:14:27.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.552 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T06:14:27.552 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 490, 2024-05-08T06:14:27.552 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 604, 2024-05-08T06:14:27.552 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 164464, 2024-05-08T06:14:27.552 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 421998, 2024-05-08T06:14:27.552 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 4, 2024-05-08T06:14:27.552 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 3, 2024-05-08T06:14:27.552 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.065503374, 2024-05-08T06:14:27.552 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.023565844, 2024-05-08T06:14:27.553 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.016392923, 2024-05-08T06:14:27.553 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.018798399, 2024-05-08T06:14:27.553 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:14:27.553 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 602, 2024-05-08T06:14:27.553 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.033619557, 2024-05-08T06:14:27.553 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000055846 2024-05-08T06:14:27.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.553 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:14:27.553 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.553 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.553 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.553 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-08T06:14:27.553 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 266496 2024-05-08T06:14:27.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.554 INFO:teuthology.orchestra.run.smithi192.stdout: "cct": { 2024-05-08T06:14:27.554 INFO:teuthology.orchestra.run.smithi192.stdout: "total_workers": 1, 2024-05-08T06:14:27.554 INFO:teuthology.orchestra.run.smithi192.stdout: "unhealthy_workers": 0 2024-05-08T06:14:27.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.554 INFO:teuthology.orchestra.run.smithi192.stdout: "finisher-MDSRank": { 2024-05-08T06:14:27.554 INFO:teuthology.orchestra.run.smithi192.stdout: "queue_len": 0, 2024-05-08T06:14:27.554 INFO:teuthology.orchestra.run.smithi192.stdout: "complete_latency": { 2024-05-08T06:14:27.554 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 2315, 2024-05-08T06:14:27.554 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.077432780, 2024-05-08T06:14:27.554 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000033448 2024-05-08T06:14:27.554 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T06:14:27.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.554 INFO:teuthology.orchestra.run.smithi192.stdout: "finisher-PurgeQueue": { 2024-05-08T06:14:27.554 INFO:teuthology.orchestra.run.smithi192.stdout: "queue_len": 0, 2024-05-08T06:14:27.555 INFO:teuthology.orchestra.run.smithi192.stdout: "complete_latency": { 2024-05-08T06:14:27.555 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.555 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.555 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.555 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T06:14:27.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.555 INFO:teuthology.orchestra.run.smithi192.stdout: "mds": { 2024-05-08T06:14:27.555 INFO:teuthology.orchestra.run.smithi192.stdout: "request": 0, 2024-05-08T06:14:27.555 INFO:teuthology.orchestra.run.smithi192.stdout: "reply": 0, 2024-05-08T06:14:27.555 INFO:teuthology.orchestra.run.smithi192.stdout: "reply_latency": { 2024-05-08T06:14:27.555 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.555 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.555 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.556 INFO:teuthology.orchestra.run.smithi192.stdout: "slow_reply": 0, 2024-05-08T06:14:27.556 INFO:teuthology.orchestra.run.smithi192.stdout: "forward": 0, 2024-05-08T06:14:27.556 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_fetch_complete": 0, 2024-05-08T06:14:27.556 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_fetch_keys": 0, 2024-05-08T06:14:27.556 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_commit": 0, 2024-05-08T06:14:27.556 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_split": 0, 2024-05-08T06:14:27.556 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_merge": 0, 2024-05-08T06:14:27.556 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes": 0, 2024-05-08T06:14:27.556 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_top": 0, 2024-05-08T06:14:27.556 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_bottom": 0, 2024-05-08T06:14:27.556 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_pin_tail": 0, 2024-05-08T06:14:27.556 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_pinned": 0, 2024-05-08T06:14:27.556 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_expired": 0, 2024-05-08T06:14:27.556 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_with_caps": 0, 2024-05-08T06:14:27.557 INFO:teuthology.orchestra.run.smithi192.stdout: "caps": 0, 2024-05-08T06:14:27.557 INFO:teuthology.orchestra.run.smithi192.stdout: "subtrees": 1, 2024-05-08T06:14:27.557 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse": 0, 2024-05-08T06:14:27.557 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_hit": 0, 2024-05-08T06:14:27.557 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_forward": 0, 2024-05-08T06:14:27.557 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_discover": 0, 2024-05-08T06:14:27.557 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_dir_fetch": 0, 2024-05-08T06:14:27.557 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_remote_ino": 0, 2024-05-08T06:14:27.557 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_lock": 0, 2024-05-08T06:14:27.557 INFO:teuthology.orchestra.run.smithi192.stdout: "load_cent": 0, 2024-05-08T06:14:27.557 INFO:teuthology.orchestra.run.smithi192.stdout: "q": 0, 2024-05-08T06:14:27.557 INFO:teuthology.orchestra.run.smithi192.stdout: "exported": 0, 2024-05-08T06:14:27.557 INFO:teuthology.orchestra.run.smithi192.stdout: "exported_inodes": 0, 2024-05-08T06:14:27.557 INFO:teuthology.orchestra.run.smithi192.stdout: "imported": 0, 2024-05-08T06:14:27.557 INFO:teuthology.orchestra.run.smithi192.stdout: "imported_inodes": 0, 2024-05-08T06:14:27.557 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_dir_fetch": 0, 2024-05-08T06:14:27.558 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_backtrace_fetch": 0, 2024-05-08T06:14:27.558 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_peer_discover": 0, 2024-05-08T06:14:27.558 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rfiles": 0, 2024-05-08T06:14:27.558 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rbytes": 0, 2024-05-08T06:14:27.558 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rsnaps": 0, 2024-05-08T06:14:27.558 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T06:14:27.558 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_set_tag": 0, 2024-05-08T06:14:27.558 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T06:14:27.558 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_inotable_repaired": 0, 2024-05-08T06:14:27.558 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dir_inodes": 0, 2024-05-08T06:14:27.558 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T06:14:27.558 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T06:14:27.558 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_file_inodes": 0, 2024-05-08T06:14:27.558 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_inode_file_caps": 0, 2024-05-08T06:14:27.559 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T06:14:27.559 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_grant": 0, 2024-05-08T06:14:27.559 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T06:14:27.559 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T06:14:27.559 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T06:14:27.559 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_caps": 0, 2024-05-08T06:14:27.559 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_caps_dirty": 0, 2024-05-08T06:14:27.559 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_cap_release": 0, 2024-05-08T06:14:27.559 INFO:teuthology.orchestra.run.smithi192.stdout: "process_request_cap_release": 0 2024-05-08T06:14:27.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.559 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_cache": { 2024-05-08T06:14:27.559 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_update": 0, 2024-05-08T06:14:27.560 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_update_receipt": 0, 2024-05-08T06:14:27.560 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_try_discover": 0, 2024-05-08T06:14:27.560 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_send_discover": 0, 2024-05-08T06:14:27.560 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_handle_discover": 0, 2024-05-08T06:14:27.560 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays": 0, 2024-05-08T06:14:27.560 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays_delayed": 0, 2024-05-08T06:14:27.560 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays_enqueuing": 0, 2024-05-08T06:14:27.560 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_created": 0, 2024-05-08T06:14:27.560 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_enqueued": 0, 2024-05-08T06:14:27.560 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_reintegrated": 0, 2024-05-08T06:14:27.560 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_migrated": 0, 2024-05-08T06:14:27.560 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_processing": 0, 2024-05-08T06:14:27.560 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_enqueued": 0, 2024-05-08T06:14:27.560 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_prioritized": 0, 2024-05-08T06:14:27.560 INFO:teuthology.orchestra.run.smithi192.stdout: "recovery_started": 0, 2024-05-08T06:14:27.560 INFO:teuthology.orchestra.run.smithi192.stdout: "recovery_completed": 0, 2024-05-08T06:14:27.561 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_quiesce_path": 0, 2024-05-08T06:14:27.561 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_quiesce_inode": 0, 2024-05-08T06:14:27.561 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T06:14:27.561 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_exportdir": 0, 2024-05-08T06:14:27.561 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_flush": 0, 2024-05-08T06:14:27.561 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_fragmentdir": 0, 2024-05-08T06:14:27.561 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_fragstats": 0, 2024-05-08T06:14:27.561 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_inodestats": 0 2024-05-08T06:14:27.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.561 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_log": { 2024-05-08T06:14:27.561 INFO:teuthology.orchestra.run.smithi192.stdout: "evlrg": 0, 2024-05-08T06:14:27.562 INFO:teuthology.orchestra.run.smithi192.stdout: "evadd": 0, 2024-05-08T06:14:27.562 INFO:teuthology.orchestra.run.smithi192.stdout: "evex": 0, 2024-05-08T06:14:27.562 INFO:teuthology.orchestra.run.smithi192.stdout: "evtrm": 0, 2024-05-08T06:14:27.562 INFO:teuthology.orchestra.run.smithi192.stdout: "ev": 6, 2024-05-08T06:14:27.562 INFO:teuthology.orchestra.run.smithi192.stdout: "evexg": 0, 2024-05-08T06:14:27.562 INFO:teuthology.orchestra.run.smithi192.stdout: "evexd": 0, 2024-05-08T06:14:27.562 INFO:teuthology.orchestra.run.smithi192.stdout: "segadd": 0, 2024-05-08T06:14:27.562 INFO:teuthology.orchestra.run.smithi192.stdout: "segex": 0, 2024-05-08T06:14:27.562 INFO:teuthology.orchestra.run.smithi192.stdout: "segtrm": 0, 2024-05-08T06:14:27.562 INFO:teuthology.orchestra.run.smithi192.stdout: "seg": 2, 2024-05-08T06:14:27.562 INFO:teuthology.orchestra.run.smithi192.stdout: "segmjr": 2, 2024-05-08T06:14:27.562 INFO:teuthology.orchestra.run.smithi192.stdout: "segexg": 0, 2024-05-08T06:14:27.562 INFO:teuthology.orchestra.run.smithi192.stdout: "segexd": 0, 2024-05-08T06:14:27.562 INFO:teuthology.orchestra.run.smithi192.stdout: "expos": 4194304, 2024-05-08T06:14:27.562 INFO:teuthology.orchestra.run.smithi192.stdout: "wrpos": 4195956, 2024-05-08T06:14:27.563 INFO:teuthology.orchestra.run.smithi192.stdout: "rdpos": 4195815, 2024-05-08T06:14:27.563 INFO:teuthology.orchestra.run.smithi192.stdout: "jlat": { 2024-05-08T06:14:27.563 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.563 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.563 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.563 INFO:teuthology.orchestra.run.smithi192.stdout: "replayed": 6 2024-05-08T06:14:27.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.563 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_mem": { 2024-05-08T06:14:27.563 INFO:teuthology.orchestra.run.smithi192.stdout: "ino": 3, 2024-05-08T06:14:27.563 INFO:teuthology.orchestra.run.smithi192.stdout: "ino+": 3, 2024-05-08T06:14:27.563 INFO:teuthology.orchestra.run.smithi192.stdout: "ino-": 0, 2024-05-08T06:14:27.563 INFO:teuthology.orchestra.run.smithi192.stdout: "dir": 1, 2024-05-08T06:14:27.563 INFO:teuthology.orchestra.run.smithi192.stdout: "dir+": 1, 2024-05-08T06:14:27.563 INFO:teuthology.orchestra.run.smithi192.stdout: "dir-": 0, 2024-05-08T06:14:27.564 INFO:teuthology.orchestra.run.smithi192.stdout: "dn": 0, 2024-05-08T06:14:27.564 INFO:teuthology.orchestra.run.smithi192.stdout: "dn+": 0, 2024-05-08T06:14:27.564 INFO:teuthology.orchestra.run.smithi192.stdout: "dn-": 0, 2024-05-08T06:14:27.564 INFO:teuthology.orchestra.run.smithi192.stdout: "cap": 0, 2024-05-08T06:14:27.564 INFO:teuthology.orchestra.run.smithi192.stdout: "cap+": 0, 2024-05-08T06:14:27.564 INFO:teuthology.orchestra.run.smithi192.stdout: "cap-": 0, 2024-05-08T06:14:27.564 INFO:teuthology.orchestra.run.smithi192.stdout: "rss": 50188, 2024-05-08T06:14:27.564 INFO:teuthology.orchestra.run.smithi192.stdout: "heap": 215416 2024-05-08T06:14:27.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.564 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_server": { 2024-05-08T06:14:27.564 INFO:teuthology.orchestra.run.smithi192.stdout: "dispatch_client_request": 0, 2024-05-08T06:14:27.564 INFO:teuthology.orchestra.run.smithi192.stdout: "dispatch_server_request": 0, 2024-05-08T06:14:27.564 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_request": 0, 2024-05-08T06:14:27.564 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_session": 0, 2024-05-08T06:14:27.564 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_peer_request": 0, 2024-05-08T06:14:27.564 INFO:teuthology.orchestra.run.smithi192.stdout: "req_create_latency": { 2024-05-08T06:14:27.565 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.565 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.565 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.565 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getattr_latency": { 2024-05-08T06:14:27.565 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.565 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.565 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.565 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getfilelock_latency": { 2024-05-08T06:14:27.565 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.565 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.565 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.565 INFO:teuthology.orchestra.run.smithi192.stdout: "req_link_latency": { 2024-05-08T06:14:27.565 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.566 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.566 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.566 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookup_latency": { 2024-05-08T06:14:27.566 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.566 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.566 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.566 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookuphash_latency": { 2024-05-08T06:14:27.566 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.566 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.566 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.566 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupino_latency": { 2024-05-08T06:14:27.566 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.567 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.567 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.567 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupname_latency": { 2024-05-08T06:14:27.567 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.567 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.567 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.567 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupparent_latency": { 2024-05-08T06:14:27.567 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.567 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.567 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.567 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupsnap_latency": { 2024-05-08T06:14:27.567 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.567 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.567 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.568 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lssnap_latency": { 2024-05-08T06:14:27.568 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.568 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.568 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.568 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mkdir_latency": { 2024-05-08T06:14:27.568 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.568 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.569 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.569 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mknod_latency": { 2024-05-08T06:14:27.569 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.569 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.569 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.569 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mksnap_latency": { 2024-05-08T06:14:27.569 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.569 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.569 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: "req_open_latency": { 2024-05-08T06:14:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: "req_readdir_latency": { 2024-05-08T06:14:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rename_latency": { 2024-05-08T06:14:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: "req_renamesnap_latency": { 2024-05-08T06:14:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: "req_snapdiff_latency": { 2024-05-08T06:14:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmdir_latency": { 2024-05-08T06:14:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmsnap_latency": { 2024-05-08T06:14:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmxattr_latency": { 2024-05-08T06:14:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setattr_latency": { 2024-05-08T06:14:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setdirlayout_latency": { 2024-05-08T06:14:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setfilelock_latency": { 2024-05-08T06:14:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setlayout_latency": { 2024-05-08T06:14:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setxattr_latency": { 2024-05-08T06:14:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: "req_symlink_latency": { 2024-05-08T06:14:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: "req_unlink_latency": { 2024-05-08T06:14:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: "cap_revoke_eviction": 0, 2024-05-08T06:14:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: "cap_acquisition_throttle": 0, 2024-05-08T06:14:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getvxattr_latency": { 2024-05-08T06:14:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T06:14:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_sessions": { 2024-05-08T06:14:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: "session_count": 0, 2024-05-08T06:14:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: "session_add": 2, 2024-05-08T06:14:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: "session_remove": 2, 2024-05-08T06:14:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: "sessions_open": 1, 2024-05-08T06:14:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: "sessions_stale": 0, 2024-05-08T06:14:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: "total_load": 0, 2024-05-08T06:14:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: "average_load": 0, 2024-05-08T06:14:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: "avg_session_uptime": 1757, 2024-05-08T06:14:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: "mdthresh_evicted": 0 2024-05-08T06:14:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: "mempool": { 2024-05-08T06:14:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_bytes": 0, 2024-05-08T06:14:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_items": 0, 2024-05-08T06:14:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T06:14:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_items": 0, 2024-05-08T06:14:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T06:14:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_items": 0, 2024-05-08T06:14:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T06:14:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T06:14:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T06:14:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T06:14:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T06:14:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_items": 0, 2024-05-08T06:14:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T06:14:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T06:14:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_bytes": 0, 2024-05-08T06:14:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_items": 0, 2024-05-08T06:14:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_bytes": 0, 2024-05-08T06:14:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_items": 0, 2024-05-08T06:14:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T06:14:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T06:14:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T06:14:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T06:14:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T06:14:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_items": 0, 2024-05-08T06:14:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_bytes": 0, 2024-05-08T06:14:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_items": 0, 2024-05-08T06:14:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T06:14:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T06:14:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_bytes": 0, 2024-05-08T06:14:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_items": 0, 2024-05-08T06:14:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_bytes": 0, 2024-05-08T06:14:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_items": 0, 2024-05-08T06:14:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T06:14:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_items": 0, 2024-05-08T06:14:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T06:14:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_items": 0, 2024-05-08T06:14:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_bytes": 189378, 2024-05-08T06:14:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_items": 51, 2024-05-08T06:14:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_bytes": 0, 2024-05-08T06:14:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_items": 0, 2024-05-08T06:14:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_bytes": 0, 2024-05-08T06:14:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_items": 0, 2024-05-08T06:14:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_bytes": 0, 2024-05-08T06:14:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_items": 0, 2024-05-08T06:14:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_bytes": 0, 2024-05-08T06:14:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_items": 0, 2024-05-08T06:14:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_bytes": 7528, 2024-05-08T06:14:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_items": 158, 2024-05-08T06:14:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T06:14:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_items": 0, 2024-05-08T06:14:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_bytes": 0, 2024-05-08T06:14:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_items": 0, 2024-05-08T06:14:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_bytes": 8512, 2024-05-08T06:14:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_items": 13, 2024-05-08T06:14:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_bytes": 0, 2024-05-08T06:14:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_items": 0, 2024-05-08T06:14:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_bytes": 0, 2024-05-08T06:14:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_items": 0 2024-05-08T06:14:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: "objecter": { 2024-05-08T06:14:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: "op_active": 0, 2024-05-08T06:14:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: "op_laggy": 0, 2024-05-08T06:14:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: "op_send": 1374, 2024-05-08T06:14:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: "op_send_bytes": 21, 2024-05-08T06:14:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: "op_resend": 3, 2024-05-08T06:14:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: "op_reply": 1374, 2024-05-08T06:14:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: "op_latency": { 2024-05-08T06:14:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 1374, 2024-05-08T06:14:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 1.118977861, 2024-05-08T06:14:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000814394 2024-05-08T06:14:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: "op_inflight": 0, 2024-05-08T06:14:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: "oplen_avg": { 2024-05-08T06:14:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 1374, 2024-05-08T06:14:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 1375 2024-05-08T06:14:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: "op": 1374, 2024-05-08T06:14:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: "op_r": 1374, 2024-05-08T06:14:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: "op_w": 0, 2024-05-08T06:14:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: "op_rmw": 0, 2024-05-08T06:14:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: "op_pg": 0, 2024-05-08T06:14:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_stat": 875, 2024-05-08T06:14:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_create": 0, 2024-05-08T06:14:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_read": 497, 2024-05-08T06:14:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_write": 0, 2024-05-08T06:14:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_writefull": 0, 2024-05-08T06:14:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_writesame": 0, 2024-05-08T06:14:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_append": 0, 2024-05-08T06:14:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_zero": 0, 2024-05-08T06:14:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_truncate": 0, 2024-05-08T06:14:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_delete": 0, 2024-05-08T06:14:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_mapext": 0, 2024-05-08T06:14:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_sparse_read": 0, 2024-05-08T06:14:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_clonerange": 0, 2024-05-08T06:14:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_getxattr": 1, 2024-05-08T06:14:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_setxattr": 0, 2024-05-08T06:14:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_cmpxattr": 0, 2024-05-08T06:14:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_rmxattr": 0, 2024-05-08T06:14:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_resetxattrs": 0, 2024-05-08T06:14:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_call": 0, 2024-05-08T06:14:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_watch": 0, 2024-05-08T06:14:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_notify": 0, 2024-05-08T06:14:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T06:14:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_pgls": 0, 2024-05-08T06:14:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_pgls_filter": 0, 2024-05-08T06:14:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_other": 0, 2024-05-08T06:14:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_active": 0, 2024-05-08T06:14:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_send": 0, 2024-05-08T06:14:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_resend": 0, 2024-05-08T06:14:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_ping": 0, 2024-05-08T06:14:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_active": 0, 2024-05-08T06:14:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_send": 0, 2024-05-08T06:14:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_resend": 0, 2024-05-08T06:14:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_active": 0, 2024-05-08T06:14:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_send": 0, 2024-05-08T06:14:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_resend": 0, 2024-05-08T06:14:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_active": 0, 2024-05-08T06:14:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_send": 0, 2024-05-08T06:14:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_resend": 0, 2024-05-08T06:14:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: "command_active": 0, 2024-05-08T06:14:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: "command_send": 0, 2024-05-08T06:14:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: "command_resend": 0, 2024-05-08T06:14:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: "map_epoch": 82, 2024-05-08T06:14:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: "map_full": 0, 2024-05-08T06:14:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: "map_inc": 2, 2024-05-08T06:14:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_sessions": 5, 2024-05-08T06:14:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_session_open": 5, 2024-05-08T06:14:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_session_close": 0, 2024-05-08T06:14:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_laggy": 0, 2024-05-08T06:14:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_wr": 0, 2024-05-08T06:14:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_rd": 2, 2024-05-08T06:14:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_del": 0 2024-05-08T06:14:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: "oft": { 2024-05-08T06:14:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_objs": 0, 2024-05-08T06:14:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_kv_pairs": 0, 2024-05-08T06:14:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_updates": 0, 2024-05-08T06:14:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_removes": 0 2024-05-08T06:14:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: "osd-slow-ops": { 2024-05-08T06:14:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: "slow_ops_count": 0 2024-05-08T06:14:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: "purge_queue": { 2024-05-08T06:14:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_ops": 0, 2024-05-08T06:14:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T06:14:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing": 0, 2024-05-08T06:14:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_high_water": 0, 2024-05-08T06:14:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executed_ops": 0, 2024-05-08T06:14:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executed": 0, 2024-05-08T06:14:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_item_in_journal": 0 2024-05-08T06:14:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T06:14:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-05-08T06:14:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2024-05-08T06:14:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-05-08T06:14:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 1541, 2024-05-08T06:14:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 1063136, 2024-05-08T06:14:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-05-08T06:14:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 1541, 2024-05-08T06:14:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-05-08T06:14:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-05-08T06:14:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 1541, 2024-05-08T06:14:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 1063136, 2024-05-08T06:14:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-05-08T06:14:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T06:14:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-objecter_bytes": { 2024-05-08T06:14:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-05-08T06:14:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2024-05-08T06:14:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-05-08T06:14:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-05-08T06:14:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-05-08T06:14:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-05-08T06:14:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-05-08T06:14:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 1374, 2024-05-08T06:14:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 1657, 2024-05-08T06:14:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 1374, 2024-05-08T06:14:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 1657, 2024-05-08T06:14:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-05-08T06:14:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T06:14:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-objecter_ops": { 2024-05-08T06:14:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-05-08T06:14:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 1024, 2024-05-08T06:14:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-05-08T06:14:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-05-08T06:14:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-05-08T06:14:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-05-08T06:14:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-05-08T06:14:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 1374, 2024-05-08T06:14:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 1374, 2024-05-08T06:14:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 1374, 2024-05-08T06:14:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 1374, 2024-05-08T06:14:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-05-08T06:14:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T06:14:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-write_buf_throttle": { 2024-05-08T06:14:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-05-08T06:14:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 3758096384, 2024-05-08T06:14:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-05-08T06:14:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-05-08T06:14:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-05-08T06:14:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-05-08T06:14:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-05-08T06:14:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-05-08T06:14:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-05-08T06:14:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 0, 2024-05-08T06:14:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 0, 2024-05-08T06:14:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-05-08T06:14:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T06:14:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-write_buf_throttle-0x55dd434dc6e0": { 2024-05-08T06:14:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-05-08T06:14:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 3758096384, 2024-05-08T06:14:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-05-08T06:14:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-05-08T06:14:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-05-08T06:14:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-05-08T06:14:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-05-08T06:14:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-05-08T06:14:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-05-08T06:14:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 0, 2024-05-08T06:14:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 0, 2024-05-08T06:14:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-05-08T06:14:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T06:14:27.592 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T06:14:27.592 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T06:14:28.327 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-08T06:14:28.327 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-05-08T06:14:28.327 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-05-08T06:14:28.327 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-05-08T06:14:28.327 DEBUG:tasks.check_counter:Getting stats from i 2024-05-08T06:14:28.328 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-08T06:14:28.565 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T06:14:28.686 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:28 smithi192 ceph-mon[34664]: pgmap v580: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:14:28.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:28 smithi123 ceph-mon[29787]: pgmap v580: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:14:28.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:28 smithi163 ceph-mon[34520]: pgmap v580: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:14:30.256 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T06:14:30.256 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T06:14:30.256 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 443, 2024-05-08T06:14:30.256 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 413, 2024-05-08T06:14:30.256 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 128227, 2024-05-08T06:14:30.256 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 105377, 2024-05-08T06:14:30.256 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 8, 2024-05-08T06:14:30.256 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 2, 2024-05-08T06:14:30.256 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.042619426, 2024-05-08T06:14:30.257 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.016640827, 2024-05-08T06:14:30.257 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.021096315, 2024-05-08T06:14:30.257 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.003327629, 2024-05-08T06:14:30.257 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:14:30.257 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 413, 2024-05-08T06:14:30.257 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.035339421, 2024-05-08T06:14:30.257 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000085567 2024-05-08T06:14:30.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.257 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:14:30.257 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 795, 2024-05-08T06:14:30.257 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000212093, 2024-05-08T06:14:30.257 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000266 2024-05-08T06:14:30.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.258 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T06:14:30.258 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T06:14:30.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.258 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T06:14:30.258 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 212, 2024-05-08T06:14:30.258 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 201, 2024-05-08T06:14:30.258 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 834047, 2024-05-08T06:14:30.258 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 133791, 2024-05-08T06:14:30.258 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 7, 2024-05-08T06:14:30.258 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 4, 2024-05-08T06:14:30.258 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.032824491, 2024-05-08T06:14:30.258 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.010492288, 2024-05-08T06:14:30.258 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.022620918, 2024-05-08T06:14:30.259 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.004339544, 2024-05-08T06:14:30.259 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:14:30.259 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 201, 2024-05-08T06:14:30.259 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.021875757, 2024-05-08T06:14:30.259 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000108834 2024-05-08T06:14:30.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.259 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:14:30.259 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 2, 2024-05-08T06:14:30.259 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000184, 2024-05-08T06:14:30.259 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000092 2024-05-08T06:14:30.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.259 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 829968, 2024-05-08T06:14:30.259 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 97840 2024-05-08T06:14:30.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.259 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T06:14:30.260 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 28, 2024-05-08T06:14:30.260 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 144, 2024-05-08T06:14:30.260 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 5798, 2024-05-08T06:14:30.260 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 495476, 2024-05-08T06:14:30.260 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 21, 2024-05-08T06:14:30.260 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 8, 2024-05-08T06:14:30.260 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.014898169, 2024-05-08T06:14:30.260 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.005843323, 2024-05-08T06:14:30.260 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.007185671, 2024-05-08T06:14:30.260 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.000121966, 2024-05-08T06:14:30.260 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:14:30.260 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 140, 2024-05-08T06:14:30.260 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.013689511, 2024-05-08T06:14:30.260 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000097782 2024-05-08T06:14:30.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.261 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:14:30.261 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 29, 2024-05-08T06:14:30.261 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000003950, 2024-05-08T06:14:30.261 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000136 2024-05-08T06:14:30.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.261 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 2672, 2024-05-08T06:14:30.261 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 349248 2024-05-08T06:14:30.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.261 INFO:teuthology.orchestra.run.smithi192.stdout: "cct": { 2024-05-08T06:14:30.261 INFO:teuthology.orchestra.run.smithi192.stdout: "total_workers": 1, 2024-05-08T06:14:30.261 INFO:teuthology.orchestra.run.smithi192.stdout: "unhealthy_workers": 0 2024-05-08T06:14:30.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.261 INFO:teuthology.orchestra.run.smithi192.stdout: "finisher-MDSRank": { 2024-05-08T06:14:30.262 INFO:teuthology.orchestra.run.smithi192.stdout: "queue_len": 0, 2024-05-08T06:14:30.262 INFO:teuthology.orchestra.run.smithi192.stdout: "complete_latency": { 2024-05-08T06:14:30.262 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 57, 2024-05-08T06:14:30.262 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.001524919, 2024-05-08T06:14:30.262 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000026752 2024-05-08T06:14:30.262 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T06:14:30.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.262 INFO:teuthology.orchestra.run.smithi192.stdout: "finisher-PurgeQueue": { 2024-05-08T06:14:30.262 INFO:teuthology.orchestra.run.smithi192.stdout: "queue_len": 0, 2024-05-08T06:14:30.262 INFO:teuthology.orchestra.run.smithi192.stdout: "complete_latency": { 2024-05-08T06:14:30.262 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 2, 2024-05-08T06:14:30.262 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000013118, 2024-05-08T06:14:30.262 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000006559 2024-05-08T06:14:30.263 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T06:14:30.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.263 INFO:teuthology.orchestra.run.smithi192.stdout: "mds": { 2024-05-08T06:14:30.263 INFO:teuthology.orchestra.run.smithi192.stdout: "request": 2, 2024-05-08T06:14:30.263 INFO:teuthology.orchestra.run.smithi192.stdout: "reply": 1, 2024-05-08T06:14:30.263 INFO:teuthology.orchestra.run.smithi192.stdout: "reply_latency": { 2024-05-08T06:14:30.263 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 1, 2024-05-08T06:14:30.263 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000160179, 2024-05-08T06:14:30.263 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000160179 2024-05-08T06:14:30.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.263 INFO:teuthology.orchestra.run.smithi192.stdout: "slow_reply": 0, 2024-05-08T06:14:30.263 INFO:teuthology.orchestra.run.smithi192.stdout: "forward": 1, 2024-05-08T06:14:30.263 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_fetch_complete": 0, 2024-05-08T06:14:30.263 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_fetch_keys": 0, 2024-05-08T06:14:30.263 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_commit": 11, 2024-05-08T06:14:30.264 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_split": 0, 2024-05-08T06:14:30.264 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_merge": 0, 2024-05-08T06:14:30.264 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes": 17, 2024-05-08T06:14:30.264 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_top": 1, 2024-05-08T06:14:30.264 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_bottom": 16, 2024-05-08T06:14:30.264 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_pin_tail": 0, 2024-05-08T06:14:30.264 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_pinned": 13, 2024-05-08T06:14:30.264 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_expired": 8, 2024-05-08T06:14:30.264 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_with_caps": 0, 2024-05-08T06:14:30.264 INFO:teuthology.orchestra.run.smithi192.stdout: "caps": 0, 2024-05-08T06:14:30.264 INFO:teuthology.orchestra.run.smithi192.stdout: "subtrees": 3, 2024-05-08T06:14:30.264 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse": 12, 2024-05-08T06:14:30.264 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_hit": 7, 2024-05-08T06:14:30.264 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_forward": 0, 2024-05-08T06:14:30.264 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_discover": 4, 2024-05-08T06:14:30.265 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_dir_fetch": 0, 2024-05-08T06:14:30.265 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_remote_ino": 0, 2024-05-08T06:14:30.265 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_lock": 0, 2024-05-08T06:14:30.265 INFO:teuthology.orchestra.run.smithi192.stdout: "load_cent": 0, 2024-05-08T06:14:30.265 INFO:teuthology.orchestra.run.smithi192.stdout: "q": 0, 2024-05-08T06:14:30.265 INFO:teuthology.orchestra.run.smithi192.stdout: "exported": 0, 2024-05-08T06:14:30.265 INFO:teuthology.orchestra.run.smithi192.stdout: "exported_inodes": 0, 2024-05-08T06:14:30.265 INFO:teuthology.orchestra.run.smithi192.stdout: "imported": 0, 2024-05-08T06:14:30.265 INFO:teuthology.orchestra.run.smithi192.stdout: "imported_inodes": 0, 2024-05-08T06:14:30.265 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_dir_fetch": 0, 2024-05-08T06:14:30.265 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_backtrace_fetch": 0, 2024-05-08T06:14:30.265 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_peer_discover": 0, 2024-05-08T06:14:30.265 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rfiles": 4, 2024-05-08T06:14:30.265 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rbytes": 234, 2024-05-08T06:14:30.265 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rsnaps": 0, 2024-05-08T06:14:30.266 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T06:14:30.266 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_set_tag": 0, 2024-05-08T06:14:30.266 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T06:14:30.266 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_inotable_repaired": 0, 2024-05-08T06:14:30.266 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dir_inodes": 0, 2024-05-08T06:14:30.266 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T06:14:30.266 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T06:14:30.266 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_file_inodes": 0, 2024-05-08T06:14:30.266 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_inode_file_caps": 0, 2024-05-08T06:14:30.266 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T06:14:30.266 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_grant": 0, 2024-05-08T06:14:30.266 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T06:14:30.266 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T06:14:30.267 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T06:14:30.267 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_caps": 0, 2024-05-08T06:14:30.267 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_caps_dirty": 0, 2024-05-08T06:14:30.267 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_cap_release": 0, 2024-05-08T06:14:30.267 INFO:teuthology.orchestra.run.smithi192.stdout: "process_request_cap_release": 1 2024-05-08T06:14:30.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.267 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_cache": { 2024-05-08T06:14:30.267 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_update": 0, 2024-05-08T06:14:30.267 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_update_receipt": 13, 2024-05-08T06:14:30.267 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_try_discover": 8, 2024-05-08T06:14:30.267 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_send_discover": 10, 2024-05-08T06:14:30.267 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_handle_discover": 0, 2024-05-08T06:14:30.267 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays": 0, 2024-05-08T06:14:30.267 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays_delayed": 0, 2024-05-08T06:14:30.267 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays_enqueuing": 0, 2024-05-08T06:14:30.268 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_created": 0, 2024-05-08T06:14:30.268 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_enqueued": 0, 2024-05-08T06:14:30.268 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_reintegrated": 0, 2024-05-08T06:14:30.268 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_migrated": 0, 2024-05-08T06:14:30.268 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_processing": 0, 2024-05-08T06:14:30.268 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_enqueued": 0, 2024-05-08T06:14:30.268 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_prioritized": 0, 2024-05-08T06:14:30.268 INFO:teuthology.orchestra.run.smithi192.stdout: "recovery_started": 0, 2024-05-08T06:14:30.268 INFO:teuthology.orchestra.run.smithi192.stdout: "recovery_completed": 0, 2024-05-08T06:14:30.268 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_quiesce_path": 0, 2024-05-08T06:14:30.268 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_quiesce_inode": 0, 2024-05-08T06:14:30.268 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T06:14:30.268 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_exportdir": 0, 2024-05-08T06:14:30.268 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_flush": 0, 2024-05-08T06:14:30.269 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_fragmentdir": 0, 2024-05-08T06:14:30.269 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_fragstats": 0, 2024-05-08T06:14:30.269 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_inodestats": 0 2024-05-08T06:14:30.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.269 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_log": { 2024-05-08T06:14:30.269 INFO:teuthology.orchestra.run.smithi192.stdout: "evlrg": 0, 2024-05-08T06:14:30.269 INFO:teuthology.orchestra.run.smithi192.stdout: "evadd": 6, 2024-05-08T06:14:30.269 INFO:teuthology.orchestra.run.smithi192.stdout: "evex": 0, 2024-05-08T06:14:30.269 INFO:teuthology.orchestra.run.smithi192.stdout: "evtrm": 0, 2024-05-08T06:14:30.269 INFO:teuthology.orchestra.run.smithi192.stdout: "ev": 6, 2024-05-08T06:14:30.269 INFO:teuthology.orchestra.run.smithi192.stdout: "evexg": 0, 2024-05-08T06:14:30.269 INFO:teuthology.orchestra.run.smithi192.stdout: "evexd": 0, 2024-05-08T06:14:30.269 INFO:teuthology.orchestra.run.smithi192.stdout: "segadd": 2, 2024-05-08T06:14:30.269 INFO:teuthology.orchestra.run.smithi192.stdout: "segex": 0, 2024-05-08T06:14:30.269 INFO:teuthology.orchestra.run.smithi192.stdout: "segtrm": 0, 2024-05-08T06:14:30.270 INFO:teuthology.orchestra.run.smithi192.stdout: "seg": 2, 2024-05-08T06:14:30.270 INFO:teuthology.orchestra.run.smithi192.stdout: "segmjr": 2, 2024-05-08T06:14:30.270 INFO:teuthology.orchestra.run.smithi192.stdout: "segexg": 0, 2024-05-08T06:14:30.270 INFO:teuthology.orchestra.run.smithi192.stdout: "segexd": 0, 2024-05-08T06:14:30.270 INFO:teuthology.orchestra.run.smithi192.stdout: "expos": 4194304, 2024-05-08T06:14:30.270 INFO:teuthology.orchestra.run.smithi192.stdout: "wrpos": 4195956, 2024-05-08T06:14:30.270 INFO:teuthology.orchestra.run.smithi192.stdout: "rdpos": 0, 2024-05-08T06:14:30.270 INFO:teuthology.orchestra.run.smithi192.stdout: "jlat": { 2024-05-08T06:14:30.270 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 5, 2024-05-08T06:14:30.270 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.010387045, 2024-05-08T06:14:30.270 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.002077409 2024-05-08T06:14:30.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.270 INFO:teuthology.orchestra.run.smithi192.stdout: "replayed": 0 2024-05-08T06:14:30.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.271 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_mem": { 2024-05-08T06:14:30.271 INFO:teuthology.orchestra.run.smithi192.stdout: "ino": 21, 2024-05-08T06:14:30.271 INFO:teuthology.orchestra.run.smithi192.stdout: "ino+": 27, 2024-05-08T06:14:30.271 INFO:teuthology.orchestra.run.smithi192.stdout: "ino-": 6, 2024-05-08T06:14:30.271 INFO:teuthology.orchestra.run.smithi192.stdout: "dir": 20, 2024-05-08T06:14:30.271 INFO:teuthology.orchestra.run.smithi192.stdout: "dir+": 25, 2024-05-08T06:14:30.271 INFO:teuthology.orchestra.run.smithi192.stdout: "dir-": 5, 2024-05-08T06:14:30.271 INFO:teuthology.orchestra.run.smithi192.stdout: "dn": 17, 2024-05-08T06:14:30.271 INFO:teuthology.orchestra.run.smithi192.stdout: "dn+": 25, 2024-05-08T06:14:30.271 INFO:teuthology.orchestra.run.smithi192.stdout: "dn-": 8, 2024-05-08T06:14:30.271 INFO:teuthology.orchestra.run.smithi192.stdout: "cap": 0, 2024-05-08T06:14:30.271 INFO:teuthology.orchestra.run.smithi192.stdout: "cap+": 1, 2024-05-08T06:14:30.271 INFO:teuthology.orchestra.run.smithi192.stdout: "cap-": 1, 2024-05-08T06:14:30.271 INFO:teuthology.orchestra.run.smithi192.stdout: "rss": 52336, 2024-05-08T06:14:30.271 INFO:teuthology.orchestra.run.smithi192.stdout: "heap": 223608 2024-05-08T06:14:30.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.272 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_server": { 2024-05-08T06:14:30.272 INFO:teuthology.orchestra.run.smithi192.stdout: "dispatch_client_request": 3, 2024-05-08T06:14:30.272 INFO:teuthology.orchestra.run.smithi192.stdout: "dispatch_server_request": 0, 2024-05-08T06:14:30.272 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_request": 2, 2024-05-08T06:14:30.272 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_session": 8, 2024-05-08T06:14:30.272 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_peer_request": 0, 2024-05-08T06:14:30.272 INFO:teuthology.orchestra.run.smithi192.stdout: "req_create_latency": { 2024-05-08T06:14:30.272 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:30.272 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:30.272 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:30.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.272 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getattr_latency": { 2024-05-08T06:14:30.272 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 1, 2024-05-08T06:14:30.272 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000160179, 2024-05-08T06:14:30.273 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000160179 2024-05-08T06:14:30.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.273 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getfilelock_latency": { 2024-05-08T06:14:30.273 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:30.273 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:30.273 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:30.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.273 INFO:teuthology.orchestra.run.smithi192.stdout: "req_link_latency": { 2024-05-08T06:14:30.273 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:30.273 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:30.273 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:30.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.273 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookup_latency": { 2024-05-08T06:14:30.273 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:30.274 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:30.274 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:30.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.274 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookuphash_latency": { 2024-05-08T06:14:30.274 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:30.274 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:30.274 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:30.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.274 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupino_latency": { 2024-05-08T06:14:30.274 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:30.274 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:30.274 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:30.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.274 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupname_latency": { 2024-05-08T06:14:30.275 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:30.275 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:30.275 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:30.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.275 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupparent_latency": { 2024-05-08T06:14:30.275 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:30.275 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:30.275 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:30.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.275 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupsnap_latency": { 2024-05-08T06:14:30.275 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:30.275 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:30.275 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:30.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.275 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lssnap_latency": { 2024-05-08T06:14:30.275 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:30.276 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:30.276 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:30.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.276 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mkdir_latency": { 2024-05-08T06:14:30.276 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:30.276 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:30.277 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:30.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.277 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mknod_latency": { 2024-05-08T06:14:30.277 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:30.277 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:30.277 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:30.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.277 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mksnap_latency": { 2024-05-08T06:14:30.277 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:30.277 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:30.277 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:30.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.277 INFO:teuthology.orchestra.run.smithi192.stdout: "req_open_latency": { 2024-05-08T06:14:30.278 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:30.278 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:30.278 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:30.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.278 INFO:teuthology.orchestra.run.smithi192.stdout: "req_readdir_latency": { 2024-05-08T06:14:30.278 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:30.278 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:30.278 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:30.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.278 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rename_latency": { 2024-05-08T06:14:30.278 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:30.278 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:30.278 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:30.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.279 INFO:teuthology.orchestra.run.smithi192.stdout: "req_renamesnap_latency": { 2024-05-08T06:14:30.279 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:30.279 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:30.279 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:30.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.279 INFO:teuthology.orchestra.run.smithi192.stdout: "req_snapdiff_latency": { 2024-05-08T06:14:30.279 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:30.279 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:30.279 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:30.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.279 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmdir_latency": { 2024-05-08T06:14:30.279 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:30.279 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:30.279 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:30.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.280 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmsnap_latency": { 2024-05-08T06:14:30.280 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:30.280 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:30.280 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:30.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.280 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmxattr_latency": { 2024-05-08T06:14:30.280 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:30.280 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:30.280 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:30.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.280 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setattr_latency": { 2024-05-08T06:14:30.280 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:30.280 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:30.281 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:30.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.281 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setdirlayout_latency": { 2024-05-08T06:14:30.281 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:30.281 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:30.281 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:30.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.281 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setfilelock_latency": { 2024-05-08T06:14:30.281 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:30.281 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:30.281 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:30.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.281 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setlayout_latency": { 2024-05-08T06:14:30.281 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:30.281 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:30.282 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:30.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.282 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setxattr_latency": { 2024-05-08T06:14:30.282 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:30.282 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:30.282 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:30.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.282 INFO:teuthology.orchestra.run.smithi192.stdout: "req_symlink_latency": { 2024-05-08T06:14:30.282 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:30.282 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:30.282 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:30.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.282 INFO:teuthology.orchestra.run.smithi192.stdout: "req_unlink_latency": { 2024-05-08T06:14:30.282 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:30.283 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:30.283 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:30.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.283 INFO:teuthology.orchestra.run.smithi192.stdout: "cap_revoke_eviction": 0, 2024-05-08T06:14:30.283 INFO:teuthology.orchestra.run.smithi192.stdout: "cap_acquisition_throttle": 0, 2024-05-08T06:14:30.283 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getvxattr_latency": { 2024-05-08T06:14:30.283 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:30.283 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:30.283 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:30.283 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T06:14:30.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.283 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_sessions": { 2024-05-08T06:14:30.283 INFO:teuthology.orchestra.run.smithi192.stdout: "session_count": 0, 2024-05-08T06:14:30.283 INFO:teuthology.orchestra.run.smithi192.stdout: "session_add": 2, 2024-05-08T06:14:30.284 INFO:teuthology.orchestra.run.smithi192.stdout: "session_remove": 2, 2024-05-08T06:14:30.284 INFO:teuthology.orchestra.run.smithi192.stdout: "sessions_open": 0, 2024-05-08T06:14:30.284 INFO:teuthology.orchestra.run.smithi192.stdout: "sessions_stale": 0, 2024-05-08T06:14:30.284 INFO:teuthology.orchestra.run.smithi192.stdout: "total_load": 1, 2024-05-08T06:14:30.284 INFO:teuthology.orchestra.run.smithi192.stdout: "average_load": 1, 2024-05-08T06:14:30.284 INFO:teuthology.orchestra.run.smithi192.stdout: "avg_session_uptime": 73, 2024-05-08T06:14:30.284 INFO:teuthology.orchestra.run.smithi192.stdout: "mdthresh_evicted": 0 2024-05-08T06:14:30.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.284 INFO:teuthology.orchestra.run.smithi192.stdout: "mempool": { 2024-05-08T06:14:30.284 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_bytes": 0, 2024-05-08T06:14:30.284 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_items": 0, 2024-05-08T06:14:30.284 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T06:14:30.284 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_items": 0, 2024-05-08T06:14:30.284 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T06:14:30.284 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_items": 0, 2024-05-08T06:14:30.285 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T06:14:30.285 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T06:14:30.285 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T06:14:30.285 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T06:14:30.285 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T06:14:30.285 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_items": 0, 2024-05-08T06:14:30.285 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T06:14:30.285 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T06:14:30.285 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_bytes": 0, 2024-05-08T06:14:30.285 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_items": 0, 2024-05-08T06:14:30.285 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_bytes": 0, 2024-05-08T06:14:30.285 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_items": 0, 2024-05-08T06:14:30.285 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T06:14:30.285 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T06:14:30.285 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T06:14:30.286 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T06:14:30.286 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T06:14:30.286 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_items": 0, 2024-05-08T06:14:30.286 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_bytes": 0, 2024-05-08T06:14:30.286 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_items": 0, 2024-05-08T06:14:30.286 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T06:14:30.286 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T06:14:30.286 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_bytes": 0, 2024-05-08T06:14:30.286 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_items": 0, 2024-05-08T06:14:30.286 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_bytes": 0, 2024-05-08T06:14:30.286 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_items": 0, 2024-05-08T06:14:30.286 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T06:14:30.286 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_items": 0, 2024-05-08T06:14:30.286 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T06:14:30.287 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_items": 0, 2024-05-08T06:14:30.287 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_bytes": 230807, 2024-05-08T06:14:30.287 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_items": 77, 2024-05-08T06:14:30.287 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_bytes": 0, 2024-05-08T06:14:30.287 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_items": 0, 2024-05-08T06:14:30.287 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_bytes": 0, 2024-05-08T06:14:30.287 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_items": 0, 2024-05-08T06:14:30.287 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_bytes": 0, 2024-05-08T06:14:30.287 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_items": 0, 2024-05-08T06:14:30.287 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_bytes": 0, 2024-05-08T06:14:30.287 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_items": 0, 2024-05-08T06:14:30.287 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_bytes": 7528, 2024-05-08T06:14:30.287 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_items": 158, 2024-05-08T06:14:30.287 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T06:14:30.287 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_items": 0, 2024-05-08T06:14:30.287 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_bytes": 0, 2024-05-08T06:14:30.288 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_items": 0, 2024-05-08T06:14:30.288 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_bytes": 85117, 2024-05-08T06:14:30.288 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_items": 335, 2024-05-08T06:14:30.288 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_bytes": 0, 2024-05-08T06:14:30.288 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_items": 0, 2024-05-08T06:14:30.288 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_bytes": 0, 2024-05-08T06:14:30.288 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_items": 0 2024-05-08T06:14:30.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.288 INFO:teuthology.orchestra.run.smithi192.stdout: "objecter": { 2024-05-08T06:14:30.288 INFO:teuthology.orchestra.run.smithi192.stdout: "op_active": 0, 2024-05-08T06:14:30.288 INFO:teuthology.orchestra.run.smithi192.stdout: "op_laggy": 0, 2024-05-08T06:14:30.288 INFO:teuthology.orchestra.run.smithi192.stdout: "op_send": 42, 2024-05-08T06:14:30.288 INFO:teuthology.orchestra.run.smithi192.stdout: "op_send_bytes": 12044, 2024-05-08T06:14:30.288 INFO:teuthology.orchestra.run.smithi192.stdout: "op_resend": 28, 2024-05-08T06:14:30.288 INFO:teuthology.orchestra.run.smithi192.stdout: "op_reply": 42, 2024-05-08T06:14:30.288 INFO:teuthology.orchestra.run.smithi192.stdout: "op_latency": { 2024-05-08T06:14:30.289 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 42, 2024-05-08T06:14:30.289 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.335993372, 2024-05-08T06:14:30.289 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.007999842 2024-05-08T06:14:30.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.289 INFO:teuthology.orchestra.run.smithi192.stdout: "op_inflight": 0, 2024-05-08T06:14:30.289 INFO:teuthology.orchestra.run.smithi192.stdout: "oplen_avg": { 2024-05-08T06:14:30.289 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 42, 2024-05-08T06:14:30.289 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 64 2024-05-08T06:14:30.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.289 INFO:teuthology.orchestra.run.smithi192.stdout: "op": 42, 2024-05-08T06:14:30.289 INFO:teuthology.orchestra.run.smithi192.stdout: "op_r": 0, 2024-05-08T06:14:30.289 INFO:teuthology.orchestra.run.smithi192.stdout: "op_w": 42, 2024-05-08T06:14:30.289 INFO:teuthology.orchestra.run.smithi192.stdout: "op_rmw": 0, 2024-05-08T06:14:30.289 INFO:teuthology.orchestra.run.smithi192.stdout: "op_pg": 0, 2024-05-08T06:14:30.289 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_stat": 0, 2024-05-08T06:14:30.290 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_create": 10, 2024-05-08T06:14:30.290 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_read": 0, 2024-05-08T06:14:30.290 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_write": 5, 2024-05-08T06:14:30.290 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_writefull": 8, 2024-05-08T06:14:30.290 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_writesame": 0, 2024-05-08T06:14:30.290 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_append": 0, 2024-05-08T06:14:30.290 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_zero": 0, 2024-05-08T06:14:30.290 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_truncate": 0, 2024-05-08T06:14:30.290 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_delete": 6, 2024-05-08T06:14:30.290 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_mapext": 0, 2024-05-08T06:14:30.290 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_sparse_read": 0, 2024-05-08T06:14:30.290 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_clonerange": 0, 2024-05-08T06:14:30.290 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_getxattr": 0, 2024-05-08T06:14:30.290 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_setxattr": 20, 2024-05-08T06:14:30.291 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_cmpxattr": 0, 2024-05-08T06:14:30.291 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_rmxattr": 0, 2024-05-08T06:14:30.291 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_resetxattrs": 0, 2024-05-08T06:14:30.291 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_call": 0, 2024-05-08T06:14:30.291 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_watch": 0, 2024-05-08T06:14:30.291 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_notify": 0, 2024-05-08T06:14:30.291 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T06:14:30.291 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_pgls": 0, 2024-05-08T06:14:30.291 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_pgls_filter": 0, 2024-05-08T06:14:30.291 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_other": 0, 2024-05-08T06:14:30.291 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_active": 0, 2024-05-08T06:14:30.291 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_send": 0, 2024-05-08T06:14:30.291 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_resend": 0, 2024-05-08T06:14:30.291 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_ping": 0, 2024-05-08T06:14:30.292 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_active": 0, 2024-05-08T06:14:30.292 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_send": 0, 2024-05-08T06:14:30.292 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_resend": 0, 2024-05-08T06:14:30.292 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_active": 0, 2024-05-08T06:14:30.292 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_send": 0, 2024-05-08T06:14:30.292 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_resend": 0, 2024-05-08T06:14:30.292 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_active": 0, 2024-05-08T06:14:30.292 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_send": 0, 2024-05-08T06:14:30.292 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_resend": 0, 2024-05-08T06:14:30.292 INFO:teuthology.orchestra.run.smithi192.stdout: "command_active": 0, 2024-05-08T06:14:30.292 INFO:teuthology.orchestra.run.smithi192.stdout: "command_send": 0, 2024-05-08T06:14:30.292 INFO:teuthology.orchestra.run.smithi192.stdout: "command_resend": 0, 2024-05-08T06:14:30.292 INFO:teuthology.orchestra.run.smithi192.stdout: "map_epoch": 82, 2024-05-08T06:14:30.292 INFO:teuthology.orchestra.run.smithi192.stdout: "map_full": 0, 2024-05-08T06:14:30.292 INFO:teuthology.orchestra.run.smithi192.stdout: "map_inc": 2, 2024-05-08T06:14:30.293 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_sessions": 8, 2024-05-08T06:14:30.293 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_session_open": 8, 2024-05-08T06:14:30.293 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_session_close": 0, 2024-05-08T06:14:30.293 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_laggy": 0, 2024-05-08T06:14:30.293 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_wr": 14, 2024-05-08T06:14:30.293 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_rd": 0, 2024-05-08T06:14:30.293 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_del": 1 2024-05-08T06:14:30.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.293 INFO:teuthology.orchestra.run.smithi192.stdout: "oft": { 2024-05-08T06:14:30.293 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_objs": 1, 2024-05-08T06:14:30.293 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_kv_pairs": 0, 2024-05-08T06:14:30.293 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_updates": 0, 2024-05-08T06:14:30.293 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_removes": 0 2024-05-08T06:14:30.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.293 INFO:teuthology.orchestra.run.smithi192.stdout: "osd-slow-ops": { 2024-05-08T06:14:30.294 INFO:teuthology.orchestra.run.smithi192.stdout: "slow_ops_count": 0 2024-05-08T06:14:30.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.294 INFO:teuthology.orchestra.run.smithi192.stdout: "purge_queue": { 2024-05-08T06:14:30.294 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_ops": 0, 2024-05-08T06:14:30.294 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T06:14:30.294 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing": 0, 2024-05-08T06:14:30.294 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_high_water": 0, 2024-05-08T06:14:30.294 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executed_ops": 0, 2024-05-08T06:14:30.294 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executed": 0, 2024-05-08T06:14:30.294 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_item_in_journal": 0 2024-05-08T06:14:30.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.294 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T06:14:30.294 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-05-08T06:14:30.294 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2024-05-08T06:14:30.295 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-05-08T06:14:30.295 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 679, 2024-05-08T06:14:30.295 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 893762, 2024-05-08T06:14:30.295 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-05-08T06:14:30.295 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 679, 2024-05-08T06:14:30.295 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-05-08T06:14:30.295 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-05-08T06:14:30.295 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 679, 2024-05-08T06:14:30.295 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 893762, 2024-05-08T06:14:30.295 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-05-08T06:14:30.295 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:30.295 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:30.295 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:30.295 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T06:14:30.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.295 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-objecter_bytes": { 2024-05-08T06:14:30.296 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-05-08T06:14:30.296 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2024-05-08T06:14:30.296 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-05-08T06:14:30.296 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-05-08T06:14:30.296 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-05-08T06:14:30.296 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-05-08T06:14:30.296 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-05-08T06:14:30.296 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 42, 2024-05-08T06:14:30.296 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 12044, 2024-05-08T06:14:30.296 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 42, 2024-05-08T06:14:30.296 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 12044, 2024-05-08T06:14:30.296 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-05-08T06:14:30.296 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:30.297 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:30.297 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:30.297 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T06:14:30.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.297 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-objecter_ops": { 2024-05-08T06:14:30.297 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-05-08T06:14:30.297 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 1024, 2024-05-08T06:14:30.297 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-05-08T06:14:30.297 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-05-08T06:14:30.297 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-05-08T06:14:30.297 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-05-08T06:14:30.297 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-05-08T06:14:30.298 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 42, 2024-05-08T06:14:30.298 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 42, 2024-05-08T06:14:30.298 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 42, 2024-05-08T06:14:30.298 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 42, 2024-05-08T06:14:30.298 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-05-08T06:14:30.298 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:30.298 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:30.298 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:30.298 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T06:14:30.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.298 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-write_buf_throttle": { 2024-05-08T06:14:30.298 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-05-08T06:14:30.298 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 3758096384, 2024-05-08T06:14:30.298 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-05-08T06:14:30.298 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-05-08T06:14:30.299 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-05-08T06:14:30.299 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-05-08T06:14:30.299 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-05-08T06:14:30.299 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-05-08T06:14:30.299 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-05-08T06:14:30.299 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 0, 2024-05-08T06:14:30.299 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 0, 2024-05-08T06:14:30.299 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-05-08T06:14:30.299 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:30.299 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:30.299 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:30.299 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T06:14:30.299 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:30.299 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-write_buf_throttle-0x561d93b0a500": { 2024-05-08T06:14:30.299 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-05-08T06:14:30.299 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 3758096384, 2024-05-08T06:14:30.300 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-05-08T06:14:30.300 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 6, 2024-05-08T06:14:30.300 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 1652, 2024-05-08T06:14:30.300 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-05-08T06:14:30.300 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 6, 2024-05-08T06:14:30.300 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-05-08T06:14:30.300 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-05-08T06:14:30.300 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 5, 2024-05-08T06:14:30.300 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 1652, 2024-05-08T06:14:30.300 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-05-08T06:14:30.300 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:30.300 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:30.300 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:30.300 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T06:14:30.300 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T06:14:30.300 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T06:14:30.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:30 smithi192 ceph-mon[34664]: pgmap v581: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-05-08T06:14:30.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:30 smithi123 ceph-mon[29787]: pgmap v581: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-05-08T06:14:30.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:30 smithi163 ceph-mon[34520]: pgmap v581: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-05-08T06:14:31.038 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-05-08T06:14:31.038 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-05-08T06:14:31.038 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-05-08T06:14:31.038 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=13 2024-05-08T06:14:31.038 DEBUG:tasks.check_counter:Getting stats from l 2024-05-08T06:14:31.039 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-08T06:14:31.274 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T06:14:32.588 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:32 smithi192 ceph-mon[34664]: pgmap v582: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:14:32.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:32 smithi123 ceph-mon[29787]: pgmap v582: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:14:32.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:32 smithi163 ceph-mon[34520]: pgmap v582: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:14:32.943 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-08T06:14:32.943 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T06:14:32.943 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 0, 2024-05-08T06:14:32.943 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 2, 2024-05-08T06:14:32.943 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 0, 2024-05-08T06:14:32.943 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 0, 2024-05-08T06:14:32.944 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 4, 2024-05-08T06:14:32.944 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 0, 2024-05-08T06:14:32.944 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.001074059, 2024-05-08T06:14:32.944 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.000000000, 2024-05-08T06:14:32.944 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.000288612, 2024-05-08T06:14:32.944 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:14:32.944 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:14:32.944 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:32.944 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:32.944 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:32.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:32.944 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:14:32.945 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:32.945 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:32.945 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:32.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:32.945 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T06:14:32.945 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T06:14:32.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:32.945 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T06:14:32.945 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 5, 2024-05-08T06:14:32.945 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 122, 2024-05-08T06:14:32.945 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 3200, 2024-05-08T06:14:32.945 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 100176, 2024-05-08T06:14:32.945 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 3, 2024-05-08T06:14:32.945 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 2, 2024-05-08T06:14:32.946 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.007812923, 2024-05-08T06:14:32.946 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.005557676, 2024-05-08T06:14:32.946 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.000835663, 2024-05-08T06:14:32.946 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:14:32.946 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:14:32.946 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 121, 2024-05-08T06:14:32.946 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.009449977, 2024-05-08T06:14:32.946 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000078098 2024-05-08T06:14:32.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:32.946 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:14:32.946 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:32.946 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:32.946 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:32.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:32.947 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-05-08T06:14:32.947 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 100176 2024-05-08T06:14:32.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:32.947 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T06:14:32.947 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 164, 2024-05-08T06:14:32.947 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 159, 2024-05-08T06:14:32.947 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 797136, 2024-05-08T06:14:32.947 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 97328, 2024-05-08T06:14:32.947 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 2, 2024-05-08T06:14:32.947 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 1, 2024-05-08T06:14:32.947 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.030455658, 2024-05-08T06:14:32.947 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.009960607, 2024-05-08T06:14:32.947 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.020176197, 2024-05-08T06:14:32.948 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.004099462, 2024-05-08T06:14:32.948 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:14:32.948 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 158, 2024-05-08T06:14:32.948 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.008600277, 2024-05-08T06:14:32.948 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000054432 2024-05-08T06:14:32.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:32.948 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:14:32.948 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:32.948 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:32.948 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:32.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:32.948 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 797136, 2024-05-08T06:14:32.948 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 97328 2024-05-08T06:14:32.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:32.949 INFO:teuthology.orchestra.run.smithi192.stdout: "cct": { 2024-05-08T06:14:32.949 INFO:teuthology.orchestra.run.smithi192.stdout: "total_workers": 0, 2024-05-08T06:14:32.949 INFO:teuthology.orchestra.run.smithi192.stdout: "unhealthy_workers": 0 2024-05-08T06:14:32.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:32.949 INFO:teuthology.orchestra.run.smithi192.stdout: "mempool": { 2024-05-08T06:14:32.949 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_bytes": 0, 2024-05-08T06:14:32.949 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_items": 0, 2024-05-08T06:14:32.950 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T06:14:32.950 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_items": 0, 2024-05-08T06:14:32.950 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T06:14:32.950 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_items": 0, 2024-05-08T06:14:32.950 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T06:14:32.950 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T06:14:32.950 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T06:14:32.950 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T06:14:32.950 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T06:14:32.950 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_items": 0, 2024-05-08T06:14:32.950 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T06:14:32.950 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T06:14:32.951 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_bytes": 0, 2024-05-08T06:14:32.951 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_items": 0, 2024-05-08T06:14:32.951 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_bytes": 0, 2024-05-08T06:14:32.951 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_items": 0, 2024-05-08T06:14:32.951 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T06:14:32.951 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T06:14:32.951 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T06:14:32.951 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T06:14:32.951 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T06:14:32.951 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_items": 0, 2024-05-08T06:14:32.951 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_bytes": 0, 2024-05-08T06:14:32.951 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_items": 0, 2024-05-08T06:14:32.951 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T06:14:32.952 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T06:14:32.952 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_bytes": 0, 2024-05-08T06:14:32.952 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_items": 0, 2024-05-08T06:14:32.952 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_bytes": 0, 2024-05-08T06:14:32.952 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_items": 0, 2024-05-08T06:14:32.952 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T06:14:32.952 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_items": 0, 2024-05-08T06:14:32.952 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T06:14:32.952 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_items": 0, 2024-05-08T06:14:32.952 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_bytes": 148945, 2024-05-08T06:14:32.952 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_items": 29, 2024-05-08T06:14:32.952 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_bytes": 0, 2024-05-08T06:14:32.952 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_items": 0, 2024-05-08T06:14:32.952 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_bytes": 0, 2024-05-08T06:14:32.953 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_items": 0, 2024-05-08T06:14:32.953 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_bytes": 0, 2024-05-08T06:14:32.953 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_items": 0, 2024-05-08T06:14:32.953 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_bytes": 0, 2024-05-08T06:14:32.953 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_items": 0, 2024-05-08T06:14:32.953 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_bytes": 0, 2024-05-08T06:14:32.953 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_items": 0, 2024-05-08T06:14:32.953 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T06:14:32.953 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_items": 0, 2024-05-08T06:14:32.953 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_bytes": 0, 2024-05-08T06:14:32.953 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_items": 0, 2024-05-08T06:14:32.953 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_bytes": 928, 2024-05-08T06:14:32.953 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_items": 2, 2024-05-08T06:14:32.953 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_bytes": 0, 2024-05-08T06:14:32.954 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_items": 0, 2024-05-08T06:14:32.954 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_bytes": 0, 2024-05-08T06:14:32.954 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_items": 0 2024-05-08T06:14:32.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-08T06:14:32.954 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T06:14:32.954 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-05-08T06:14:32.954 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2024-05-08T06:14:32.954 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-05-08T06:14:32.954 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 165, 2024-05-08T06:14:32.954 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 773981, 2024-05-08T06:14:32.954 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-05-08T06:14:32.954 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 165, 2024-05-08T06:14:32.954 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-05-08T06:14:32.955 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-05-08T06:14:32.955 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 165, 2024-05-08T06:14:32.955 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 773981, 2024-05-08T06:14:32.955 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-05-08T06:14:32.955 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-05-08T06:14:32.955 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-05-08T06:14:32.955 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-05-08T06:14:32.955 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T06:14:32.955 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-08T06:14:32.955 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-08T06:14:33.647 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:33 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:14:33.675 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-05-08T06:14:33.675 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-05-08T06:14:33.675 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-05-08T06:14:33.675 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-05-08T06:14:33.675 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-08T06:14:33.687 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-08T06:14:33.687 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-05-08T06:14:33.687 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T06:14:33.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:33 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:14:33.739 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-05-08T06:14:33.739 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T06:14:33.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:33 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:14:33.831 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph osd blocklist ls 2024-05-08T06:14:34.255 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:14:34.603 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:34 smithi123 ceph-mon[29787]: pgmap v583: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:14:34.603 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:34 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:14:34.603 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:34 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:14:34.603 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:34 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:14:34.604 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:34 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:14:34.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:34 smithi163 ceph-mon[34520]: pgmap v583: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:14:34.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:34 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:14:34.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:34 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:14:34.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:34 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:14:34.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:34 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:14:34.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:34 smithi192 ceph-mon[34664]: pgmap v583: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T06:14:34.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:34 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:14:34.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:34 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:14:34.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:34 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:14:34.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:34 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:14:35.348 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:35 smithi192 ceph-mon[34664]: Adjusting osd_memory_target on smithi163 to 176.2M 2024-05-08T06:14:35.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:35 smithi123 ceph-mon[29787]: Adjusting osd_memory_target on smithi163 to 176.2M 2024-05-08T06:14:35.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:35 smithi123 ceph-mon[29787]: Unable to set osd_memory_target on smithi163 to 184781004: error parsing value: Value '184781004' is below minimum 939524096 2024-05-08T06:14:35.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:35 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:14:35.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:35 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:14:35.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:35 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:14:35.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:35 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:14:35.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:35 smithi123 ceph-mon[29787]: Adjusting osd_memory_target on smithi123 to 176.4M 2024-05-08T06:14:35.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:35 smithi123 ceph-mon[29787]: Unable to set osd_memory_target on smithi123 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-05-08T06:14:35.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:35 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:14:35.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:35 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:14:35.707 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:35 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:14:35.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:35 smithi163 ceph-mon[34520]: Adjusting osd_memory_target on smithi163 to 176.2M 2024-05-08T06:14:35.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:35 smithi163 ceph-mon[34520]: Unable to set osd_memory_target on smithi163 to 184781004: error parsing value: Value '184781004' is below minimum 939524096 2024-05-08T06:14:35.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:35 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:14:35.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:35 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:14:35.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:35 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:14:35.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:35 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:14:35.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:35 smithi163 ceph-mon[34520]: Adjusting osd_memory_target on smithi123 to 176.4M 2024-05-08T06:14:35.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:35 smithi163 ceph-mon[34520]: Unable to set osd_memory_target on smithi123 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-05-08T06:14:35.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:35 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:14:35.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:35 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:14:35.789 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:35 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:14:35.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:35 smithi192 ceph-mon[34664]: Unable to set osd_memory_target on smithi163 to 184781004: error parsing value: Value '184781004' is below minimum 939524096 2024-05-08T06:14:35.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:35 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:14:35.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:35 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:14:35.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:35 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:14:35.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:35 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:14:35.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:35 smithi192 ceph-mon[34664]: Adjusting osd_memory_target on smithi123 to 176.4M 2024-05-08T06:14:35.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:35 smithi192 ceph-mon[34664]: Unable to set osd_memory_target on smithi123 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-05-08T06:14:35.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:35 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:14:35.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:35 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:14:35.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:35 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' 2024-05-08T06:14:36.550 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:36 smithi192 ceph-mon[34664]: pgmap v584: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T06:14:36.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:36 smithi163 ceph-mon[34520]: pgmap v584: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T06:14:36.796 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:36 smithi123 ceph-mon[29787]: pgmap v584: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T06:14:36.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.801+0000 7f144b000700 1 -- 172.21.15.123:0/3026583794 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1444109510 msgr2=0x7f14441098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:14:36.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.801+0000 7f144b000700 1 --2- 172.21.15.123:0/3026583794 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1444109510 0x7f14441098f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f14340098f0 tx=0x7f1434009c00 comp rx=0 tx=0).stop 2024-05-08T06:14:36.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.801+0000 7f144b000700 1 -- 172.21.15.123:0/3026583794 shutdown_connections 2024-05-08T06:14:36.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.801+0000 7f144b000700 1 --2- 172.21.15.123:0/3026583794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f144406bb40 0x7f1444110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:36.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.801+0000 7f144b000700 1 --2- 172.21.15.123:0/3026583794 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1444109510 0x7f14441098f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:36.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.801+0000 7f144b000700 1 --2- 172.21.15.123:0/3026583794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f144406b180 0x7f144406b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:36.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.801+0000 7f144b000700 1 -- 172.21.15.123:0/3026583794 >> 172.21.15.123:0/3026583794 conn(0x7f1444079350 msgr2=0x7f1444079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:14:36.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.801+0000 7f144b000700 1 -- 172.21.15.123:0/3026583794 shutdown_connections 2024-05-08T06:14:36.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.801+0000 7f144b000700 1 -- 172.21.15.123:0/3026583794 wait complete. 2024-05-08T06:14:36.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.802+0000 7f144b000700 1 Processor -- start 2024-05-08T06:14:36.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.802+0000 7f144b000700 1 -- start start 2024-05-08T06:14:36.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.803+0000 7f144b000700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f144406b180 0x7f144419bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:14:36.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.803+0000 7f144b000700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f144406bb40 0x7f144419c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:14:36.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.803+0000 7f144b000700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1444109510 0x7f14441a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:14:36.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.803+0000 7f144b000700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1444113690 con 0x7f144406b180 2024-05-08T06:14:36.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.803+0000 7f144b000700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1444113510 con 0x7f144406bb40 2024-05-08T06:14:36.804 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.803+0000 7f144b000700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1444113810 con 0x7f1444109510 2024-05-08T06:14:36.805 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.803+0000 7f1443e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f144406b180 0x7f144419bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:14:36.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.803+0000 7f1443e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f144406b180 0x7f144419bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52520/0 (socket says 172.21.15.123:52520) 2024-05-08T06:14:36.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.803+0000 7f1443400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f144406bb40 0x7f144419c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:14:36.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.803+0000 7f1443e00700 1 -- 172.21.15.123:0/855019526 learned_addr learned my addr 172.21.15.123:0/855019526 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:14:36.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.803+0000 7f1449000700 1 --2- 172.21.15.123:0/855019526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1444109510 0x7f14441a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:14:36.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.804+0000 7f1443400700 1 -- 172.21.15.123:0/855019526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1444109510 msgr2=0x7f14441a07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:14:36.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.804+0000 7f1443400700 1 --2- 172.21.15.123:0/855019526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1444109510 0x7f14441a07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:36.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.804+0000 7f1443400700 1 -- 172.21.15.123:0/855019526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f144406b180 msgr2=0x7f144419bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:14:36.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.804+0000 7f1443400700 1 --2- 172.21.15.123:0/855019526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f144406b180 0x7f144419bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:36.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.804+0000 7f1443400700 1 -- 172.21.15.123:0/855019526 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14441a0f10 con 0x7f144406bb40 2024-05-08T06:14:36.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.804+0000 7f1449000700 1 --2- 172.21.15.123:0/855019526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1444109510 0x7f14441a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:14:36.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.804+0000 7f1443e00700 1 --2- 172.21.15.123:0/855019526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f144406b180 0x7f144419bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:14:36.806 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.804+0000 7f1443400700 1 --2- 172.21.15.123:0/855019526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f144406bb40 0x7f144419c440 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f143000e9f0 tx=0x7f143000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:14:36.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.804+0000 7f1440c00700 1 -- 172.21.15.123:0/855019526 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1430004220 con 0x7f144406bb40 2024-05-08T06:14:36.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.804+0000 7f144b000700 1 -- 172.21.15.123:0/855019526 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14441a1200 con 0x7f144406bb40 2024-05-08T06:14:36.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.804+0000 7f1440c00700 1 -- 172.21.15.123:0/855019526 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14300043c0 con 0x7f144406bb40 2024-05-08T06:14:36.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.804+0000 7f1440c00700 1 -- 172.21.15.123:0/855019526 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1430005020 con 0x7f144406bb40 2024-05-08T06:14:36.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.805+0000 7f144b000700 1 -- 172.21.15.123:0/855019526 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14441a8a80 con 0x7f144406bb40 2024-05-08T06:14:36.807 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.806+0000 7f144b000700 1 -- 172.21.15.123:0/855019526 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f144406a190 con 0x7f144406bb40 2024-05-08T06:14:36.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.809+0000 7f1440c00700 1 -- 172.21.15.123:0/855019526 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7f1430004750 con 0x7f144406bb40 2024-05-08T06:14:36.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.810+0000 7f1440c00700 1 --2- 172.21.15.123:0/855019526 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f14280b2880 0x7f14280b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:14:36.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.810+0000 7f1440c00700 1 -- 172.21.15.123:0/855019526 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f14300e21e0 con 0x7f144406bb40 2024-05-08T06:14:36.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.811+0000 7f1443e00700 1 --2- 172.21.15.123:0/855019526 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f14280b2880 0x7f14280b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:14:36.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.811+0000 7f1443e00700 1 --2- 172.21.15.123:0/855019526 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f14280b2880 0x7f14280b4d40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1434009fd0 tx=0x7f1434019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:14:36.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:36.815+0000 7f1440c00700 1 -- 172.21.15.123:0/855019526 <== mon.2 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f14300a9eb0 con 0x7f144406bb40 2024-05-08T06:14:37.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:37.105+0000 7f144b000700 1 -- 172.21.15.123:0/855019526 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f14441a8c50 con 0x7f144406bb40 2024-05-08T06:14:37.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:37.106+0000 7f1440c00700 1 -- 172.21.15.123:0/855019526 <== mon.2 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v82) ==== 81+0+599 (secure 0 0 0) 0x7f14300a97b0 con 0x7f144406bb40 2024-05-08T06:14:37.108 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:6801/3057688649 2024-05-09T05:57:30.887936+0000 2024-05-08T06:14:37.108 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:6800/3057688649 2024-05-09T05:57:30.887936+0000 2024-05-08T06:14:37.108 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:6800/2769724312 2024-05-09T05:56:55.849036+0000 2024-05-08T06:14:37.108 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:0/399884043 2024-05-09T05:57:30.887936+0000 2024-05-08T06:14:37.108 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:0/3406061897 2024-05-09T05:57:30.887936+0000 2024-05-08T06:14:37.108 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:0/644457268 2024-05-09T05:56:55.849036+0000 2024-05-08T06:14:37.108 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:0/3055092197 2024-05-09T05:56:55.849036+0000 2024-05-08T06:14:37.108 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:0/343390357 2024-05-09T05:57:30.887936+0000 2024-05-08T06:14:37.108 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:6801/2769724312 2024-05-09T05:56:55.849036+0000 2024-05-08T06:14:37.108 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123:0/2263171562 2024-05-09T05:56:55.849036+0000 2024-05-08T06:14:37.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:37.110+0000 7f144b000700 1 -- 172.21.15.123:0/855019526 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f14280b2880 msgr2=0x7f14280b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:14:37.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:37.110+0000 7f144b000700 1 --2- 172.21.15.123:0/855019526 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f14280b2880 0x7f14280b4d40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1434009fd0 tx=0x7f1434019040 comp rx=0 tx=0).stop 2024-05-08T06:14:37.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:37.110+0000 7f144b000700 1 -- 172.21.15.123:0/855019526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f144406bb40 msgr2=0x7f144419c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:14:37.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:37.110+0000 7f144b000700 1 --2- 172.21.15.123:0/855019526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f144406bb40 0x7f144419c440 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f143000e9f0 tx=0x7f143000ed00 comp rx=0 tx=0).stop 2024-05-08T06:14:37.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:37.110+0000 7f144b000700 1 -- 172.21.15.123:0/855019526 shutdown_connections 2024-05-08T06:14:37.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:37.110+0000 7f144b000700 1 --2- 172.21.15.123:0/855019526 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f14280b2880 0x7f14280b4d40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:37.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:37.110+0000 7f144b000700 1 --2- 172.21.15.123:0/855019526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f144406b180 0x7f144419bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:37.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:37.110+0000 7f144b000700 1 --2- 172.21.15.123:0/855019526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1444109510 0x7f14441a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:37.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:37.110+0000 7f144b000700 1 --2- 172.21.15.123:0/855019526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f144406bb40 0x7f144419c440 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:37.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:37.110+0000 7f144b000700 1 -- 172.21.15.123:0/855019526 >> 172.21.15.123:0/855019526 conn(0x7f1444079350 msgr2=0x7f144410e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:14:37.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:37.110+0000 7f144b000700 1 -- 172.21.15.123:0/855019526 shutdown_connections 2024-05-08T06:14:37.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:37.110+0000 7f144b000700 1 -- 172.21.15.123:0/855019526 wait complete. 2024-05-08T06:14:37.112 INFO:teuthology.orchestra.run.smithi123.stderr:listed 10 entries 2024-05-08T06:14:37.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:37 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/855019526' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T06:14:37.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:37 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/855019526' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T06:14:37.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:37 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/855019526' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T06:14:38.094 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T06:14:38.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:14:38.095 DEBUG:teuthology.orchestra.run.smithi123:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-08T06:14:38.189 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2024-05-08T06:14:38.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:14:38.189 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T06:14:38.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:14:38.264 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2024-05-08T06:14:38.300 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T06:14:38.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:14:38.301 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T06:14:38.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:14:38.410 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link delete ceph-brx 2024-05-08T06:14:38.520 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-08T06:14:38.535 INFO:tasks.cephadm:Teardown begin 2024-05-08T06:14:38.535 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T06:14:38.536 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T06:14:38.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:38 smithi123 ceph-mon[29787]: pgmap v585: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:14:38.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:38 smithi163 ceph-mon[34520]: pgmap v585: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:14:38.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:38 smithi192 ceph-mon[34664]: pgmap v585: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T06:14:38.923 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:14:40.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:40 smithi163 ceph-mon[34520]: pgmap v586: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-08T06:14:40.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:40 smithi192 ceph-mon[34664]: pgmap v586: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-08T06:14:40.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:40 smithi123 ceph-mon[29787]: pgmap v586: 97 pgs: 97 active+clean; 4.8 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-08T06:14:41.528 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.527+0000 7faebac00700 1 -- 172.21.15.123:0/730131540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faeb406c360 msgr2=0x7faeb410fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:14:41.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.527+0000 7faebac00700 1 --2- 172.21.15.123:0/730131540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faeb406c360 0x7faeb410fa00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7faea0009990 tx=0x7faea0009ca0 comp rx=0 tx=0).stop 2024-05-08T06:14:41.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.527+0000 7faebac00700 1 -- 172.21.15.123:0/730131540 shutdown_connections 2024-05-08T06:14:41.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.527+0000 7faebac00700 1 --2- 172.21.15.123:0/730131540 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faeb410ff40 0x7faeb4112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:41.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.527+0000 7faebac00700 1 --2- 172.21.15.123:0/730131540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeb406ba40 0x7faeb406be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:41.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.527+0000 7faebac00700 1 --2- 172.21.15.123:0/730131540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faeb406c360 0x7faeb410fa00 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:41.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.527+0000 7faebac00700 1 -- 172.21.15.123:0/730131540 >> 172.21.15.123:0/730131540 conn(0x7faeb4079310 msgr2=0x7faeb4079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:14:41.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.527+0000 7faebac00700 1 -- 172.21.15.123:0/730131540 shutdown_connections 2024-05-08T06:14:41.529 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.527+0000 7faebac00700 1 -- 172.21.15.123:0/730131540 wait complete. 2024-05-08T06:14:41.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.529+0000 7faebac00700 1 Processor -- start 2024-05-08T06:14:41.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.529+0000 7faebac00700 1 -- start start 2024-05-08T06:14:41.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.529+0000 7faebac00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faeb406ba40 0x7faeb419bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:14:41.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.529+0000 7faebac00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeb406c360 0x7faeb419c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:14:41.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.530+0000 7faebac00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faeb410ff40 0x7faeb41a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:14:41.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.530+0000 7faebac00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faeb41158d0 con 0x7faeb406ba40 2024-05-08T06:14:41.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.530+0000 7faebac00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7faeb4115750 con 0x7faeb406c360 2024-05-08T06:14:41.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.530+0000 7faebac00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7faeb4115a50 con 0x7faeb410ff40 2024-05-08T06:14:41.531 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.530+0000 7faeb3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faeb406ba40 0x7faeb419bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:14:41.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.530+0000 7faeb3e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faeb406ba40 0x7faeb419bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52540/0 (socket says 172.21.15.123:52540) 2024-05-08T06:14:41.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.530+0000 7faeb3e00700 1 -- 172.21.15.123:0/3658424372 learned_addr learned my addr 172.21.15.123:0/3658424372 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:14:41.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.530+0000 7faeb8c00700 1 --2- 172.21.15.123:0/3658424372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faeb410ff40 0x7faeb41a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:14:41.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.530+0000 7faeb3400700 1 --2- 172.21.15.123:0/3658424372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeb406c360 0x7faeb419c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:14:41.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.530+0000 7faeb3e00700 1 -- 172.21.15.123:0/3658424372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faeb410ff40 msgr2=0x7faeb41a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:14:41.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.530+0000 7faeb3e00700 1 --2- 172.21.15.123:0/3658424372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faeb410ff40 0x7faeb41a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:41.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.530+0000 7faeb3e00700 1 -- 172.21.15.123:0/3658424372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeb406c360 msgr2=0x7faeb419c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:14:41.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.530+0000 7faeb3e00700 1 --2- 172.21.15.123:0/3658424372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeb406c360 0x7faeb419c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:41.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.530+0000 7faeb3e00700 1 -- 172.21.15.123:0/3658424372 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeb41a0db0 con 0x7faeb406ba40 2024-05-08T06:14:41.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.530+0000 7faeb8c00700 1 --2- 172.21.15.123:0/3658424372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faeb410ff40 0x7faeb41a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:14:41.532 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.530+0000 7faeb3400700 1 --2- 172.21.15.123:0/3658424372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeb406c360 0x7faeb419c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:14:41.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.531+0000 7faeb3e00700 1 --2- 172.21.15.123:0/3658424372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faeb406ba40 0x7faeb419bda0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7faea400e9a0 tx=0x7faea400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:14:41.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.531+0000 7faeb0c00700 1 -- 172.21.15.123:0/3658424372 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea400cbd0 con 0x7faeb406ba40 2024-05-08T06:14:41.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.531+0000 7faeb0c00700 1 -- 172.21.15.123:0/3658424372 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faea400cd70 con 0x7faeb406ba40 2024-05-08T06:14:41.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.531+0000 7faebac00700 1 -- 172.21.15.123:0/3658424372 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeb41a10a0 con 0x7faeb406ba40 2024-05-08T06:14:41.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.531+0000 7faeb0c00700 1 -- 172.21.15.123:0/3658424372 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea4018680 con 0x7faeb406ba40 2024-05-08T06:14:41.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.531+0000 7faebac00700 1 -- 172.21.15.123:0/3658424372 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeb41a8920 con 0x7faeb406ba40 2024-05-08T06:14:41.533 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.532+0000 7faebac00700 1 -- 172.21.15.123:0/3658424372 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faeb406a190 con 0x7faeb406ba40 2024-05-08T06:14:41.535 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.534+0000 7faeb0c00700 1 -- 172.21.15.123:0/3658424372 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7faea4010430 con 0x7faeb406ba40 2024-05-08T06:14:41.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.536+0000 7faeb0c00700 1 --2- 172.21.15.123:0/3658424372 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fae980b2900 0x7fae980b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:14:41.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.536+0000 7faeb0c00700 1 -- 172.21.15.123:0/3658424372 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7faea40e2470 con 0x7faeb406ba40 2024-05-08T06:14:41.537 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.536+0000 7faeb3400700 1 --2- 172.21.15.123:0/3658424372 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fae980b2900 0x7fae980b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:14:41.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.537+0000 7faeb3400700 1 --2- 172.21.15.123:0/3658424372 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fae980b2900 0x7fae980b4dc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7faea000b2e0 tx=0x7faea001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:14:41.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.541+0000 7faeb0c00700 1 -- 172.21.15.123:0/3658424372 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faea40aa140 con 0x7faeb406ba40 2024-05-08T06:14:41.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.827+0000 7faebac00700 1 -- 172.21.15.123:0/3658424372 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faeb419d130 con 0x7faeb406ba40 2024-05-08T06:14:41.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.829+0000 7faeb0c00700 1 -- 172.21.15.123:0/3658424372 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11885 (secure 0 0 0) 0x7faeb419d130 con 0x7faeb406ba40 2024-05-08T06:14:41.829 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:14:41.829 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3646476522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3646476522},{"type":"v1","addr":"172.21.15.123:6835","nonce":3646476522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2763706497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2763706497},{"type":"v1","addr":"172.21.15.123:6837","nonce":2763706497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/274635265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":274635265},{"type":"v1","addr":"172.21.15.123:6839","nonce":274635265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/175423115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":175423115},{"type":"v1","addr":"172.21.15.192:6833","nonce":175423115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/3263009472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3263009472},{"type":"v1","addr":"172.21.15.163:6833","nonce":3263009472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3308612826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3308612826},{"type":"v1","addr":"172.21.15.163:6835","nonce":3308612826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T06:05:54.755550+0000","modified":"2024-05-08T06:10:12.600417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24512,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/202513680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":202513680},{"type":"v1","addr":"172.21.15.192:6835","nonce":202513680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/2483545969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2483545969},{"type":"v1","addr":"172.21.15.192:6837","nonce":2483545969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/660511838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":660511838},{"type":"v1","addr":"172.21.15.163:6837","nonce":660511838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.192:6839/3598682360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":3598682360},{"type":"v1","addr":"172.21.15.192:6839","nonce":3598682360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6841/1571726231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":1571726231},{"type":"v1","addr":"172.21.15.123:6841","nonce":1571726231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.163:6839/1568161515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":1568161515},{"type":"v1","addr":"172.21.15.163:6839","nonce":1568161515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24512,24496,24524]},"id":1}]} 2024-05-08T06:14:41.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.832+0000 7faebac00700 1 -- 172.21.15.123:0/3658424372 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fae980b2900 msgr2=0x7fae980b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:14:41.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.832+0000 7faebac00700 1 --2- 172.21.15.123:0/3658424372 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fae980b2900 0x7fae980b4dc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7faea000b2e0 tx=0x7faea001a040 comp rx=0 tx=0).stop 2024-05-08T06:14:41.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.832+0000 7faebac00700 1 -- 172.21.15.123:0/3658424372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faeb406ba40 msgr2=0x7faeb419bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:14:41.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.832+0000 7faebac00700 1 --2- 172.21.15.123:0/3658424372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faeb406ba40 0x7faeb419bda0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7faea400e9a0 tx=0x7faea400ecb0 comp rx=0 tx=0).stop 2024-05-08T06:14:41.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.832+0000 7faebac00700 1 -- 172.21.15.123:0/3658424372 shutdown_connections 2024-05-08T06:14:41.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.832+0000 7faebac00700 1 --2- 172.21.15.123:0/3658424372 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fae980b2900 0x7fae980b4dc0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:41.832 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.832+0000 7faebac00700 1 --2- 172.21.15.123:0/3658424372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faeb406ba40 0x7faeb419bda0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:41.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.832+0000 7faebac00700 1 --2- 172.21.15.123:0/3658424372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faeb410ff40 0x7faeb41a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:41.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.832+0000 7faebac00700 1 --2- 172.21.15.123:0/3658424372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeb406c360 0x7faeb419c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:41.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.832+0000 7faebac00700 1 -- 172.21.15.123:0/3658424372 >> 172.21.15.123:0/3658424372 conn(0x7faeb4079310 msgr2=0x7faeb41113f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:14:41.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.832+0000 7faebac00700 1 -- 172.21.15.123:0/3658424372 shutdown_connections 2024-05-08T06:14:41.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:41.833+0000 7faebac00700 1 -- 172.21.15.123:0/3658424372 wait complete. 2024-05-08T06:14:41.834 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 24 2024-05-08T06:14:42.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:42 smithi163 ceph-mon[34520]: pgmap v587: 97 pgs: 97 active+clean; 4.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:14:42.788 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:42 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/3658424372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:14:42.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:42 smithi192 ceph-mon[34664]: pgmap v587: 97 pgs: 97 active+clean; 4.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:14:42.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:42 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/3658424372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:14:42.807 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:42 smithi123 ceph-mon[29787]: pgmap v587: 97 pgs: 97 active+clean; 4.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:14:42.807 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:42 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/3658424372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:14:42.811 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T06:14:43.257 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:14:44.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:44 smithi123 ceph-mon[29787]: pgmap v588: 97 pgs: 97 active+clean; 4.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-08T06:14:45.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:44 smithi163 ceph-mon[34520]: pgmap v588: 97 pgs: 97 active+clean; 4.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-08T06:14:45.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:44 smithi192 ceph-mon[34664]: pgmap v588: 97 pgs: 97 active+clean; 4.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-08T06:14:45.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.730+0000 7f47a5400700 1 -- 172.21.15.123:0/3416319649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47a0075ff0 msgr2=0x7f47a0114130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:14:45.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.730+0000 7f47a5400700 1 --2- 172.21.15.123:0/3416319649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47a0075ff0 0x7f47a0114130 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4788009990 tx=0x7f4788009ca0 comp rx=0 tx=0).stop 2024-05-08T06:14:45.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.730+0000 7f47a5400700 1 -- 172.21.15.123:0/3416319649 shutdown_connections 2024-05-08T06:14:45.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.730+0000 7f47a5400700 1 --2- 172.21.15.123:0/3416319649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47a0114670 0x7f47a0116a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:45.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.730+0000 7f47a5400700 1 --2- 172.21.15.123:0/3416319649 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47a00756d0 0x7f47a0075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:45.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.730+0000 7f47a5400700 1 --2- 172.21.15.123:0/3416319649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47a0075ff0 0x7f47a0114130 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:45.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.730+0000 7f47a5400700 1 -- 172.21.15.123:0/3416319649 >> 172.21.15.123:0/3416319649 conn(0x7f47a007a600 msgr2=0x7f47a007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:14:45.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.730+0000 7f47a5400700 1 -- 172.21.15.123:0/3416319649 shutdown_connections 2024-05-08T06:14:45.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.730+0000 7f47a5400700 1 -- 172.21.15.123:0/3416319649 wait complete. 2024-05-08T06:14:45.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.731+0000 7f47a5400700 1 Processor -- start 2024-05-08T06:14:45.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.731+0000 7f47a5400700 1 -- start start 2024-05-08T06:14:45.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.731+0000 7f47a5400700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47a00756d0 0x7f47a01a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:14:45.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.731+0000 7f47a5400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47a0075ff0 0x7f47a01a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:14:45.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.731+0000 7f47a5400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47a0114670 0x7f47a01a4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:14:45.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.731+0000 7f47a5400700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f47a0119cd0 con 0x7f47a0114670 2024-05-08T06:14:45.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.731+0000 7f47a5400700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f47a0119b50 con 0x7f47a00756d0 2024-05-08T06:14:45.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.731+0000 7f47a5400700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f47a0119e50 con 0x7f47a0075ff0 2024-05-08T06:14:45.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.732+0000 7f479f200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47a0114670 0x7f47a01a4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:14:45.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.732+0000 7f4795e00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47a0075ff0 0x7f47a01a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:14:45.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.732+0000 7f479f200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47a0114670 0x7f47a01a4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53170/0 (socket says 172.21.15.123:53170) 2024-05-08T06:14:45.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.732+0000 7f479f200700 1 -- 172.21.15.123:0/3833840192 learned_addr learned my addr 172.21.15.123:0/3833840192 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:14:45.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.732+0000 7f479e800700 1 --2- 172.21.15.123:0/3833840192 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47a00756d0 0x7f47a01a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:14:45.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.732+0000 7f479f200700 1 -- 172.21.15.123:0/3833840192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47a0075ff0 msgr2=0x7f47a01a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:14:45.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.732+0000 7f479f200700 1 --2- 172.21.15.123:0/3833840192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47a0075ff0 0x7f47a01a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:45.734 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.732+0000 7f479f200700 1 -- 172.21.15.123:0/3833840192 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47a00756d0 msgr2=0x7f47a01a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:14:45.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.732+0000 7f479f200700 1 --2- 172.21.15.123:0/3833840192 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47a00756d0 0x7f47a01a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:45.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.732+0000 7f479f200700 1 -- 172.21.15.123:0/3833840192 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a01a51b0 con 0x7f47a0114670 2024-05-08T06:14:45.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.732+0000 7f4795e00700 1 --2- 172.21.15.123:0/3833840192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47a0075ff0 0x7f47a01a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:14:45.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.732+0000 7f479e800700 1 --2- 172.21.15.123:0/3833840192 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47a00756d0 0x7f47a01a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:14:45.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.733+0000 7f479f200700 1 --2- 172.21.15.123:0/3833840192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47a0114670 0x7f47a01a4a70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f479800ef90 tx=0x7f479800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:14:45.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.733+0000 7f4797e00700 1 -- 172.21.15.123:0/3833840192 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47980092e0 con 0x7f47a0114670 2024-05-08T06:14:45.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.733+0000 7f4797e00700 1 -- 172.21.15.123:0/3833840192 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4798004bb0 con 0x7f47a0114670 2024-05-08T06:14:45.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.733+0000 7f4797e00700 1 -- 172.21.15.123:0/3833840192 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798004280 con 0x7f47a0114670 2024-05-08T06:14:45.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.733+0000 7f47a5400700 1 -- 172.21.15.123:0/3833840192 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a01a5440 con 0x7f47a0114670 2024-05-08T06:14:45.735 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.733+0000 7f47a5400700 1 -- 172.21.15.123:0/3833840192 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a01acd20 con 0x7f47a0114670 2024-05-08T06:14:45.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.734+0000 7f47a5400700 1 -- 172.21.15.123:0/3833840192 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47a006a190 con 0x7f47a0114670 2024-05-08T06:14:45.736 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.737+0000 7f4797e00700 1 -- 172.21.15.123:0/3833840192 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7f479801b030 con 0x7f47a0114670 2024-05-08T06:14:45.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.738+0000 7f4797e00700 1 --2- 172.21.15.123:0/3833840192 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f47840b2900 0x7f47840b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:14:45.738 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.738+0000 7f4797e00700 1 -- 172.21.15.123:0/3833840192 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f47980e2660 con 0x7f47a0114670 2024-05-08T06:14:45.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.738+0000 7f479e800700 1 --2- 172.21.15.123:0/3833840192 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f47840b2900 0x7f47840b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:14:45.739 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.739+0000 7f479e800700 1 --2- 172.21.15.123:0/3833840192 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f47840b2900 0x7f47840b4dc0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4790011490 tx=0x7f4790009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:14:45.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:45.741+0000 7f4797e00700 1 -- 172.21.15.123:0/3833840192 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4798022020 con 0x7f47a0114670 2024-05-08T06:14:46.024 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:46.023+0000 7f47a5400700 1 -- 172.21.15.123:0/3833840192 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47a01acef0 con 0x7f47a0114670 2024-05-08T06:14:46.025 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:46.025+0000 7f4797e00700 1 -- 172.21.15.123:0/3833840192 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11885 (secure 0 0 0) 0x7f4798010040 con 0x7f47a0114670 2024-05-08T06:14:46.025 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:14:46.026 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3646476522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3646476522},{"type":"v1","addr":"172.21.15.123:6835","nonce":3646476522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2763706497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2763706497},{"type":"v1","addr":"172.21.15.123:6837","nonce":2763706497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/274635265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":274635265},{"type":"v1","addr":"172.21.15.123:6839","nonce":274635265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/175423115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":175423115},{"type":"v1","addr":"172.21.15.192:6833","nonce":175423115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/3263009472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3263009472},{"type":"v1","addr":"172.21.15.163:6833","nonce":3263009472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3308612826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3308612826},{"type":"v1","addr":"172.21.15.163:6835","nonce":3308612826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T06:05:54.755550+0000","modified":"2024-05-08T06:10:12.600417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24512,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/202513680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":202513680},{"type":"v1","addr":"172.21.15.192:6835","nonce":202513680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/2483545969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2483545969},{"type":"v1","addr":"172.21.15.192:6837","nonce":2483545969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/660511838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":660511838},{"type":"v1","addr":"172.21.15.163:6837","nonce":660511838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.192:6839/3598682360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":3598682360},{"type":"v1","addr":"172.21.15.192:6839","nonce":3598682360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6841/1571726231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6840","nonce":1571726231},{"type":"v1","addr":"172.21.15.123:6841","nonce":1571726231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.163:6839/1568161515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":1568161515},{"type":"v1","addr":"172.21.15.163:6839","nonce":1568161515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24512,24496,24524]},"id":1}]} 2024-05-08T06:14:46.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:46.029+0000 7f47a5400700 1 -- 172.21.15.123:0/3833840192 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f47840b2900 msgr2=0x7f47840b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:14:46.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:46.029+0000 7f47a5400700 1 --2- 172.21.15.123:0/3833840192 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f47840b2900 0x7f47840b4dc0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4790011490 tx=0x7f4790009260 comp rx=0 tx=0).stop 2024-05-08T06:14:46.028 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:46.029+0000 7f47a5400700 1 -- 172.21.15.123:0/3833840192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47a0114670 msgr2=0x7f47a01a4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:14:46.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:46.029+0000 7f47a5400700 1 --2- 172.21.15.123:0/3833840192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47a0114670 0x7f47a01a4a70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f479800ef90 tx=0x7f479800c3a0 comp rx=0 tx=0).stop 2024-05-08T06:14:46.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:46.029+0000 7f47a5400700 1 -- 172.21.15.123:0/3833840192 shutdown_connections 2024-05-08T06:14:46.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:46.029+0000 7f47a5400700 1 --2- 172.21.15.123:0/3833840192 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f47840b2900 0x7f47840b4dc0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:46.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:46.029+0000 7f47a5400700 1 --2- 172.21.15.123:0/3833840192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47a0114670 0x7f47a01a4a70 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:46.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:46.029+0000 7f47a5400700 1 --2- 172.21.15.123:0/3833840192 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47a00756d0 0x7f47a01a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:46.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:46.029+0000 7f47a5400700 1 --2- 172.21.15.123:0/3833840192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47a0075ff0 0x7f47a01a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:46.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:46.029+0000 7f47a5400700 1 -- 172.21.15.123:0/3833840192 >> 172.21.15.123:0/3833840192 conn(0x7f47a007a600 msgr2=0x7f47a0113430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:14:46.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:46.029+0000 7f47a5400700 1 -- 172.21.15.123:0/3833840192 shutdown_connections 2024-05-08T06:14:46.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:46.029+0000 7f47a5400700 1 -- 172.21.15.123:0/3833840192 wait complete. 2024-05-08T06:14:46.031 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 24 2024-05-08T06:14:46.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:46 smithi123 ceph-mon[29787]: pgmap v589: 97 pgs: 97 active+clean; 4.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-08T06:14:46.705 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:46 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/3833840192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:14:47.028 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T06:14:47.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:46 smithi163 ceph-mon[34520]: pgmap v589: 97 pgs: 97 active+clean; 4.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-08T06:14:47.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:46 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/3833840192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:14:47.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:46 smithi192 ceph-mon[34664]: pgmap v589: 97 pgs: 97 active+clean; 4.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-08T06:14:47.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:46 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/3833840192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:14:47.479 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:14:48.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:48 smithi123 ceph-mon[29787]: pgmap v590: 97 pgs: 97 active+clean; 4.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-08T06:14:49.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:48 smithi163 ceph-mon[34520]: pgmap v590: 97 pgs: 97 active+clean; 4.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-08T06:14:49.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:48 smithi192 ceph-mon[34664]: pgmap v590: 97 pgs: 97 active+clean; 4.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-08T06:14:50.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.188+0000 7f7775a00700 1 -- 172.21.15.123:0/1932738293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7770108850 msgr2=0x7f77701103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:14:50.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.188+0000 7f7775a00700 1 --2- 172.21.15.123:0/1932738293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7770108850 0x7f77701103d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f776800b0d0 tx=0x7f776800b3e0 comp rx=0 tx=0).stop 2024-05-08T06:14:50.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.188+0000 7f7775a00700 1 -- 172.21.15.123:0/1932738293 shutdown_connections 2024-05-08T06:14:50.188 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.188+0000 7f7775a00700 1 --2- 172.21.15.123:0/1932738293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7770108850 0x7f77701103d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:50.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.188+0000 7f7775a00700 1 --2- 172.21.15.123:0/1932738293 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7770103750 0x7f7770103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:50.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.188+0000 7f7775a00700 1 --2- 172.21.15.123:0/1932738293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7770104100 0x7f7770108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:50.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.188+0000 7f7775a00700 1 -- 172.21.15.123:0/1932738293 >> 172.21.15.123:0/1932738293 conn(0x7f777007a600 msgr2=0x7f777007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:14:50.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.188+0000 7f7775a00700 1 -- 172.21.15.123:0/1932738293 shutdown_connections 2024-05-08T06:14:50.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.188+0000 7f7775a00700 1 -- 172.21.15.123:0/1932738293 wait complete. 2024-05-08T06:14:50.189 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.189+0000 7f7775a00700 1 Processor -- start 2024-05-08T06:14:50.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.189+0000 7f7775a00700 1 -- start start 2024-05-08T06:14:50.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.190+0000 7f7775a00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7770103750 0x7f77701a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:14:50.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.190+0000 7f7775a00700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7770104100 0x7f77701a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:14:50.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.190+0000 7f7775a00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7770108850 0x7f777019bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:14:50.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.190+0000 7f7775a00700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7770113640 con 0x7f7770108850 2024-05-08T06:14:50.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.190+0000 7f7775a00700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f77701134c0 con 0x7f7770103750 2024-05-08T06:14:50.190 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.190+0000 7f7775a00700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f77701137c0 con 0x7f7770104100 2024-05-08T06:14:50.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.190+0000 7f776f400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7770108850 0x7f777019bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:14:50.191 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.190+0000 7f776ea00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7770103750 0x7f77701a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:14:50.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.190+0000 7f776f400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7770108850 0x7f777019bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53188/0 (socket says 172.21.15.123:53188) 2024-05-08T06:14:50.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.190+0000 7f776f400700 1 -- 172.21.15.123:0/2292034221 learned_addr learned my addr 172.21.15.123:0/2292034221 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:14:50.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.190+0000 7f776ea00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7770103750 0x7f77701a18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:36716/0 (socket says 172.21.15.123:36716) 2024-05-08T06:14:50.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.190+0000 7f7766000700 1 --2- 172.21.15.123:0/2292034221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7770104100 0x7f77701a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:14:50.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.191+0000 7f776f400700 1 -- 172.21.15.123:0/2292034221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7770104100 msgr2=0x7f77701a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:14:50.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.191+0000 7f776f400700 1 --2- 172.21.15.123:0/2292034221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7770104100 0x7f77701a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:50.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.191+0000 7f776f400700 1 -- 172.21.15.123:0/2292034221 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7770103750 msgr2=0x7f77701a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:14:50.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.191+0000 7f776f400700 1 --2- 172.21.15.123:0/2292034221 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7770103750 0x7f77701a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:50.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.191+0000 7f776f400700 1 -- 172.21.15.123:0/2292034221 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f777019c260 con 0x7f7770108850 2024-05-08T06:14:50.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.191+0000 7f776ea00700 1 --2- 172.21.15.123:0/2292034221 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7770103750 0x7f77701a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:14:50.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.191+0000 7f776f400700 1 --2- 172.21.15.123:0/2292034221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7770108850 0x7f777019bb00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7768007a00 tx=0x7f7768007a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:14:50.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.191+0000 7f7767e00700 1 -- 172.21.15.123:0/2292034221 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776801f070 con 0x7f7770108850 2024-05-08T06:14:50.192 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.191+0000 7f7775a00700 1 -- 172.21.15.123:0/2292034221 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f777019c4f0 con 0x7f7770108850 2024-05-08T06:14:50.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.191+0000 7f7767e00700 1 -- 172.21.15.123:0/2292034221 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7768004c50 con 0x7f7770108850 2024-05-08T06:14:50.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.191+0000 7f7767e00700 1 -- 172.21.15.123:0/2292034221 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776801b690 con 0x7f7770108850 2024-05-08T06:14:50.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.192+0000 7f7775a00700 1 -- 172.21.15.123:0/2292034221 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77701a87c0 con 0x7f7770108850 2024-05-08T06:14:50.193 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.193+0000 7f7775a00700 1 -- 172.21.15.123:0/2292034221 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f777006a190 con 0x7f7770108850 2024-05-08T06:14:50.195 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.195+0000 7f7767e00700 1 -- 172.21.15.123:0/2292034221 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7f7768022070 con 0x7f7770108850 2024-05-08T06:14:50.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.196+0000 7f7767e00700 1 --2- 172.21.15.123:0/2292034221 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f77540b2880 0x7f77540b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:14:50.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.197+0000 7f7767e00700 1 -- 172.21.15.123:0/2292034221 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f77680e3190 con 0x7f7770108850 2024-05-08T06:14:50.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.197+0000 7f776ea00700 1 --2- 172.21.15.123:0/2292034221 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f77540b2880 0x7f77540b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:14:50.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.198+0000 7f776ea00700 1 --2- 172.21.15.123:0/2292034221 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f77540b2880 0x7f77540b4d40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7770102ce0 tx=0x7f7760009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:14:50.199 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.200+0000 7f7767e00700 1 -- 172.21.15.123:0/2292034221 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f77680aaef0 con 0x7f7770108850 2024-05-08T06:14:50.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.435+0000 7f7775a00700 1 -- 172.21.15.123:0/2292034221 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f77701a8990 con 0x7f7770108850 2024-05-08T06:14:50.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.438+0000 7f7767e00700 1 -- 172.21.15.123:0/2292034221 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22719 (secure 0 0 0) 0x7f77680aa7f0 con 0x7f7770108850 2024-05-08T06:14:50.438 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-08T06:14:50.439 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":82,"fsid":"85cfb2e6-0cff-11ef-bc97-c7b262605968","created":"2024-05-08T05:56:12.544750+0000","modified":"2024-05-08T06:06:32.471993+0000","last_up_change":"2024-05-08T06:04:33.654727+0000","last_in_change":"2024-05-08T06:04:13.809050+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T06:00:17.050518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T06:05:44.815334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T06:05:49.604457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a14fde9b-bfb1-40a7-a105-305e8981a778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6803","nonce":3608618058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6805","nonce":3608618058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6809","nonce":3608618058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3608618058},{"type":"v1","addr":"172.21.15.123:6807","nonce":3608618058}]},"public_addr":"172.21.15.123:6803/3608618058","cluster_addr":"172.21.15.123:6805/3608618058","heartbeat_back_addr":"172.21.15.123:6809/3608618058","heartbeat_front_addr":"172.21.15.123:6807/3608618058","state":["exists","up"]},{"osd":1,"uuid":"e023ae3d-a82e-4bb6-8fef-889dd07fe183","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6801","nonce":1884753449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6803","nonce":1884753449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6807","nonce":1884753449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1884753449},{"type":"v1","addr":"172.21.15.163:6805","nonce":1884753449}]},"public_addr":"172.21.15.163:6801/1884753449","cluster_addr":"172.21.15.163:6803/1884753449","heartbeat_back_addr":"172.21.15.163:6807/1884753449","heartbeat_front_addr":"172.21.15.163:6805/1884753449","state":["exists","up"]},{"osd":2,"uuid":"69f4cfca-63cb-4862-9480-d79891b0f338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6801","nonce":4161570404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6803","nonce":4161570404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6807","nonce":4161570404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4161570404},{"type":"v1","addr":"172.21.15.192:6805","nonce":4161570404}]},"public_addr":"172.21.15.192:6801/4161570404","cluster_addr":"172.21.15.192:6803/4161570404","heartbeat_back_addr":"172.21.15.192:6807/4161570404","heartbeat_front_addr":"172.21.15.192:6805/4161570404","state":["exists","up"]},{"osd":3,"uuid":"71bb5ecd-c88c-45fb-8b51-0eefb53941c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6811","nonce":1243003258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6813","nonce":1243003258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6817","nonce":1243003258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1243003258},{"type":"v1","addr":"172.21.15.123:6815","nonce":1243003258}]},"public_addr":"172.21.15.123:6811/1243003258","cluster_addr":"172.21.15.123:6813/1243003258","heartbeat_back_addr":"172.21.15.123:6817/1243003258","heartbeat_front_addr":"172.21.15.123:6815/1243003258","state":["exists","up"]},{"osd":4,"uuid":"5a7a3462-e74f-4d15-a85c-709b5d423745","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6809","nonce":591680273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6811","nonce":591680273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6815","nonce":591680273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":591680273},{"type":"v1","addr":"172.21.15.163:6813","nonce":591680273}]},"public_addr":"172.21.15.163:6809/591680273","cluster_addr":"172.21.15.163:6811/591680273","heartbeat_back_addr":"172.21.15.163:6815/591680273","heartbeat_front_addr":"172.21.15.163:6813/591680273","state":["exists","up"]},{"osd":5,"uuid":"85e3f1ab-f8c6-4025-bab9-9ddb6bd3e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6809","nonce":1734821180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6811","nonce":1734821180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6815","nonce":1734821180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1734821180},{"type":"v1","addr":"172.21.15.192:6813","nonce":1734821180}]},"public_addr":"172.21.15.192:6809/1734821180","cluster_addr":"172.21.15.192:6811/1734821180","heartbeat_back_addr":"172.21.15.192:6815/1734821180","heartbeat_front_addr":"172.21.15.192:6813/1734821180","state":["exists","up"]},{"osd":6,"uuid":"f21a5624-6f8f-48be-bc6d-0a883d6b7b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6819","nonce":1002138878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6821","nonce":1002138878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6825","nonce":1002138878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1002138878},{"type":"v1","addr":"172.21.15.123:6823","nonce":1002138878}]},"public_addr":"172.21.15.123:6819/1002138878","cluster_addr":"172.21.15.123:6821/1002138878","heartbeat_back_addr":"172.21.15.123:6825/1002138878","heartbeat_front_addr":"172.21.15.123:6823/1002138878","state":["exists","up"]},{"osd":7,"uuid":"6eb16729-d260-4041-9829-3e868795b111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6817","nonce":354411978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6819","nonce":354411978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6823","nonce":354411978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":354411978},{"type":"v1","addr":"172.21.15.163:6821","nonce":354411978}]},"public_addr":"172.21.15.163:6817/354411978","cluster_addr":"172.21.15.163:6819/354411978","heartbeat_back_addr":"172.21.15.163:6823/354411978","heartbeat_front_addr":"172.21.15.163:6821/354411978","state":["exists","up"]},{"osd":8,"uuid":"122a281e-18b8-4728-97db-458e7432b2b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6817","nonce":3669753689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6819","nonce":3669753689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6823","nonce":3669753689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3669753689},{"type":"v1","addr":"172.21.15.192:6821","nonce":3669753689}]},"public_addr":"172.21.15.192:6817/3669753689","cluster_addr":"172.21.15.192:6819/3669753689","heartbeat_back_addr":"172.21.15.192:6823/3669753689","heartbeat_front_addr":"172.21.15.192:6821/3669753689","state":["exists","up"]},{"osd":9,"uuid":"9fa022a9-b873-4e07-806c-641a706a150f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6827","nonce":3104436473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6829","nonce":3104436473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6833","nonce":3104436473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":3104436473},{"type":"v1","addr":"172.21.15.123:6831","nonce":3104436473}]},"public_addr":"172.21.15.123:6827/3104436473","cluster_addr":"172.21.15.123:6829/3104436473","heartbeat_back_addr":"172.21.15.123:6833/3104436473","heartbeat_front_addr":"172.21.15.123:6831/3104436473","state":["exists","up"]},{"osd":10,"uuid":"fb59339b-e11b-4ffc-b202-7f67cb21da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6825","nonce":2628487883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6827","nonce":2628487883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6831","nonce":2628487883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2628487883},{"type":"v1","addr":"172.21.15.163:6829","nonce":2628487883}]},"public_addr":"172.21.15.163:6825/2628487883","cluster_addr":"172.21.15.163:6827/2628487883","heartbeat_back_addr":"172.21.15.163:6831/2628487883","heartbeat_front_addr":"172.21.15.163:6829/2628487883","state":["exists","up"]},{"osd":11,"uuid":"be7a5f36-c1be-41d7-80ee-004d0546eb9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6825","nonce":1339159347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6827","nonce":1339159347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6831","nonce":1339159347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1339159347},{"type":"v1","addr":"172.21.15.192:6829","nonce":1339159347}]},"public_addr":"172.21.15.192:6825/1339159347","cluster_addr":"172.21.15.192:6827/1339159347","heartbeat_back_addr":"172.21.15.192:6831/1339159347","heartbeat_front_addr":"172.21.15.192:6829/1339159347","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:59:24.923234+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:59:49.867071+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:00:13.454062+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:00:46.983594+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:01:12.820452+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:01:37.451382+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:02:13.325181+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:02:39.453436+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:03:03.837509+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:03:39.460579+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:04:06.314082+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T06:04:31.723154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6801/3057688649":"2024-05-09T05:57:30.887936+0000","172.21.15.123:6800/3057688649":"2024-05-09T05:57:30.887936+0000","172.21.15.123:6800/2769724312":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/399884043":"2024-05-09T05:57:30.887936+0000","172.21.15.123:0/3406061897":"2024-05-09T05:57:30.887936+0000","172.21.15.123:0/644457268":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/3055092197":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/343390357":"2024-05-09T05:57:30.887936+0000","172.21.15.123:6801/2769724312":"2024-05-09T05:56:55.849036+0000","172.21.15.123:0/2263171562":"2024-05-09T05:56:55.849036+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T06:14:50.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.441+0000 7f7775a00700 1 -- 172.21.15.123:0/2292034221 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f77540b2880 msgr2=0x7f77540b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:14:50.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.441+0000 7f7775a00700 1 --2- 172.21.15.123:0/2292034221 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f77540b2880 0x7f77540b4d40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7770102ce0 tx=0x7f7760009260 comp rx=0 tx=0).stop 2024-05-08T06:14:50.441 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.441+0000 7f7775a00700 1 -- 172.21.15.123:0/2292034221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7770108850 msgr2=0x7f777019bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:14:50.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.441+0000 7f7775a00700 1 --2- 172.21.15.123:0/2292034221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7770108850 0x7f777019bb00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7768007a00 tx=0x7f7768007a30 comp rx=0 tx=0).stop 2024-05-08T06:14:50.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.441+0000 7f7775a00700 1 -- 172.21.15.123:0/2292034221 shutdown_connections 2024-05-08T06:14:50.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.441+0000 7f7775a00700 1 --2- 172.21.15.123:0/2292034221 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f77540b2880 0x7f77540b4d40 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:50.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.441+0000 7f7775a00700 1 --2- 172.21.15.123:0/2292034221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7770108850 0x7f777019bb00 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:50.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.441+0000 7f7775a00700 1 --2- 172.21.15.123:0/2292034221 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7770103750 0x7f77701a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:50.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.441+0000 7f7775a00700 1 --2- 172.21.15.123:0/2292034221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7770104100 0x7f77701a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:50.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.441+0000 7f7775a00700 1 -- 172.21.15.123:0/2292034221 >> 172.21.15.123:0/2292034221 conn(0x7f777007a600 msgr2=0x7f7770101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:14:50.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.441+0000 7f7775a00700 1 -- 172.21.15.123:0/2292034221 shutdown_connections 2024-05-08T06:14:50.442 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:50.442+0000 7f7775a00700 1 -- 172.21.15.123:0/2292034221 wait complete. 2024-05-08T06:14:50.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:50 smithi123 ceph-mon[29787]: pgmap v591: 97 pgs: 97 active+clean; 4.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-08T06:14:50.706 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:50 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/2292034221' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:14:51.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:50 smithi163 ceph-mon[34520]: pgmap v591: 97 pgs: 97 active+clean; 4.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-08T06:14:51.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:50 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/2292034221' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:14:51.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:50 smithi192 ceph-mon[34664]: pgmap v591: 97 pgs: 97 active+clean; 4.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-08T06:14:51.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:50 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/2292034221' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:14:51.491 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph fs fail cephfs 2024-05-08T06:14:51.940 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:14:52.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:52 smithi123 ceph-mon[29787]: pgmap v592: 97 pgs: 97 active+clean; 4.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:14:53.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:52 smithi163 ceph-mon[34520]: pgmap v592: 97 pgs: 97 active+clean; 4.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:14:53.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:52 smithi192 ceph-mon[34664]: pgmap v592: 97 pgs: 97 active+clean; 4.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-08T06:14:54.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.549+0000 7fe3f5600700 1 -- 172.21.15.123:0/4021371646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3f00758d0 msgr2=0x7fe3f0075cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:14:54.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.549+0000 7fe3f5600700 1 --2- 172.21.15.123:0/4021371646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3f00758d0 0x7fe3f0075cb0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0009990 tx=0x7fe3e0009ca0 comp rx=0 tx=0).stop 2024-05-08T06:14:54.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.549+0000 7fe3f5600700 1 -- 172.21.15.123:0/4021371646 shutdown_connections 2024-05-08T06:14:54.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.549+0000 7fe3f5600700 1 --2- 172.21.15.123:0/4021371646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3f00758d0 0x7fe3f0075cb0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:54.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.549+0000 7fe3f5600700 1 --2- 172.21.15.123:0/4021371646 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3f010fc00 0x7fe3f0112070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:54.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.549+0000 7fe3f5600700 1 --2- 172.21.15.123:0/4021371646 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3f00761f0 0x7fe3f010f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:54.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.549+0000 7fe3f5600700 1 -- 172.21.15.123:0/4021371646 >> 172.21.15.123:0/4021371646 conn(0x7fe3f00feeb0 msgr2=0x7fe3f01012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:14:54.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.549+0000 7fe3f5600700 1 -- 172.21.15.123:0/4021371646 shutdown_connections 2024-05-08T06:14:54.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.549+0000 7fe3f5600700 1 -- 172.21.15.123:0/4021371646 wait complete. 2024-05-08T06:14:54.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.550+0000 7fe3f5600700 1 Processor -- start 2024-05-08T06:14:54.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.550+0000 7fe3f5600700 1 -- start start 2024-05-08T06:14:54.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.551+0000 7fe3f5600700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3f00758d0 0x7fe3f01a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:14:54.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.551+0000 7fe3f5600700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3f00761f0 0x7fe3f01a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:14:54.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.551+0000 7fe3f5600700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3f010fc00 0x7fe3f019baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:14:54.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.551+0000 7fe3f5600700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f0115780 con 0x7fe3f010fc00 2024-05-08T06:14:54.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.551+0000 7fe3f5600700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f0115600 con 0x7fe3f00758d0 2024-05-08T06:14:54.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.551+0000 7fe3f5600700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f0115900 con 0x7fe3f00761f0 2024-05-08T06:14:54.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.551+0000 7fe3f4c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3f010fc00 0x7fe3f019baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:14:54.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.551+0000 7fe3f4c00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3f010fc00 0x7fe3f019baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53196/0 (socket says 172.21.15.123:53196) 2024-05-08T06:14:54.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.551+0000 7fe3f4c00700 1 -- 172.21.15.123:0/2618598218 learned_addr learned my addr 172.21.15.123:0/2618598218 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:14:54.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.551+0000 7fe3e7400700 1 --2- 172.21.15.123:0/2618598218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3f00761f0 0x7fe3f01a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:14:54.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.551+0000 7fe3efe00700 1 --2- 172.21.15.123:0/2618598218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3f00758d0 0x7fe3f01a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:14:54.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.552+0000 7fe3f4c00700 1 -- 172.21.15.123:0/2618598218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3f00761f0 msgr2=0x7fe3f01a1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:14:54.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.552+0000 7fe3f4c00700 1 --2- 172.21.15.123:0/2618598218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3f00761f0 0x7fe3f01a1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:54.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.552+0000 7fe3f4c00700 1 -- 172.21.15.123:0/2618598218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3f00758d0 msgr2=0x7fe3f01a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:14:54.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.552+0000 7fe3f4c00700 1 --2- 172.21.15.123:0/2618598218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3f00758d0 0x7fe3f01a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:54.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.552+0000 7fe3f4c00700 1 -- 172.21.15.123:0/2618598218 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3f019c200 con 0x7fe3f010fc00 2024-05-08T06:14:54.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.552+0000 7fe3f4c00700 1 --2- 172.21.15.123:0/2618598218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3f010fc00 0x7fe3f019baa0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8007ee0 tx=0x7fe3e800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:14:54.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.552+0000 7fe3ed600700 1 -- 172.21.15.123:0/2618598218 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e800dd20 con 0x7fe3f010fc00 2024-05-08T06:14:54.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.552+0000 7fe3ed600700 1 -- 172.21.15.123:0/2618598218 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3e8004020 con 0x7fe3f010fc00 2024-05-08T06:14:54.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.552+0000 7fe3f5600700 1 -- 172.21.15.123:0/2618598218 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3f019c4f0 con 0x7fe3f010fc00 2024-05-08T06:14:54.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.552+0000 7fe3ed600700 1 -- 172.21.15.123:0/2618598218 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e8015660 con 0x7fe3f010fc00 2024-05-08T06:14:54.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.553+0000 7fe3f5600700 1 -- 172.21.15.123:0/2618598218 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3f01a87b0 con 0x7fe3f010fc00 2024-05-08T06:14:54.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.554+0000 7fe3f5600700 1 -- 172.21.15.123:0/2618598218 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3f006a190 con 0x7fe3f010fc00 2024-05-08T06:14:54.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.556+0000 7fe3ed600700 1 -- 172.21.15.123:0/2618598218 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7fe3e80041d0 con 0x7fe3f010fc00 2024-05-08T06:14:54.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.557+0000 7fe3ed600700 1 --2- 172.21.15.123:0/2618598218 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe3d40b2880 0x7fe3d40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:14:54.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.558+0000 7fe3ed600700 1 -- 172.21.15.123:0/2618598218 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fe3e80e1b50 con 0x7fe3f010fc00 2024-05-08T06:14:54.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.558+0000 7fe3efe00700 1 --2- 172.21.15.123:0/2618598218 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe3d40b2880 0x7fe3d40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:14:54.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.558+0000 7fe3efe00700 1 --2- 172.21.15.123:0/2618598218 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe3d40b2880 0x7fe3d40b4d40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe3e000b2e0 tx=0x7fe3e001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:14:54.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.562+0000 7fe3ed600700 1 -- 172.21.15.123:0/2618598218 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe3e8010070 con 0x7fe3f010fc00 2024-05-08T06:14:54.846 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.845+0000 7fe3f5600700 1 -- 172.21.15.123:0/2618598218 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe3f002e510 con 0x7fe3f010fc00 2024-05-08T06:14:54.850 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.850+0000 7fe3ed600700 1 -- 172.21.15.123:0/2618598218 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) ==== 161+0+0 (secure 0 0 0) 0x7fe3e80a9820 con 0x7fe3f010fc00 2024-05-08T06:14:54.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.855+0000 7fe3f5600700 1 -- 172.21.15.123:0/2618598218 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe3d40b2880 msgr2=0x7fe3d40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:14:54.855 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.855+0000 7fe3f5600700 1 --2- 172.21.15.123:0/2618598218 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe3d40b2880 0x7fe3d40b4d40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe3e000b2e0 tx=0x7fe3e001a040 comp rx=0 tx=0).stop 2024-05-08T06:14:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.855+0000 7fe3f5600700 1 -- 172.21.15.123:0/2618598218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3f010fc00 msgr2=0x7fe3f019baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:14:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.855+0000 7fe3f5600700 1 --2- 172.21.15.123:0/2618598218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3f010fc00 0x7fe3f019baa0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8007ee0 tx=0x7fe3e800d3b0 comp rx=0 tx=0).stop 2024-05-08T06:14:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.855+0000 7fe3f5600700 1 -- 172.21.15.123:0/2618598218 shutdown_connections 2024-05-08T06:14:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.855+0000 7fe3f5600700 1 --2- 172.21.15.123:0/2618598218 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe3d40b2880 0x7fe3d40b4d40 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.855+0000 7fe3f5600700 1 --2- 172.21.15.123:0/2618598218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3f010fc00 0x7fe3f019baa0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.855+0000 7fe3f5600700 1 --2- 172.21.15.123:0/2618598218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3f00758d0 0x7fe3f01a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.855+0000 7fe3f5600700 1 --2- 172.21.15.123:0/2618598218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3f00761f0 0x7fe3f01a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.855+0000 7fe3f5600700 1 -- 172.21.15.123:0/2618598218 >> 172.21.15.123:0/2618598218 conn(0x7fe3f00feeb0 msgr2=0x7fe3f00ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:14:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.855+0000 7fe3f5600700 1 -- 172.21.15.123:0/2618598218 shutdown_connections 2024-05-08T06:14:54.856 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:54.855+0000 7fe3f5600700 1 -- 172.21.15.123:0/2618598218 wait complete. 2024-05-08T06:14:54.857 INFO:teuthology.orchestra.run.smithi123.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T06:14:54.905 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:54 smithi163 ceph-mon[34520]: pgmap v593: 97 pgs: 97 active+clean; 4.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-08T06:14:54.905 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:54 smithi192 ceph-mon[34664]: pgmap v593: 97 pgs: 97 active+clean; 4.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-08T06:14:54.906 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 08 06:14:54 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-f[65135]: ignoring --setuser ceph since I am not root 2024-05-08T06:14:54.906 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 08 06:14:54 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-f[65135]: ignoring --setgroup ceph since I am not root 2024-05-08T06:14:54.906 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 08 06:14:54 smithi192 ceph-mds[65139]: -- 172.21.15.192:0/103664499 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556975d39a40 con 0x556975d0c800 2024-05-08T06:14:54.906 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 08 06:14:54 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-f[65135]: starting mds.f at 2024-05-08T06:14:54.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:54 smithi123 ceph-mon[29787]: pgmap v593: 97 pgs: 97 active+clean; 4.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-08T06:14:54.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:54 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mon-a[29783]: 2024-05-08T06:14:54.847+0000 7f7533c00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T06:14:55.287 INFO:journalctl@ceph.mds.a.smithi123.stdout:May 08 06:14:54 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-a[71273]: ignoring --setuser ceph since I am not root 2024-05-08T06:14:55.287 INFO:journalctl@ceph.mds.a.smithi123.stdout:May 08 06:14:54 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-a[71273]: ignoring --setgroup ceph since I am not root 2024-05-08T06:14:55.287 INFO:journalctl@ceph.mds.a.smithi123.stdout:May 08 06:14:54 smithi123 ceph-mds[71277]: -- 172.21.15.123:0/890635055 <== mon.2 v2:172.21.15.163:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563c30e79a40 con 0x563c30e4cc00 2024-05-08T06:14:55.288 INFO:journalctl@ceph.mds.a.smithi123.stdout:May 08 06:14:54 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-a[71273]: starting mds.a at 2024-05-08T06:14:55.288 INFO:journalctl@ceph.mds.h.smithi163.stdout:May 08 06:14:54 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-h[64723]: ignoring --setuser ceph since I am not root 2024-05-08T06:14:55.288 INFO:journalctl@ceph.mds.h.smithi163.stdout:May 08 06:14:54 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-h[64723]: ignoring --setgroup ceph since I am not root 2024-05-08T06:14:55.288 INFO:journalctl@ceph.mds.h.smithi163.stdout:May 08 06:14:54 smithi163 ceph-mds[64727]: -- 172.21.15.163:0/1310095309 <== mon.2 v2:172.21.15.163:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56423810ba40 con 0x5642380de800 2024-05-08T06:14:55.288 INFO:journalctl@ceph.mds.h.smithi163.stdout:May 08 06:14:54 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-h[64723]: starting mds.h at 2024-05-08T06:14:55.289 INFO:journalctl@ceph.mds.b.smithi163.stdout:May 08 06:14:54 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-b[65037]: ignoring --setuser ceph since I am not root 2024-05-08T06:14:55.289 INFO:journalctl@ceph.mds.b.smithi163.stdout:May 08 06:14:54 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-b[65037]: ignoring --setgroup ceph since I am not root 2024-05-08T06:14:55.289 INFO:journalctl@ceph.mds.b.smithi163.stdout:May 08 06:14:54 smithi163 ceph-mds[65041]: -- 172.21.15.163:0/2116149269 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5638aeb9fa40 con 0x5638aeb73000 2024-05-08T06:14:55.289 INFO:journalctl@ceph.mds.b.smithi163.stdout:May 08 06:14:54 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-b[65037]: starting mds.b at 2024-05-08T06:14:55.300 INFO:journalctl@ceph.mds.i.smithi192.stdout:May 08 06:14:54 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-i[65766]: ignoring --setuser ceph since I am not root 2024-05-08T06:14:55.300 INFO:journalctl@ceph.mds.i.smithi192.stdout:May 08 06:14:54 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-i[65766]: ignoring --setgroup ceph since I am not root 2024-05-08T06:14:55.300 INFO:journalctl@ceph.mds.i.smithi192.stdout:May 08 06:14:54 smithi192 ceph-mds[65770]: -- 172.21.15.192:0/2919345107 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556397a09a40 con 0x5563979dd000 2024-05-08T06:14:55.301 INFO:journalctl@ceph.mds.i.smithi192.stdout:May 08 06:14:54 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-i[65766]: starting mds.i at 2024-05-08T06:14:55.301 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 08 06:14:54 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-c[65457]: ignoring --setuser ceph since I am not root 2024-05-08T06:14:55.301 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 08 06:14:54 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-c[65457]: ignoring --setgroup ceph since I am not root 2024-05-08T06:14:55.301 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 08 06:14:54 smithi192 ceph-mds[65461]: -- 172.21.15.192:0/3813373489 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556502773a40 con 0x556502746800 2024-05-08T06:14:55.301 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 08 06:14:54 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-c[65457]: starting mds.c at 2024-05-08T06:14:55.885 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T06:14:55.952 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:55 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/2618598218' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-08T06:14:55.953 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:55 smithi123 ceph-mon[29787]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-08T06:14:55.953 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:55 smithi123 ceph-mon[29787]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T06:14:55.953 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:55 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/2618598218' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-08T06:14:55.953 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:55 smithi123 ceph-mon[29787]: osdmap e83: 12 total, 12 up, 12 in 2024-05-08T06:14:55.953 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:55 smithi123 ceph-mon[29787]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-08T06:14:56.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:55 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/2618598218' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-08T06:14:56.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:55 smithi163 ceph-mon[34520]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-08T06:14:56.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:55 smithi163 ceph-mon[34520]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T06:14:56.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:55 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/2618598218' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-08T06:14:56.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:55 smithi163 ceph-mon[34520]: osdmap e83: 12 total, 12 up, 12 in 2024-05-08T06:14:56.039 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:55 smithi163 ceph-mon[34520]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-08T06:14:56.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:55 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/2618598218' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-08T06:14:56.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:55 smithi192 ceph-mon[34664]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-08T06:14:56.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:55 smithi192 ceph-mon[34664]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T06:14:56.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:55 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/2618598218' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-08T06:14:56.050 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:55 smithi192 ceph-mon[34664]: osdmap e83: 12 total, 12 up, 12 in 2024-05-08T06:14:56.051 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:55 smithi192 ceph-mon[34664]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-08T06:14:56.327 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:14:57.028 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:56 smithi123 ceph-mon[29787]: pgmap v595: 97 pgs: 97 active+clean; 4.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-08T06:14:57.029 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:56 smithi123 ceph-mon[29787]: mds.? [v2:172.21.15.192:6834/1408472376,v1:172.21.15.192:6835/1408472376] up:boot 2024-05-08T06:14:57.029 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:56 smithi123 ceph-mon[29787]: mds.? [v2:172.21.15.163:6836/3332757128,v1:172.21.15.163:6838/3332757128] up:boot 2024-05-08T06:14:57.029 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:56 smithi123 ceph-mon[29787]: mds.? [v2:172.21.15.163:6837/1416213539,v1:172.21.15.163:6839/1416213539] up:boot 2024-05-08T06:14:57.029 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:56 smithi123 ceph-mon[29787]: mds.? [v2:172.21.15.192:6836/3256886030,v1:172.21.15.192:6838/3256886030] up:boot 2024-05-08T06:14:57.029 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:56 smithi123 ceph-mon[29787]: mds.? [v2:172.21.15.192:6837/4093990288,v1:172.21.15.192:6839/4093990288] up:boot 2024-05-08T06:14:57.029 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:56 smithi123 ceph-mon[29787]: mds.? [v2:172.21.15.123:6840/290797036,v1:172.21.15.123:6841/290797036] up:boot 2024-05-08T06:14:57.029 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:56 smithi123 ceph-mon[29787]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-08T06:14:57.029 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:56 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T06:14:57.029 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:56 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T06:14:57.029 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:56 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T06:14:57.029 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:56 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T06:14:57.029 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:56 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T06:14:57.030 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:56 smithi123 ceph-mon[29787]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T06:14:57.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:56 smithi163 ceph-mon[34520]: pgmap v595: 97 pgs: 97 active+clean; 4.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-08T06:14:57.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:56 smithi163 ceph-mon[34520]: mds.? [v2:172.21.15.192:6834/1408472376,v1:172.21.15.192:6835/1408472376] up:boot 2024-05-08T06:14:57.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:56 smithi163 ceph-mon[34520]: mds.? [v2:172.21.15.163:6836/3332757128,v1:172.21.15.163:6838/3332757128] up:boot 2024-05-08T06:14:57.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:56 smithi163 ceph-mon[34520]: mds.? [v2:172.21.15.163:6837/1416213539,v1:172.21.15.163:6839/1416213539] up:boot 2024-05-08T06:14:57.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:56 smithi163 ceph-mon[34520]: mds.? [v2:172.21.15.192:6836/3256886030,v1:172.21.15.192:6838/3256886030] up:boot 2024-05-08T06:14:57.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:56 smithi163 ceph-mon[34520]: mds.? [v2:172.21.15.192:6837/4093990288,v1:172.21.15.192:6839/4093990288] up:boot 2024-05-08T06:14:57.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:56 smithi163 ceph-mon[34520]: mds.? [v2:172.21.15.123:6840/290797036,v1:172.21.15.123:6841/290797036] up:boot 2024-05-08T06:14:57.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:56 smithi163 ceph-mon[34520]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-08T06:14:57.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:56 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T06:14:57.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:56 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T06:14:57.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:56 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T06:14:57.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:56 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T06:14:57.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:56 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T06:14:57.289 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:56 smithi163 ceph-mon[34520]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T06:14:57.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:56 smithi192 ceph-mon[34664]: pgmap v595: 97 pgs: 97 active+clean; 4.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-08T06:14:57.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:56 smithi192 ceph-mon[34664]: mds.? [v2:172.21.15.192:6834/1408472376,v1:172.21.15.192:6835/1408472376] up:boot 2024-05-08T06:14:57.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:56 smithi192 ceph-mon[34664]: mds.? [v2:172.21.15.163:6836/3332757128,v1:172.21.15.163:6838/3332757128] up:boot 2024-05-08T06:14:57.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:56 smithi192 ceph-mon[34664]: mds.? [v2:172.21.15.163:6837/1416213539,v1:172.21.15.163:6839/1416213539] up:boot 2024-05-08T06:14:57.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:56 smithi192 ceph-mon[34664]: mds.? [v2:172.21.15.192:6836/3256886030,v1:172.21.15.192:6838/3256886030] up:boot 2024-05-08T06:14:57.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:56 smithi192 ceph-mon[34664]: mds.? [v2:172.21.15.192:6837/4093990288,v1:172.21.15.192:6839/4093990288] up:boot 2024-05-08T06:14:57.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:56 smithi192 ceph-mon[34664]: mds.? [v2:172.21.15.123:6840/290797036,v1:172.21.15.123:6841/290797036] up:boot 2024-05-08T06:14:57.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:56 smithi192 ceph-mon[34664]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-08T06:14:57.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:56 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T06:14:57.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:56 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T06:14:57.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:56 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T06:14:57.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:56 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T06:14:57.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:56 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T06:14:57.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:56 smithi192 ceph-mon[34664]: from='mgr.14152 172.21.15.123:0/1573854817' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T06:14:58.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.825+0000 7fe508200700 1 -- 172.21.15.123:0/2535407339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe500114670 msgr2=0x7fe500116a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:14:58.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.825+0000 7fe508200700 1 --2- 172.21.15.123:0/2535407339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe500114670 0x7fe500116a60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe4f800b0a0 tx=0x7fe4f800b3b0 comp rx=0 tx=0).stop 2024-05-08T06:14:58.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.825+0000 7fe508200700 1 -- 172.21.15.123:0/2535407339 shutdown_connections 2024-05-08T06:14:58.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.825+0000 7fe508200700 1 --2- 172.21.15.123:0/2535407339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe500114670 0x7fe500116a60 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:58.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.825+0000 7fe508200700 1 --2- 172.21.15.123:0/2535407339 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5000756d0 0x7fe500075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:58.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.825+0000 7fe508200700 1 --2- 172.21.15.123:0/2535407339 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe500075ff0 0x7fe500114130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:58.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.825+0000 7fe508200700 1 -- 172.21.15.123:0/2535407339 >> 172.21.15.123:0/2535407339 conn(0x7fe50007a600 msgr2=0x7fe50007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:14:58.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.825+0000 7fe508200700 1 -- 172.21.15.123:0/2535407339 shutdown_connections 2024-05-08T06:14:58.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.825+0000 7fe508200700 1 -- 172.21.15.123:0/2535407339 wait complete. 2024-05-08T06:14:58.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.826+0000 7fe508200700 1 Processor -- start 2024-05-08T06:14:58.827 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.826+0000 7fe508200700 1 -- start start 2024-05-08T06:14:58.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.826+0000 7fe508200700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5000756d0 0x7fe5001a0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:14:58.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.826+0000 7fe508200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe500075ff0 0x7fe5001a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:14:58.828 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.826+0000 7fe508200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe500114670 0x7fe5001a4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:14:58.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.826+0000 7fe508200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe500119c80 con 0x7fe500114670 2024-05-08T06:14:58.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.826+0000 7fe508200700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe500119b00 con 0x7fe500075ff0 2024-05-08T06:14:58.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.826+0000 7fe508200700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe500119e00 con 0x7fe5000756d0 2024-05-08T06:14:58.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.827+0000 7fe504e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe500075ff0 0x7fe5001a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:14:58.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.827+0000 7fe505800700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5000756d0 0x7fe5001a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:14:58.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.827+0000 7fe504e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe500075ff0 0x7fe5001a0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:57296/0 (socket says 172.21.15.123:57296) 2024-05-08T06:14:58.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.827+0000 7fe504e00700 1 -- 172.21.15.123:0/2177399970 learned_addr learned my addr 172.21.15.123:0/2177399970 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:14:58.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.827+0000 7fe506200700 1 --2- 172.21.15.123:0/2177399970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe500114670 0x7fe5001a4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:14:58.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.827+0000 7fe504e00700 1 -- 172.21.15.123:0/2177399970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5000756d0 msgr2=0x7fe5001a0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:14:58.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.827+0000 7fe504e00700 1 --2- 172.21.15.123:0/2177399970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5000756d0 0x7fe5001a0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:58.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.827+0000 7fe504e00700 1 -- 172.21.15.123:0/2177399970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe500114670 msgr2=0x7fe5001a4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:14:58.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.827+0000 7fe504e00700 1 --2- 172.21.15.123:0/2177399970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe500114670 0x7fe5001a4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:58.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.827+0000 7fe504e00700 1 -- 172.21.15.123:0/2177399970 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5001a5220 con 0x7fe500075ff0 2024-05-08T06:14:58.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.827+0000 7fe506200700 1 --2- 172.21.15.123:0/2177399970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe500114670 0x7fe5001a4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:14:58.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.827+0000 7fe504e00700 1 --2- 172.21.15.123:0/2177399970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe500075ff0 0x7fe5001a0750 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec00e9a0 tx=0x7fe4ec00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:14:58.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.828+0000 7fe4fe000700 1 -- 172.21.15.123:0/2177399970 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec00cbd0 con 0x7fe500075ff0 2024-05-08T06:14:58.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.828+0000 7fe4fe000700 1 -- 172.21.15.123:0/2177399970 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4ec00cd70 con 0x7fe500075ff0 2024-05-08T06:14:58.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.828+0000 7fe508200700 1 -- 172.21.15.123:0/2177399970 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5001a5510 con 0x7fe500075ff0 2024-05-08T06:14:58.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.828+0000 7fe4fe000700 1 -- 172.21.15.123:0/2177399970 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec018680 con 0x7fe500075ff0 2024-05-08T06:14:58.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.828+0000 7fe508200700 1 -- 172.21.15.123:0/2177399970 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5001acd90 con 0x7fe500075ff0 2024-05-08T06:14:58.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.829+0000 7fe508200700 1 -- 172.21.15.123:0/2177399970 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe50006a190 con 0x7fe500075ff0 2024-05-08T06:14:58.831 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.831+0000 7fe4fe000700 1 -- 172.21.15.123:0/2177399970 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7fe4ec010430 con 0x7fe500075ff0 2024-05-08T06:14:58.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.833+0000 7fe4fe000700 1 --2- 172.21.15.123:0/2177399970 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe4e80b2880 0x7fe4e80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:14:58.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.833+0000 7fe4fe000700 1 -- 172.21.15.123:0/2177399970 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8469+0+0 (secure 0 0 0) 0x7fe4ec0e2a20 con 0x7fe500075ff0 2024-05-08T06:14:58.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.833+0000 7fe505800700 1 --2- 172.21.15.123:0/2177399970 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe4e80b2880 0x7fe4e80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:14:58.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.834+0000 7fe505800700 1 --2- 172.21.15.123:0/2177399970 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe4e80b2880 0x7fe4e80b4d40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009c60 tx=0x7fe4f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:14:58.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:58.835+0000 7fe4fe000700 1 -- 172.21.15.123:0/2177399970 <== mon.2 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe4ec0aa5f0 con 0x7fe500075ff0 2024-05-08T06:14:59.120 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:59.119+0000 7fe508200700 1 -- 172.21.15.123:0/2177399970 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fe5001acf60 con 0x7fe500075ff0 2024-05-08T06:14:59.120 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:58 smithi123 ceph-mon[29787]: pgmap v596: 97 pgs: 97 active+clean; 4.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:14:59.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:58 smithi163 ceph-mon[34520]: pgmap v596: 97 pgs: 97 active+clean; 4.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:14:59.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:58 smithi192 ceph-mon[34664]: pgmap v596: 97 pgs: 97 active+clean; 4.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T06:14:59.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:59.886+0000 7fe4fe000700 1 -- 172.21.15.123:0/2177399970 <== mon.2 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) ==== 104+0+0 (secure 0 0 0) 0x7fe4ec0a9ef0 con 0x7fe500075ff0 2024-05-08T06:14:59.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:59.890+0000 7fe508200700 1 -- 172.21.15.123:0/2177399970 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe4e80b2880 msgr2=0x7fe4e80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:14:59.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:59.890+0000 7fe508200700 1 --2- 172.21.15.123:0/2177399970 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe4e80b2880 0x7fe4e80b4d40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009c60 tx=0x7fe4f4009260 comp rx=0 tx=0).stop 2024-05-08T06:14:59.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:59.890+0000 7fe508200700 1 -- 172.21.15.123:0/2177399970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe500075ff0 msgr2=0x7fe5001a0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:14:59.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:59.890+0000 7fe508200700 1 --2- 172.21.15.123:0/2177399970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe500075ff0 0x7fe5001a0750 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec00e9a0 tx=0x7fe4ec00ecb0 comp rx=0 tx=0).stop 2024-05-08T06:14:59.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:59.890+0000 7fe508200700 1 -- 172.21.15.123:0/2177399970 shutdown_connections 2024-05-08T06:14:59.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:59.890+0000 7fe508200700 1 --2- 172.21.15.123:0/2177399970 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7fe4e80b2880 0x7fe4e80b4d40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:59.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:59.890+0000 7fe508200700 1 --2- 172.21.15.123:0/2177399970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe500114670 0x7fe5001a4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:59.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:59.890+0000 7fe508200700 1 --2- 172.21.15.123:0/2177399970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5000756d0 0x7fe5001a0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:59.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:59.890+0000 7fe508200700 1 --2- 172.21.15.123:0/2177399970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe500075ff0 0x7fe5001a0750 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:14:59.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:59.890+0000 7fe508200700 1 -- 172.21.15.123:0/2177399970 >> 172.21.15.123:0/2177399970 conn(0x7fe50007a600 msgr2=0x7fe5001134a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:14:59.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:59.890+0000 7fe508200700 1 -- 172.21.15.123:0/2177399970 shutdown_connections 2024-05-08T06:14:59.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:14:59.890+0000 7fe508200700 1 -- 172.21.15.123:0/2177399970 wait complete. 2024-05-08T06:15:00.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:59 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/2177399970' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T06:15:00.206 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:14:59 smithi123 ceph-mon[29787]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T06:15:00.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:59 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/2177399970' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T06:15:00.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:14:59 smithi163 ceph-mon[34520]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T06:15:00.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:59 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/2177399970' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T06:15:00.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:14:59 smithi192 ceph-mon[34664]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T06:15:00.925 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T06:15:01.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:15:00 smithi123 ceph-mon[29787]: pgmap v597: 97 pgs: 97 active+clean; 4.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-05-08T06:15:01.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:15:00 smithi123 ceph-mon[29787]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-08T06:15:01.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:15:00 smithi123 ceph-mon[29787]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-08T06:15:01.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:15:00 smithi123 ceph-mon[29787]: Cluster is now healthy 2024-05-08T06:15:01.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:15:00 smithi123 ceph-mon[29787]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-08T06:15:01.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:15:00 smithi123 ceph-mon[29787]: fsmap 12 up:standby 2024-05-08T06:15:01.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:15:00 smithi163 ceph-mon[34520]: pgmap v597: 97 pgs: 97 active+clean; 4.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-05-08T06:15:01.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:15:00 smithi163 ceph-mon[34520]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-08T06:15:01.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:15:00 smithi163 ceph-mon[34520]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-08T06:15:01.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:15:00 smithi163 ceph-mon[34520]: Cluster is now healthy 2024-05-08T06:15:01.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:15:00 smithi163 ceph-mon[34520]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-08T06:15:01.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:15:00 smithi163 ceph-mon[34520]: fsmap 12 up:standby 2024-05-08T06:15:01.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:15:00 smithi192 ceph-mon[34664]: pgmap v597: 97 pgs: 97 active+clean; 4.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-05-08T06:15:01.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:15:00 smithi192 ceph-mon[34664]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-08T06:15:01.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:15:00 smithi192 ceph-mon[34664]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-08T06:15:01.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:15:00 smithi192 ceph-mon[34664]: Cluster is now healthy 2024-05-08T06:15:01.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:15:00 smithi192 ceph-mon[34664]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-08T06:15:01.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:15:00 smithi192 ceph-mon[34664]: fsmap 12 up:standby 2024-05-08T06:15:01.362 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:15:03.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:15:02 smithi123 ceph-mon[29787]: pgmap v598: 97 pgs: 97 active+clean; 4.8 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-08T06:15:03.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:15:02 smithi163 ceph-mon[34520]: pgmap v598: 97 pgs: 97 active+clean; 4.8 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-08T06:15:03.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:15:02 smithi192 ceph-mon[34664]: pgmap v598: 97 pgs: 97 active+clean; 4.8 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-08T06:15:03.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.809+0000 7f46cb200700 1 -- 172.21.15.123:0/834833438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46cc105f90 msgr2=0x7f46cc1124a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:15:03.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.809+0000 7f46cb200700 1 --2- 172.21.15.123:0/834833438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46cc105f90 0x7f46cc1124a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f46c400b0a0 tx=0x7f46c400b3b0 comp rx=0 tx=0).stop 2024-05-08T06:15:03.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.809+0000 7f46cb200700 1 -- 172.21.15.123:0/834833438 shutdown_connections 2024-05-08T06:15:03.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.809+0000 7f46cb200700 1 --2- 172.21.15.123:0/834833438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46cc105f90 0x7f46cc1124a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:15:03.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.809+0000 7f46cb200700 1 --2- 172.21.15.123:0/834833438 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46cc10b660 0x7f46cc10ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:15:03.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.809+0000 7f46cb200700 1 --2- 172.21.15.123:0/834833438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46cc1055d0 0x7f46cc105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:15:03.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.809+0000 7f46cb200700 1 -- 172.21.15.123:0/834833438 >> 172.21.15.123:0/834833438 conn(0x7f46cc1010c0 msgr2=0x7f46cc1034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:15:03.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.809+0000 7f46cb200700 1 -- 172.21.15.123:0/834833438 shutdown_connections 2024-05-08T06:15:03.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.809+0000 7f46cb200700 1 -- 172.21.15.123:0/834833438 wait complete. 2024-05-08T06:15:03.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.810+0000 7f46cb200700 1 Processor -- start 2024-05-08T06:15:03.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.811+0000 7f46cb200700 1 -- start start 2024-05-08T06:15:03.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.811+0000 7f46cb200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46cc1055d0 0x7f46cc1a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:15:03.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.811+0000 7f46cb200700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46cc105f90 0x7f46cc1a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:15:03.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.811+0000 7f46cb200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46cc10b660 0x7f46cc19b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:15:03.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.811+0000 7f46cb200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc115820 con 0x7f46cc10b660 2024-05-08T06:15:03.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.811+0000 7f46cb200700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc1156a0 con 0x7f46cc1055d0 2024-05-08T06:15:03.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.811+0000 7f46cb200700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc1159a0 con 0x7f46cc105f90 2024-05-08T06:15:03.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.811+0000 7f46ca800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46cc10b660 0x7f46cc19b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:15:03.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.811+0000 7f46c9e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46cc1055d0 0x7f46cc1a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:15:03.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.811+0000 7f46c9e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46cc1055d0 0x7f46cc1a1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.123:57314/0 (socket says 172.21.15.123:57314) 2024-05-08T06:15:03.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.811+0000 7f46c9e00700 1 -- 172.21.15.123:0/852613246 learned_addr learned my addr 172.21.15.123:0/852613246 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:15:03.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.811+0000 7f46ca800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46cc10b660 0x7f46cc19b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60124/0 (socket says 172.21.15.123:60124) 2024-05-08T06:15:03.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.811+0000 7f46c9400700 1 --2- 172.21.15.123:0/852613246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46cc105f90 0x7f46cc1a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:15:03.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.812+0000 7f46ca800700 1 -- 172.21.15.123:0/852613246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46cc105f90 msgr2=0x7f46cc1a1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:15:03.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.812+0000 7f46ca800700 1 --2- 172.21.15.123:0/852613246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46cc105f90 0x7f46cc1a1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:15:03.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.812+0000 7f46ca800700 1 -- 172.21.15.123:0/852613246 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46cc1055d0 msgr2=0x7f46cc1a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:15:03.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.812+0000 7f46ca800700 1 --2- 172.21.15.123:0/852613246 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46cc1055d0 0x7f46cc1a1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:15:03.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.812+0000 7f46ca800700 1 -- 172.21.15.123:0/852613246 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46cc19c140 con 0x7f46cc10b660 2024-05-08T06:15:03.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.812+0000 7f46c9e00700 1 --2- 172.21.15.123:0/852613246 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46cc1055d0 0x7f46cc1a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:15:03.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.812+0000 7f46ca800700 1 --2- 172.21.15.123:0/852613246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46cc10b660 0x7f46cc19b9e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f46c400b070 tx=0x7f46c4004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:15:03.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.813+0000 7f46c2a00700 1 -- 172.21.15.123:0/852613246 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c4021070 con 0x7f46cc10b660 2024-05-08T06:15:03.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.813+0000 7f46c2a00700 1 -- 172.21.15.123:0/852613246 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46c4007e30 con 0x7f46cc10b660 2024-05-08T06:15:03.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.813+0000 7f46cb200700 1 -- 172.21.15.123:0/852613246 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46cc19c3d0 con 0x7f46cc10b660 2024-05-08T06:15:03.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.813+0000 7f46c2a00700 1 -- 172.21.15.123:0/852613246 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c40045f0 con 0x7f46cc10b660 2024-05-08T06:15:03.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.813+0000 7f46cb200700 1 -- 172.21.15.123:0/852613246 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46cc1a86f0 con 0x7f46cc10b660 2024-05-08T06:15:03.814 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.814+0000 7f46cb200700 1 -- 172.21.15.123:0/852613246 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46cc06a190 con 0x7f46cc10b660 2024-05-08T06:15:03.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.816+0000 7f46c2a00700 1 -- 172.21.15.123:0/852613246 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7f46c4007980 con 0x7f46cc10b660 2024-05-08T06:15:03.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.818+0000 7f46c2a00700 1 --2- 172.21.15.123:0/852613246 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f46b00b2880 0x7f46b00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:15:03.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.819+0000 7f46c2a00700 1 -- 172.21.15.123:0/852613246 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8469+0+0 (secure 0 0 0) 0x7f46c400f020 con 0x7f46cc10b660 2024-05-08T06:15:03.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.819+0000 7f46c9e00700 1 --2- 172.21.15.123:0/852613246 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f46b00b2880 0x7f46b00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:15:03.819 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.819+0000 7f46c9e00700 1 --2- 172.21.15.123:0/852613246 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f46b00b2880 0x7f46b00b4d40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f46cc19cb20 tx=0x7f46bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:15:03.823 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:03.824+0000 7f46c2a00700 1 -- 172.21.15.123:0/852613246 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f46c402a020 con 0x7f46cc10b660 2024-05-08T06:15:04.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:04.057+0000 7f46cb200700 1 -- 172.21.15.123:0/852613246 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f46cc19cf10 con 0x7f46cc10b660 2024-05-08T06:15:04.887 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:04.887+0000 7f46c2a00700 1 -- 172.21.15.123:0/852613246 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v84) ==== 181+0+0 (secure 0 0 0) 0x7f46c40ab290 con 0x7f46cc10b660 2024-05-08T06:15:04.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:04.893+0000 7f46cb200700 1 -- 172.21.15.123:0/852613246 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f46b00b2880 msgr2=0x7f46b00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:15:04.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:04.893+0000 7f46cb200700 1 --2- 172.21.15.123:0/852613246 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f46b00b2880 0x7f46b00b4d40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f46cc19cb20 tx=0x7f46bc009260 comp rx=0 tx=0).stop 2024-05-08T06:15:04.893 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:04.893+0000 7f46cb200700 1 -- 172.21.15.123:0/852613246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46cc10b660 msgr2=0x7f46cc19b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:15:04.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:04.893+0000 7f46cb200700 1 --2- 172.21.15.123:0/852613246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46cc10b660 0x7f46cc19b9e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f46c400b070 tx=0x7f46c4004c80 comp rx=0 tx=0).stop 2024-05-08T06:15:04.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:04.894+0000 7f46cb200700 1 -- 172.21.15.123:0/852613246 shutdown_connections 2024-05-08T06:15:04.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:04.894+0000 7f46cb200700 1 --2- 172.21.15.123:0/852613246 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f46b00b2880 0x7f46b00b4d40 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:15:04.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:04.894+0000 7f46cb200700 1 --2- 172.21.15.123:0/852613246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46cc10b660 0x7f46cc19b9e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:15:04.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:04.894+0000 7f46cb200700 1 --2- 172.21.15.123:0/852613246 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46cc1055d0 0x7f46cc1a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:15:04.894 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:04.894+0000 7f46cb200700 1 --2- 172.21.15.123:0/852613246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46cc105f90 0x7f46cc1a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:15:04.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:04.894+0000 7f46cb200700 1 -- 172.21.15.123:0/852613246 >> 172.21.15.123:0/852613246 conn(0x7f46cc1010c0 msgr2=0x7f46cc102700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:15:04.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:04.894+0000 7f46cb200700 1 -- 172.21.15.123:0/852613246 shutdown_connections 2024-05-08T06:15:04.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:04.895+0000 7f46cb200700 1 -- 172.21.15.123:0/852613246 wait complete. 2024-05-08T06:15:04.900 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_metadata' removed 2024-05-08T06:15:05.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:15:04 smithi123 ceph-mon[29787]: pgmap v599: 97 pgs: 97 active+clean; 4.8 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 716 B/s rd, 1 op/s 2024-05-08T06:15:05.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:15:04 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/852613246' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T06:15:05.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:15:04 smithi163 ceph-mon[34520]: pgmap v599: 97 pgs: 97 active+clean; 4.8 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 716 B/s rd, 1 op/s 2024-05-08T06:15:05.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:15:04 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/852613246' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T06:15:05.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:15:04 smithi192 ceph-mon[34664]: pgmap v599: 97 pgs: 97 active+clean; 4.8 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 716 B/s rd, 1 op/s 2024-05-08T06:15:05.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:15:04 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/852613246' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T06:15:05.929 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T06:15:06.005 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:15:05 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/852613246' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T06:15:06.005 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:15:05 smithi123 ceph-mon[29787]: osdmap e84: 12 total, 12 up, 12 in 2024-05-08T06:15:06.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:15:05 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/852613246' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T06:15:06.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:15:05 smithi163 ceph-mon[34520]: osdmap e84: 12 total, 12 up, 12 in 2024-05-08T06:15:06.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:15:05 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/852613246' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T06:15:06.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:15:05 smithi192 ceph-mon[34664]: osdmap e84: 12 total, 12 up, 12 in 2024-05-08T06:15:06.384 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:15:07.027 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:15:06 smithi123 ceph-mon[29787]: pgmap v601: 65 pgs: 65 active+clean; 577 KiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:15:07.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:15:06 smithi163 ceph-mon[34520]: pgmap v601: 65 pgs: 65 active+clean; 577 KiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:15:07.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:15:06 smithi192 ceph-mon[34664]: pgmap v601: 65 pgs: 65 active+clean; 577 KiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:15:08.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.911+0000 7f1c01200700 1 -- 172.21.15.123:0/2257483102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bfc109ad0 msgr2=0x7f1bfc109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:15:08.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.911+0000 7f1c01200700 1 --2- 172.21.15.123:0/2257483102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bfc109ad0 0x7f1bfc109f50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1be8009990 tx=0x7f1be8009ca0 comp rx=0 tx=0).stop 2024-05-08T06:15:08.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.912+0000 7f1c01200700 1 -- 172.21.15.123:0/2257483102 shutdown_connections 2024-05-08T06:15:08.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.912+0000 7f1c01200700 1 --2- 172.21.15.123:0/2257483102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bfc10a490 0x7f1bfc1169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:15:08.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.912+0000 7f1c01200700 1 --2- 172.21.15.123:0/2257483102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bfc10fb60 0x7f1bfc10ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:15:08.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.912+0000 7f1c01200700 1 --2- 172.21.15.123:0/2257483102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bfc109ad0 0x7f1bfc109f50 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:15:08.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.912+0000 7f1c01200700 1 -- 172.21.15.123:0/2257483102 >> 172.21.15.123:0/2257483102 conn(0x7f1bfc076c70 msgr2=0x7f1bfc1082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:15:08.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.912+0000 7f1c01200700 1 -- 172.21.15.123:0/2257483102 shutdown_connections 2024-05-08T06:15:08.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.912+0000 7f1c01200700 1 -- 172.21.15.123:0/2257483102 wait complete. 2024-05-08T06:15:08.913 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.913+0000 7f1c01200700 1 Processor -- start 2024-05-08T06:15:08.914 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.913+0000 7f1c01200700 1 -- start start 2024-05-08T06:15:08.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.914+0000 7f1c01200700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bfc109ad0 0x7f1bfc1a0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:15:08.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.914+0000 7f1c01200700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bfc10a490 0x7f1bfc1a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:15:08.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.914+0000 7f1c01200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bfc10fb60 0x7f1bfc1a4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:15:08.915 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.914+0000 7f1c01200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1bfc119d60 con 0x7f1bfc10fb60 2024-05-08T06:15:08.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.914+0000 7f1c01200700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1bfc119be0 con 0x7f1bfc109ad0 2024-05-08T06:15:08.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.914+0000 7f1c01200700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1bfc119ee0 con 0x7f1bfc10a490 2024-05-08T06:15:08.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.914+0000 7f1bfa800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bfc10fb60 0x7f1bfc1a4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:15:08.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.914+0000 7f1bf9e00700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bfc109ad0 0x7f1bfc1a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:15:08.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.914+0000 7f1bf9400700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bfc10a490 0x7f1bfc1a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:15:08.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.914+0000 7f1bfa800700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bfc10fb60 0x7f1bfc1a4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36222/0 (socket says 172.21.15.123:36222) 2024-05-08T06:15:08.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.914+0000 7f1bfa800700 1 -- 172.21.15.123:0/124722734 learned_addr learned my addr 172.21.15.123:0/124722734 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-08T06:15:08.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.915+0000 7f1bf9e00700 1 -- 172.21.15.123:0/124722734 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bfc10a490 msgr2=0x7f1bfc1a07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:15:08.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.915+0000 7f1bf9e00700 1 --2- 172.21.15.123:0/124722734 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bfc10a490 0x7f1bfc1a07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:15:08.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.915+0000 7f1bf9e00700 1 -- 172.21.15.123:0/124722734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bfc10fb60 msgr2=0x7f1bfc1a4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:15:08.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.915+0000 7f1bf9e00700 1 --2- 172.21.15.123:0/124722734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bfc10fb60 0x7f1bfc1a4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:15:08.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.915+0000 7f1bf9e00700 1 -- 172.21.15.123:0/124722734 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bfc1a52a0 con 0x7f1bfc109ad0 2024-05-08T06:15:08.916 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.915+0000 7f1bfa800700 1 --2- 172.21.15.123:0/124722734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bfc10fb60 0x7f1bfc1a4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T06:15:08.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.915+0000 7f1bf9e00700 1 --2- 172.21.15.123:0/124722734 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bfc109ad0 0x7f1bfc1a0290 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1bec00e9f0 tx=0x7f1bec00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:15:08.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.915+0000 7f1bf2a00700 1 -- 172.21.15.123:0/124722734 <== mon.2 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bec004220 con 0x7f1bfc109ad0 2024-05-08T06:15:08.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.915+0000 7f1c01200700 1 -- 172.21.15.123:0/124722734 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bfc1a5590 con 0x7f1bfc109ad0 2024-05-08T06:15:08.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.915+0000 7f1bf2a00700 1 -- 172.21.15.123:0/124722734 <== mon.2 v2:172.21.15.163:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bec0043c0 con 0x7f1bfc109ad0 2024-05-08T06:15:08.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.915+0000 7f1bf2a00700 1 -- 172.21.15.123:0/124722734 <== mon.2 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bec005020 con 0x7f1bfc109ad0 2024-05-08T06:15:08.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.916+0000 7f1c01200700 1 -- 172.21.15.123:0/124722734 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bfc1ace10 con 0x7f1bfc109ad0 2024-05-08T06:15:08.917 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.917+0000 7f1c01200700 1 -- 172.21.15.123:0/124722734 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bfc112060 con 0x7f1bfc109ad0 2024-05-08T06:15:08.919 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.919+0000 7f1bf2a00700 1 -- 172.21.15.123:0/124722734 <== mon.2 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 20) ==== 149653+0+0 (secure 0 0 0) 0x7f1bec004750 con 0x7f1bfc109ad0 2024-05-08T06:15:08.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.921+0000 7f1bf2a00700 1 --2- 172.21.15.123:0/124722734 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f1be00b2880 0x7f1be00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:15:08.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.921+0000 7f1bf2a00700 1 -- 172.21.15.123:0/124722734 <== mon.2 v2:172.21.15.163:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7944+0+0 (secure 0 0 0) 0x7f1bec0e20e0 con 0x7f1bfc109ad0 2024-05-08T06:15:08.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.921+0000 7f1bf9400700 1 --2- 172.21.15.123:0/124722734 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f1be00b2880 0x7f1be00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:15:08.921 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.922+0000 7f1bf9400700 1 --2- 172.21.15.123:0/124722734 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f1be00b2880 0x7f1be00b4d40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc10f300 tx=0x7f1be800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:15:08.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:08.923+0000 7f1bf2a00700 1 -- 172.21.15.123:0/124722734 <== mon.2 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1bec0a9f60 con 0x7f1bfc109ad0 2024-05-08T06:15:09.152 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:09.152+0000 7f1c01200700 1 -- 172.21.15.123:0/124722734 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1bfc1a15f0 con 0x7f1bfc109ad0 2024-05-08T06:15:09.152 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:15:08 smithi123 ceph-mon[29787]: pgmap v602: 65 pgs: 65 active+clean; 577 KiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:15:09.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:15:08 smithi163 ceph-mon[34520]: pgmap v602: 65 pgs: 65 active+clean; 577 KiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:15:09.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:15:08 smithi192 ceph-mon[34664]: pgmap v602: 65 pgs: 65 active+clean; 577 KiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:15:09.943 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:09.942+0000 7f1bf2a00700 1 -- 172.21.15.123:0/124722734 <== mon.2 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v85) ==== 169+0+0 (secure 0 0 0) 0x7f1bec0a9860 con 0x7f1bfc109ad0 2024-05-08T06:15:09.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:09.948+0000 7f1c01200700 1 -- 172.21.15.123:0/124722734 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f1be00b2880 msgr2=0x7f1be00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:15:09.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:09.948+0000 7f1c01200700 1 --2- 172.21.15.123:0/124722734 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f1be00b2880 0x7f1be00b4d40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc10f300 tx=0x7f1be800b260 comp rx=0 tx=0).stop 2024-05-08T06:15:09.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:09.948+0000 7f1c01200700 1 -- 172.21.15.123:0/124722734 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bfc109ad0 msgr2=0x7f1bfc1a0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:15:09.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:09.948+0000 7f1c01200700 1 --2- 172.21.15.123:0/124722734 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bfc109ad0 0x7f1bfc1a0290 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1bec00e9f0 tx=0x7f1bec00ed00 comp rx=0 tx=0).stop 2024-05-08T06:15:09.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:09.949+0000 7f1c01200700 1 -- 172.21.15.123:0/124722734 shutdown_connections 2024-05-08T06:15:09.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:09.949+0000 7f1c01200700 1 --2- 172.21.15.123:0/124722734 >> [v2:172.21.15.123:6800/4169246006,v1:172.21.15.123:6801/4169246006] conn(0x7f1be00b2880 0x7f1be00b4d40 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:15:09.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:09.949+0000 7f1c01200700 1 --2- 172.21.15.123:0/124722734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bfc10fb60 0x7f1bfc1a4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:15:09.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:09.949+0000 7f1c01200700 1 --2- 172.21.15.123:0/124722734 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bfc109ad0 0x7f1bfc1a0290 secure :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1bec00e9f0 tx=0x7f1bec00ed00 comp rx=0 tx=0).stop 2024-05-08T06:15:09.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:09.949+0000 7f1c01200700 1 --2- 172.21.15.123:0/124722734 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bfc10a490 0x7f1bfc1a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:15:09.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:09.949+0000 7f1c01200700 1 -- 172.21.15.123:0/124722734 >> 172.21.15.123:0/124722734 conn(0x7f1bfc076c70 msgr2=0x7f1bfc077390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:15:09.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:09.950+0000 7f1c01200700 1 -- 172.21.15.123:0/124722734 shutdown_connections 2024-05-08T06:15:09.951 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-08T06:15:09.950+0000 7f1c01200700 1 -- 172.21.15.123:0/124722734 wait complete. 2024-05-08T06:15:09.952 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_data' removed 2024-05-08T06:15:10.202 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:15:09 smithi192 ceph-mon[34664]: from='client.? 172.21.15.123:0/124722734' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T06:15:10.202 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:15:09 smithi192 ceph-mon[34664]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T06:15:10.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:15:09 smithi123 ceph-mon[29787]: from='client.? 172.21.15.123:0/124722734' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T06:15:10.205 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:15:09 smithi123 ceph-mon[29787]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T06:15:10.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:15:09 smithi163 ceph-mon[34520]: from='client.? 172.21.15.123:0/124722734' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T06:15:10.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:15:09 smithi163 ceph-mon[34520]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T06:15:10.861 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T06:15:10.954 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T06:15:10.987 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T06:15:11.019 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-08T06:15:11.019 DEBUG:teuthology.orchestra.run.smithi123:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-08T06:15:11.076 DEBUG:teuthology.orchestra.run.smithi163:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-08T06:15:11.094 DEBUG:teuthology.orchestra.run.smithi192:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-08T06:15:11.112 INFO:tasks.cephadm:Stopping all daemons... 2024-05-08T06:15:11.112 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-08T06:15:11.112 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mon.a 2024-05-08T06:15:11.190 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:15:10 smithi123 ceph-mon[29787]: pgmap v603: 65 pgs: 65 active+clean; 577 KiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:15:11.191 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:15:10 smithi123 ceph-mon[29787]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T06:15:11.191 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:15:10 smithi123 ceph-mon[29787]: osdmap e85: 12 total, 12 up, 12 in 2024-05-08T06:15:11.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:15:10 smithi163 ceph-mon[34520]: pgmap v603: 65 pgs: 65 active+clean; 577 KiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:15:11.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:15:10 smithi163 ceph-mon[34520]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T06:15:11.288 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:15:10 smithi163 ceph-mon[34520]: osdmap e85: 12 total, 12 up, 12 in 2024-05-08T06:15:11.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:15:10 smithi192 ceph-mon[34664]: pgmap v603: 65 pgs: 65 active+clean; 577 KiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:15:11.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:15:10 smithi192 ceph-mon[34664]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T06:15:11.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:15:10 smithi192 ceph-mon[34664]: osdmap e85: 12 total, 12 up, 12 in 2024-05-08T06:15:11.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:15:11 smithi123 systemd[1]: Stopping Ceph mon.a for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T06:15:12.193 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:15:11 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mon-a[29783]: 2024-05-08T06:15:11.735+0000 7f753aa00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:15:12.193 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:15:11 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mon-a[29783]: 2024-05-08T06:15:11.735+0000 7f753aa00700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-08T06:15:12.193 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:15:11 smithi123 podman[95538]: 2024-05-08 06:15:11.798613196 +0000 UTC m=+0.250371060 container died f0e5e4ca808065702ad1f6b524525c89e14c8ee4d9920e81144d7e309f8874d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mon-a, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-08T06:15:12.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:15:12 smithi123 podman[95538]: 2024-05-08 06:15:12.328746203 +0000 UTC m=+0.780504074 container cleanup f0e5e4ca808065702ad1f6b524525c89e14c8ee4d9920e81144d7e309f8874d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f) 2024-05-08T06:15:12.455 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:15:12 smithi123 bash[95538]: ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mon-a 2024-05-08T06:15:12.955 INFO:journalctl@ceph.mon.a.smithi123.stdout:May 08 06:15:12 smithi123 podman[95552]: 2024-05-08 06:15:12.672203014 +0000 UTC m=+0.858421037 container remove f0e5e4ca808065702ad1f6b524525c89e14c8ee4d9920e81144d7e309f8874d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mon-a, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-08T06:15:13.198 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mon.a.service' 2024-05-08T06:15:13.341 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:15:13.342 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-08T06:15:13.342 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-08T06:15:13.342 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mon.b 2024-05-08T06:15:13.662 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:15:13 smithi163 systemd[1]: Stopping Ceph mon.b for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T06:15:14.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:15:13 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mon-b[34515]: 2024-05-08T06:15:13.663+0000 7fd3b039f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:15:14.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:15:13 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mon-b[34515]: 2024-05-08T06:15:13.663+0000 7fd3b039f700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-08T06:15:14.038 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:15:13 smithi163 podman[68445]: 2024-05-08 06:15:13.775273445 +0000 UTC m=+0.236872100 container died 72a80a2675fe76851a0b33442a0c56c468c74517efd98d98c6f35e97024d9f3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mon-b, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True) 2024-05-08T06:15:14.380 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:15:14 smithi163 podman[68445]: 2024-05-08 06:15:14.171760007 +0000 UTC m=+0.633358653 container cleanup 72a80a2675fe76851a0b33442a0c56c468c74517efd98d98c6f35e97024d9f3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mon-b, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD) 2024-05-08T06:15:14.381 INFO:journalctl@ceph.mon.b.smithi163.stdout:May 08 06:15:14 smithi163 bash[68445]: ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mon-b 2024-05-08T06:15:14.585 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mon.b.service' 2024-05-08T06:15:14.628 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:15:14.628 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-08T06:15:14.628 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-08T06:15:14.629 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mon.c 2024-05-08T06:15:14.951 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:15:14 smithi192 systemd[1]: Stopping Ceph mon.c for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T06:15:15.300 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:15:14 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mon-c[34660]: 2024-05-08T06:15:14.950+0000 7fd17e46a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:15:15.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:15:14 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mon-c[34660]: 2024-05-08T06:15:14.950+0000 7fd17e46a700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-08T06:15:15.301 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:15:15 smithi192 podman[69176]: 2024-05-08 06:15:15.124439606 +0000 UTC m=+0.299905499 container died adfc8c6ace0c13c8f69f9eb6f7f84c47df58aee294cc7da4d39cb19c17483b6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mon-c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T06:15:15.737 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:15:15 smithi192 podman[69176]: 2024-05-08 06:15:15.535947912 +0000 UTC m=+0.711413806 container cleanup adfc8c6ace0c13c8f69f9eb6f7f84c47df58aee294cc7da4d39cb19c17483b6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mon-c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-05-08T06:15:15.737 INFO:journalctl@ceph.mon.c.smithi192.stdout:May 08 06:15:15 smithi192 bash[69176]: ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mon-c 2024-05-08T06:15:15.954 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mon.c.service' 2024-05-08T06:15:16.003 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:15:16.003 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-08T06:15:16.003 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-08T06:15:16.003 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mgr.x 2024-05-08T06:15:16.455 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 06:15:16 smithi123 systemd[1]: Stopping Ceph mgr.x for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T06:15:16.956 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 06:15:16 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T06:15:16.532+0000 7f78abc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:15:16.956 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 06:15:16 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x[30027]: 2024-05-08T06:15:16.532+0000 7f78abc00700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-05-08T06:15:16.956 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 06:15:16 smithi123 podman[95657]: 2024-05-08 06:15:16.626260553 +0000 UTC m=+0.242712561 container died 1df64aa6e04a75c924ebcf1acc4968a5d0a94432f61ef7eeae365e7eba7ec580 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-08T06:15:17.520 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 06:15:17 smithi123 podman[95657]: 2024-05-08 06:15:17.243101258 +0000 UTC m=+0.859553260 container cleanup 1df64aa6e04a75c924ebcf1acc4968a5d0a94432f61ef7eeae365e7eba7ec580 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-08T06:15:17.520 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 06:15:17 smithi123 bash[95657]: ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x 2024-05-08T06:15:17.774 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 06:15:17 smithi123 podman[95671]: 2024-05-08 06:15:17.520019097 +0000 UTC m=+0.878935959 container remove 1df64aa6e04a75c924ebcf1acc4968a5d0a94432f61ef7eeae365e7eba7ec580 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-x, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-08T06:15:18.028 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mgr.x.service' 2024-05-08T06:15:18.093 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 06:15:18 smithi123 systemd[1]: ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mgr.x.service: Deactivated successfully. 2024-05-08T06:15:18.094 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 06:15:18 smithi123 systemd[1]: Stopped Ceph mgr.x for 85cfb2e6-0cff-11ef-bc97-c7b262605968. 2024-05-08T06:15:18.094 INFO:journalctl@ceph.mgr.x.smithi123.stdout:May 08 06:15:18 smithi123 systemd[1]: ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mgr.x.service: Consumed 1min 57.074s CPU time. 2024-05-08T06:15:18.128 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:15:18.129 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-08T06:15:18.129 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-08T06:15:18.129 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mgr.y 2024-05-08T06:15:18.538 INFO:journalctl@ceph.mgr.y.smithi163.stdout:May 08 06:15:18 smithi163 systemd[1]: Stopping Ceph mgr.y for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T06:15:18.918 INFO:journalctl@ceph.mgr.y.smithi163.stdout:May 08 06:15:18 smithi163 podman[68568]: 2024-05-08 06:15:18.621941955 +0000 UTC m=+0.299044813 container died de9650aaecc952dbe3635e67aac53dea6e863bb45f515f226d568ba6ce841d9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-y, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T06:15:19.288 INFO:journalctl@ceph.mgr.y.smithi163.stdout:May 08 06:15:19 smithi163 podman[68568]: 2024-05-08 06:15:19.002501402 +0000 UTC m=+0.679604259 container cleanup de9650aaecc952dbe3635e67aac53dea6e863bb45f515f226d568ba6ce841d9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, ceph=True, org.label-schema.build-date=20240507, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T06:15:19.288 INFO:journalctl@ceph.mgr.y.smithi163.stdout:May 08 06:15:19 smithi163 bash[68568]: ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-y 2024-05-08T06:15:19.563 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mgr.y.service' 2024-05-08T06:15:19.596 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:15:19.596 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-08T06:15:19.597 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-08T06:15:19.597 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mgr.z 2024-05-08T06:15:19.937 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 08 06:15:19 smithi192 systemd[1]: Stopping Ceph mgr.z for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T06:15:20.243 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 08 06:15:19 smithi192 podman[69299]: 2024-05-08 06:15:19.93831517 +0000 UTC m=+0.149826504 container died 85c10a5c884bb25b4a5e75d28456fa7ad2b7042ae3645654bee97bb1e77783e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T06:15:20.550 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 08 06:15:20 smithi192 podman[69299]: 2024-05-08 06:15:20.335476143 +0000 UTC m=+0.546987476 container cleanup 85c10a5c884bb25b4a5e75d28456fa7ad2b7042ae3645654bee97bb1e77783e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-z, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux ) 2024-05-08T06:15:20.551 INFO:journalctl@ceph.mgr.z.smithi192.stdout:May 08 06:15:20 smithi192 bash[69299]: ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mgr-z 2024-05-08T06:15:20.759 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mgr.z.service' 2024-05-08T06:15:20.792 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:15:20.792 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-08T06:15:20.792 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-08T06:15:20.792 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@osd.0 2024-05-08T06:15:21.205 INFO:journalctl@ceph.osd.0.smithi123.stdout:May 08 06:15:20 smithi123 systemd[1]: Stopping Ceph osd.0 for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T06:15:21.705 INFO:journalctl@ceph.osd.0.smithi123.stdout:May 08 06:15:21 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-0[44030]: 2024-05-08T06:15:21.427+0000 7fea73a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:15:21.705 INFO:journalctl@ceph.osd.0.smithi123.stdout:May 08 06:15:21 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-0[44030]: 2024-05-08T06:15:21.427+0000 7fea73a00700 -1 osd.0 85 *** Got signal Terminated *** 2024-05-08T06:15:21.705 INFO:journalctl@ceph.osd.0.smithi123.stdout:May 08 06:15:21 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-0[44030]: 2024-05-08T06:15:21.427+0000 7fea73a00700 -1 osd.0 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T06:15:27.205 INFO:journalctl@ceph.osd.0.smithi123.stdout:May 08 06:15:26 smithi123 podman[95776]: 2024-05-08 06:15:26.816197279 +0000 UTC m=+5.560784565 container died fa983cfb74970776b68c1425c5f9df3537b7a5e97bcd8de3bac29b1102a5f6f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-08T06:15:27.833 INFO:journalctl@ceph.osd.0.smithi123.stdout:May 08 06:15:27 smithi123 podman[95776]: 2024-05-08 06:15:27.506618542 +0000 UTC m=+6.251205882 container cleanup fa983cfb74970776b68c1425c5f9df3537b7a5e97bcd8de3bac29b1102a5f6f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-0, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-08T06:15:27.833 INFO:journalctl@ceph.osd.0.smithi123.stdout:May 08 06:15:27 smithi123 bash[95776]: ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-0 2024-05-08T06:15:28.204 INFO:journalctl@ceph.osd.0.smithi123.stdout:May 08 06:15:27 smithi123 podman[95795]: 2024-05-08 06:15:27.833335507 +0000 UTC m=+1.008394129 container remove fa983cfb74970776b68c1425c5f9df3537b7a5e97bcd8de3bac29b1102a5f6f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-08T06:15:28.772 INFO:journalctl@ceph.osd.0.smithi123.stdout:May 08 06:15:28 smithi123 podman[95885]: 2024-05-08 06:15:28.426263298 +0000 UTC m=+0.035186214 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:15:28.772 INFO:journalctl@ceph.osd.0.smithi123.stdout:May 08 06:15:28 smithi123 podman[95885]: 2024-05-08 06:15:28.525850427 +0000 UTC m=+0.134773378 container create bc8528e159b7be4ece5ce4641a11a0d6519b9801a5c5102374e69a59c1011892 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-08T06:15:29.205 INFO:journalctl@ceph.osd.0.smithi123.stdout:May 08 06:15:28 smithi123 podman[95885]: 2024-05-08 06:15:28.905002251 +0000 UTC m=+0.513925209 container init bc8528e159b7be4ece5ce4641a11a0d6519b9801a5c5102374e69a59c1011892 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, io.buildah.version=1.33.5) 2024-05-08T06:15:29.205 INFO:journalctl@ceph.osd.0.smithi123.stdout:May 08 06:15:28 smithi123 podman[95885]: 2024-05-08 06:15:28.916220751 +0000 UTC m=+0.525143709 container start bc8528e159b7be4ece5ce4641a11a0d6519b9801a5c5102374e69a59c1011892 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T06:15:29.205 INFO:journalctl@ceph.osd.0.smithi123.stdout:May 08 06:15:29 smithi123 podman[95885]: 2024-05-08 06:15:29.037311782 +0000 UTC m=+0.646234766 container attach bc8528e159b7be4ece5ce4641a11a0d6519b9801a5c5102374e69a59c1011892 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True) 2024-05-08T06:15:29.705 INFO:journalctl@ceph.osd.0.smithi123.stdout:May 08 06:15:29 smithi123 podman[95885]: 2024-05-08 06:15:29.365164733 +0000 UTC m=+0.974087648 container died bc8528e159b7be4ece5ce4641a11a0d6519b9801a5c5102374e69a59c1011892 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5) 2024-05-08T06:15:30.339 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@osd.0.service' 2024-05-08T06:15:30.454 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:15:30.455 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-08T06:15:30.455 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-08T06:15:30.455 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@osd.1 2024-05-08T06:15:30.765 INFO:journalctl@ceph.osd.1.smithi163.stdout:May 08 06:15:30 smithi163 systemd[1]: Stopping Ceph osd.1 for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T06:15:31.038 INFO:journalctl@ceph.osd.1.smithi163.stdout:May 08 06:15:30 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-1[39131]: 2024-05-08T06:15:30.764+0000 7f861173a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:15:31.038 INFO:journalctl@ceph.osd.1.smithi163.stdout:May 08 06:15:30 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-1[39131]: 2024-05-08T06:15:30.764+0000 7f861173a700 -1 osd.1 85 *** Got signal Terminated *** 2024-05-08T06:15:31.038 INFO:journalctl@ceph.osd.1.smithi163.stdout:May 08 06:15:30 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-1[39131]: 2024-05-08T06:15:30.764+0000 7f861173a700 -1 osd.1 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T06:15:36.500 INFO:journalctl@ceph.osd.1.smithi163.stdout:May 08 06:15:36 smithi163 podman[68707]: 2024-05-08 06:15:36.206833304 +0000 UTC m=+5.560805839 container died 0279cbe930da3bf73c9a9457f0e120455ec6f39a3a3f1d326880d0197b5004ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-1, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True) 2024-05-08T06:15:36.788 INFO:journalctl@ceph.osd.1.smithi163.stdout:May 08 06:15:36 smithi163 podman[68707]: 2024-05-08 06:15:36.582353688 +0000 UTC m=+5.936326229 container cleanup 0279cbe930da3bf73c9a9457f0e120455ec6f39a3a3f1d326880d0197b5004ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-1, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T06:15:36.788 INFO:journalctl@ceph.osd.1.smithi163.stdout:May 08 06:15:36 smithi163 bash[68707]: ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-1 2024-05-08T06:15:37.194 INFO:journalctl@ceph.osd.1.smithi163.stdout:May 08 06:15:36 smithi163 podman[68724]: 2024-05-08 06:15:36.851080009 +0000 UTC m=+0.640733075 container remove 0279cbe930da3bf73c9a9457f0e120455ec6f39a3a3f1d326880d0197b5004ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-1, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-05-08T06:15:37.470 INFO:journalctl@ceph.osd.1.smithi163.stdout:May 08 06:15:37 smithi163 podman[68813]: 2024-05-08 06:15:37.097939225 +0000 UTC m=+0.018097619 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:15:37.470 INFO:journalctl@ceph.osd.1.smithi163.stdout:May 08 06:15:37 smithi163 podman[68813]: 2024-05-08 06:15:37.210188659 +0000 UTC m=+0.130347053 container create d4732415e099d9b57a0911aab27e39284714e98b895d38ce6105b50a8131fa73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True) 2024-05-08T06:15:37.774 INFO:journalctl@ceph.osd.1.smithi163.stdout:May 08 06:15:37 smithi163 podman[68813]: 2024-05-08 06:15:37.47104243 +0000 UTC m=+0.391200823 container init d4732415e099d9b57a0911aab27e39284714e98b895d38ce6105b50a8131fa73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240507, ceph=True, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-08T06:15:37.774 INFO:journalctl@ceph.osd.1.smithi163.stdout:May 08 06:15:37 smithi163 podman[68813]: 2024-05-08 06:15:37.474357836 +0000 UTC m=+0.394516232 container start d4732415e099d9b57a0911aab27e39284714e98b895d38ce6105b50a8131fa73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-08T06:15:37.774 INFO:journalctl@ceph.osd.1.smithi163.stdout:May 08 06:15:37 smithi163 podman[68813]: 2024-05-08 06:15:37.554563797 +0000 UTC m=+0.474722217 container attach d4732415e099d9b57a0911aab27e39284714e98b895d38ce6105b50a8131fa73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T06:15:37.774 INFO:journalctl@ceph.osd.1.smithi163.stdout:May 08 06:15:37 smithi163 podman[68813]: 2024-05-08 06:15:37.774960955 +0000 UTC m=+0.695119355 container died d4732415e099d9b57a0911aab27e39284714e98b895d38ce6105b50a8131fa73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-08T06:15:38.395 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@osd.1.service' 2024-05-08T06:15:38.433 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:15:38.433 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-08T06:15:38.433 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-08T06:15:38.433 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@osd.2 2024-05-08T06:15:38.747 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 08 06:15:38 smithi192 systemd[1]: Stopping Ceph osd.2 for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T06:15:39.050 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 08 06:15:38 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-2[39865]: 2024-05-08T06:15:38.747+0000 7fed85528700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:15:39.050 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 08 06:15:38 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-2[39865]: 2024-05-08T06:15:38.747+0000 7fed85528700 -1 osd.2 85 *** Got signal Terminated *** 2024-05-08T06:15:39.050 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 08 06:15:38 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-2[39865]: 2024-05-08T06:15:38.747+0000 7fed85528700 -1 osd.2 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T06:15:44.506 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 08 06:15:44 smithi192 podman[69422]: 2024-05-08 06:15:44.22451001 +0000 UTC m=+5.601109244 container died df36c3c198f8953067e6570bfa0ec136188ac1af637644c95de4e5611d09903b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, ceph=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-08T06:15:44.800 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 08 06:15:44 smithi192 podman[69422]: 2024-05-08 06:15:44.606943549 +0000 UTC m=+5.983542792 container cleanup df36c3c198f8953067e6570bfa0ec136188ac1af637644c95de4e5611d09903b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-2, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-08T06:15:44.800 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 08 06:15:44 smithi192 bash[69422]: ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-2 2024-05-08T06:15:45.180 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 08 06:15:44 smithi192 podman[69440]: 2024-05-08 06:15:44.875048821 +0000 UTC m=+0.648178847 container remove df36c3c198f8953067e6570bfa0ec136188ac1af637644c95de4e5611d09903b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-2, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5) 2024-05-08T06:15:45.519 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 08 06:15:45 smithi192 podman[69528]: 2024-05-08 06:15:45.084452695 +0000 UTC m=+0.017703529 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:15:45.519 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 08 06:15:45 smithi192 podman[69528]: 2024-05-08 06:15:45.192454172 +0000 UTC m=+0.125705036 container create 610abdc7e61af29c1d7383706f0302b8ac90f0763ab57f5fd8bfdc7c422f5271 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-08T06:15:45.519 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 08 06:15:45 smithi192 podman[69528]: 2024-05-08 06:15:45.4282311 +0000 UTC m=+0.361481921 container init 610abdc7e61af29c1d7383706f0302b8ac90f0763ab57f5fd8bfdc7c422f5271 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T06:15:45.519 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 08 06:15:45 smithi192 podman[69528]: 2024-05-08 06:15:45.431415732 +0000 UTC m=+0.364666553 container start 610abdc7e61af29c1d7383706f0302b8ac90f0763ab57f5fd8bfdc7c422f5271 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-08T06:15:45.520 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 08 06:15:45 smithi192 podman[69528]: 2024-05-08 06:15:45.520069983 +0000 UTC m=+0.453320814 container attach 610abdc7e61af29c1d7383706f0302b8ac90f0763ab57f5fd8bfdc7c422f5271 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-08T06:15:45.800 INFO:journalctl@ceph.osd.2.smithi192.stdout:May 08 06:15:45 smithi192 podman[69528]: 2024-05-08 06:15:45.724607877 +0000 UTC m=+0.657858689 container died 610abdc7e61af29c1d7383706f0302b8ac90f0763ab57f5fd8bfdc7c422f5271 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-08T06:15:46.316 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@osd.2.service' 2024-05-08T06:15:46.354 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:15:46.355 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-08T06:15:46.355 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-08T06:15:46.355 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@osd.3 2024-05-08T06:15:46.705 INFO:journalctl@ceph.osd.3.smithi123.stdout:May 08 06:15:46 smithi123 systemd[1]: Stopping Ceph osd.3 for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T06:15:47.205 INFO:journalctl@ceph.osd.3.smithi123.stdout:May 08 06:15:46 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-3[50712]: 2024-05-08T06:15:46.949+0000 7fac2a600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:15:47.205 INFO:journalctl@ceph.osd.3.smithi123.stdout:May 08 06:15:46 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-3[50712]: 2024-05-08T06:15:46.949+0000 7fac2a600700 -1 osd.3 85 *** Got signal Terminated *** 2024-05-08T06:15:47.205 INFO:journalctl@ceph.osd.3.smithi123.stdout:May 08 06:15:46 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-3[50712]: 2024-05-08T06:15:46.949+0000 7fac2a600700 -1 osd.3 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T06:15:47.955 INFO:journalctl@ceph.osd.3.smithi123.stdout:May 08 06:15:47 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-3[50712]: 2024-05-08T06:15:47.527+0000 7fac20e00700 -1 osd.3 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.268152+0000 front 2024-05-08T06:15:23.268428+0000 (oldest deadline 2024-05-08T06:15:47.368127+0000) 2024-05-08T06:15:48.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:15:47 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:15:47.834+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:15:48.705 INFO:journalctl@ceph.osd.3.smithi123.stdout:May 08 06:15:48 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-3[50712]: 2024-05-08T06:15:48.525+0000 7fac20e00700 -1 osd.3 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.268152+0000 front 2024-05-08T06:15:23.268428+0000 (oldest deadline 2024-05-08T06:15:47.368127+0000) 2024-05-08T06:15:49.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:15:48 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:15:48.811+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.840183+0000 front 2024-05-08T06:15:25.840948+0000 (oldest deadline 2024-05-08T06:15:48.740113+0000) 2024-05-08T06:15:49.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:15:48 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:15:48.806+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:15:49.566 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:15:49 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:15:49.288+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:15:49.955 INFO:journalctl@ceph.osd.3.smithi123.stdout:May 08 06:15:49 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-3[50712]: 2024-05-08T06:15:49.564+0000 7fac20e00700 -1 osd.3 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.268152+0000 front 2024-05-08T06:15:23.268428+0000 (oldest deadline 2024-05-08T06:15:47.368127+0000) 2024-05-08T06:15:49.955 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:15:49 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:15:49.790+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.840183+0000 front 2024-05-08T06:15:25.840948+0000 (oldest deadline 2024-05-08T06:15:48.740113+0000) 2024-05-08T06:15:50.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:15:49 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:15:49.728+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:15:50.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:15:49 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:15:49.833+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:15:50.578 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:15:50 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:15:50.326+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:15:50.746 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:15:50 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:15:50.426+0000 7f46870a6700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.208042+0000 front 2024-05-08T06:15:25.208125+0000 (oldest deadline 2024-05-08T06:15:49.907884+0000) 2024-05-08T06:15:50.955 INFO:journalctl@ceph.osd.3.smithi123.stdout:May 08 06:15:50 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-3[50712]: 2024-05-08T06:15:50.578+0000 7fac20e00700 -1 osd.3 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.268152+0000 front 2024-05-08T06:15:23.268428+0000 (oldest deadline 2024-05-08T06:15:47.368127+0000) 2024-05-08T06:15:50.955 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:15:50 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:15:50.816+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.840183+0000 front 2024-05-08T06:15:25.840948+0000 (oldest deadline 2024-05-08T06:15:48.740113+0000) 2024-05-08T06:15:51.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:15:50 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:15:50.747+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:15:51.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:15:50 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:15:50.876+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:15:51.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:15:51 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:15:51.299+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:15:51.705 INFO:journalctl@ceph.osd.3.smithi123.stdout:May 08 06:15:51 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-3[50712]: 2024-05-08T06:15:51.546+0000 7fac20e00700 -1 osd.3 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.268152+0000 front 2024-05-08T06:15:23.268428+0000 (oldest deadline 2024-05-08T06:15:47.368127+0000) 2024-05-08T06:15:51.706 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:15:51 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:15:51.412+0000 7f46870a6700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.208042+0000 front 2024-05-08T06:15:25.208125+0000 (oldest deadline 2024-05-08T06:15:49.907884+0000) 2024-05-08T06:15:52.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:15:51 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:15:51.707+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:15:52.206 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:15:51 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:15:51.858+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.840183+0000 front 2024-05-08T06:15:25.840948+0000 (oldest deadline 2024-05-08T06:15:48.740113+0000) 2024-05-08T06:15:52.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:15:51 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:15:51.835+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:15:52.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:15:52 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:15:52.338+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:15:52.705 INFO:journalctl@ceph.osd.3.smithi123.stdout:May 08 06:15:52 smithi123 podman[95990]: 2024-05-08 06:15:52.348695371 +0000 UTC m=+5.556864894 container died 2bdbe31172318a95be78f663495c4e872e6be01dacbde6f787fa4233deec198e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-3, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-05-08T06:15:52.723 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:15:52 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:15:52.391+0000 7f46870a6700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.208042+0000 front 2024-05-08T06:15:25.208125+0000 (oldest deadline 2024-05-08T06:15:49.907884+0000) 2024-05-08T06:15:53.002 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:15:52 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:15:52.899+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.840183+0000 front 2024-05-08T06:15:25.840948+0000 (oldest deadline 2024-05-08T06:15:48.740113+0000) 2024-05-08T06:15:53.003 INFO:journalctl@ceph.osd.3.smithi123.stdout:May 08 06:15:52 smithi123 podman[95990]: 2024-05-08 06:15:52.993914622 +0000 UTC m=+6.202084176 container cleanup 2bdbe31172318a95be78f663495c4e872e6be01dacbde6f787fa4233deec198e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-3, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T06:15:53.003 INFO:journalctl@ceph.osd.3.smithi123.stdout:May 08 06:15:52 smithi123 bash[95990]: ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-3 2024-05-08T06:15:53.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:15:52 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:15:52.723+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:15:53.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:15:52 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:15:52.866+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:15:53.705 INFO:journalctl@ceph.osd.3.smithi123.stdout:May 08 06:15:53 smithi123 podman[96022]: 2024-05-08 06:15:53.262221551 +0000 UTC m=+0.898166845 container remove 2bdbe31172318a95be78f663495c4e872e6be01dacbde6f787fa4233deec198e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-08T06:15:53.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:15:53 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:15:53.359+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:15:53.722 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:15:53 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:15:53.403+0000 7f46870a6700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.208042+0000 front 2024-05-08T06:15:25.208125+0000 (oldest deadline 2024-05-08T06:15:49.907884+0000) 2024-05-08T06:15:54.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:15:53 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:15:53.723+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:15:54.205 INFO:journalctl@ceph.osd.3.smithi123.stdout:May 08 06:15:54 smithi123 podman[96110]: 2024-05-08 06:15:54.070697703 +0000 UTC m=+0.033252814 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:15:54.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:15:53 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:15:53.928+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.840183+0000 front 2024-05-08T06:15:25.840948+0000 (oldest deadline 2024-05-08T06:15:48.740113+0000) 2024-05-08T06:15:54.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:15:53 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:15:53.868+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:15:54.571 INFO:journalctl@ceph.osd.3.smithi123.stdout:May 08 06:15:54 smithi123 podman[96110]: 2024-05-08 06:15:54.224131523 +0000 UTC m=+0.186686685 container create 2127b5e2e4a346f5488b38bcd16a8d2326c38c4fded564f18fb7812a1b7bf490 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f) 2024-05-08T06:15:54.571 INFO:journalctl@ceph.osd.3.smithi123.stdout:May 08 06:15:54 smithi123 podman[96110]: 2024-05-08 06:15:54.570333889 +0000 UTC m=+0.532889021 container init 2127b5e2e4a346f5488b38bcd16a8d2326c38c4fded564f18fb7812a1b7bf490 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-08T06:15:54.571 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:15:54 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:15:54.313+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:15:54.720 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:15:54 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:15:54.413+0000 7f46870a6700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.208042+0000 front 2024-05-08T06:15:25.208125+0000 (oldest deadline 2024-05-08T06:15:49.907884+0000) 2024-05-08T06:15:54.950 INFO:journalctl@ceph.osd.3.smithi123.stdout:May 08 06:15:54 smithi123 podman[96110]: 2024-05-08 06:15:54.582177265 +0000 UTC m=+0.544732378 container start 2127b5e2e4a346f5488b38bcd16a8d2326c38c4fded564f18fb7812a1b7bf490 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-08T06:15:54.950 INFO:journalctl@ceph.osd.3.smithi123.stdout:May 08 06:15:54 smithi123 podman[96110]: 2024-05-08 06:15:54.657425291 +0000 UTC m=+0.619980469 container attach 2127b5e2e4a346f5488b38bcd16a8d2326c38c4fded564f18fb7812a1b7bf490 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-3-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True) 2024-05-08T06:15:55.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:15:54 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:15:54.720+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:15:55.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:15:54 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:15:54.949+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.840183+0000 front 2024-05-08T06:15:25.840948+0000 (oldest deadline 2024-05-08T06:15:48.740113+0000) 2024-05-08T06:15:55.205 INFO:journalctl@ceph.osd.3.smithi123.stdout:May 08 06:15:55 smithi123 podman[96110]: 2024-05-08 06:15:55.033856873 +0000 UTC m=+0.996412015 container died 2127b5e2e4a346f5488b38bcd16a8d2326c38c4fded564f18fb7812a1b7bf490 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T06:15:55.287 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:15:54 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:15:54.898+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:15:55.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:15:55 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:15:55.346+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:15:55.740 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:15:55 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:15:55.424+0000 7f46870a6700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.208042+0000 front 2024-05-08T06:15:25.208125+0000 (oldest deadline 2024-05-08T06:15:49.907884+0000) 2024-05-08T06:15:55.925 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@osd.3.service' 2024-05-08T06:15:56.029 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:15:56.029 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-08T06:15:56.029 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-08T06:15:56.029 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@osd.4 2024-05-08T06:15:56.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:15:55 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:15:55.739+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:15:56.189 INFO:journalctl@ceph.osd.4.smithi163.stdout:May 08 06:15:56 smithi163 systemd[1]: Stopping Ceph osd.4 for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T06:15:56.189 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:15:55 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:15:55.934+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:15:56.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:15:55 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:15:55.960+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.840183+0000 front 2024-05-08T06:15:25.840948+0000 (oldest deadline 2024-05-08T06:15:48.740113+0000) 2024-05-08T06:15:56.538 INFO:journalctl@ceph.osd.4.smithi163.stdout:May 08 06:15:56 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-4[46064]: 2024-05-08T06:15:56.352+0000 7f16e4e05700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:15:56.538 INFO:journalctl@ceph.osd.4.smithi163.stdout:May 08 06:15:56 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-4[46064]: 2024-05-08T06:15:56.352+0000 7f16e4e05700 -1 osd.4 85 *** Got signal Terminated *** 2024-05-08T06:15:56.538 INFO:journalctl@ceph.osd.4.smithi163.stdout:May 08 06:15:56 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-4[46064]: 2024-05-08T06:15:56.352+0000 7f16e4e05700 -1 osd.4 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T06:15:56.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:15:56 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:15:56.390+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:15:56.741 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:15:56 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:15:56.456+0000 7f46870a6700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.208042+0000 front 2024-05-08T06:15:25.208125+0000 (oldest deadline 2024-05-08T06:15:49.907884+0000) 2024-05-08T06:15:56.741 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:15:56 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:15:56.456+0000 7f46870a6700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.608958+0000 front 2024-05-08T06:15:34.609053+0000 (oldest deadline 2024-05-08T06:15:56.308959+0000) 2024-05-08T06:15:56.741 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:15:56 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:15:56.653+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:15:57.004 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:15:56 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:15:56.741+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:15:57.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:15:56 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:15:56.921+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.840183+0000 front 2024-05-08T06:15:25.840948+0000 (oldest deadline 2024-05-08T06:15:48.740113+0000) 2024-05-08T06:15:57.209 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:15:56 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:15:56.928+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:15:57.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:15:57 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:15:57.428+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:15:57.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:15:57 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:15:57.428+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:15:57.765 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:15:57 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:15:57.650+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:15:57.765 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:15:57 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:15:57.450+0000 7f46870a6700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.208042+0000 front 2024-05-08T06:15:25.208125+0000 (oldest deadline 2024-05-08T06:15:49.907884+0000) 2024-05-08T06:15:57.765 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:15:57 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:15:57.450+0000 7f46870a6700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.608958+0000 front 2024-05-08T06:15:34.609053+0000 (oldest deadline 2024-05-08T06:15:56.308959+0000) 2024-05-08T06:15:58.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:15:57 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:15:57.764+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:15:58.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:15:57 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:15:57.893+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.840183+0000 front 2024-05-08T06:15:25.840948+0000 (oldest deadline 2024-05-08T06:15:48.740113+0000) 2024-05-08T06:15:58.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:15:57 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:15:57.934+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:15:58.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:15:58 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:15:58.412+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:15:58.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:15:58 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:15:58.412+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:15:58.800 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:15:58 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:15:58.642+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:15:58.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:15:58 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:15:58.723+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:15:58.801 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:15:58 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:15:58.495+0000 7f46870a6700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.208042+0000 front 2024-05-08T06:15:25.208125+0000 (oldest deadline 2024-05-08T06:15:49.907884+0000) 2024-05-08T06:15:58.801 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:15:58 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:15:58.495+0000 7f46870a6700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.608958+0000 front 2024-05-08T06:15:34.609053+0000 (oldest deadline 2024-05-08T06:15:56.308959+0000) 2024-05-08T06:15:59.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:15:58 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:15:58.912+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.840183+0000 front 2024-05-08T06:15:25.840948+0000 (oldest deadline 2024-05-08T06:15:48.740113+0000) 2024-05-08T06:15:59.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:15:58 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:15:58.950+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:15:59.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:15:59 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:15:59.417+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:15:59.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:15:59 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:15:59.417+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:15:59.800 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:15:59 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:15:59.606+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:15:59.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:15:59 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:15:59.757+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:15:59.801 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:15:59 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:15:59.510+0000 7f46870a6700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.208042+0000 front 2024-05-08T06:15:25.208125+0000 (oldest deadline 2024-05-08T06:15:49.907884+0000) 2024-05-08T06:15:59.801 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:15:59 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:15:59.510+0000 7f46870a6700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.608958+0000 front 2024-05-08T06:15:34.609053+0000 (oldest deadline 2024-05-08T06:15:56.308959+0000) 2024-05-08T06:16:00.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:15:59 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:15:59.928+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.840183+0000 front 2024-05-08T06:15:25.840948+0000 (oldest deadline 2024-05-08T06:15:48.740113+0000) 2024-05-08T06:16:00.290 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:15:59 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:15:59.964+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:00.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:00 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:00.419+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:00.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:00 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:00.419+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:00.757 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:16:00 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:16:00.483+0000 7f46870a6700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.208042+0000 front 2024-05-08T06:15:25.208125+0000 (oldest deadline 2024-05-08T06:15:49.907884+0000) 2024-05-08T06:16:00.757 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:16:00 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:16:00.483+0000 7f46870a6700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.608958+0000 front 2024-05-08T06:15:34.609053+0000 (oldest deadline 2024-05-08T06:15:56.308959+0000) 2024-05-08T06:16:00.758 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:00 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:00.565+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:16:00.788 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:00 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:00.343+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:33.831967+0000 front 2024-05-08T06:15:33.832034+0000 (oldest deadline 2024-05-08T06:15:59.732138+0000) 2024-05-08T06:16:01.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:00 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:00.757+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:01.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:00 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:00.880+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.840183+0000 front 2024-05-08T06:15:25.840948+0000 (oldest deadline 2024-05-08T06:15:48.740113+0000) 2024-05-08T06:16:01.226 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:00 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:00.926+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:01.226 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:00 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:00.926+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:01.538 INFO:journalctl@ceph.osd.4.smithi163.stdout:May 08 06:16:01 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-4[46064]: 2024-05-08T06:16:01.226+0000 7f16e13f9700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.365125+0000 front 2024-05-08T06:15:34.365093+0000 (oldest deadline 2024-05-08T06:16:00.265222+0000) 2024-05-08T06:16:01.538 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:01 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:01.376+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:33.831967+0000 front 2024-05-08T06:15:33.832034+0000 (oldest deadline 2024-05-08T06:15:59.732138+0000) 2024-05-08T06:16:01.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:01 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:01.429+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:01.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:01 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:01.429+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:01.803 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:01 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:01.538+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:16:01.803 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:01 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:01.734+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:01.803 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:01 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:01.734+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:01.803 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:16:01 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:16:01.508+0000 7f46870a6700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.208042+0000 front 2024-05-08T06:15:25.208125+0000 (oldest deadline 2024-05-08T06:15:49.907884+0000) 2024-05-08T06:16:01.803 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:16:01 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:16:01.508+0000 7f46870a6700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.608958+0000 front 2024-05-08T06:15:34.609053+0000 (oldest deadline 2024-05-08T06:15:56.308959+0000) 2024-05-08T06:16:01.974 INFO:journalctl@ceph.osd.4.smithi163.stdout:May 08 06:16:01 smithi163 podman[68931]: 2024-05-08 06:16:01.709313432 +0000 UTC m=+5.487272852 container died 6e390b3ce4b13036957bcaadf1becc0d745b3ac3ef30fc3b5c1d420801d4412a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-4, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T06:16:02.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:01 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:01.838+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.840183+0000 front 2024-05-08T06:15:25.840948+0000 (oldest deadline 2024-05-08T06:15:48.740113+0000) 2024-05-08T06:16:02.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:01 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:01.838+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.643693+0000 front 2024-05-08T06:15:35.643815+0000 (oldest deadline 2024-05-08T06:16:01.543986+0000) 2024-05-08T06:16:02.288 INFO:journalctl@ceph.osd.4.smithi163.stdout:May 08 06:16:02 smithi163 podman[68931]: 2024-05-08 06:16:02.11059964 +0000 UTC m=+5.888559051 container cleanup 6e390b3ce4b13036957bcaadf1becc0d745b3ac3ef30fc3b5c1d420801d4412a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-08T06:16:02.288 INFO:journalctl@ceph.osd.4.smithi163.stdout:May 08 06:16:02 smithi163 bash[68931]: ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-4 2024-05-08T06:16:02.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:01 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:01.974+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:02.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:01 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:01.974+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:02.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:02 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:02.424+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:02.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:02 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:02.424+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:02.730 INFO:journalctl@ceph.osd.4.smithi163.stdout:May 08 06:16:02 smithi163 podman[68960]: 2024-05-08 06:16:02.387074125 +0000 UTC m=+0.675653214 container remove 6e390b3ce4b13036957bcaadf1becc0d745b3ac3ef30fc3b5c1d420801d4412a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-4, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-08T06:16:02.730 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:02 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:02.406+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:33.831967+0000 front 2024-05-08T06:15:33.832034+0000 (oldest deadline 2024-05-08T06:15:59.732138+0000) 2024-05-08T06:16:03.031 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:02 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:02.935+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:03.031 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:02 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:02.935+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:03.032 INFO:journalctl@ceph.osd.4.smithi163.stdout:May 08 06:16:02 smithi163 podman[69050]: 2024-05-08 06:16:02.633961642 +0000 UTC m=+0.017844486 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:16:03.032 INFO:journalctl@ceph.osd.4.smithi163.stdout:May 08 06:16:02 smithi163 podman[69050]: 2024-05-08 06:16:02.76158414 +0000 UTC m=+0.145467006 container create 8023de44fec39be66014632ec49b62574392aa709e5f3b4ee5272bad2c6ac862 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-08T06:16:03.032 INFO:journalctl@ceph.osd.4.smithi163.stdout:May 08 06:16:03 smithi163 podman[69050]: 2024-05-08 06:16:03.03243473 +0000 UTC m=+0.416317553 container init 8023de44fec39be66014632ec49b62574392aa709e5f3b4ee5272bad2c6ac862 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-4-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS) 2024-05-08T06:16:03.050 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:02 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:02.564+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:16:03.050 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:16:02 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:16:02.552+0000 7f46870a6700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.208042+0000 front 2024-05-08T06:15:25.208125+0000 (oldest deadline 2024-05-08T06:15:49.907884+0000) 2024-05-08T06:16:03.051 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:16:02 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:16:02.552+0000 7f46870a6700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.608958+0000 front 2024-05-08T06:15:34.609053+0000 (oldest deadline 2024-05-08T06:15:56.308959+0000) 2024-05-08T06:16:03.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:02 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:02.778+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:03.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:02 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:02.778+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:03.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:02 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:02.859+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.840183+0000 front 2024-05-08T06:15:25.840948+0000 (oldest deadline 2024-05-08T06:15:48.740113+0000) 2024-05-08T06:16:03.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:02 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:02.859+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.643693+0000 front 2024-05-08T06:15:35.643815+0000 (oldest deadline 2024-05-08T06:16:01.543986+0000) 2024-05-08T06:16:03.288 INFO:journalctl@ceph.osd.4.smithi163.stdout:May 08 06:16:03 smithi163 podman[69050]: 2024-05-08 06:16:03.035605059 +0000 UTC m=+0.419487884 container start 8023de44fec39be66014632ec49b62574392aa709e5f3b4ee5272bad2c6ac862 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True) 2024-05-08T06:16:03.288 INFO:journalctl@ceph.osd.4.smithi163.stdout:May 08 06:16:03 smithi163 podman[69050]: 2024-05-08 06:16:03.107388181 +0000 UTC m=+0.491271020 container attach 8023de44fec39be66014632ec49b62574392aa709e5f3b4ee5272bad2c6ac862 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-08T06:16:03.609 INFO:journalctl@ceph.osd.4.smithi163.stdout:May 08 06:16:03 smithi163 podman[69050]: 2024-05-08 06:16:03.335576557 +0000 UTC m=+0.719459394 container died 8023de44fec39be66014632ec49b62574392aa709e5f3b4ee5272bad2c6ac862 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T06:16:03.609 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:03 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:03.378+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:33.831967+0000 front 2024-05-08T06:15:33.832034+0000 (oldest deadline 2024-05-08T06:15:59.732138+0000) 2024-05-08T06:16:03.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:03 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:03.416+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:03.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:03 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:03.416+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:03.781 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:03 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:03.581+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:16:03.781 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:16:03 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:16:03.520+0000 7f46870a6700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.208042+0000 front 2024-05-08T06:15:25.208125+0000 (oldest deadline 2024-05-08T06:15:49.907884+0000) 2024-05-08T06:16:03.782 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:16:03 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:16:03.520+0000 7f46870a6700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.608958+0000 front 2024-05-08T06:15:34.609053+0000 (oldest deadline 2024-05-08T06:15:56.308959+0000) 2024-05-08T06:16:03.836 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@osd.4.service' 2024-05-08T06:16:03.861 INFO:journalctl@ceph.osd.4.smithi163.stdout:May 08 06:16:03 smithi163 podman[69076]: 2024-05-08 06:16:03.819176443 +0000 UTC m=+0.478397215 container remove 8023de44fec39be66014632ec49b62574392aa709e5f3b4ee5272bad2c6ac862 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-08T06:16:03.862 INFO:journalctl@ceph.osd.4.smithi163.stdout:May 08 06:16:03 smithi163 systemd[1]: ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@osd.4.service: Deactivated successfully. 2024-05-08T06:16:03.862 INFO:journalctl@ceph.osd.4.smithi163.stdout:May 08 06:16:03 smithi163 systemd[1]: Stopped Ceph osd.4 for 85cfb2e6-0cff-11ef-bc97-c7b262605968. 2024-05-08T06:16:03.862 INFO:journalctl@ceph.osd.4.smithi163.stdout:May 08 06:16:03 smithi163 systemd[1]: ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@osd.4.service: Consumed 9.747s CPU time. 2024-05-08T06:16:03.874 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:16:03.874 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-08T06:16:03.874 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-08T06:16:03.874 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@osd.5 2024-05-08T06:16:04.050 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:16:03 smithi192 systemd[1]: Stopping Ceph osd.5 for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T06:16:04.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:03 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:03.781+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:04.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:03 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:03.781+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:04.134 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:03 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:03.881+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.840183+0000 front 2024-05-08T06:15:25.840948+0000 (oldest deadline 2024-05-08T06:15:48.740113+0000) 2024-05-08T06:16:04.134 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:03 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:03.881+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.643693+0000 front 2024-05-08T06:15:35.643815+0000 (oldest deadline 2024-05-08T06:16:01.543986+0000) 2024-05-08T06:16:04.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:03 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:03.968+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:04.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:03 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:03.968+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:04.523 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:16:04 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:16:04.184+0000 7f468aab2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:16:04.523 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:16:04 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:16:04.184+0000 7f468aab2700 -1 osd.5 85 *** Got signal Terminated *** 2024-05-08T06:16:04.523 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:16:04 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:16:04.184+0000 7f468aab2700 -1 osd.5 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T06:16:04.523 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:16:04 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:16:04.523+0000 7f46870a6700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.208042+0000 front 2024-05-08T06:15:25.208125+0000 (oldest deadline 2024-05-08T06:15:49.907884+0000) 2024-05-08T06:16:04.524 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:16:04 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:16:04.523+0000 7f46870a6700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.608958+0000 front 2024-05-08T06:15:34.609053+0000 (oldest deadline 2024-05-08T06:15:56.308959+0000) 2024-05-08T06:16:04.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:04 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:04.388+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:04.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:04 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:04.388+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:04.775 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:04 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:04.603+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:16:04.788 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:04 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:04.332+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:33.831967+0000 front 2024-05-08T06:15:33.832034+0000 (oldest deadline 2024-05-08T06:15:59.732138+0000) 2024-05-08T06:16:05.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:04 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:04.775+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:05.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:04 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:04.775+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:05.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:04 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:04.775+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:05.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:04 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:04.874+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.840183+0000 front 2024-05-08T06:15:25.840948+0000 (oldest deadline 2024-05-08T06:15:48.740113+0000) 2024-05-08T06:16:05.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:04 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:04.874+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.643693+0000 front 2024-05-08T06:15:35.643815+0000 (oldest deadline 2024-05-08T06:16:01.543986+0000) 2024-05-08T06:16:05.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:04 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:04.928+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:05.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:04 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:04.928+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:05.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:05 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:05.436+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:05.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:05 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:05.436+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:05.734 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:05 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:05.557+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:16:05.734 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:16:05 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:16:05.482+0000 7f46870a6700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.208042+0000 front 2024-05-08T06:15:25.208125+0000 (oldest deadline 2024-05-08T06:15:49.907884+0000) 2024-05-08T06:16:05.734 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:16:05 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:16:05.482+0000 7f46870a6700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.608958+0000 front 2024-05-08T06:15:34.609053+0000 (oldest deadline 2024-05-08T06:15:56.308959+0000) 2024-05-08T06:16:05.734 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:16:05 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:16:05.482+0000 7f46870a6700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:40.910289+0000 front 2024-05-08T06:15:40.910238+0000 (oldest deadline 2024-05-08T06:16:05.010337+0000) 2024-05-08T06:16:05.788 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:05 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:05.298+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:33.831967+0000 front 2024-05-08T06:15:33.832034+0000 (oldest deadline 2024-05-08T06:15:59.732138+0000) 2024-05-08T06:16:06.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:05 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:05.734+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:06.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:05 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:05.734+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:06.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:05 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:05.734+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:06.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:05 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:05.885+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.840183+0000 front 2024-05-08T06:15:25.840948+0000 (oldest deadline 2024-05-08T06:15:48.740113+0000) 2024-05-08T06:16:06.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:05 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:05.885+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.643693+0000 front 2024-05-08T06:15:35.643815+0000 (oldest deadline 2024-05-08T06:16:01.543986+0000) 2024-05-08T06:16:06.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:05 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:05.885+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:41.544986+0000 front 2024-05-08T06:15:41.544809+0000 (oldest deadline 2024-05-08T06:16:05.645045+0000) 2024-05-08T06:16:06.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:05 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:05.960+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:06.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:05 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:05.960+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:06.765 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:06 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:06.537+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:16:06.765 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:16:06 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:16:06.511+0000 7f46870a6700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.208042+0000 front 2024-05-08T06:15:25.208125+0000 (oldest deadline 2024-05-08T06:15:49.907884+0000) 2024-05-08T06:16:06.765 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:16:06 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:16:06.511+0000 7f46870a6700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.608958+0000 front 2024-05-08T06:15:34.609053+0000 (oldest deadline 2024-05-08T06:15:56.308959+0000) 2024-05-08T06:16:06.765 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:16:06 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:16:06.511+0000 7f46870a6700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:40.910289+0000 front 2024-05-08T06:15:40.910238+0000 (oldest deadline 2024-05-08T06:16:05.010337+0000) 2024-05-08T06:16:06.788 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:06 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:06.316+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:33.831967+0000 front 2024-05-08T06:15:33.832034+0000 (oldest deadline 2024-05-08T06:15:59.732138+0000) 2024-05-08T06:16:06.788 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:06 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:06.316+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:39.732695+0000 front 2024-05-08T06:15:39.732814+0000 (oldest deadline 2024-05-08T06:16:05.632727+0000) 2024-05-08T06:16:06.869 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:06 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:06.480+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:06.869 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:06 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:06.480+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:07.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:06 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:06.765+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:07.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:06 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:06.765+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:07.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:06 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:06.765+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:07.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:06 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:06.868+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.840183+0000 front 2024-05-08T06:15:25.840948+0000 (oldest deadline 2024-05-08T06:15:48.740113+0000) 2024-05-08T06:16:07.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:06 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:06.868+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.643693+0000 front 2024-05-08T06:15:35.643815+0000 (oldest deadline 2024-05-08T06:16:01.543986+0000) 2024-05-08T06:16:07.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:06 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:06.868+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:41.544986+0000 front 2024-05-08T06:15:41.544809+0000 (oldest deadline 2024-05-08T06:16:05.645045+0000) 2024-05-08T06:16:07.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:06 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:06.961+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:07.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:06 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:06.961+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:07.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:07 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:07.436+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:07.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:07 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:07.436+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:07.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:07 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:07.436+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.109835+0000 front 2024-05-08T06:15:42.109956+0000 (oldest deadline 2024-05-08T06:16:06.809996+0000) 2024-05-08T06:16:07.783 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:07 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:07.495+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:16:07.784 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:16:07 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:16:07.557+0000 7f46870a6700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.208042+0000 front 2024-05-08T06:15:25.208125+0000 (oldest deadline 2024-05-08T06:15:49.907884+0000) 2024-05-08T06:16:07.784 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:16:07 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:16:07.557+0000 7f46870a6700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.608958+0000 front 2024-05-08T06:15:34.609053+0000 (oldest deadline 2024-05-08T06:15:56.308959+0000) 2024-05-08T06:16:07.784 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:16:07 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:16:07.557+0000 7f46870a6700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:40.910289+0000 front 2024-05-08T06:15:40.910238+0000 (oldest deadline 2024-05-08T06:16:05.010337+0000) 2024-05-08T06:16:07.788 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:07 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:07.297+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:33.831967+0000 front 2024-05-08T06:15:33.832034+0000 (oldest deadline 2024-05-08T06:15:59.732138+0000) 2024-05-08T06:16:07.788 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:07 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:07.297+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:39.732695+0000 front 2024-05-08T06:15:39.732814+0000 (oldest deadline 2024-05-08T06:16:05.632727+0000) 2024-05-08T06:16:08.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:07 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:07.784+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:08.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:07 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:07.784+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:08.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:07 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:07.784+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:08.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:07 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:07.895+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.840183+0000 front 2024-05-08T06:15:25.840948+0000 (oldest deadline 2024-05-08T06:15:48.740113+0000) 2024-05-08T06:16:08.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:07 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:07.895+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.643693+0000 front 2024-05-08T06:15:35.643815+0000 (oldest deadline 2024-05-08T06:16:01.543986+0000) 2024-05-08T06:16:08.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:07 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:07.895+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:41.544986+0000 front 2024-05-08T06:15:41.544809+0000 (oldest deadline 2024-05-08T06:16:05.645045+0000) 2024-05-08T06:16:08.276 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:08 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:08.000+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:08.276 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:08 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:08.000+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:08.276 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:08 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:08.000+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:08.538 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:08 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:08.276+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:33.831967+0000 front 2024-05-08T06:15:33.832034+0000 (oldest deadline 2024-05-08T06:15:59.732138+0000) 2024-05-08T06:16:08.538 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:08 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:08.276+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:39.732695+0000 front 2024-05-08T06:15:39.732814+0000 (oldest deadline 2024-05-08T06:16:05.632727+0000) 2024-05-08T06:16:08.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:08 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:08.450+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:08.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:08 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:08.450+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:08.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:08 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:08.450+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.109835+0000 front 2024-05-08T06:15:42.109956+0000 (oldest deadline 2024-05-08T06:16:06.809996+0000) 2024-05-08T06:16:08.800 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:08 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:08.540+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:16:08.800 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:08 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:08.540+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.363256+0000 front 2024-05-08T06:15:42.363239+0000 (oldest deadline 2024-05-08T06:16:07.663434+0000) 2024-05-08T06:16:08.800 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:16:08 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:16:08.556+0000 7f46870a6700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.208042+0000 front 2024-05-08T06:15:25.208125+0000 (oldest deadline 2024-05-08T06:15:49.907884+0000) 2024-05-08T06:16:08.801 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:16:08 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:16:08.556+0000 7f46870a6700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.608958+0000 front 2024-05-08T06:15:34.609053+0000 (oldest deadline 2024-05-08T06:15:56.308959+0000) 2024-05-08T06:16:08.801 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:16:08 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5[46852]: 2024-05-08T06:16:08.556+0000 7f46870a6700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:40.910289+0000 front 2024-05-08T06:15:40.910238+0000 (oldest deadline 2024-05-08T06:16:05.010337+0000) 2024-05-08T06:16:08.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:08 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:08.748+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:08.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:08 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:08.748+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:08.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:08 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:08.748+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:09.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:08 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:08.930+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.840183+0000 front 2024-05-08T06:15:25.840948+0000 (oldest deadline 2024-05-08T06:15:48.740113+0000) 2024-05-08T06:16:09.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:08 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:08.930+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.643693+0000 front 2024-05-08T06:15:35.643815+0000 (oldest deadline 2024-05-08T06:16:01.543986+0000) 2024-05-08T06:16:09.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:08 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:08.930+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:41.544986+0000 front 2024-05-08T06:15:41.544809+0000 (oldest deadline 2024-05-08T06:16:05.645045+0000) 2024-05-08T06:16:09.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:09 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:08.999+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:09.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:09 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:08.999+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:09.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:09 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:08.999+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:09.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:09 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:09.415+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:09.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:09 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:09.415+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:09.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:09 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:09.415+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.109835+0000 front 2024-05-08T06:15:42.109956+0000 (oldest deadline 2024-05-08T06:16:06.809996+0000) 2024-05-08T06:16:09.777 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:16:09 smithi192 podman[69647]: 2024-05-08 06:16:09.499320352 +0000 UTC m=+5.431235579 container died 8b3bd3b46b9859e82b64d8f7b8a0465aeb2d9951d79564620dd13b06543d6a6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, ceph=True) 2024-05-08T06:16:09.777 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:09 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:09.503+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:16:09.777 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:09 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:09.503+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.363256+0000 front 2024-05-08T06:15:42.363239+0000 (oldest deadline 2024-05-08T06:16:07.663434+0000) 2024-05-08T06:16:09.777 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:09 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:09.718+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:09.777 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:09 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:09.718+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:09.777 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:09 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:09.718+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:09.788 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:09 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:09.296+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:33.831967+0000 front 2024-05-08T06:15:33.832034+0000 (oldest deadline 2024-05-08T06:15:59.732138+0000) 2024-05-08T06:16:09.788 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:09 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:09.297+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:39.732695+0000 front 2024-05-08T06:15:39.732814+0000 (oldest deadline 2024-05-08T06:16:05.632727+0000) 2024-05-08T06:16:10.050 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:16:09 smithi192 podman[69647]: 2024-05-08 06:16:09.877007139 +0000 UTC m=+5.808922362 container cleanup 8b3bd3b46b9859e82b64d8f7b8a0465aeb2d9951d79564620dd13b06543d6a6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5) 2024-05-08T06:16:10.050 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:16:09 smithi192 bash[69647]: ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5 2024-05-08T06:16:10.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:09 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:09.907+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.840183+0000 front 2024-05-08T06:15:25.840948+0000 (oldest deadline 2024-05-08T06:15:48.740113+0000) 2024-05-08T06:16:10.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:09 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:09.907+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.643693+0000 front 2024-05-08T06:15:35.643815+0000 (oldest deadline 2024-05-08T06:16:01.543986+0000) 2024-05-08T06:16:10.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:09 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:09.907+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:41.544986+0000 front 2024-05-08T06:15:41.544809+0000 (oldest deadline 2024-05-08T06:16:05.645045+0000) 2024-05-08T06:16:10.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:10 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:10.018+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:10.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:10 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:10.018+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:10.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:10 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:10.018+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:10.498 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:16:10 smithi192 podman[69663]: 2024-05-08 06:16:10.16185336 +0000 UTC m=+0.659537979 container remove 8b3bd3b46b9859e82b64d8f7b8a0465aeb2d9951d79564620dd13b06543d6a6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507) 2024-05-08T06:16:10.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:10 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:10.430+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:10.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:10 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:10.430+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:10.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:10 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:10.430+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.109835+0000 front 2024-05-08T06:15:42.109956+0000 (oldest deadline 2024-05-08T06:16:06.809996+0000) 2024-05-08T06:16:10.755 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:10 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:10.687+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:10.755 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:10 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:10.687+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:10.755 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:10 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:10.687+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:10.756 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:16:10 smithi192 podman[69753]: 2024-05-08 06:16:10.401387369 +0000 UTC m=+0.018565532 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:16:10.756 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:16:10 smithi192 podman[69753]: 2024-05-08 06:16:10.512692685 +0000 UTC m=+0.129870833 container create 80dfb7a41d15e2e5e752f17a7b5eda050ee7910edf1ee9782d28ab76ea4898ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T06:16:10.756 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:10 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:10.503+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:16:10.756 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:10 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:10.503+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.363256+0000 front 2024-05-08T06:15:42.363239+0000 (oldest deadline 2024-05-08T06:16:07.663434+0000) 2024-05-08T06:16:10.788 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:10 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:10.310+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:33.831967+0000 front 2024-05-08T06:15:33.832034+0000 (oldest deadline 2024-05-08T06:15:59.732138+0000) 2024-05-08T06:16:10.788 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:10 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:10.310+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:39.732695+0000 front 2024-05-08T06:15:39.732814+0000 (oldest deadline 2024-05-08T06:16:05.632727+0000) 2024-05-08T06:16:11.050 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:16:10 smithi192 podman[69753]: 2024-05-08 06:16:10.75661085 +0000 UTC m=+0.373788999 container init 80dfb7a41d15e2e5e752f17a7b5eda050ee7910edf1ee9782d28ab76ea4898ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f) 2024-05-08T06:16:11.050 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:16:10 smithi192 podman[69753]: 2024-05-08 06:16:10.76014834 +0000 UTC m=+0.377326494 container start 80dfb7a41d15e2e5e752f17a7b5eda050ee7910edf1ee9782d28ab76ea4898ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True) 2024-05-08T06:16:11.050 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:16:10 smithi192 podman[69753]: 2024-05-08 06:16:10.856887307 +0000 UTC m=+0.474065474 container attach 80dfb7a41d15e2e5e752f17a7b5eda050ee7910edf1ee9782d28ab76ea4898ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-08T06:16:11.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:10 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:10.918+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.840183+0000 front 2024-05-08T06:15:25.840948+0000 (oldest deadline 2024-05-08T06:15:48.740113+0000) 2024-05-08T06:16:11.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:10 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:10.918+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.643693+0000 front 2024-05-08T06:15:35.643815+0000 (oldest deadline 2024-05-08T06:16:01.543986+0000) 2024-05-08T06:16:11.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:10 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:10.918+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:41.544986+0000 front 2024-05-08T06:15:41.544809+0000 (oldest deadline 2024-05-08T06:16:05.645045+0000) 2024-05-08T06:16:11.288 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:11 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:11.268+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:33.831967+0000 front 2024-05-08T06:15:33.832034+0000 (oldest deadline 2024-05-08T06:15:59.732138+0000) 2024-05-08T06:16:11.288 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:11 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:11.268+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:39.732695+0000 front 2024-05-08T06:15:39.732814+0000 (oldest deadline 2024-05-08T06:16:05.632727+0000) 2024-05-08T06:16:11.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:11 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:11.022+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:11.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:11 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:11.022+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:11.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:11 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:11.022+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:11.350 INFO:journalctl@ceph.osd.5.smithi192.stdout:May 08 06:16:11 smithi192 podman[69753]: 2024-05-08 06:16:11.052570493 +0000 UTC m=+0.669748643 container died 80dfb7a41d15e2e5e752f17a7b5eda050ee7910edf1ee9782d28ab76ea4898ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, ceph=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-08T06:16:11.601 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:11 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:11.457+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:16:11.601 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:11 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:11.457+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.363256+0000 front 2024-05-08T06:15:42.363239+0000 (oldest deadline 2024-05-08T06:16:07.663434+0000) 2024-05-08T06:16:11.627 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@osd.5.service' 2024-05-08T06:16:11.666 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:16:11.666 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-08T06:16:11.666 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-08T06:16:11.666 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@osd.6 2024-05-08T06:16:11.741 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:11 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:11.470+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:11.741 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:11 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:11.470+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:11.741 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:11 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:11.470+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.109835+0000 front 2024-05-08T06:15:42.109956+0000 (oldest deadline 2024-05-08T06:16:06.809996+0000) 2024-05-08T06:16:12.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:11 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:11.731+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:12.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:11 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:11.731+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:12.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:11 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:11.731+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:12.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:11 smithi123 systemd[1]: Stopping Ceph osd.6 for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T06:16:12.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:11 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:11.945+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.840183+0000 front 2024-05-08T06:15:25.840948+0000 (oldest deadline 2024-05-08T06:15:48.740113+0000) 2024-05-08T06:16:12.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:11 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:11.945+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.643693+0000 front 2024-05-08T06:15:35.643815+0000 (oldest deadline 2024-05-08T06:16:01.543986+0000) 2024-05-08T06:16:12.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:11 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:11.945+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:41.544986+0000 front 2024-05-08T06:15:41.544809+0000 (oldest deadline 2024-05-08T06:16:05.645045+0000) 2024-05-08T06:16:12.288 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:12 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:12.253+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:33.831967+0000 front 2024-05-08T06:15:33.832034+0000 (oldest deadline 2024-05-08T06:15:59.732138+0000) 2024-05-08T06:16:12.288 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:12 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:12.253+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:39.732695+0000 front 2024-05-08T06:15:39.732814+0000 (oldest deadline 2024-05-08T06:16:05.632727+0000) 2024-05-08T06:16:12.289 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:12 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:12.023+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:12.289 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:12 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:12.023+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:12.289 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:12 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:12.023+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:12.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:12 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:12.445+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:12.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:12 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:12.445+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:12.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:12 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:12.445+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.109835+0000 front 2024-05-08T06:15:42.109956+0000 (oldest deadline 2024-05-08T06:16:06.809996+0000) 2024-05-08T06:16:12.706 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:12 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:12.253+0000 7fd763800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:16:12.706 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:12 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:12.253+0000 7fd763800700 -1 osd.6 85 *** Got signal Terminated *** 2024-05-08T06:16:12.706 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:12 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:12.253+0000 7fd763800700 -1 osd.6 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T06:16:12.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:12 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:12.707+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:12.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:12 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:12.707+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:12.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:12 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:12.707+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:12.800 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:12 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:12.477+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:16:12.801 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:12 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:12.477+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.363256+0000 front 2024-05-08T06:15:42.363239+0000 (oldest deadline 2024-05-08T06:16:07.663434+0000) 2024-05-08T06:16:13.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:12 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:12.940+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.840183+0000 front 2024-05-08T06:15:25.840948+0000 (oldest deadline 2024-05-08T06:15:48.740113+0000) 2024-05-08T06:16:13.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:12 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:12.940+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.643693+0000 front 2024-05-08T06:15:35.643815+0000 (oldest deadline 2024-05-08T06:16:01.543986+0000) 2024-05-08T06:16:13.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:12 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:12.940+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:41.544986+0000 front 2024-05-08T06:15:41.544809+0000 (oldest deadline 2024-05-08T06:16:05.645045+0000) 2024-05-08T06:16:13.288 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:13 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:13.247+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:33.831967+0000 front 2024-05-08T06:15:33.832034+0000 (oldest deadline 2024-05-08T06:15:59.732138+0000) 2024-05-08T06:16:13.288 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:13 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:13.247+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:39.732695+0000 front 2024-05-08T06:15:39.732814+0000 (oldest deadline 2024-05-08T06:16:05.632727+0000) 2024-05-08T06:16:13.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:13 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:13.024+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:13.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:13 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:13.024+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:13.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:13 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:13.024+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:13.743 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:13 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:13.490+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:16:13.743 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:13 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:13.490+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.363256+0000 front 2024-05-08T06:15:42.363239+0000 (oldest deadline 2024-05-08T06:16:07.663434+0000) 2024-05-08T06:16:13.954 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:13 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:13.495+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:13.955 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:13 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:13.495+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:13.955 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:13 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:13.495+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.109835+0000 front 2024-05-08T06:15:42.109956+0000 (oldest deadline 2024-05-08T06:16:06.809996+0000) 2024-05-08T06:16:13.955 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:13 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:13.495+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:49.712308+0000 front 2024-05-08T06:15:49.712030+0000 (oldest deadline 2024-05-08T06:16:13.113280+0000) 2024-05-08T06:16:14.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:13 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:13.743+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:14.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:13 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:13.743+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:14.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:13 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:13.743+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:14.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:13 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:13.954+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.840183+0000 front 2024-05-08T06:15:25.840948+0000 (oldest deadline 2024-05-08T06:15:48.740113+0000) 2024-05-08T06:16:14.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:13 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:13.954+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.643693+0000 front 2024-05-08T06:15:35.643815+0000 (oldest deadline 2024-05-08T06:16:01.543986+0000) 2024-05-08T06:16:14.205 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:13 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:13.954+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:41.544986+0000 front 2024-05-08T06:15:41.544809+0000 (oldest deadline 2024-05-08T06:16:05.645045+0000) 2024-05-08T06:16:14.288 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:14 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:14.240+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:33.831967+0000 front 2024-05-08T06:15:33.832034+0000 (oldest deadline 2024-05-08T06:15:59.732138+0000) 2024-05-08T06:16:14.288 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:14 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:14.240+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:39.732695+0000 front 2024-05-08T06:15:39.732814+0000 (oldest deadline 2024-05-08T06:16:05.632727+0000) 2024-05-08T06:16:14.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:14 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:14.025+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:14.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:14 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:14.025+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:14.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:14 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:14.025+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:14.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:14 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:14.448+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:14.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:14 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:14.448+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:14.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:14 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:14.448+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.109835+0000 front 2024-05-08T06:15:42.109956+0000 (oldest deadline 2024-05-08T06:16:06.809996+0000) 2024-05-08T06:16:14.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:14 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:14.448+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:49.712308+0000 front 2024-05-08T06:15:49.712030+0000 (oldest deadline 2024-05-08T06:16:13.113280+0000) 2024-05-08T06:16:14.708 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:14 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:14.447+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:16:14.708 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:14 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:14.447+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.363256+0000 front 2024-05-08T06:15:42.363239+0000 (oldest deadline 2024-05-08T06:16:07.663434+0000) 2024-05-08T06:16:15.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:14 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:14.708+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:15.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:14 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:14.708+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:15.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:14 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:14.708+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:15.284 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:15 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:15.027+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:15.284 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:15 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:15.027+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:15.284 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:15 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:15.027+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:15.431 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:14 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:14.997+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.840183+0000 front 2024-05-08T06:15:25.840948+0000 (oldest deadline 2024-05-08T06:15:48.740113+0000) 2024-05-08T06:16:15.431 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:14 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:14.997+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.643693+0000 front 2024-05-08T06:15:35.643815+0000 (oldest deadline 2024-05-08T06:16:01.543986+0000) 2024-05-08T06:16:15.431 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:14 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:14.997+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:41.544986+0000 front 2024-05-08T06:15:41.544809+0000 (oldest deadline 2024-05-08T06:16:05.645045+0000) 2024-05-08T06:16:15.538 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:15 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:15.283+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:33.831967+0000 front 2024-05-08T06:15:33.832034+0000 (oldest deadline 2024-05-08T06:15:59.732138+0000) 2024-05-08T06:16:15.538 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:15 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:15.283+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:39.732695+0000 front 2024-05-08T06:15:39.732814+0000 (oldest deadline 2024-05-08T06:16:05.632727+0000) 2024-05-08T06:16:15.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:15 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:15.430+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:15.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:15 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:15.430+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:15.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:15 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:15.430+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.109835+0000 front 2024-05-08T06:15:42.109956+0000 (oldest deadline 2024-05-08T06:16:06.809996+0000) 2024-05-08T06:16:15.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:15 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:15.430+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:49.712308+0000 front 2024-05-08T06:15:49.712030+0000 (oldest deadline 2024-05-08T06:16:13.113280+0000) 2024-05-08T06:16:15.755 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:15 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:15.407+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:16:15.755 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:15 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:15.407+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.363256+0000 front 2024-05-08T06:15:42.363239+0000 (oldest deadline 2024-05-08T06:16:07.663434+0000) 2024-05-08T06:16:15.755 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:15 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:15.407+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:51.065366+0000 front 2024-05-08T06:15:51.065285+0000 (oldest deadline 2024-05-08T06:16:14.565065+0000) 2024-05-08T06:16:16.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:15 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:15.755+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:16.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:15 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:15.755+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:16.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:15 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:15.755+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:16.401 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:15 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:15.966+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.840183+0000 front 2024-05-08T06:15:25.840948+0000 (oldest deadline 2024-05-08T06:15:48.740113+0000) 2024-05-08T06:16:16.402 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:15 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:15.966+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.643693+0000 front 2024-05-08T06:15:35.643815+0000 (oldest deadline 2024-05-08T06:16:01.543986+0000) 2024-05-08T06:16:16.402 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:15 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:15.966+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:41.544986+0000 front 2024-05-08T06:15:41.544809+0000 (oldest deadline 2024-05-08T06:16:05.645045+0000) 2024-05-08T06:16:16.538 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:16 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:16.294+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:33.831967+0000 front 2024-05-08T06:15:33.832034+0000 (oldest deadline 2024-05-08T06:15:59.732138+0000) 2024-05-08T06:16:16.538 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:16 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:16.294+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:39.732695+0000 front 2024-05-08T06:15:39.732814+0000 (oldest deadline 2024-05-08T06:16:05.632727+0000) 2024-05-08T06:16:16.538 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:16 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:16.294+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:50.934177+0000 front 2024-05-08T06:15:50.933889+0000 (oldest deadline 2024-05-08T06:16:15.633660+0000) 2024-05-08T06:16:16.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:16 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:16.076+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:16.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:16 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:16.076+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:16.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:16 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:16.076+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:16.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:16 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:16.076+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:47.576642+0000 front 2024-05-08T06:15:47.576333+0000 (oldest deadline 2024-05-08T06:16:15.176725+0000) 2024-05-08T06:16:16.550 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:16 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:16.384+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:16:16.550 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:16 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:16.384+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.363256+0000 front 2024-05-08T06:15:42.363239+0000 (oldest deadline 2024-05-08T06:16:07.663434+0000) 2024-05-08T06:16:16.550 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:16 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:16.384+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:51.065366+0000 front 2024-05-08T06:15:51.065285+0000 (oldest deadline 2024-05-08T06:16:14.565065+0000) 2024-05-08T06:16:16.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:16 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:16.401+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:16.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:16 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:16.401+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:16.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:16 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:16.401+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.109835+0000 front 2024-05-08T06:15:42.109956+0000 (oldest deadline 2024-05-08T06:16:06.809996+0000) 2024-05-08T06:16:16.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:16 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:16.401+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:49.712308+0000 front 2024-05-08T06:15:49.712030+0000 (oldest deadline 2024-05-08T06:16:13.113280+0000) 2024-05-08T06:16:17.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:16 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:16.765+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:17.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:16 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:16.765+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:17.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:16 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:16.765+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:17.112 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:16 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:16.963+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:25.840183+0000 front 2024-05-08T06:15:25.840948+0000 (oldest deadline 2024-05-08T06:15:48.740113+0000) 2024-05-08T06:16:17.112 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:16 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:16.963+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.643693+0000 front 2024-05-08T06:15:35.643815+0000 (oldest deadline 2024-05-08T06:16:01.543986+0000) 2024-05-08T06:16:17.112 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:16 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:16.963+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:41.544986+0000 front 2024-05-08T06:15:41.544809+0000 (oldest deadline 2024-05-08T06:16:05.645045+0000) 2024-05-08T06:16:17.112 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:16 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6[57722]: 2024-05-08T06:16:16.963+0000 7fd75a000700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:50.947001+0000 front 2024-05-08T06:15:50.947210+0000 (oldest deadline 2024-05-08T06:16:16.748265+0000) 2024-05-08T06:16:17.538 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:17 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:17.259+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:33.831967+0000 front 2024-05-08T06:15:33.832034+0000 (oldest deadline 2024-05-08T06:15:59.732138+0000) 2024-05-08T06:16:17.538 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:17 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:17.259+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:39.732695+0000 front 2024-05-08T06:15:39.732814+0000 (oldest deadline 2024-05-08T06:16:05.632727+0000) 2024-05-08T06:16:17.538 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:17 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:17.259+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:50.934177+0000 front 2024-05-08T06:15:50.933889+0000 (oldest deadline 2024-05-08T06:16:15.633660+0000) 2024-05-08T06:16:17.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:17 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:17.064+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:17.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:17 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:17.064+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:17.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:17 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:17.064+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:17.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:17 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:17.064+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:47.576642+0000 front 2024-05-08T06:15:47.576333+0000 (oldest deadline 2024-05-08T06:16:15.176725+0000) 2024-05-08T06:16:17.704 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:17 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:17.413+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:17.704 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:17 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:17.413+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:17.704 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:17 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:17.413+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.109835+0000 front 2024-05-08T06:15:42.109956+0000 (oldest deadline 2024-05-08T06:16:06.809996+0000) 2024-05-08T06:16:17.704 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:17 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:17.413+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:49.712308+0000 front 2024-05-08T06:15:49.712030+0000 (oldest deadline 2024-05-08T06:16:13.113280+0000) 2024-05-08T06:16:17.719 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:17 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:17.398+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:16:17.719 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:17 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:17.398+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.363256+0000 front 2024-05-08T06:15:42.363239+0000 (oldest deadline 2024-05-08T06:16:07.663434+0000) 2024-05-08T06:16:17.719 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:17 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:17.398+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:51.065366+0000 front 2024-05-08T06:15:51.065285+0000 (oldest deadline 2024-05-08T06:16:14.565065+0000) 2024-05-08T06:16:17.955 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:17 smithi123 podman[96217]: 2024-05-08 06:16:17.70365257 +0000 UTC m=+5.609471024 container died 2f0d57bdb785628e1f712f17d22dc2fad104096eafaa7c17e8547315268660ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-08T06:16:18.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:17 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:17.719+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:18.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:17 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:17.719+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:18.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:17 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:17.719+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:18.538 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:18 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:18.234+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:33.831967+0000 front 2024-05-08T06:15:33.832034+0000 (oldest deadline 2024-05-08T06:15:59.732138+0000) 2024-05-08T06:16:18.538 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:18 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:18.234+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:39.732695+0000 front 2024-05-08T06:15:39.732814+0000 (oldest deadline 2024-05-08T06:16:05.632727+0000) 2024-05-08T06:16:18.538 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:18 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:18.234+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:50.934177+0000 front 2024-05-08T06:15:50.933889+0000 (oldest deadline 2024-05-08T06:16:15.633660+0000) 2024-05-08T06:16:18.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:18 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:18.106+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:18.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:18 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:18.106+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:18.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:18 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:18.106+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:18.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:18 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:18.106+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:47.576642+0000 front 2024-05-08T06:15:47.576333+0000 (oldest deadline 2024-05-08T06:16:15.176725+0000) 2024-05-08T06:16:18.705 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:18 smithi123 podman[96217]: 2024-05-08 06:16:18.348835733 +0000 UTC m=+6.254654249 container cleanup 2f0d57bdb785628e1f712f17d22dc2fad104096eafaa7c17e8547315268660ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507) 2024-05-08T06:16:18.706 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:18 smithi123 bash[96217]: ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6 2024-05-08T06:16:18.706 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:18 smithi123 podman[96246]: 2024-05-08 06:16:18.582076389 +0000 UTC m=+0.873581557 container remove 2f0d57bdb785628e1f712f17d22dc2fad104096eafaa7c17e8547315268660ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-08T06:16:18.706 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:18 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:18.391+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:18.706 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:18 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:18.391+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:18.706 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:18 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:18.391+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.109835+0000 front 2024-05-08T06:15:42.109956+0000 (oldest deadline 2024-05-08T06:16:06.809996+0000) 2024-05-08T06:16:18.707 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:18 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:18.391+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:49.712308+0000 front 2024-05-08T06:15:49.712030+0000 (oldest deadline 2024-05-08T06:16:13.113280+0000) 2024-05-08T06:16:18.762 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:18 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:18.377+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:16:18.767 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:18 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:18.377+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.363256+0000 front 2024-05-08T06:15:42.363239+0000 (oldest deadline 2024-05-08T06:16:07.663434+0000) 2024-05-08T06:16:18.767 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:18 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:18.377+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:51.065366+0000 front 2024-05-08T06:15:51.065285+0000 (oldest deadline 2024-05-08T06:16:14.565065+0000) 2024-05-08T06:16:19.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:18 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:18.763+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:19.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:18 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:18.763+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:19.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:18 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:18.763+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:19.538 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:19 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:19.229+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:33.831967+0000 front 2024-05-08T06:15:33.832034+0000 (oldest deadline 2024-05-08T06:15:59.732138+0000) 2024-05-08T06:16:19.538 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:19 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:19.229+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:39.732695+0000 front 2024-05-08T06:15:39.732814+0000 (oldest deadline 2024-05-08T06:16:05.632727+0000) 2024-05-08T06:16:19.538 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:19 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:19.229+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:50.934177+0000 front 2024-05-08T06:15:50.933889+0000 (oldest deadline 2024-05-08T06:16:15.633660+0000) 2024-05-08T06:16:19.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:19 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:19.093+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:19.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:19 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:19.093+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:19.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:19 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:19.093+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:19.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:19 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:19.093+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:47.576642+0000 front 2024-05-08T06:15:47.576333+0000 (oldest deadline 2024-05-08T06:16:15.176725+0000) 2024-05-08T06:16:19.688 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:19 smithi123 podman[96334]: 2024-05-08 06:16:19.346894107 +0000 UTC m=+0.032756884 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:16:19.688 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:19 smithi123 podman[96334]: 2024-05-08 06:16:19.466358417 +0000 UTC m=+0.152221228 container create f866e5c1e12cf143e4686407f3d7dbb96c4814c31922c1777b54c02317b9ea62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f) 2024-05-08T06:16:19.688 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:19 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:19.371+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:19.689 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:19 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:19.371+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:19.689 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:19 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:19.371+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.109835+0000 front 2024-05-08T06:15:42.109956+0000 (oldest deadline 2024-05-08T06:16:06.809996+0000) 2024-05-08T06:16:19.689 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:19 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:19.371+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:49.712308+0000 front 2024-05-08T06:15:49.712030+0000 (oldest deadline 2024-05-08T06:16:13.113280+0000) 2024-05-08T06:16:19.773 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:19 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:19.392+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:16:19.773 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:19 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:19.392+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.363256+0000 front 2024-05-08T06:15:42.363239+0000 (oldest deadline 2024-05-08T06:16:07.663434+0000) 2024-05-08T06:16:19.773 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:19 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:19.392+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:51.065366+0000 front 2024-05-08T06:15:51.065285+0000 (oldest deadline 2024-05-08T06:16:14.565065+0000) 2024-05-08T06:16:19.955 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:19 smithi123 podman[96334]: 2024-05-08 06:16:19.834903419 +0000 UTC m=+0.520766240 container init f866e5c1e12cf143e4686407f3d7dbb96c4814c31922c1777b54c02317b9ea62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T06:16:19.955 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:19 smithi123 podman[96334]: 2024-05-08 06:16:19.845259941 +0000 UTC m=+0.531122747 container start f866e5c1e12cf143e4686407f3d7dbb96c4814c31922c1777b54c02317b9ea62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-08T06:16:19.955 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:19 smithi123 podman[96334]: 2024-05-08 06:16:19.935386401 +0000 UTC m=+0.621249226 container attach f866e5c1e12cf143e4686407f3d7dbb96c4814c31922c1777b54c02317b9ea62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6-deactivate, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-08T06:16:20.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:19 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:19.773+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:20.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:19 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:19.773+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:20.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:19 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:19.773+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:20.538 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:20 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:20.234+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:33.831967+0000 front 2024-05-08T06:15:33.832034+0000 (oldest deadline 2024-05-08T06:15:59.732138+0000) 2024-05-08T06:16:20.538 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:20 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:20.234+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:39.732695+0000 front 2024-05-08T06:15:39.732814+0000 (oldest deadline 2024-05-08T06:16:05.632727+0000) 2024-05-08T06:16:20.538 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:20 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:20.234+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:50.934177+0000 front 2024-05-08T06:15:50.933889+0000 (oldest deadline 2024-05-08T06:16:15.633660+0000) 2024-05-08T06:16:20.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:20 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:20.059+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:20.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:20 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:20.059+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:20.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:20 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:20.059+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:20.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:20 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:20.059+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:47.576642+0000 front 2024-05-08T06:15:47.576333+0000 (oldest deadline 2024-05-08T06:16:15.176725+0000) 2024-05-08T06:16:20.661 INFO:journalctl@ceph.osd.6.smithi123.stdout:May 08 06:16:20 smithi123 podman[96334]: 2024-05-08 06:16:20.296617281 +0000 UTC m=+0.982480099 container died f866e5c1e12cf143e4686407f3d7dbb96c4814c31922c1777b54c02317b9ea62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-08T06:16:20.661 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:20 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:20.397+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:20.661 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:20 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:20.397+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:20.662 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:20 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:20.397+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.109835+0000 front 2024-05-08T06:15:42.109956+0000 (oldest deadline 2024-05-08T06:16:06.809996+0000) 2024-05-08T06:16:20.662 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:20 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:20.397+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:49.712308+0000 front 2024-05-08T06:15:49.712030+0000 (oldest deadline 2024-05-08T06:16:13.113280+0000) 2024-05-08T06:16:20.756 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:20 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:20.368+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:16:20.756 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:20 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:20.368+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.363256+0000 front 2024-05-08T06:15:42.363239+0000 (oldest deadline 2024-05-08T06:16:07.663434+0000) 2024-05-08T06:16:20.756 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:20 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:20.368+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:51.065366+0000 front 2024-05-08T06:15:51.065285+0000 (oldest deadline 2024-05-08T06:16:14.565065+0000) 2024-05-08T06:16:21.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:20 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:20.756+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:21.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:20 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:20.756+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:21.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:20 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:20.756+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:21.254 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@osd.6.service' 2024-05-08T06:16:21.369 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:16:21.369 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-08T06:16:21.369 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-08T06:16:21.369 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@osd.7 2024-05-08T06:16:21.396 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:21 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:21.280+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:33.831967+0000 front 2024-05-08T06:15:33.832034+0000 (oldest deadline 2024-05-08T06:15:59.732138+0000) 2024-05-08T06:16:21.396 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:21 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:21.280+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:39.732695+0000 front 2024-05-08T06:15:39.732814+0000 (oldest deadline 2024-05-08T06:16:05.632727+0000) 2024-05-08T06:16:21.396 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:21 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:21.280+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:50.934177+0000 front 2024-05-08T06:15:50.933889+0000 (oldest deadline 2024-05-08T06:16:15.633660+0000) 2024-05-08T06:16:21.396 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:21 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:21.051+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:21.396 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:21 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:21.051+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:21.396 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:21 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:21.051+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:21.396 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:21 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:21.051+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:47.576642+0000 front 2024-05-08T06:15:47.576333+0000 (oldest deadline 2024-05-08T06:16:15.176725+0000) 2024-05-08T06:16:21.670 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:21 smithi163 systemd[1]: Stopping Ceph osd.7 for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T06:16:21.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:21 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:21.381+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:21.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:21 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:21.381+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:21.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:21 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:21.381+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.109835+0000 front 2024-05-08T06:15:42.109956+0000 (oldest deadline 2024-05-08T06:16:06.809996+0000) 2024-05-08T06:16:21.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:21 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:21.381+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:49.712308+0000 front 2024-05-08T06:15:49.712030+0000 (oldest deadline 2024-05-08T06:16:13.113280+0000) 2024-05-08T06:16:21.708 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:21 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:21.375+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:16:21.708 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:21 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:21.375+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.363256+0000 front 2024-05-08T06:15:42.363239+0000 (oldest deadline 2024-05-08T06:16:07.663434+0000) 2024-05-08T06:16:21.708 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:21 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:21.375+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:51.065366+0000 front 2024-05-08T06:15:51.065285+0000 (oldest deadline 2024-05-08T06:16:14.565065+0000) 2024-05-08T06:16:22.015 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:21 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:21.670+0000 7f73e67b6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:16:22.015 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:21 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:21.670+0000 7f73e67b6700 -1 osd.7 85 *** Got signal Terminated *** 2024-05-08T06:16:22.015 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:21 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:21.670+0000 7f73e67b6700 -1 osd.7 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T06:16:22.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:21 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:21.709+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:22.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:21 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:21.709+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:22.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:21 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:21.709+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:22.288 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:22 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:22.244+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:33.831967+0000 front 2024-05-08T06:15:33.832034+0000 (oldest deadline 2024-05-08T06:15:59.732138+0000) 2024-05-08T06:16:22.288 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:22 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:22.244+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:39.732695+0000 front 2024-05-08T06:15:39.732814+0000 (oldest deadline 2024-05-08T06:16:05.632727+0000) 2024-05-08T06:16:22.288 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:22 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:22.244+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:50.934177+0000 front 2024-05-08T06:15:50.933889+0000 (oldest deadline 2024-05-08T06:16:15.633660+0000) 2024-05-08T06:16:22.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:22 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:22.015+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:22.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:22 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:22.015+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:22.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:22 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:22.015+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:22.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:22 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:22.015+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:47.576642+0000 front 2024-05-08T06:15:47.576333+0000 (oldest deadline 2024-05-08T06:16:15.176725+0000) 2024-05-08T06:16:22.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:22 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:22.385+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:22.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:22 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:22.385+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:22.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:22 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:22.385+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.109835+0000 front 2024-05-08T06:15:42.109956+0000 (oldest deadline 2024-05-08T06:16:06.809996+0000) 2024-05-08T06:16:22.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:22 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:22.385+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:49.712308+0000 front 2024-05-08T06:15:49.712030+0000 (oldest deadline 2024-05-08T06:16:13.113280+0000) 2024-05-08T06:16:22.757 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:22 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:22.409+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:16:22.758 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:22 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:22.409+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.363256+0000 front 2024-05-08T06:15:42.363239+0000 (oldest deadline 2024-05-08T06:16:07.663434+0000) 2024-05-08T06:16:22.758 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:22 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:22.409+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:51.065366+0000 front 2024-05-08T06:15:51.065285+0000 (oldest deadline 2024-05-08T06:16:14.565065+0000) 2024-05-08T06:16:23.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:22 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:22.757+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:23.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:22 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:22.757+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:23.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:22 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:22.757+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:23.288 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:23 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:23.209+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:33.831967+0000 front 2024-05-08T06:15:33.832034+0000 (oldest deadline 2024-05-08T06:15:59.732138+0000) 2024-05-08T06:16:23.288 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:23 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:23.209+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:39.732695+0000 front 2024-05-08T06:15:39.732814+0000 (oldest deadline 2024-05-08T06:16:05.632727+0000) 2024-05-08T06:16:23.288 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:23 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:23.209+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:50.934177+0000 front 2024-05-08T06:15:50.933889+0000 (oldest deadline 2024-05-08T06:16:15.633660+0000) 2024-05-08T06:16:23.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:22 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:22.993+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:23.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:22 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:22.993+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:23.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:22 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:22.993+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:23.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:22 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:22.993+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:47.576642+0000 front 2024-05-08T06:15:47.576333+0000 (oldest deadline 2024-05-08T06:16:15.176725+0000) 2024-05-08T06:16:23.289 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:22 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:22.993+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.677497+0000 front 2024-05-08T06:15:58.677473+0000 (oldest deadline 2024-05-08T06:16:22.177585+0000) 2024-05-08T06:16:23.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:23 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:23.408+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:23.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:23 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:23.408+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:23.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:23 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:23.408+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.109835+0000 front 2024-05-08T06:15:42.109956+0000 (oldest deadline 2024-05-08T06:16:06.809996+0000) 2024-05-08T06:16:23.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:23 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:23.408+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:49.712308+0000 front 2024-05-08T06:15:49.712030+0000 (oldest deadline 2024-05-08T06:16:13.113280+0000) 2024-05-08T06:16:23.708 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:23 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:23.432+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:16:23.708 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:23 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:23.432+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.363256+0000 front 2024-05-08T06:15:42.363239+0000 (oldest deadline 2024-05-08T06:16:07.663434+0000) 2024-05-08T06:16:23.708 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:23 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:23.432+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:51.065366+0000 front 2024-05-08T06:15:51.065285+0000 (oldest deadline 2024-05-08T06:16:14.565065+0000) 2024-05-08T06:16:23.708 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:23 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:23.432+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.566476+0000 front 2024-05-08T06:15:58.566601+0000 (oldest deadline 2024-05-08T06:16:22.666446+0000) 2024-05-08T06:16:24.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:23 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:23.708+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:24.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:23 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:23.708+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:24.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:23 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:23.708+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:24.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:23 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:23.708+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:16:00.051637+0000 front 2024-05-08T06:16:00.051514+0000 (oldest deadline 2024-05-08T06:16:22.951535+0000) 2024-05-08T06:16:24.288 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:24 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:24.213+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:33.831967+0000 front 2024-05-08T06:15:33.832034+0000 (oldest deadline 2024-05-08T06:15:59.732138+0000) 2024-05-08T06:16:24.288 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:24 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:24.213+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:39.732695+0000 front 2024-05-08T06:15:39.732814+0000 (oldest deadline 2024-05-08T06:16:05.632727+0000) 2024-05-08T06:16:24.288 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:24 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:24.213+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:50.934177+0000 front 2024-05-08T06:15:50.933889+0000 (oldest deadline 2024-05-08T06:16:15.633660+0000) 2024-05-08T06:16:24.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:24 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:24.003+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:24.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:24 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:24.003+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:24.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:24 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:24.003+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:24.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:24 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:24.003+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:47.576642+0000 front 2024-05-08T06:15:47.576333+0000 (oldest deadline 2024-05-08T06:16:15.176725+0000) 2024-05-08T06:16:24.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:24 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:24.003+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.677497+0000 front 2024-05-08T06:15:58.677473+0000 (oldest deadline 2024-05-08T06:16:22.177585+0000) 2024-05-08T06:16:24.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:24 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:24.453+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:24.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:24 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:24.453+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:24.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:24 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:24.453+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.109835+0000 front 2024-05-08T06:15:42.109956+0000 (oldest deadline 2024-05-08T06:16:06.809996+0000) 2024-05-08T06:16:24.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:24 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:24.453+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:49.712308+0000 front 2024-05-08T06:15:49.712030+0000 (oldest deadline 2024-05-08T06:16:13.113280+0000) 2024-05-08T06:16:24.706 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:24 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:24.453+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.817347+0000 front 2024-05-08T06:15:58.817382+0000 (oldest deadline 2024-05-08T06:16:24.117220+0000) 2024-05-08T06:16:24.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:24 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:24.672+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:24.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:24 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:24.672+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:24.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:24 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:24.672+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:24.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:24 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:24.672+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:16:00.051637+0000 front 2024-05-08T06:16:00.051514+0000 (oldest deadline 2024-05-08T06:16:22.951535+0000) 2024-05-08T06:16:24.800 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:24 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:24.444+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:16:24.801 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:24 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:24.444+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.363256+0000 front 2024-05-08T06:15:42.363239+0000 (oldest deadline 2024-05-08T06:16:07.663434+0000) 2024-05-08T06:16:24.801 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:24 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:24.444+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:51.065366+0000 front 2024-05-08T06:15:51.065285+0000 (oldest deadline 2024-05-08T06:16:14.565065+0000) 2024-05-08T06:16:24.801 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:24 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:24.444+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.566476+0000 front 2024-05-08T06:15:58.566601+0000 (oldest deadline 2024-05-08T06:16:22.666446+0000) 2024-05-08T06:16:25.288 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:25 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:25.210+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:33.831967+0000 front 2024-05-08T06:15:33.832034+0000 (oldest deadline 2024-05-08T06:15:59.732138+0000) 2024-05-08T06:16:25.288 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:25 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:25.210+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:39.732695+0000 front 2024-05-08T06:15:39.732814+0000 (oldest deadline 2024-05-08T06:16:05.632727+0000) 2024-05-08T06:16:25.288 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:25 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:25.210+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:50.934177+0000 front 2024-05-08T06:15:50.933889+0000 (oldest deadline 2024-05-08T06:16:15.633660+0000) 2024-05-08T06:16:25.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:25 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:25.035+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:25.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:25 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:25.035+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:25.289 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:25 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:25.035+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:25.289 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:25 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:25.035+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:47.576642+0000 front 2024-05-08T06:15:47.576333+0000 (oldest deadline 2024-05-08T06:16:15.176725+0000) 2024-05-08T06:16:25.289 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:25 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:25.035+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.677497+0000 front 2024-05-08T06:15:58.677473+0000 (oldest deadline 2024-05-08T06:16:22.177585+0000) 2024-05-08T06:16:25.752 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:25 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:25.470+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:16:25.752 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:25 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:25.470+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.363256+0000 front 2024-05-08T06:15:42.363239+0000 (oldest deadline 2024-05-08T06:16:07.663434+0000) 2024-05-08T06:16:25.753 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:25 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:25.470+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:51.065366+0000 front 2024-05-08T06:15:51.065285+0000 (oldest deadline 2024-05-08T06:16:14.565065+0000) 2024-05-08T06:16:25.753 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:25 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:25.470+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.566476+0000 front 2024-05-08T06:15:58.566601+0000 (oldest deadline 2024-05-08T06:16:22.666446+0000) 2024-05-08T06:16:25.753 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:25 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:25.649+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:25.753 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:25 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:25.649+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:25.753 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:25 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:25.649+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:25.753 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:25 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:25.649+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:16:00.051637+0000 front 2024-05-08T06:16:00.051514+0000 (oldest deadline 2024-05-08T06:16:22.951535+0000) 2024-05-08T06:16:25.955 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:25 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:25.481+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:25.955 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:25 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:25.481+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:25.955 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:25 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:25.481+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.109835+0000 front 2024-05-08T06:15:42.109956+0000 (oldest deadline 2024-05-08T06:16:06.809996+0000) 2024-05-08T06:16:25.955 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:25 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:25.481+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:49.712308+0000 front 2024-05-08T06:15:49.712030+0000 (oldest deadline 2024-05-08T06:16:13.113280+0000) 2024-05-08T06:16:25.956 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:25 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:25.481+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.817347+0000 front 2024-05-08T06:15:58.817382+0000 (oldest deadline 2024-05-08T06:16:24.117220+0000) 2024-05-08T06:16:26.289 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:26 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:26.203+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:33.831967+0000 front 2024-05-08T06:15:33.832034+0000 (oldest deadline 2024-05-08T06:15:59.732138+0000) 2024-05-08T06:16:26.289 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:26 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:26.203+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:39.732695+0000 front 2024-05-08T06:15:39.732814+0000 (oldest deadline 2024-05-08T06:16:05.632727+0000) 2024-05-08T06:16:26.290 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:26 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:26.203+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:50.934177+0000 front 2024-05-08T06:15:50.933889+0000 (oldest deadline 2024-05-08T06:16:15.633660+0000) 2024-05-08T06:16:26.290 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:26 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7[52604]: 2024-05-08T06:16:26.203+0000 7f73e2daa700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:16:00.735672+0000 front 2024-05-08T06:16:00.735732+0000 (oldest deadline 2024-05-08T06:16:26.035804+0000) 2024-05-08T06:16:26.290 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:25 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:25.998+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:26.290 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:25 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:25.998+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:26.290 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:25 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:25.998+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:26.290 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:25 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:25.998+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:47.576642+0000 front 2024-05-08T06:15:47.576333+0000 (oldest deadline 2024-05-08T06:16:15.176725+0000) 2024-05-08T06:16:26.290 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:25 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:25.998+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.677497+0000 front 2024-05-08T06:15:58.677473+0000 (oldest deadline 2024-05-08T06:16:22.177585+0000) 2024-05-08T06:16:26.615 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:26 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:26.450+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:16:26.615 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:26 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:26.450+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.363256+0000 front 2024-05-08T06:15:42.363239+0000 (oldest deadline 2024-05-08T06:16:07.663434+0000) 2024-05-08T06:16:26.615 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:26 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:26.450+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:51.065366+0000 front 2024-05-08T06:15:51.065285+0000 (oldest deadline 2024-05-08T06:16:14.565065+0000) 2024-05-08T06:16:26.615 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:26 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:26.450+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.566476+0000 front 2024-05-08T06:15:58.566601+0000 (oldest deadline 2024-05-08T06:16:22.666446+0000) 2024-05-08T06:16:26.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:26 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:26.441+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:26.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:26 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:26.441+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:26.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:26 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:26.441+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.109835+0000 front 2024-05-08T06:15:42.109956+0000 (oldest deadline 2024-05-08T06:16:06.809996+0000) 2024-05-08T06:16:26.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:26 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:26.441+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:49.712308+0000 front 2024-05-08T06:15:49.712030+0000 (oldest deadline 2024-05-08T06:16:13.113280+0000) 2024-05-08T06:16:26.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:26 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:26.441+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.817347+0000 front 2024-05-08T06:15:58.817382+0000 (oldest deadline 2024-05-08T06:16:24.117220+0000) 2024-05-08T06:16:27.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:26 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:26.615+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:27.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:26 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:26.615+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:27.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:26 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:26.615+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:27.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:26 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:26.615+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:16:00.051637+0000 front 2024-05-08T06:16:00.051514+0000 (oldest deadline 2024-05-08T06:16:22.951535+0000) 2024-05-08T06:16:27.409 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:27 smithi163 podman[69155]: 2024-05-08 06:16:27.141481006 +0000 UTC m=+5.578020163 container died 0e5766329585f838868b6077bd00d080f672ac0d35d2b32046bbca5ad1cf18dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-08T06:16:27.409 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:27 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:27.042+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:27.409 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:27 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:27.042+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:27.409 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:27 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:27.042+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:27.409 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:27 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:27.042+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:47.576642+0000 front 2024-05-08T06:15:47.576333+0000 (oldest deadline 2024-05-08T06:16:15.176725+0000) 2024-05-08T06:16:27.409 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:27 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:27.042+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.677497+0000 front 2024-05-08T06:15:58.677473+0000 (oldest deadline 2024-05-08T06:16:22.177585+0000) 2024-05-08T06:16:27.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:27 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:27.434+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:27.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:27 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:27.434+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:27.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:27 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:27.434+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.109835+0000 front 2024-05-08T06:15:42.109956+0000 (oldest deadline 2024-05-08T06:16:06.809996+0000) 2024-05-08T06:16:27.706 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:27 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:27.434+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:49.712308+0000 front 2024-05-08T06:15:49.712030+0000 (oldest deadline 2024-05-08T06:16:13.113280+0000) 2024-05-08T06:16:27.706 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:27 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:27.434+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.817347+0000 front 2024-05-08T06:15:58.817382+0000 (oldest deadline 2024-05-08T06:16:24.117220+0000) 2024-05-08T06:16:27.787 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:27 smithi163 podman[69155]: 2024-05-08 06:16:27.528960096 +0000 UTC m=+5.965499250 container cleanup 0e5766329585f838868b6077bd00d080f672ac0d35d2b32046bbca5ad1cf18dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-08T06:16:27.787 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:27 smithi163 bash[69155]: ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7 2024-05-08T06:16:27.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:27 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:27.566+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:27.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:27 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:27.566+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:27.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:27 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:27.566+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:27.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:27 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:27.566+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:16:00.051637+0000 front 2024-05-08T06:16:00.051514+0000 (oldest deadline 2024-05-08T06:16:22.951535+0000) 2024-05-08T06:16:27.801 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:27 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:27.443+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:16:27.801 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:27 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:27.443+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.363256+0000 front 2024-05-08T06:15:42.363239+0000 (oldest deadline 2024-05-08T06:16:07.663434+0000) 2024-05-08T06:16:27.801 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:27 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:27.443+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:51.065366+0000 front 2024-05-08T06:15:51.065285+0000 (oldest deadline 2024-05-08T06:16:14.565065+0000) 2024-05-08T06:16:27.801 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:27 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:27.443+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.566476+0000 front 2024-05-08T06:15:58.566601+0000 (oldest deadline 2024-05-08T06:16:22.666446+0000) 2024-05-08T06:16:28.038 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:27 smithi163 podman[69183]: 2024-05-08 06:16:27.78849791 +0000 UTC m=+0.645366919 container remove 0e5766329585f838868b6077bd00d080f672ac0d35d2b32046bbca5ad1cf18dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-08T06:16:28.382 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:28 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:28.056+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:28.383 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:28 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:28.056+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:28.383 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:28 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:28.056+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:28.383 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:28 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:28.056+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:47.576642+0000 front 2024-05-08T06:15:47.576333+0000 (oldest deadline 2024-05-08T06:16:15.176725+0000) 2024-05-08T06:16:28.383 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:28 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:28.056+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.677497+0000 front 2024-05-08T06:15:58.677473+0000 (oldest deadline 2024-05-08T06:16:22.177585+0000) 2024-05-08T06:16:28.383 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:28 smithi163 podman[69275]: 2024-05-08 06:16:28.081034548 +0000 UTC m=+0.099726781 container create 101c6de0cb2a775fa4b0f2032ae64535e1ca7bb3d8b401d095091782f979b3c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f) 2024-05-08T06:16:28.383 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:28 smithi163 podman[69275]: 2024-05-08 06:16:27.999983366 +0000 UTC m=+0.018675616 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:16:28.383 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:28 smithi163 podman[69275]: 2024-05-08 06:16:28.275120409 +0000 UTC m=+0.293812641 container init 101c6de0cb2a775fa4b0f2032ae64535e1ca7bb3d8b401d095091782f979b3c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-08T06:16:28.383 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:28 smithi163 podman[69275]: 2024-05-08 06:16:28.278153438 +0000 UTC m=+0.296845671 container start 101c6de0cb2a775fa4b0f2032ae64535e1ca7bb3d8b401d095091782f979b3c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240507) 2024-05-08T06:16:28.383 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:28 smithi163 podman[69275]: 2024-05-08 06:16:28.38334601 +0000 UTC m=+0.402038248 container attach 101c6de0cb2a775fa4b0f2032ae64535e1ca7bb3d8b401d095091782f979b3c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-08T06:16:28.788 INFO:journalctl@ceph.osd.7.smithi163.stdout:May 08 06:16:28 smithi163 podman[69275]: 2024-05-08 06:16:28.565602938 +0000 UTC m=+0.584295170 container died 101c6de0cb2a775fa4b0f2032ae64535e1ca7bb3d8b401d095091782f979b3c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2) 2024-05-08T06:16:28.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:28 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:28.577+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:28.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:28 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:28.577+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:28.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:28 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:28.577+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:28.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:28 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:28.577+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:16:00.051637+0000 front 2024-05-08T06:16:00.051514+0000 (oldest deadline 2024-05-08T06:16:22.951535+0000) 2024-05-08T06:16:28.801 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:28 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:28.438+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:16:28.801 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:28 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:28.438+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.363256+0000 front 2024-05-08T06:15:42.363239+0000 (oldest deadline 2024-05-08T06:16:07.663434+0000) 2024-05-08T06:16:28.801 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:28 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:28.438+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:51.065366+0000 front 2024-05-08T06:15:51.065285+0000 (oldest deadline 2024-05-08T06:16:14.565065+0000) 2024-05-08T06:16:28.801 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:28 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:28.438+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.566476+0000 front 2024-05-08T06:15:58.566601+0000 (oldest deadline 2024-05-08T06:16:22.666446+0000) 2024-05-08T06:16:28.955 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:28 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:28.463+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:28.955 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:28 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:28.463+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:28.955 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:28 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:28.463+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.109835+0000 front 2024-05-08T06:15:42.109956+0000 (oldest deadline 2024-05-08T06:16:06.809996+0000) 2024-05-08T06:16:28.955 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:28 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:28.463+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:49.712308+0000 front 2024-05-08T06:15:49.712030+0000 (oldest deadline 2024-05-08T06:16:13.113280+0000) 2024-05-08T06:16:28.956 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:28 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:28.463+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.817347+0000 front 2024-05-08T06:15:58.817382+0000 (oldest deadline 2024-05-08T06:16:24.117220+0000) 2024-05-08T06:16:29.130 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:29 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:29.030+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:29.130 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:29 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:29.030+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:29.130 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:29 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:29.030+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:29.130 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:29 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:29.030+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:47.576642+0000 front 2024-05-08T06:15:47.576333+0000 (oldest deadline 2024-05-08T06:16:15.176725+0000) 2024-05-08T06:16:29.130 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:29 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:29.030+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.677497+0000 front 2024-05-08T06:15:58.677473+0000 (oldest deadline 2024-05-08T06:16:22.177585+0000) 2024-05-08T06:16:29.156 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@osd.7.service' 2024-05-08T06:16:29.233 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:16:29.234 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-08T06:16:29.234 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-08T06:16:29.234 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@osd.8 2024-05-08T06:16:29.550 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:29 smithi192 systemd[1]: Stopping Ceph osd.8 for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T06:16:29.550 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:29 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:29.402+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:16:29.550 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:29 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:29.402+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.363256+0000 front 2024-05-08T06:15:42.363239+0000 (oldest deadline 2024-05-08T06:16:07.663434+0000) 2024-05-08T06:16:29.550 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:29 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:29.402+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:51.065366+0000 front 2024-05-08T06:15:51.065285+0000 (oldest deadline 2024-05-08T06:16:14.565065+0000) 2024-05-08T06:16:29.550 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:29 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:29.402+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.566476+0000 front 2024-05-08T06:15:58.566601+0000 (oldest deadline 2024-05-08T06:16:22.666446+0000) 2024-05-08T06:16:29.955 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:29 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:29.469+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:29.955 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:29 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:29.469+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:29.955 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:29 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:29.469+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.109835+0000 front 2024-05-08T06:15:42.109956+0000 (oldest deadline 2024-05-08T06:16:06.809996+0000) 2024-05-08T06:16:29.955 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:29 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:29.469+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:49.712308+0000 front 2024-05-08T06:15:49.712030+0000 (oldest deadline 2024-05-08T06:16:13.113280+0000) 2024-05-08T06:16:29.956 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:29 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:29.469+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.817347+0000 front 2024-05-08T06:15:58.817382+0000 (oldest deadline 2024-05-08T06:16:24.117220+0000) 2024-05-08T06:16:30.050 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:29 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:29.554+0000 7fd029fd4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:16:30.050 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:29 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:29.554+0000 7fd029fd4700 -1 osd.8 85 *** Got signal Terminated *** 2024-05-08T06:16:30.050 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:29 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:29.554+0000 7fd029fd4700 -1 osd.8 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T06:16:30.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:29 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:29.623+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:30.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:29 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:29.623+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:30.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:29 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:29.623+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:30.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:29 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:29.623+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:16:00.051637+0000 front 2024-05-08T06:16:00.051514+0000 (oldest deadline 2024-05-08T06:16:22.951535+0000) 2024-05-08T06:16:30.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:30 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:30.021+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:30.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:30 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:30.021+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:30.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:30 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:30.021+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:30.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:30 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:30.021+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:47.576642+0000 front 2024-05-08T06:15:47.576333+0000 (oldest deadline 2024-05-08T06:16:15.176725+0000) 2024-05-08T06:16:30.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:30 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:30.021+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.677497+0000 front 2024-05-08T06:15:58.677473+0000 (oldest deadline 2024-05-08T06:16:22.177585+0000) 2024-05-08T06:16:30.617 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:30 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:30.355+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:16:30.617 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:30 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:30.355+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.363256+0000 front 2024-05-08T06:15:42.363239+0000 (oldest deadline 2024-05-08T06:16:07.663434+0000) 2024-05-08T06:16:30.617 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:30 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:30.355+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:51.065366+0000 front 2024-05-08T06:15:51.065285+0000 (oldest deadline 2024-05-08T06:16:14.565065+0000) 2024-05-08T06:16:30.617 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:30 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:30.355+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.566476+0000 front 2024-05-08T06:15:58.566601+0000 (oldest deadline 2024-05-08T06:16:22.666446+0000) 2024-05-08T06:16:30.955 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:30 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:30.519+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:30.955 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:30 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:30.519+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:30.955 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:30 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:30.519+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.109835+0000 front 2024-05-08T06:15:42.109956+0000 (oldest deadline 2024-05-08T06:16:06.809996+0000) 2024-05-08T06:16:30.955 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:30 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:30.519+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:49.712308+0000 front 2024-05-08T06:15:49.712030+0000 (oldest deadline 2024-05-08T06:16:13.113280+0000) 2024-05-08T06:16:30.955 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:30 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:30.519+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.817347+0000 front 2024-05-08T06:15:58.817382+0000 (oldest deadline 2024-05-08T06:16:24.117220+0000) 2024-05-08T06:16:30.956 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:30 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:30.519+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:06.418637+0000 front 2024-05-08T06:16:06.418700+0000 (oldest deadline 2024-05-08T06:16:30.419197+0000) 2024-05-08T06:16:31.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:30 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:30.617+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:31.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:30 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:30.617+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:31.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:30 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:30.617+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:31.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:30 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:30.617+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:16:00.051637+0000 front 2024-05-08T06:16:00.051514+0000 (oldest deadline 2024-05-08T06:16:22.951535+0000) 2024-05-08T06:16:31.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:30 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:30.617+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:04.652467+0000 front 2024-05-08T06:16:04.652398+0000 (oldest deadline 2024-05-08T06:16:30.552552+0000) 2024-05-08T06:16:31.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:31 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:31.043+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:31.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:31 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:31.043+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:31.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:31 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:31.043+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:31.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:31 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:31.043+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:47.576642+0000 front 2024-05-08T06:15:47.576333+0000 (oldest deadline 2024-05-08T06:16:15.176725+0000) 2024-05-08T06:16:31.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:31 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:31.043+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.677497+0000 front 2024-05-08T06:15:58.677473+0000 (oldest deadline 2024-05-08T06:16:22.177585+0000) 2024-05-08T06:16:31.666 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:31 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:31.401+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:16:31.666 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:31 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:31.401+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.363256+0000 front 2024-05-08T06:15:42.363239+0000 (oldest deadline 2024-05-08T06:16:07.663434+0000) 2024-05-08T06:16:31.667 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:31 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:31.401+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:51.065366+0000 front 2024-05-08T06:15:51.065285+0000 (oldest deadline 2024-05-08T06:16:14.565065+0000) 2024-05-08T06:16:31.667 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:31 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:31.401+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.566476+0000 front 2024-05-08T06:15:58.566601+0000 (oldest deadline 2024-05-08T06:16:22.666446+0000) 2024-05-08T06:16:31.955 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:31 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:31.479+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:31.955 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:31 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:31.479+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:31.955 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:31 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:31.479+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.109835+0000 front 2024-05-08T06:15:42.109956+0000 (oldest deadline 2024-05-08T06:16:06.809996+0000) 2024-05-08T06:16:31.955 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:31 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:31.479+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:49.712308+0000 front 2024-05-08T06:15:49.712030+0000 (oldest deadline 2024-05-08T06:16:13.113280+0000) 2024-05-08T06:16:31.955 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:31 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:31.479+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.817347+0000 front 2024-05-08T06:15:58.817382+0000 (oldest deadline 2024-05-08T06:16:24.117220+0000) 2024-05-08T06:16:31.955 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:31 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:31.479+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:06.418637+0000 front 2024-05-08T06:16:06.418700+0000 (oldest deadline 2024-05-08T06:16:30.419197+0000) 2024-05-08T06:16:32.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:31 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:31.667+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:32.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:31 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:31.667+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:32.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:31 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:31.667+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:32.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:31 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:31.667+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:16:00.051637+0000 front 2024-05-08T06:16:00.051514+0000 (oldest deadline 2024-05-08T06:16:22.951535+0000) 2024-05-08T06:16:32.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:31 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:31.667+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:04.652467+0000 front 2024-05-08T06:16:04.652398+0000 (oldest deadline 2024-05-08T06:16:30.552552+0000) 2024-05-08T06:16:32.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:32 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:32.068+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:32.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:32 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:32.068+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:32.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:32 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:32.068+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:32.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:32 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:32.068+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:47.576642+0000 front 2024-05-08T06:15:47.576333+0000 (oldest deadline 2024-05-08T06:16:15.176725+0000) 2024-05-08T06:16:32.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:32 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:32.068+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.677497+0000 front 2024-05-08T06:15:58.677473+0000 (oldest deadline 2024-05-08T06:16:22.177585+0000) 2024-05-08T06:16:32.800 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:32 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:32.445+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:16:32.800 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:32 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:32.445+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.363256+0000 front 2024-05-08T06:15:42.363239+0000 (oldest deadline 2024-05-08T06:16:07.663434+0000) 2024-05-08T06:16:32.800 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:32 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:32.445+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:51.065366+0000 front 2024-05-08T06:15:51.065285+0000 (oldest deadline 2024-05-08T06:16:14.565065+0000) 2024-05-08T06:16:32.800 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:32 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:32.445+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.566476+0000 front 2024-05-08T06:15:58.566601+0000 (oldest deadline 2024-05-08T06:16:22.666446+0000) 2024-05-08T06:16:32.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:32 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:32.649+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:32.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:32 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:32.649+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:32.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:32 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:32.649+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:32.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:32 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:32.649+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:16:00.051637+0000 front 2024-05-08T06:16:00.051514+0000 (oldest deadline 2024-05-08T06:16:22.951535+0000) 2024-05-08T06:16:32.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:32 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:32.649+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:04.652467+0000 front 2024-05-08T06:16:04.652398+0000 (oldest deadline 2024-05-08T06:16:30.552552+0000) 2024-05-08T06:16:32.955 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:32 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:32.514+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:32.955 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:32 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:32.514+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:32.955 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:32 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:32.514+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.109835+0000 front 2024-05-08T06:15:42.109956+0000 (oldest deadline 2024-05-08T06:16:06.809996+0000) 2024-05-08T06:16:32.955 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:32 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:32.514+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:49.712308+0000 front 2024-05-08T06:15:49.712030+0000 (oldest deadline 2024-05-08T06:16:13.113280+0000) 2024-05-08T06:16:32.956 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:32 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:32.514+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.817347+0000 front 2024-05-08T06:15:58.817382+0000 (oldest deadline 2024-05-08T06:16:24.117220+0000) 2024-05-08T06:16:32.956 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:32 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:32.514+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:06.418637+0000 front 2024-05-08T06:16:06.418700+0000 (oldest deadline 2024-05-08T06:16:30.419197+0000) 2024-05-08T06:16:33.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:33 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:33.047+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:33.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:33 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:33.047+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:33.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:33 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:33.047+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:33.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:33 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:33.047+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:47.576642+0000 front 2024-05-08T06:15:47.576333+0000 (oldest deadline 2024-05-08T06:16:15.176725+0000) 2024-05-08T06:16:33.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:33 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:33.047+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.677497+0000 front 2024-05-08T06:15:58.677473+0000 (oldest deadline 2024-05-08T06:16:22.177585+0000) 2024-05-08T06:16:33.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:33 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:33.047+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:06.178933+0000 front 2024-05-08T06:16:06.178920+0000 (oldest deadline 2024-05-08T06:16:32.078839+0000) 2024-05-08T06:16:33.800 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:33 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:33.407+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:16:33.800 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:33 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:33.407+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.363256+0000 front 2024-05-08T06:15:42.363239+0000 (oldest deadline 2024-05-08T06:16:07.663434+0000) 2024-05-08T06:16:33.801 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:33 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:33.407+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:51.065366+0000 front 2024-05-08T06:15:51.065285+0000 (oldest deadline 2024-05-08T06:16:14.565065+0000) 2024-05-08T06:16:33.801 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:33 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:33.407+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.566476+0000 front 2024-05-08T06:15:58.566601+0000 (oldest deadline 2024-05-08T06:16:22.666446+0000) 2024-05-08T06:16:33.801 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:33 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:33.407+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:07.367475+0000 front 2024-05-08T06:16:07.367495+0000 (oldest deadline 2024-05-08T06:16:33.267599+0000) 2024-05-08T06:16:33.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:33 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:33.643+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:33.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:33 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:33.643+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:33.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:33 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:33.643+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:33.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:33 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:33.643+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:16:00.051637+0000 front 2024-05-08T06:16:00.051514+0000 (oldest deadline 2024-05-08T06:16:22.951535+0000) 2024-05-08T06:16:33.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:33 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:33.643+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:04.652467+0000 front 2024-05-08T06:16:04.652398+0000 (oldest deadline 2024-05-08T06:16:30.552552+0000) 2024-05-08T06:16:33.955 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:33 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:33.522+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:33.955 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:33 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:33.522+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:33.956 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:33 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:33.522+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.109835+0000 front 2024-05-08T06:15:42.109956+0000 (oldest deadline 2024-05-08T06:16:06.809996+0000) 2024-05-08T06:16:33.956 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:33 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:33.522+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:49.712308+0000 front 2024-05-08T06:15:49.712030+0000 (oldest deadline 2024-05-08T06:16:13.113280+0000) 2024-05-08T06:16:33.956 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:33 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:33.522+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.817347+0000 front 2024-05-08T06:15:58.817382+0000 (oldest deadline 2024-05-08T06:16:24.117220+0000) 2024-05-08T06:16:33.956 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:33 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:33.522+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:06.418637+0000 front 2024-05-08T06:16:06.418700+0000 (oldest deadline 2024-05-08T06:16:30.419197+0000) 2024-05-08T06:16:34.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:34 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:34.085+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:34.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:34 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:34.085+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:34.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:34 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:34.085+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:34.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:34 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:34.085+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:47.576642+0000 front 2024-05-08T06:15:47.576333+0000 (oldest deadline 2024-05-08T06:16:15.176725+0000) 2024-05-08T06:16:34.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:34 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:34.085+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.677497+0000 front 2024-05-08T06:15:58.677473+0000 (oldest deadline 2024-05-08T06:16:22.177585+0000) 2024-05-08T06:16:34.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:34 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:34.085+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:06.178933+0000 front 2024-05-08T06:16:06.178920+0000 (oldest deadline 2024-05-08T06:16:32.078839+0000) 2024-05-08T06:16:34.800 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:34 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:34.411+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:31.162330+0000 front 2024-05-08T06:15:31.162426+0000 (oldest deadline 2024-05-08T06:15:56.462462+0000) 2024-05-08T06:16:34.801 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:34 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:34.411+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.363256+0000 front 2024-05-08T06:15:42.363239+0000 (oldest deadline 2024-05-08T06:16:07.663434+0000) 2024-05-08T06:16:34.801 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:34 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:34.411+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:51.065366+0000 front 2024-05-08T06:15:51.065285+0000 (oldest deadline 2024-05-08T06:16:14.565065+0000) 2024-05-08T06:16:34.801 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:34 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:34.411+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.566476+0000 front 2024-05-08T06:15:58.566601+0000 (oldest deadline 2024-05-08T06:16:22.666446+0000) 2024-05-08T06:16:34.801 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:34 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8[53417]: 2024-05-08T06:16:34.411+0000 7fd0225c0700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:07.367475+0000 front 2024-05-08T06:16:07.367495+0000 (oldest deadline 2024-05-08T06:16:33.267599+0000) 2024-05-08T06:16:34.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:34 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:34.633+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:34.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:34 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:34.633+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:34.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:34 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:34.633+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:34.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:34 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:34.633+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:16:00.051637+0000 front 2024-05-08T06:16:00.051514+0000 (oldest deadline 2024-05-08T06:16:22.951535+0000) 2024-05-08T06:16:34.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:34 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:34.633+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:04.652467+0000 front 2024-05-08T06:16:04.652398+0000 (oldest deadline 2024-05-08T06:16:30.552552+0000) 2024-05-08T06:16:34.955 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:34 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:34.479+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:34.955 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:34 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:34.479+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:34.955 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:34 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:34.479+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.109835+0000 front 2024-05-08T06:15:42.109956+0000 (oldest deadline 2024-05-08T06:16:06.809996+0000) 2024-05-08T06:16:34.955 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:34 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:34.479+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:49.712308+0000 front 2024-05-08T06:15:49.712030+0000 (oldest deadline 2024-05-08T06:16:13.113280+0000) 2024-05-08T06:16:34.955 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:34 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:34.479+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.817347+0000 front 2024-05-08T06:15:58.817382+0000 (oldest deadline 2024-05-08T06:16:24.117220+0000) 2024-05-08T06:16:34.956 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:34 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:34.479+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:06.418637+0000 front 2024-05-08T06:16:06.418700+0000 (oldest deadline 2024-05-08T06:16:30.419197+0000) 2024-05-08T06:16:35.179 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:34 smithi192 podman[69871]: 2024-05-08 06:16:34.879754909 +0000 UTC m=+5.450006387 container died eb91d693cf9a1608e779836b9de68d31411c6990f5ee13c7c9030ea8b8e943c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=) 2024-05-08T06:16:35.505 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:35 smithi192 podman[69871]: 2024-05-08 06:16:35.279547455 +0000 UTC m=+5.849798937 container cleanup eb91d693cf9a1608e779836b9de68d31411c6990f5ee13c7c9030ea8b8e943c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-08T06:16:35.505 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:35 smithi192 bash[69871]: ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8 2024-05-08T06:16:35.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:35 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:35.104+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:35.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:35 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:35.104+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:35.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:35 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:35.104+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:35.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:35 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:35.104+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:47.576642+0000 front 2024-05-08T06:15:47.576333+0000 (oldest deadline 2024-05-08T06:16:15.176725+0000) 2024-05-08T06:16:35.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:35 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:35.104+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.677497+0000 front 2024-05-08T06:15:58.677473+0000 (oldest deadline 2024-05-08T06:16:22.177585+0000) 2024-05-08T06:16:35.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:35 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:35.104+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:06.178933+0000 front 2024-05-08T06:16:06.178920+0000 (oldest deadline 2024-05-08T06:16:32.078839+0000) 2024-05-08T06:16:35.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:35 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:35.443+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:35.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:35 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:35.443+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:35.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:35 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:35.443+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.109835+0000 front 2024-05-08T06:15:42.109956+0000 (oldest deadline 2024-05-08T06:16:06.809996+0000) 2024-05-08T06:16:35.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:35 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:35.443+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:49.712308+0000 front 2024-05-08T06:15:49.712030+0000 (oldest deadline 2024-05-08T06:16:13.113280+0000) 2024-05-08T06:16:35.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:35 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:35.443+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.817347+0000 front 2024-05-08T06:15:58.817382+0000 (oldest deadline 2024-05-08T06:16:24.117220+0000) 2024-05-08T06:16:35.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:35 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:35.443+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:06.418637+0000 front 2024-05-08T06:16:06.418700+0000 (oldest deadline 2024-05-08T06:16:30.419197+0000) 2024-05-08T06:16:35.800 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:35 smithi192 podman[69889]: 2024-05-08 06:16:35.506075139 +0000 UTC m=+0.619947580 container remove eb91d693cf9a1608e779836b9de68d31411c6990f5ee13c7c9030ea8b8e943c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f) 2024-05-08T06:16:35.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:35 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:35.615+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:35.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:35 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:35.615+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:35.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:35 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:35.615+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:35.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:35 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:35.615+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:16:00.051637+0000 front 2024-05-08T06:16:00.051514+0000 (oldest deadline 2024-05-08T06:16:22.951535+0000) 2024-05-08T06:16:35.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:35 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:35.615+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:04.652467+0000 front 2024-05-08T06:16:04.652398+0000 (oldest deadline 2024-05-08T06:16:30.552552+0000) 2024-05-08T06:16:36.200 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:35 smithi192 podman[69981]: 2024-05-08 06:16:35.853378026 +0000 UTC m=+0.018097783 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:16:36.200 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:35 smithi192 podman[69981]: 2024-05-08 06:16:35.965313732 +0000 UTC m=+0.130033489 container create 3a2ca55b16c4f82ac041e5da04ab5589e8fda888ffbe6ec032d8e81dc0e6d8c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True) 2024-05-08T06:16:36.494 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:36 smithi192 podman[69981]: 2024-05-08 06:16:36.201113867 +0000 UTC m=+0.365833625 container init 3a2ca55b16c4f82ac041e5da04ab5589e8fda888ffbe6ec032d8e81dc0e6d8c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-08T06:16:36.494 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:36 smithi192 podman[69981]: 2024-05-08 06:16:36.204422554 +0000 UTC m=+0.369142316 container start 3a2ca55b16c4f82ac041e5da04ab5589e8fda888ffbe6ec032d8e81dc0e6d8c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f) 2024-05-08T06:16:36.494 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:36 smithi192 podman[69981]: 2024-05-08 06:16:36.309490232 +0000 UTC m=+0.474210012 container attach 3a2ca55b16c4f82ac041e5da04ab5589e8fda888ffbe6ec032d8e81dc0e6d8c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, ceph=True) 2024-05-08T06:16:36.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:36 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:36.055+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:36.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:36 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:36.055+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:36.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:36 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:36.055+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:36.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:36 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:36.055+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:47.576642+0000 front 2024-05-08T06:15:47.576333+0000 (oldest deadline 2024-05-08T06:16:15.176725+0000) 2024-05-08T06:16:36.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:36 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:36.055+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.677497+0000 front 2024-05-08T06:15:58.677473+0000 (oldest deadline 2024-05-08T06:16:22.177585+0000) 2024-05-08T06:16:36.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:36 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:36.055+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:06.178933+0000 front 2024-05-08T06:16:06.178920+0000 (oldest deadline 2024-05-08T06:16:32.078839+0000) 2024-05-08T06:16:36.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:36 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:36.431+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:36.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:36 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:36.431+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:36.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:36 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:36.431+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.109835+0000 front 2024-05-08T06:15:42.109956+0000 (oldest deadline 2024-05-08T06:16:06.809996+0000) 2024-05-08T06:16:36.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:36 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:36.431+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:49.712308+0000 front 2024-05-08T06:15:49.712030+0000 (oldest deadline 2024-05-08T06:16:13.113280+0000) 2024-05-08T06:16:36.706 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:36 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:36.431+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.817347+0000 front 2024-05-08T06:15:58.817382+0000 (oldest deadline 2024-05-08T06:16:24.117220+0000) 2024-05-08T06:16:36.706 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:36 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:36.431+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:06.418637+0000 front 2024-05-08T06:16:06.418700+0000 (oldest deadline 2024-05-08T06:16:30.419197+0000) 2024-05-08T06:16:36.770 INFO:journalctl@ceph.osd.8.smithi192.stdout:May 08 06:16:36 smithi192 podman[69981]: 2024-05-08 06:16:36.495270519 +0000 UTC m=+0.659990275 container died 3a2ca55b16c4f82ac041e5da04ab5589e8fda888ffbe6ec032d8e81dc0e6d8c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux ) 2024-05-08T06:16:36.770 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:36 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:36.590+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:36.770 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:36 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:36.590+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:36.770 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:36 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:36.590+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:36.771 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:36 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:36.590+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:16:00.051637+0000 front 2024-05-08T06:16:00.051514+0000 (oldest deadline 2024-05-08T06:16:22.951535+0000) 2024-05-08T06:16:36.771 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:36 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:36.590+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:04.652467+0000 front 2024-05-08T06:16:04.652398+0000 (oldest deadline 2024-05-08T06:16:30.552552+0000) 2024-05-08T06:16:37.088 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@osd.8.service' 2024-05-08T06:16:37.126 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:16:37.127 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-08T06:16:37.127 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-08T06:16:37.127 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@osd.9 2024-05-08T06:16:37.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:37 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:37.086+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:37.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:37 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:37.086+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:37.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:37 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:37.086+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:37.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:37 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:37.086+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:47.576642+0000 front 2024-05-08T06:15:47.576333+0000 (oldest deadline 2024-05-08T06:16:15.176725+0000) 2024-05-08T06:16:37.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:37 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:37.086+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.677497+0000 front 2024-05-08T06:15:58.677473+0000 (oldest deadline 2024-05-08T06:16:22.177585+0000) 2024-05-08T06:16:37.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:37 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:37.086+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:06.178933+0000 front 2024-05-08T06:16:06.178920+0000 (oldest deadline 2024-05-08T06:16:32.078839+0000) 2024-05-08T06:16:37.674 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:37 smithi123 systemd[1]: Stopping Ceph osd.9 for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T06:16:37.674 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:37 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:37.452+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:37.674 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:37 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:37.452+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:37.674 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:37 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:37.452+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.109835+0000 front 2024-05-08T06:15:42.109956+0000 (oldest deadline 2024-05-08T06:16:06.809996+0000) 2024-05-08T06:16:37.674 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:37 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:37.452+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:49.712308+0000 front 2024-05-08T06:15:49.712030+0000 (oldest deadline 2024-05-08T06:16:13.113280+0000) 2024-05-08T06:16:37.674 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:37 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:37.452+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.817347+0000 front 2024-05-08T06:15:58.817382+0000 (oldest deadline 2024-05-08T06:16:24.117220+0000) 2024-05-08T06:16:37.674 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:37 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:37.452+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:06.418637+0000 front 2024-05-08T06:16:06.418700+0000 (oldest deadline 2024-05-08T06:16:30.419197+0000) 2024-05-08T06:16:37.675 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:37 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:37.672+0000 7f820aa00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:16:37.675 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:37 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:37.672+0000 7f820aa00700 -1 osd.9 85 *** Got signal Terminated *** 2024-05-08T06:16:37.675 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:37 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:37.672+0000 7f820aa00700 -1 osd.9 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T06:16:37.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:37 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:37.548+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:37.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:37 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:37.548+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:37.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:37 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:37.548+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:37.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:37 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:37.548+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:16:00.051637+0000 front 2024-05-08T06:16:00.051514+0000 (oldest deadline 2024-05-08T06:16:22.951535+0000) 2024-05-08T06:16:37.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:37 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:37.548+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:04.652467+0000 front 2024-05-08T06:16:04.652398+0000 (oldest deadline 2024-05-08T06:16:30.552552+0000) 2024-05-08T06:16:38.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:38 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:38.040+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:38.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:38 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:38.040+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:38.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:38 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:38.040+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:38.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:38 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:38.040+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:47.576642+0000 front 2024-05-08T06:15:47.576333+0000 (oldest deadline 2024-05-08T06:16:15.176725+0000) 2024-05-08T06:16:38.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:38 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:38.040+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.677497+0000 front 2024-05-08T06:15:58.677473+0000 (oldest deadline 2024-05-08T06:16:22.177585+0000) 2024-05-08T06:16:38.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:38 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:38.040+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:06.178933+0000 front 2024-05-08T06:16:06.178920+0000 (oldest deadline 2024-05-08T06:16:32.078839+0000) 2024-05-08T06:16:38.955 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:38 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:38.460+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:38.956 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:38 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:38.460+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:38.956 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:38 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:38.460+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.109835+0000 front 2024-05-08T06:15:42.109956+0000 (oldest deadline 2024-05-08T06:16:06.809996+0000) 2024-05-08T06:16:38.956 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:38 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:38.460+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:49.712308+0000 front 2024-05-08T06:15:49.712030+0000 (oldest deadline 2024-05-08T06:16:13.113280+0000) 2024-05-08T06:16:38.956 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:38 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:38.460+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.817347+0000 front 2024-05-08T06:15:58.817382+0000 (oldest deadline 2024-05-08T06:16:24.117220+0000) 2024-05-08T06:16:38.956 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:38 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:38.460+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:06.418637+0000 front 2024-05-08T06:16:06.418700+0000 (oldest deadline 2024-05-08T06:16:30.419197+0000) 2024-05-08T06:16:39.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:38 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:38.567+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:39.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:38 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:38.567+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:39.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:38 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:38.567+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:39.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:38 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:38.567+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:16:00.051637+0000 front 2024-05-08T06:16:00.051514+0000 (oldest deadline 2024-05-08T06:16:22.951535+0000) 2024-05-08T06:16:39.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:38 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:38.567+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:04.652467+0000 front 2024-05-08T06:16:04.652398+0000 (oldest deadline 2024-05-08T06:16:30.552552+0000) 2024-05-08T06:16:39.247 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:38 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:38.994+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:39.258 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:38 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:38.994+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:39.259 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:38 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:38.994+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:39.259 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:38 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:38.994+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:47.576642+0000 front 2024-05-08T06:15:47.576333+0000 (oldest deadline 2024-05-08T06:16:15.176725+0000) 2024-05-08T06:16:39.259 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:38 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:38.994+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.677497+0000 front 2024-05-08T06:15:58.677473+0000 (oldest deadline 2024-05-08T06:16:22.177585+0000) 2024-05-08T06:16:39.259 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:38 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:38.994+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:06.178933+0000 front 2024-05-08T06:16:06.178920+0000 (oldest deadline 2024-05-08T06:16:32.078839+0000) 2024-05-08T06:16:39.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:39 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:39.448+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:39.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:39 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:39.448+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:39.706 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:39 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:39.448+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.109835+0000 front 2024-05-08T06:15:42.109956+0000 (oldest deadline 2024-05-08T06:16:06.809996+0000) 2024-05-08T06:16:39.706 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:39 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:39.448+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:49.712308+0000 front 2024-05-08T06:15:49.712030+0000 (oldest deadline 2024-05-08T06:16:13.113280+0000) 2024-05-08T06:16:39.706 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:39 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:39.448+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.817347+0000 front 2024-05-08T06:15:58.817382+0000 (oldest deadline 2024-05-08T06:16:24.117220+0000) 2024-05-08T06:16:39.706 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:39 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:39.448+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:06.418637+0000 front 2024-05-08T06:16:06.418700+0000 (oldest deadline 2024-05-08T06:16:30.419197+0000) 2024-05-08T06:16:39.706 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:39 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:39.448+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-05-08T06:16:15.621745+0000 front 2024-05-08T06:16:15.621942+0000 (oldest deadline 2024-05-08T06:16:38.521397+0000) 2024-05-08T06:16:39.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:39 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:39.533+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:39.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:39 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:39.533+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:39.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:39 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:39.533+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:39.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:39 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:39.533+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:16:00.051637+0000 front 2024-05-08T06:16:00.051514+0000 (oldest deadline 2024-05-08T06:16:22.951535+0000) 2024-05-08T06:16:39.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:39 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:39.533+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:04.652467+0000 front 2024-05-08T06:16:04.652398+0000 (oldest deadline 2024-05-08T06:16:30.552552+0000) 2024-05-08T06:16:39.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:39 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:39.533+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-05-08T06:16:14.454557+0000 front 2024-05-08T06:16:14.454569+0000 (oldest deadline 2024-05-08T06:16:39.154226+0000) 2024-05-08T06:16:40.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:40 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:40.035+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:40.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:40 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:40.035+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:40.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:40 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:40.035+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:40.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:40 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:40.035+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:47.576642+0000 front 2024-05-08T06:15:47.576333+0000 (oldest deadline 2024-05-08T06:16:15.176725+0000) 2024-05-08T06:16:40.288 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:40 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:40.035+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.677497+0000 front 2024-05-08T06:15:58.677473+0000 (oldest deadline 2024-05-08T06:16:22.177585+0000) 2024-05-08T06:16:40.289 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:40 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:40.035+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:06.178933+0000 front 2024-05-08T06:16:06.178920+0000 (oldest deadline 2024-05-08T06:16:32.078839+0000) 2024-05-08T06:16:40.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:40 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:40.414+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:40.706 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:40 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:40.414+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:40.706 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:40 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:40.414+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.109835+0000 front 2024-05-08T06:15:42.109956+0000 (oldest deadline 2024-05-08T06:16:06.809996+0000) 2024-05-08T06:16:40.706 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:40 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:40.414+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:49.712308+0000 front 2024-05-08T06:15:49.712030+0000 (oldest deadline 2024-05-08T06:16:13.113280+0000) 2024-05-08T06:16:40.706 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:40 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:40.414+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.817347+0000 front 2024-05-08T06:15:58.817382+0000 (oldest deadline 2024-05-08T06:16:24.117220+0000) 2024-05-08T06:16:40.706 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:40 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:40.414+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:06.418637+0000 front 2024-05-08T06:16:06.418700+0000 (oldest deadline 2024-05-08T06:16:30.419197+0000) 2024-05-08T06:16:40.706 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:40 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:40.414+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-05-08T06:16:15.621745+0000 front 2024-05-08T06:16:15.621942+0000 (oldest deadline 2024-05-08T06:16:38.521397+0000) 2024-05-08T06:16:41.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:40 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:40.560+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:41.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:40 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:40.560+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:41.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:40 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:40.560+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:41.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:40 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:40.560+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:16:00.051637+0000 front 2024-05-08T06:16:00.051514+0000 (oldest deadline 2024-05-08T06:16:22.951535+0000) 2024-05-08T06:16:41.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:40 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:40.560+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:04.652467+0000 front 2024-05-08T06:16:04.652398+0000 (oldest deadline 2024-05-08T06:16:30.552552+0000) 2024-05-08T06:16:41.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:40 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:40.560+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-05-08T06:16:14.454557+0000 front 2024-05-08T06:16:14.454569+0000 (oldest deadline 2024-05-08T06:16:39.154226+0000) 2024-05-08T06:16:41.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:41 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:41.058+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:41.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:41 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:41.058+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:41.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:41 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:41.058+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:41.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:41 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:41.058+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:47.576642+0000 front 2024-05-08T06:15:47.576333+0000 (oldest deadline 2024-05-08T06:16:15.176725+0000) 2024-05-08T06:16:41.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:41 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:41.058+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.677497+0000 front 2024-05-08T06:15:58.677473+0000 (oldest deadline 2024-05-08T06:16:22.177585+0000) 2024-05-08T06:16:41.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:41 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:41.058+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:06.178933+0000 front 2024-05-08T06:16:06.178920+0000 (oldest deadline 2024-05-08T06:16:32.078839+0000) 2024-05-08T06:16:41.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:41 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:41.426+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:41.706 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:41 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:41.426+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:41.706 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:41 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:41.426+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.109835+0000 front 2024-05-08T06:15:42.109956+0000 (oldest deadline 2024-05-08T06:16:06.809996+0000) 2024-05-08T06:16:41.706 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:41 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:41.426+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:49.712308+0000 front 2024-05-08T06:15:49.712030+0000 (oldest deadline 2024-05-08T06:16:13.113280+0000) 2024-05-08T06:16:41.706 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:41 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:41.426+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.817347+0000 front 2024-05-08T06:15:58.817382+0000 (oldest deadline 2024-05-08T06:16:24.117220+0000) 2024-05-08T06:16:41.706 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:41 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:41.426+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:06.418637+0000 front 2024-05-08T06:16:06.418700+0000 (oldest deadline 2024-05-08T06:16:30.419197+0000) 2024-05-08T06:16:41.706 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:41 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:41.426+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-05-08T06:16:15.621745+0000 front 2024-05-08T06:16:15.621942+0000 (oldest deadline 2024-05-08T06:16:38.521397+0000) 2024-05-08T06:16:42.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:41 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:41.599+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:42.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:41 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:41.599+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:42.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:41 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:41.599+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:42.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:41 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:41.599+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:16:00.051637+0000 front 2024-05-08T06:16:00.051514+0000 (oldest deadline 2024-05-08T06:16:22.951535+0000) 2024-05-08T06:16:42.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:41 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:41.599+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:04.652467+0000 front 2024-05-08T06:16:04.652398+0000 (oldest deadline 2024-05-08T06:16:30.552552+0000) 2024-05-08T06:16:42.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:41 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:41.599+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-05-08T06:16:14.454557+0000 front 2024-05-08T06:16:14.454569+0000 (oldest deadline 2024-05-08T06:16:39.154226+0000) 2024-05-08T06:16:42.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:42 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:42.099+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:42.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:42 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:42.099+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:42.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:42 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:42.099+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:42.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:42 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:42.099+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:47.576642+0000 front 2024-05-08T06:15:47.576333+0000 (oldest deadline 2024-05-08T06:16:15.176725+0000) 2024-05-08T06:16:42.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:42 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:42.099+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.677497+0000 front 2024-05-08T06:15:58.677473+0000 (oldest deadline 2024-05-08T06:16:22.177585+0000) 2024-05-08T06:16:42.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:42 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:42.099+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:06.178933+0000 front 2024-05-08T06:16:06.178920+0000 (oldest deadline 2024-05-08T06:16:32.078839+0000) 2024-05-08T06:16:42.705 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:42 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:42.432+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:23.906797+0000 front 2024-05-08T06:15:23.907027+0000 (oldest deadline 2024-05-08T06:15:48.606378+0000) 2024-05-08T06:16:42.706 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:42 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:42.432+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:32.708065+0000 front 2024-05-08T06:15:32.708115+0000 (oldest deadline 2024-05-08T06:15:56.808280+0000) 2024-05-08T06:16:42.706 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:42 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:42.432+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.109835+0000 front 2024-05-08T06:15:42.109956+0000 (oldest deadline 2024-05-08T06:16:06.809996+0000) 2024-05-08T06:16:42.706 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:42 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:42.432+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:49.712308+0000 front 2024-05-08T06:15:49.712030+0000 (oldest deadline 2024-05-08T06:16:13.113280+0000) 2024-05-08T06:16:42.706 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:42 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:42.432+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.817347+0000 front 2024-05-08T06:15:58.817382+0000 (oldest deadline 2024-05-08T06:16:24.117220+0000) 2024-05-08T06:16:42.706 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:42 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:42.432+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:06.418637+0000 front 2024-05-08T06:16:06.418700+0000 (oldest deadline 2024-05-08T06:16:30.419197+0000) 2024-05-08T06:16:42.706 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:42 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9[64476]: 2024-05-08T06:16:42.432+0000 7f8201200700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-05-08T06:16:15.621745+0000 front 2024-05-08T06:16:15.621942+0000 (oldest deadline 2024-05-08T06:16:38.521397+0000) 2024-05-08T06:16:43.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:42 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:42.568+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:43.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:42 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:42.568+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:43.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:42 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:42.568+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:43.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:42 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:42.568+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:16:00.051637+0000 front 2024-05-08T06:16:00.051514+0000 (oldest deadline 2024-05-08T06:16:22.951535+0000) 2024-05-08T06:16:43.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:42 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:42.568+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:04.652467+0000 front 2024-05-08T06:16:04.652398+0000 (oldest deadline 2024-05-08T06:16:30.552552+0000) 2024-05-08T06:16:43.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:42 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:42.568+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-05-08T06:16:14.454557+0000 front 2024-05-08T06:16:14.454569+0000 (oldest deadline 2024-05-08T06:16:39.154226+0000) 2024-05-08T06:16:43.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:43 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:43.069+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:43.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:43 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:43.069+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:43.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:43 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:43.069+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:43.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:43 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:43.069+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:47.576642+0000 front 2024-05-08T06:15:47.576333+0000 (oldest deadline 2024-05-08T06:16:15.176725+0000) 2024-05-08T06:16:43.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:43 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:43.069+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.677497+0000 front 2024-05-08T06:15:58.677473+0000 (oldest deadline 2024-05-08T06:16:22.177585+0000) 2024-05-08T06:16:43.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:43 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:43.069+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:06.178933+0000 front 2024-05-08T06:16:06.178920+0000 (oldest deadline 2024-05-08T06:16:32.078839+0000) 2024-05-08T06:16:43.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:43 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:43.069+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-05-08T06:16:15.981053+0000 front 2024-05-08T06:16:15.981041+0000 (oldest deadline 2024-05-08T06:16:42.381201+0000) 2024-05-08T06:16:43.619 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:43 smithi123 podman[96438]: 2024-05-08 06:16:43.288857136 +0000 UTC m=+5.755826698 container died 023b70f352c75222e08c30f70a5c190ad56ae0d9a668884d1f59007ad6cbea5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-08T06:16:44.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:43 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:43.598+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:44.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:43 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:43.598+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:44.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:43 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:43.598+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:44.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:43 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:43.598+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:16:00.051637+0000 front 2024-05-08T06:16:00.051514+0000 (oldest deadline 2024-05-08T06:16:22.951535+0000) 2024-05-08T06:16:44.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:43 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:43.598+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:04.652467+0000 front 2024-05-08T06:16:04.652398+0000 (oldest deadline 2024-05-08T06:16:30.552552+0000) 2024-05-08T06:16:44.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:43 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:43.598+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-05-08T06:16:14.454557+0000 front 2024-05-08T06:16:14.454569+0000 (oldest deadline 2024-05-08T06:16:39.154226+0000) 2024-05-08T06:16:44.205 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:43 smithi123 podman[96438]: 2024-05-08 06:16:43.873656432 +0000 UTC m=+6.340625985 container cleanup 023b70f352c75222e08c30f70a5c190ad56ae0d9a668884d1f59007ad6cbea5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-08T06:16:44.205 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:43 smithi123 bash[96438]: ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9 2024-05-08T06:16:44.205 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:44 smithi123 podman[96459]: 2024-05-08 06:16:44.091581513 +0000 UTC m=+0.789288526 container remove 023b70f352c75222e08c30f70a5c190ad56ae0d9a668884d1f59007ad6cbea5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True) 2024-05-08T06:16:44.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:44 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:44.116+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:44.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:44 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:44.116+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:44.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:44 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:44.116+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:44.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:44 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:44.116+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:47.576642+0000 front 2024-05-08T06:15:47.576333+0000 (oldest deadline 2024-05-08T06:16:15.176725+0000) 2024-05-08T06:16:44.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:44 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:44.116+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.677497+0000 front 2024-05-08T06:15:58.677473+0000 (oldest deadline 2024-05-08T06:16:22.177585+0000) 2024-05-08T06:16:44.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:44 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:44.116+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:06.178933+0000 front 2024-05-08T06:16:06.178920+0000 (oldest deadline 2024-05-08T06:16:32.078839+0000) 2024-05-08T06:16:44.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:44 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:44.116+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-05-08T06:16:15.981053+0000 front 2024-05-08T06:16:15.981041+0000 (oldest deadline 2024-05-08T06:16:42.381201+0000) 2024-05-08T06:16:44.952 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:44 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:44.578+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:44.952 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:44 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:44.578+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:44.952 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:44 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:44.578+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:44.952 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:44 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:44.578+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:16:00.051637+0000 front 2024-05-08T06:16:00.051514+0000 (oldest deadline 2024-05-08T06:16:22.951535+0000) 2024-05-08T06:16:44.952 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:44 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:44.578+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:04.652467+0000 front 2024-05-08T06:16:04.652398+0000 (oldest deadline 2024-05-08T06:16:30.552552+0000) 2024-05-08T06:16:44.952 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:44 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:44.578+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-05-08T06:16:14.454557+0000 front 2024-05-08T06:16:14.454569+0000 (oldest deadline 2024-05-08T06:16:39.154226+0000) 2024-05-08T06:16:45.154 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:44 smithi123 podman[96552]: 2024-05-08 06:16:44.748263442 +0000 UTC m=+0.033422322 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:16:45.154 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:44 smithi123 podman[96552]: 2024-05-08 06:16:44.851081437 +0000 UTC m=+0.136240354 container create 7c5aa8a128f544e12e549212ca92b37e88660a77c8bc04c6d79c3d3bc71b169e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-08T06:16:45.154 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:45 smithi123 podman[96552]: 2024-05-08 06:16:45.1544667 +0000 UTC m=+0.439625620 container init 7c5aa8a128f544e12e549212ca92b37e88660a77c8bc04c6d79c3d3bc71b169e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-08T06:16:45.455 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:45 smithi123 podman[96552]: 2024-05-08 06:16:45.164968737 +0000 UTC m=+0.450127656 container start 7c5aa8a128f544e12e549212ca92b37e88660a77c8bc04c6d79c3d3bc71b169e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True) 2024-05-08T06:16:45.455 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:45 smithi123 podman[96552]: 2024-05-08 06:16:45.278323141 +0000 UTC m=+0.563482061 container attach 7c5aa8a128f544e12e549212ca92b37e88660a77c8bc04c6d79c3d3bc71b169e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-08T06:16:45.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:45 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:45.138+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:45.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:45 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:45.138+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:45.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:45 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:45.138+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:45.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:45 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:45.138+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:47.576642+0000 front 2024-05-08T06:15:47.576333+0000 (oldest deadline 2024-05-08T06:16:15.176725+0000) 2024-05-08T06:16:45.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:45 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:45.138+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.677497+0000 front 2024-05-08T06:15:58.677473+0000 (oldest deadline 2024-05-08T06:16:22.177585+0000) 2024-05-08T06:16:45.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:45 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:45.138+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:06.178933+0000 front 2024-05-08T06:16:06.178920+0000 (oldest deadline 2024-05-08T06:16:32.078839+0000) 2024-05-08T06:16:45.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:45 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:45.138+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-05-08T06:16:15.981053+0000 front 2024-05-08T06:16:15.981041+0000 (oldest deadline 2024-05-08T06:16:42.381201+0000) 2024-05-08T06:16:45.951 INFO:journalctl@ceph.osd.9.smithi123.stdout:May 08 06:16:45 smithi123 podman[96583]: 2024-05-08 06:16:45.647400138 +0000 UTC m=+0.032235442 container died 7c5aa8a128f544e12e549212ca92b37e88660a77c8bc04c6d79c3d3bc71b169e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-08T06:16:46.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:45 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:45.577+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:46.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:45 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:45.577+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:46.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:45 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:45.577+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:46.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:45 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:45.577+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:16:00.051637+0000 front 2024-05-08T06:16:00.051514+0000 (oldest deadline 2024-05-08T06:16:22.951535+0000) 2024-05-08T06:16:46.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:45 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:45.577+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:04.652467+0000 front 2024-05-08T06:16:04.652398+0000 (oldest deadline 2024-05-08T06:16:30.552552+0000) 2024-05-08T06:16:46.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:45 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:45.577+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-05-08T06:16:14.454557+0000 front 2024-05-08T06:16:14.454569+0000 (oldest deadline 2024-05-08T06:16:39.154226+0000) 2024-05-08T06:16:46.457 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@osd.9.service' 2024-05-08T06:16:46.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:46 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:46.145+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:46.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:46 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:46.145+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:46.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:46 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:46.145+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:46.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:46 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:46.145+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:47.576642+0000 front 2024-05-08T06:15:47.576333+0000 (oldest deadline 2024-05-08T06:16:15.176725+0000) 2024-05-08T06:16:46.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:46 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:46.145+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.677497+0000 front 2024-05-08T06:15:58.677473+0000 (oldest deadline 2024-05-08T06:16:22.177585+0000) 2024-05-08T06:16:46.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:46 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:46.145+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:06.178933+0000 front 2024-05-08T06:16:06.178920+0000 (oldest deadline 2024-05-08T06:16:32.078839+0000) 2024-05-08T06:16:46.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:46 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:46.145+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-05-08T06:16:15.981053+0000 front 2024-05-08T06:16:15.981041+0000 (oldest deadline 2024-05-08T06:16:42.381201+0000) 2024-05-08T06:16:46.553 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:16:46.554 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-08T06:16:46.554 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-08T06:16:46.554 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@osd.10 2024-05-08T06:16:46.849 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:46 smithi163 systemd[1]: Stopping Ceph osd.10 for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T06:16:46.849 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:46 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:46.849+0000 7f931accd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:16:46.849 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:46 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:46.849+0000 7f931accd700 -1 osd.10 85 *** Got signal Terminated *** 2024-05-08T06:16:46.849 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:46 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:46.849+0000 7f931accd700 -1 osd.10 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T06:16:47.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:46 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:46.558+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:47.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:46 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:46.558+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:47.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:46 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:46.558+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:47.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:46 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:46.558+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:16:00.051637+0000 front 2024-05-08T06:16:00.051514+0000 (oldest deadline 2024-05-08T06:16:22.951535+0000) 2024-05-08T06:16:47.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:46 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:46.558+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:04.652467+0000 front 2024-05-08T06:16:04.652398+0000 (oldest deadline 2024-05-08T06:16:30.552552+0000) 2024-05-08T06:16:47.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:46 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:46.558+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-05-08T06:16:14.454557+0000 front 2024-05-08T06:16:14.454569+0000 (oldest deadline 2024-05-08T06:16:39.154226+0000) 2024-05-08T06:16:47.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:47 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:47.102+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:47.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:47 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:47.102+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:47.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:47 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:47.102+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:47.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:47 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:47.102+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:47.576642+0000 front 2024-05-08T06:15:47.576333+0000 (oldest deadline 2024-05-08T06:16:15.176725+0000) 2024-05-08T06:16:47.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:47 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:47.102+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.677497+0000 front 2024-05-08T06:15:58.677473+0000 (oldest deadline 2024-05-08T06:16:22.177585+0000) 2024-05-08T06:16:47.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:47 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:47.102+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:06.178933+0000 front 2024-05-08T06:16:06.178920+0000 (oldest deadline 2024-05-08T06:16:32.078839+0000) 2024-05-08T06:16:47.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:47 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:47.102+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-05-08T06:16:15.981053+0000 front 2024-05-08T06:16:15.981041+0000 (oldest deadline 2024-05-08T06:16:42.381201+0000) 2024-05-08T06:16:47.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:47 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:47.531+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:47.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:47 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:47.531+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:47.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:47 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:47.531+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:47.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:47 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:47.531+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:16:00.051637+0000 front 2024-05-08T06:16:00.051514+0000 (oldest deadline 2024-05-08T06:16:22.951535+0000) 2024-05-08T06:16:47.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:47 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:47.531+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:04.652467+0000 front 2024-05-08T06:16:04.652398+0000 (oldest deadline 2024-05-08T06:16:30.552552+0000) 2024-05-08T06:16:47.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:47 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:47.531+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-05-08T06:16:14.454557+0000 front 2024-05-08T06:16:14.454569+0000 (oldest deadline 2024-05-08T06:16:39.154226+0000) 2024-05-08T06:16:48.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:48 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:48.141+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:48.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:48 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:48.141+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:48.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:48 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:48.141+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:48.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:48 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:48.141+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:47.576642+0000 front 2024-05-08T06:15:47.576333+0000 (oldest deadline 2024-05-08T06:16:15.176725+0000) 2024-05-08T06:16:48.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:48 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:48.141+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.677497+0000 front 2024-05-08T06:15:58.677473+0000 (oldest deadline 2024-05-08T06:16:22.177585+0000) 2024-05-08T06:16:48.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:48 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:48.141+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:06.178933+0000 front 2024-05-08T06:16:06.178920+0000 (oldest deadline 2024-05-08T06:16:32.078839+0000) 2024-05-08T06:16:48.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:48 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:48.141+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-05-08T06:16:15.981053+0000 front 2024-05-08T06:16:15.981041+0000 (oldest deadline 2024-05-08T06:16:42.381201+0000) 2024-05-08T06:16:49.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:48 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:48.555+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:49.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:48 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:48.555+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:49.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:48 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:48.555+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:49.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:48 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:48.555+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:16:00.051637+0000 front 2024-05-08T06:16:00.051514+0000 (oldest deadline 2024-05-08T06:16:22.951535+0000) 2024-05-08T06:16:49.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:48 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:48.555+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:04.652467+0000 front 2024-05-08T06:16:04.652398+0000 (oldest deadline 2024-05-08T06:16:30.552552+0000) 2024-05-08T06:16:49.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:48 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:48.555+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-05-08T06:16:14.454557+0000 front 2024-05-08T06:16:14.454569+0000 (oldest deadline 2024-05-08T06:16:39.154226+0000) 2024-05-08T06:16:49.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:49 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:49.186+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:49.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:49 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:49.186+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:49.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:49 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:49.186+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:49.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:49 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:49.186+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:47.576642+0000 front 2024-05-08T06:15:47.576333+0000 (oldest deadline 2024-05-08T06:16:15.176725+0000) 2024-05-08T06:16:49.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:49 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:49.186+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.677497+0000 front 2024-05-08T06:15:58.677473+0000 (oldest deadline 2024-05-08T06:16:22.177585+0000) 2024-05-08T06:16:49.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:49 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:49.186+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:06.178933+0000 front 2024-05-08T06:16:06.178920+0000 (oldest deadline 2024-05-08T06:16:32.078839+0000) 2024-05-08T06:16:49.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:49 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:49.186+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-05-08T06:16:15.981053+0000 front 2024-05-08T06:16:15.981041+0000 (oldest deadline 2024-05-08T06:16:42.381201+0000) 2024-05-08T06:16:49.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:49 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:49.510+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:49.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:49 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:49.510+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:49.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:49 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:49.510+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:49.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:49 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:49.510+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:16:00.051637+0000 front 2024-05-08T06:16:00.051514+0000 (oldest deadline 2024-05-08T06:16:22.951535+0000) 2024-05-08T06:16:49.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:49 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:49.510+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:04.652467+0000 front 2024-05-08T06:16:04.652398+0000 (oldest deadline 2024-05-08T06:16:30.552552+0000) 2024-05-08T06:16:49.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:49 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:49.510+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-05-08T06:16:14.454557+0000 front 2024-05-08T06:16:14.454569+0000 (oldest deadline 2024-05-08T06:16:39.154226+0000) 2024-05-08T06:16:50.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:50 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:50.151+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:50.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:50 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:50.151+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:50.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:50 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:50.151+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:50.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:50 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:50.151+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:47.576642+0000 front 2024-05-08T06:15:47.576333+0000 (oldest deadline 2024-05-08T06:16:15.176725+0000) 2024-05-08T06:16:50.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:50 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:50.151+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.677497+0000 front 2024-05-08T06:15:58.677473+0000 (oldest deadline 2024-05-08T06:16:22.177585+0000) 2024-05-08T06:16:50.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:50 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:50.151+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:06.178933+0000 front 2024-05-08T06:16:06.178920+0000 (oldest deadline 2024-05-08T06:16:32.078839+0000) 2024-05-08T06:16:50.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:50 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:50.151+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-05-08T06:16:15.981053+0000 front 2024-05-08T06:16:15.981041+0000 (oldest deadline 2024-05-08T06:16:42.381201+0000) 2024-05-08T06:16:50.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:50 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:50.461+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:50.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:50 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:50.461+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:50.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:50 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:50.461+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:50.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:50 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:50.461+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:16:00.051637+0000 front 2024-05-08T06:16:00.051514+0000 (oldest deadline 2024-05-08T06:16:22.951535+0000) 2024-05-08T06:16:50.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:50 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:50.461+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:04.652467+0000 front 2024-05-08T06:16:04.652398+0000 (oldest deadline 2024-05-08T06:16:30.552552+0000) 2024-05-08T06:16:50.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:50 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:50.461+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-05-08T06:16:14.454557+0000 front 2024-05-08T06:16:14.454569+0000 (oldest deadline 2024-05-08T06:16:39.154226+0000) 2024-05-08T06:16:50.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:50 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:50.462+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6820 osd.7 since back 2024-05-08T06:16:22.655162+0000 front 2024-05-08T06:16:22.655245+0000 (oldest deadline 2024-05-08T06:16:49.655496+0000) 2024-05-08T06:16:51.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:51 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:51.199+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.773393+0000 front 2024-05-08T06:15:24.773505+0000 (oldest deadline 2024-05-08T06:15:47.673192+0000) 2024-05-08T06:16:51.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:51 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:51.199+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:34.074441+0000 front 2024-05-08T06:15:34.074462+0000 (oldest deadline 2024-05-08T06:15:59.974572+0000) 2024-05-08T06:16:51.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:51 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:51.199+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.275505+0000 front 2024-05-08T06:15:42.275657+0000 (oldest deadline 2024-05-08T06:16:07.575651+0000) 2024-05-08T06:16:51.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:51 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:51.199+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6814 osd.3 since back 2024-05-08T06:15:47.576642+0000 front 2024-05-08T06:15:47.576333+0000 (oldest deadline 2024-05-08T06:16:15.176725+0000) 2024-05-08T06:16:51.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:51 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:51.199+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:15:58.677497+0000 front 2024-05-08T06:15:58.677473+0000 (oldest deadline 2024-05-08T06:16:22.177585+0000) 2024-05-08T06:16:51.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:51 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:51.199+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:06.178933+0000 front 2024-05-08T06:16:06.178920+0000 (oldest deadline 2024-05-08T06:16:32.078839+0000) 2024-05-08T06:16:51.539 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:51 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10[59671]: 2024-05-08T06:16:51.199+0000 7f93132b9700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-05-08T06:16:15.981053+0000 front 2024-05-08T06:16:15.981041+0000 (oldest deadline 2024-05-08T06:16:42.381201+0000) 2024-05-08T06:16:51.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:51 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:51.502+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:51.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:51 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:51.502+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:51.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:51 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:51.502+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:51.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:51 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:51.502+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:16:00.051637+0000 front 2024-05-08T06:16:00.051514+0000 (oldest deadline 2024-05-08T06:16:22.951535+0000) 2024-05-08T06:16:51.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:51 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:51.502+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:04.652467+0000 front 2024-05-08T06:16:04.652398+0000 (oldest deadline 2024-05-08T06:16:30.552552+0000) 2024-05-08T06:16:51.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:51 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:51.502+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-05-08T06:16:14.454557+0000 front 2024-05-08T06:16:14.454569+0000 (oldest deadline 2024-05-08T06:16:39.154226+0000) 2024-05-08T06:16:51.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:51 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:51.502+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6820 osd.7 since back 2024-05-08T06:16:22.655162+0000 front 2024-05-08T06:16:22.655245+0000 (oldest deadline 2024-05-08T06:16:49.655496+0000) 2024-05-08T06:16:52.538 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:52 smithi163 podman[69379]: 2024-05-08 06:16:52.25435231 +0000 UTC m=+5.509004397 container died 4bdbdd3fe5d1c0c34d6733bd7be7ce7d12ff12c46279ee6925c23e7b077490f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True) 2024-05-08T06:16:52.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:52 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:52.541+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:52.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:52 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:52.541+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:52.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:52 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:52.541+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:52.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:52 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:52.541+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:16:00.051637+0000 front 2024-05-08T06:16:00.051514+0000 (oldest deadline 2024-05-08T06:16:22.951535+0000) 2024-05-08T06:16:52.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:52 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:52.541+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:04.652467+0000 front 2024-05-08T06:16:04.652398+0000 (oldest deadline 2024-05-08T06:16:30.552552+0000) 2024-05-08T06:16:52.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:52 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:52.541+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-05-08T06:16:14.454557+0000 front 2024-05-08T06:16:14.454569+0000 (oldest deadline 2024-05-08T06:16:39.154226+0000) 2024-05-08T06:16:52.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:52 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:52.541+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6820 osd.7 since back 2024-05-08T06:16:22.655162+0000 front 2024-05-08T06:16:22.655245+0000 (oldest deadline 2024-05-08T06:16:49.655496+0000) 2024-05-08T06:16:53.038 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:52 smithi163 podman[69379]: 2024-05-08 06:16:52.561867247 +0000 UTC m=+5.816519342 container cleanup 4bdbdd3fe5d1c0c34d6733bd7be7ce7d12ff12c46279ee6925c23e7b077490f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD) 2024-05-08T06:16:53.038 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:52 smithi163 bash[69379]: ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10 2024-05-08T06:16:53.038 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:52 smithi163 podman[69396]: 2024-05-08 06:16:52.767431456 +0000 UTC m=+0.508858559 container remove 4bdbdd3fe5d1c0c34d6733bd7be7ce7d12ff12c46279ee6925c23e7b077490f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0) 2024-05-08T06:16:53.386 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:53 smithi163 podman[69485]: 2024-05-08 06:16:53.003639355 +0000 UTC m=+0.018415831 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:16:53.387 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:53 smithi163 podman[69485]: 2024-05-08 06:16:53.118380412 +0000 UTC m=+0.133156885 container create 7abd6370e10e3e739c0cff57ee1e2ab3d5c25aecad3c38b6ac035256d18c210b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, ceph=True) 2024-05-08T06:16:53.387 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:53 smithi163 podman[69485]: 2024-05-08 06:16:53.296428755 +0000 UTC m=+0.311205230 container init 7abd6370e10e3e739c0cff57ee1e2ab3d5c25aecad3c38b6ac035256d18c210b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T06:16:53.387 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:53 smithi163 podman[69485]: 2024-05-08 06:16:53.299451412 +0000 UTC m=+0.314227886 container start 7abd6370e10e3e739c0cff57ee1e2ab3d5c25aecad3c38b6ac035256d18c210b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-08T06:16:53.387 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:53 smithi163 podman[69485]: 2024-05-08 06:16:53.387496544 +0000 UTC m=+0.402273035 container attach 7abd6370e10e3e739c0cff57ee1e2ab3d5c25aecad3c38b6ac035256d18c210b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507) 2024-05-08T06:16:53.788 INFO:journalctl@ceph.osd.10.smithi163.stdout:May 08 06:16:53 smithi163 podman[69485]: 2024-05-08 06:16:53.594261925 +0000 UTC m=+0.609038411 container died 7abd6370e10e3e739c0cff57ee1e2ab3d5c25aecad3c38b6ac035256d18c210b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T06:16:53.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:53 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:53.539+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:53.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:53 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:53.539+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:53.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:53 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:53.539+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:53.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:53 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:53.539+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:16:00.051637+0000 front 2024-05-08T06:16:00.051514+0000 (oldest deadline 2024-05-08T06:16:22.951535+0000) 2024-05-08T06:16:53.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:53 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:53.539+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:04.652467+0000 front 2024-05-08T06:16:04.652398+0000 (oldest deadline 2024-05-08T06:16:30.552552+0000) 2024-05-08T06:16:53.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:53 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:53.539+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-05-08T06:16:14.454557+0000 front 2024-05-08T06:16:14.454569+0000 (oldest deadline 2024-05-08T06:16:39.154226+0000) 2024-05-08T06:16:53.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:53 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:53.540+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6820 osd.7 since back 2024-05-08T06:16:22.655162+0000 front 2024-05-08T06:16:22.655245+0000 (oldest deadline 2024-05-08T06:16:49.655496+0000) 2024-05-08T06:16:54.147 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@osd.10.service' 2024-05-08T06:16:54.183 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:16:54.184 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-08T06:16:54.184 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-08T06:16:54.184 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@osd.11 2024-05-08T06:16:54.506 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:54 smithi192 systemd[1]: Stopping Ceph osd.11 for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T06:16:54.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:54 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:54.507+0000 7f06d689f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:16:54.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:54 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:54.507+0000 7f06d689f700 -1 osd.11 85 *** Got signal Terminated *** 2024-05-08T06:16:54.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:54 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:54.507+0000 7f06d689f700 -1 osd.11 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T06:16:54.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:54 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:54.543+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:54.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:54 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:54.543+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:54.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:54 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:54.543+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:54.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:54 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:54.543+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:16:00.051637+0000 front 2024-05-08T06:16:00.051514+0000 (oldest deadline 2024-05-08T06:16:22.951535+0000) 2024-05-08T06:16:54.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:54 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:54.543+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:04.652467+0000 front 2024-05-08T06:16:04.652398+0000 (oldest deadline 2024-05-08T06:16:30.552552+0000) 2024-05-08T06:16:54.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:54 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:54.543+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-05-08T06:16:14.454557+0000 front 2024-05-08T06:16:14.454569+0000 (oldest deadline 2024-05-08T06:16:39.154226+0000) 2024-05-08T06:16:54.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:54 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:54.543+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6820 osd.7 since back 2024-05-08T06:16:22.655162+0000 front 2024-05-08T06:16:22.655245+0000 (oldest deadline 2024-05-08T06:16:49.655496+0000) 2024-05-08T06:16:55.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:55 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:55.522+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:55.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:55 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:55.522+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:55.800 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:55 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:55.522+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:55.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:55 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:55.522+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:16:00.051637+0000 front 2024-05-08T06:16:00.051514+0000 (oldest deadline 2024-05-08T06:16:22.951535+0000) 2024-05-08T06:16:55.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:55 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:55.522+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:04.652467+0000 front 2024-05-08T06:16:04.652398+0000 (oldest deadline 2024-05-08T06:16:30.552552+0000) 2024-05-08T06:16:55.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:55 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:55.522+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-05-08T06:16:14.454557+0000 front 2024-05-08T06:16:14.454569+0000 (oldest deadline 2024-05-08T06:16:39.154226+0000) 2024-05-08T06:16:55.801 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:55 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:55.522+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6820 osd.7 since back 2024-05-08T06:16:22.655162+0000 front 2024-05-08T06:16:22.655245+0000 (oldest deadline 2024-05-08T06:16:49.655496+0000) 2024-05-08T06:16:57.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:56 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:56.569+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:57.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:56 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:56.569+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:57.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:56 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:56.569+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:57.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:56 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:56.569+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:16:00.051637+0000 front 2024-05-08T06:16:00.051514+0000 (oldest deadline 2024-05-08T06:16:22.951535+0000) 2024-05-08T06:16:57.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:56 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:56.569+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:04.652467+0000 front 2024-05-08T06:16:04.652398+0000 (oldest deadline 2024-05-08T06:16:30.552552+0000) 2024-05-08T06:16:57.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:56 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:56.569+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-05-08T06:16:14.454557+0000 front 2024-05-08T06:16:14.454569+0000 (oldest deadline 2024-05-08T06:16:39.154226+0000) 2024-05-08T06:16:57.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:56 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:56.569+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6820 osd.7 since back 2024-05-08T06:16:22.655162+0000 front 2024-05-08T06:16:22.655245+0000 (oldest deadline 2024-05-08T06:16:49.655496+0000) 2024-05-08T06:16:57.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:56 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:56.569+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6820 osd.8 since back 2024-05-08T06:16:29.655768+0000 front 2024-05-08T06:16:29.655784+0000 (oldest deadline 2024-05-08T06:16:55.555947+0000) 2024-05-08T06:16:58.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:57 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:57.557+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:58.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:57 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:57.557+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:58.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:57 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:57.557+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:58.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:57 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:57.557+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:16:00.051637+0000 front 2024-05-08T06:16:00.051514+0000 (oldest deadline 2024-05-08T06:16:22.951535+0000) 2024-05-08T06:16:58.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:57 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:57.557+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:04.652467+0000 front 2024-05-08T06:16:04.652398+0000 (oldest deadline 2024-05-08T06:16:30.552552+0000) 2024-05-08T06:16:58.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:57 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:57.557+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-05-08T06:16:14.454557+0000 front 2024-05-08T06:16:14.454569+0000 (oldest deadline 2024-05-08T06:16:39.154226+0000) 2024-05-08T06:16:58.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:57 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:57.557+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6820 osd.7 since back 2024-05-08T06:16:22.655162+0000 front 2024-05-08T06:16:22.655245+0000 (oldest deadline 2024-05-08T06:16:49.655496+0000) 2024-05-08T06:16:58.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:57 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:57.557+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6820 osd.8 since back 2024-05-08T06:16:29.655768+0000 front 2024-05-08T06:16:29.655784+0000 (oldest deadline 2024-05-08T06:16:55.555947+0000) 2024-05-08T06:16:59.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:58 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:58.591+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6806 osd.0 since back 2024-05-08T06:15:24.545702+0000 front 2024-05-08T06:15:24.545570+0000 (oldest deadline 2024-05-08T06:15:49.245549+0000) 2024-05-08T06:16:59.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:58 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:58.591+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6804 osd.1 since back 2024-05-08T06:15:35.647366+0000 front 2024-05-08T06:15:35.647284+0000 (oldest deadline 2024-05-08T06:16:01.547422+0000) 2024-05-08T06:16:59.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:58 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:58.591+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-05-08T06:15:42.048392+0000 front 2024-05-08T06:15:42.048272+0000 (oldest deadline 2024-05-08T06:16:04.348382+0000) 2024-05-08T06:16:59.050 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:58 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:58.591+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6812 osd.4 since back 2024-05-08T06:16:00.051637+0000 front 2024-05-08T06:16:00.051514+0000 (oldest deadline 2024-05-08T06:16:22.951535+0000) 2024-05-08T06:16:59.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:58 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:58.591+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-05-08T06:16:04.652467+0000 front 2024-05-08T06:16:04.652398+0000 (oldest deadline 2024-05-08T06:16:30.552552+0000) 2024-05-08T06:16:59.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:58 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:58.591+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.123:6822 osd.6 since back 2024-05-08T06:16:14.454557+0000 front 2024-05-08T06:16:14.454569+0000 (oldest deadline 2024-05-08T06:16:39.154226+0000) 2024-05-08T06:16:59.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:58 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:58.591+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.163:6820 osd.7 since back 2024-05-08T06:16:22.655162+0000 front 2024-05-08T06:16:22.655245+0000 (oldest deadline 2024-05-08T06:16:49.655496+0000) 2024-05-08T06:16:59.051 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:58 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11[60456]: 2024-05-08T06:16:58.591+0000 7f06cee8b700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.192:6820 osd.8 since back 2024-05-08T06:16:29.655768+0000 front 2024-05-08T06:16:29.655784+0000 (oldest deadline 2024-05-08T06:16:55.555947+0000) 2024-05-08T06:17:00.147 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:16:59 smithi192 podman[70100]: 2024-05-08 06:16:59.828112008 +0000 UTC m=+5.444426586 container died e1982ccf0aa30eb71ead1afbccf1965b62581bf694818f96fe411c32f26ca396 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-08T06:17:00.147 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:17:00 smithi192 podman[70100]: 2024-05-08 06:17:00.148147384 +0000 UTC m=+5.764461963 container cleanup e1982ccf0aa30eb71ead1afbccf1965b62581bf694818f96fe411c32f26ca396 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T06:17:00.550 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:17:00 smithi192 bash[70100]: ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11 2024-05-08T06:17:00.550 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:17:00 smithi192 podman[70118]: 2024-05-08 06:17:00.349297252 +0000 UTC m=+0.524516825 container remove e1982ccf0aa30eb71ead1afbccf1965b62581bf694818f96fe411c32f26ca396 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-08T06:17:01.034 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:17:00 smithi192 podman[70207]: 2024-05-08 06:17:00.758340578 +0000 UTC m=+0.108838057 container create ab44c005a4adc037b8e4d498c306a4a7eab06e2509c014e24be6adf38b12aa95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-08T06:17:01.035 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:17:00 smithi192 podman[70207]: 2024-05-08 06:17:00.667146908 +0000 UTC m=+0.017644354 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:17:01.035 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:17:00 smithi192 podman[70207]: 2024-05-08 06:17:00.977547929 +0000 UTC m=+0.328045371 container init ab44c005a4adc037b8e4d498c306a4a7eab06e2509c014e24be6adf38b12aa95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-08T06:17:01.035 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:17:00 smithi192 podman[70207]: 2024-05-08 06:17:00.98105294 +0000 UTC m=+0.331550376 container start ab44c005a4adc037b8e4d498c306a4a7eab06e2509c014e24be6adf38b12aa95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11-deactivate, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=) 2024-05-08T06:17:01.035 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:17:01 smithi192 podman[70207]: 2024-05-08 06:17:01.035370351 +0000 UTC m=+0.385867795 container attach ab44c005a4adc037b8e4d498c306a4a7eab06e2509c014e24be6adf38b12aa95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_CLEAN=True, ceph=True, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T06:17:01.300 INFO:journalctl@ceph.osd.11.smithi192.stdout:May 08 06:17:01 smithi192 podman[70207]: 2024-05-08 06:17:01.283424683 +0000 UTC m=+0.633922120 container died ab44c005a4adc037b8e4d498c306a4a7eab06e2509c014e24be6adf38b12aa95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-08T06:17:01.823 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@osd.11.service' 2024-05-08T06:17:01.860 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:17:01.861 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-08T06:17:01.861 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-08T06:17:01.861 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mds.a 2024-05-08T06:17:02.205 INFO:journalctl@ceph.mds.a.smithi123.stdout:May 08 06:17:01 smithi123 systemd[1]: Stopping Ceph mds.a for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T06:17:02.705 INFO:journalctl@ceph.mds.a.smithi123.stdout:May 08 06:17:02 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-a[71273]: 2024-05-08T06:17:02.450+0000 7f97f7e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:17:02.705 INFO:journalctl@ceph.mds.a.smithi123.stdout:May 08 06:17:02 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-a[71273]: 2024-05-08T06:17:02.450+0000 7f97f7e00700 -1 mds.a *** got signal Terminated *** 2024-05-08T06:17:06.787 INFO:journalctl@ceph.mds.a.smithi123.stdout:May 08 06:17:06 smithi123 podman[96670]: 2024-05-08 06:17:06.469125691 +0000 UTC m=+4.188477058 container died daf390136276f52772900b181b2ebb6da473dd321aa6044ae27bf3dfb450ca07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507) 2024-05-08T06:17:07.046 INFO:journalctl@ceph.mds.a.smithi123.stdout:May 08 06:17:07 smithi123 podman[96670]: 2024-05-08 06:17:07.015700056 +0000 UTC m=+4.735051453 container cleanup daf390136276f52772900b181b2ebb6da473dd321aa6044ae27bf3dfb450ca07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-08T06:17:07.046 INFO:journalctl@ceph.mds.a.smithi123.stdout:May 08 06:17:07 smithi123 bash[96670]: ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-a 2024-05-08T06:17:07.705 INFO:journalctl@ceph.mds.a.smithi123.stdout:May 08 06:17:07 smithi123 podman[96684]: 2024-05-08 06:17:07.408991841 +0000 UTC m=+0.924736835 container remove daf390136276f52772900b181b2ebb6da473dd321aa6044ae27bf3dfb450ca07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-a, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-08T06:17:07.915 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mds.a.service' 2024-05-08T06:17:08.023 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:17:08.023 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-08T06:17:08.023 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-08T06:17:08.023 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mds.d 2024-05-08T06:17:08.205 INFO:journalctl@ceph.mds.d.smithi123.stdout:May 08 06:17:08 smithi123 systemd[1]: Stopping Ceph mds.d for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T06:17:08.955 INFO:journalctl@ceph.mds.d.smithi123.stdout:May 08 06:17:08 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-d[70097]: 2024-05-08T06:17:08.584+0000 7f7528a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:17:08.955 INFO:journalctl@ceph.mds.d.smithi123.stdout:May 08 06:17:08 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-d[70097]: 2024-05-08T06:17:08.584+0000 7f7528a00700 -1 mds.d *** got signal Terminated *** 2024-05-08T06:17:10.374 INFO:journalctl@ceph.mds.d.smithi123.stdout:May 08 06:17:10 smithi123 podman[96789]: 2024-05-08 06:17:10.069320839 +0000 UTC m=+1.613974704 container died 781cf0787d16ef22888fed6f87d22dce3c3cfb75687b6f76a3ddcaa9f69c761c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-d, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-08T06:17:10.625 INFO:journalctl@ceph.mds.d.smithi123.stdout:May 08 06:17:10 smithi123 podman[96789]: 2024-05-08 06:17:10.619462798 +0000 UTC m=+2.164116671 container cleanup 781cf0787d16ef22888fed6f87d22dce3c3cfb75687b6f76a3ddcaa9f69c761c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-d, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-08T06:17:10.625 INFO:journalctl@ceph.mds.d.smithi123.stdout:May 08 06:17:10 smithi123 bash[96789]: ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-d 2024-05-08T06:17:11.205 INFO:journalctl@ceph.mds.d.smithi123.stdout:May 08 06:17:10 smithi123 podman[96803]: 2024-05-08 06:17:10.929159895 +0000 UTC m=+0.839909097 container remove 781cf0787d16ef22888fed6f87d22dce3c3cfb75687b6f76a3ddcaa9f69c761c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T06:17:11.456 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mds.d.service' 2024-05-08T06:17:11.564 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:17:11.564 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-08T06:17:11.564 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-08T06:17:11.564 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mds.g 2024-05-08T06:17:12.155 INFO:journalctl@ceph.mds.g.smithi123.stdout:May 08 06:17:11 smithi123 systemd[1]: Stopping Ceph mds.g for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T06:17:12.455 INFO:journalctl@ceph.mds.g.smithi123.stdout:May 08 06:17:12 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-g[70874]: 2024-05-08T06:17:12.154+0000 7f4288600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:17:12.455 INFO:journalctl@ceph.mds.g.smithi123.stdout:May 08 06:17:12 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-g[70874]: 2024-05-08T06:17:12.154+0000 7f4288600700 -1 mds.g *** got signal Terminated *** 2024-05-08T06:17:14.069 INFO:journalctl@ceph.mds.g.smithi123.stdout:May 08 06:17:13 smithi123 podman[96910]: 2024-05-08 06:17:13.742698459 +0000 UTC m=+1.747617617 container died 18d9e4aac5d989c69b945c509096468800ed63b867bef9622a1e6c3903954993 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-g, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=) 2024-05-08T06:17:14.455 INFO:journalctl@ceph.mds.g.smithi123.stdout:May 08 06:17:14 smithi123 podman[96910]: 2024-05-08 06:17:14.273668464 +0000 UTC m=+2.278587632 container cleanup 18d9e4aac5d989c69b945c509096468800ed63b867bef9622a1e6c3903954993 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-g, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-08T06:17:14.455 INFO:journalctl@ceph.mds.g.smithi123.stdout:May 08 06:17:14 smithi123 bash[96910]: ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-g 2024-05-08T06:17:14.955 INFO:journalctl@ceph.mds.g.smithi123.stdout:May 08 06:17:14 smithi123 podman[96924]: 2024-05-08 06:17:14.508565811 +0000 UTC m=+0.749467306 container remove 18d9e4aac5d989c69b945c509096468800ed63b867bef9622a1e6c3903954993 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-g, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507) 2024-05-08T06:17:15.054 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mds.g.service' 2024-05-08T06:17:15.235 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:17:15.236 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-08T06:17:15.236 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-08T06:17:15.236 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mds.j 2024-05-08T06:17:15.705 INFO:journalctl@ceph.mds.j.smithi123.stdout:May 08 06:17:15 smithi123 systemd[1]: Stopping Ceph mds.j for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T06:17:16.205 INFO:journalctl@ceph.mds.j.smithi123.stdout:May 08 06:17:15 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-j[69685]: 2024-05-08T06:17:15.790+0000 7fbca8200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:17:16.205 INFO:journalctl@ceph.mds.j.smithi123.stdout:May 08 06:17:15 smithi123 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-j[69685]: 2024-05-08T06:17:15.790+0000 7fbca8200700 -1 mds.j *** got signal Terminated *** 2024-05-08T06:17:17.398 INFO:journalctl@ceph.mds.j.smithi123.stdout:May 08 06:17:17 smithi123 podman[97035]: 2024-05-08 06:17:17.046011731 +0000 UTC m=+1.410120586 container died 4b34666c1f3629a84af127336806c575b608c4d522179b618676b8077ddc1961 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-j, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-08T06:17:17.705 INFO:journalctl@ceph.mds.j.smithi123.stdout:May 08 06:17:17 smithi123 podman[97035]: 2024-05-08 06:17:17.594310943 +0000 UTC m=+1.958419774 container cleanup 4b34666c1f3629a84af127336806c575b608c4d522179b618676b8077ddc1961 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-j, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-08T06:17:17.705 INFO:journalctl@ceph.mds.j.smithi123.stdout:May 08 06:17:17 smithi123 bash[97035]: ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-j 2024-05-08T06:17:18.205 INFO:journalctl@ceph.mds.j.smithi123.stdout:May 08 06:17:17 smithi123 podman[97056]: 2024-05-08 06:17:17.845407739 +0000 UTC m=+0.782938552 container remove 4b34666c1f3629a84af127336806c575b608c4d522179b618676b8077ddc1961 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-j, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-08T06:17:18.355 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mds.j.service' 2024-05-08T06:17:18.464 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:17:18.464 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-08T06:17:18.465 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-08T06:17:18.465 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mds.b 2024-05-08T06:17:18.766 INFO:journalctl@ceph.mds.b.smithi163.stdout:May 08 06:17:18 smithi163 systemd[1]: Stopping Ceph mds.b for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T06:17:19.038 INFO:journalctl@ceph.mds.b.smithi163.stdout:May 08 06:17:18 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-b[65037]: 2024-05-08T06:17:18.766+0000 7f9f2e35d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:17:19.038 INFO:journalctl@ceph.mds.b.smithi163.stdout:May 08 06:17:18 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-b[65037]: 2024-05-08T06:17:18.766+0000 7f9f2e35d700 -1 mds.b *** got signal Terminated *** 2024-05-08T06:17:23.038 INFO:journalctl@ceph.mds.b.smithi163.stdout:May 08 06:17:22 smithi163 podman[69591]: 2024-05-08 06:17:22.774063531 +0000 UTC m=+4.112835081 container died 222c13906a21dea588a675641b306a4ab79d1ea56abd064146c95193658bb259 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-08T06:17:23.309 INFO:journalctl@ceph.mds.b.smithi163.stdout:May 08 06:17:23 smithi163 podman[69591]: 2024-05-08 06:17:23.116661374 +0000 UTC m=+4.455432938 container cleanup 222c13906a21dea588a675641b306a4ab79d1ea56abd064146c95193658bb259 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507) 2024-05-08T06:17:23.309 INFO:journalctl@ceph.mds.b.smithi163.stdout:May 08 06:17:23 smithi163 bash[69591]: ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-b 2024-05-08T06:17:23.531 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mds.b.service' 2024-05-08T06:17:23.566 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:17:23.566 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-08T06:17:23.566 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-08T06:17:23.566 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mds.e 2024-05-08T06:17:23.895 INFO:journalctl@ceph.mds.e.smithi163.stdout:May 08 06:17:23 smithi163 systemd[1]: Stopping Ceph mds.e for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T06:17:24.288 INFO:journalctl@ceph.mds.e.smithi163.stdout:May 08 06:17:23 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-e[64415]: 2024-05-08T06:17:23.895+0000 7f4e803b9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:17:24.288 INFO:journalctl@ceph.mds.e.smithi163.stdout:May 08 06:17:23 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-e[64415]: 2024-05-08T06:17:23.895+0000 7f4e803b9700 -1 mds.e *** got signal Terminated *** 2024-05-08T06:17:25.588 INFO:journalctl@ceph.mds.e.smithi163.stdout:May 08 06:17:25 smithi163 podman[69726]: 2024-05-08 06:17:25.320325915 +0000 UTC m=+1.534434400 container died 7203fdc45ce121a996826bbc71bd21e6a8f510438d3dd0d663567f9e9da08cd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-e, ceph=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=) 2024-05-08T06:17:25.929 INFO:journalctl@ceph.mds.e.smithi163.stdout:May 08 06:17:25 smithi163 podman[69726]: 2024-05-08 06:17:25.671097829 +0000 UTC m=+1.885206302 container cleanup 7203fdc45ce121a996826bbc71bd21e6a8f510438d3dd0d663567f9e9da08cd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-e, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True) 2024-05-08T06:17:25.930 INFO:journalctl@ceph.mds.e.smithi163.stdout:May 08 06:17:25 smithi163 bash[69726]: ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-e 2024-05-08T06:17:26.098 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mds.e.service' 2024-05-08T06:17:26.130 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:17:26.130 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-08T06:17:26.130 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-08T06:17:26.131 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mds.h 2024-05-08T06:17:26.441 INFO:journalctl@ceph.mds.h.smithi163.stdout:May 08 06:17:26 smithi163 systemd[1]: Stopping Ceph mds.h for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T06:17:26.788 INFO:journalctl@ceph.mds.h.smithi163.stdout:May 08 06:17:26 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-h[64723]: 2024-05-08T06:17:26.441+0000 7f38602a6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:17:26.788 INFO:journalctl@ceph.mds.h.smithi163.stdout:May 08 06:17:26 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-h[64723]: 2024-05-08T06:17:26.441+0000 7f38602a6700 -1 mds.h *** got signal Terminated *** 2024-05-08T06:17:30.765 INFO:journalctl@ceph.mds.h.smithi163.stdout:May 08 06:17:30 smithi163 podman[69846]: 2024-05-08 06:17:30.447676206 +0000 UTC m=+4.101179738 container died ae165ce0973ef5b6f757325a4c249793fb30007b6633d3c114b68954f5a5bb62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-h, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-08T06:17:31.038 INFO:journalctl@ceph.mds.h.smithi163.stdout:May 08 06:17:30 smithi163 podman[69846]: 2024-05-08 06:17:30.766733526 +0000 UTC m=+4.420237057 container cleanup ae165ce0973ef5b6f757325a4c249793fb30007b6633d3c114b68954f5a5bb62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-h, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f) 2024-05-08T06:17:31.038 INFO:journalctl@ceph.mds.h.smithi163.stdout:May 08 06:17:30 smithi163 bash[69846]: ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-h 2024-05-08T06:17:31.038 INFO:journalctl@ceph.mds.h.smithi163.stdout:May 08 06:17:30 smithi163 podman[69860]: 2024-05-08 06:17:30.95124765 +0000 UTC m=+0.499326754 container remove ae165ce0973ef5b6f757325a4c249793fb30007b6633d3c114b68954f5a5bb62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T06:17:31.202 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mds.h.service' 2024-05-08T06:17:31.232 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:17:31.233 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-08T06:17:31.233 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-08T06:17:31.233 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mds.k 2024-05-08T06:17:31.788 INFO:journalctl@ceph.mds.k.smithi163.stdout:May 08 06:17:31 smithi163 systemd[1]: Stopping Ceph mds.k for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T06:17:31.788 INFO:journalctl@ceph.mds.k.smithi163.stdout:May 08 06:17:31 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-k[64098]: 2024-05-08T06:17:31.553+0000 7f676a705700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:17:31.788 INFO:journalctl@ceph.mds.k.smithi163.stdout:May 08 06:17:31 smithi163 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-k[64098]: 2024-05-08T06:17:31.553+0000 7f676a705700 -1 mds.k *** got signal Terminated *** 2024-05-08T06:17:35.836 INFO:journalctl@ceph.mds.k.smithi163.stdout:May 08 06:17:35 smithi163 podman[69970]: 2024-05-08 06:17:35.542436635 +0000 UTC m=+4.096019418 container died aafe0eeceb55efafd793cde31ed4c999b597055d96977065ebf905adb10c3e4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-k, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True) 2024-05-08T06:17:36.167 INFO:journalctl@ceph.mds.k.smithi163.stdout:May 08 06:17:35 smithi163 podman[69970]: 2024-05-08 06:17:35.836864475 +0000 UTC m=+4.390447271 container cleanup aafe0eeceb55efafd793cde31ed4c999b597055d96977065ebf905adb10c3e4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-k, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-08T06:17:36.167 INFO:journalctl@ceph.mds.k.smithi163.stdout:May 08 06:17:35 smithi163 bash[69970]: ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-k 2024-05-08T06:17:36.167 INFO:journalctl@ceph.mds.k.smithi163.stdout:May 08 06:17:36 smithi163 podman[69982]: 2024-05-08 06:17:36.05469356 +0000 UTC m=+0.505983272 container remove aafe0eeceb55efafd793cde31ed4c999b597055d96977065ebf905adb10c3e4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-k, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T06:17:36.181 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mds.k.service' 2024-05-08T06:17:36.212 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:17:36.212 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-08T06:17:36.213 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-08T06:17:36.213 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mds.c 2024-05-08T06:17:36.504 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 08 06:17:36 smithi192 systemd[1]: Stopping Ceph mds.c for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T06:17:36.801 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 08 06:17:36 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-c[65457]: 2024-05-08T06:17:36.502+0000 7fed0ac0b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:17:36.802 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 08 06:17:36 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-c[65457]: 2024-05-08T06:17:36.502+0000 7fed0ac0b700 -1 mds.c *** got signal Terminated *** 2024-05-08T06:17:39.050 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 08 06:17:38 smithi192 podman[70311]: 2024-05-08 06:17:38.93021192 +0000 UTC m=+2.523504062 container died 61baa468f6d082634a12e77df3d294e5a779b9de9505af78c9c28408fa70034d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-08T06:17:39.737 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 08 06:17:39 smithi192 podman[70311]: 2024-05-08 06:17:39.503944665 +0000 UTC m=+3.097236811 container cleanup 61baa468f6d082634a12e77df3d294e5a779b9de9505af78c9c28408fa70034d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T06:17:39.738 INFO:journalctl@ceph.mds.c.smithi192.stdout:May 08 06:17:39 smithi192 bash[70311]: ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-c 2024-05-08T06:17:39.877 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mds.c.service' 2024-05-08T06:17:39.913 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:17:39.913 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-08T06:17:39.913 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-08T06:17:39.913 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mds.f 2024-05-08T06:17:40.300 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 08 06:17:40 smithi192 systemd[1]: Stopping Ceph mds.f for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T06:17:40.800 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 08 06:17:40 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-f[65135]: 2024-05-08T06:17:40.339+0000 7f1afc960700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:17:40.800 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 08 06:17:40 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-f[65135]: 2024-05-08T06:17:40.339+0000 7f1afc960700 -1 mds.f *** got signal Terminated *** 2024-05-08T06:17:43.223 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 08 06:17:42 smithi192 podman[70444]: 2024-05-08 06:17:42.921609389 +0000 UTC m=+2.784877518 container died 67904393a6f4cd214ebcbcd064a173ad5316ccc7a474f8d4fdcee2507f9de675 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-08T06:17:43.223 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 08 06:17:43 smithi192 podman[70444]: 2024-05-08 06:17:43.224103293 +0000 UTC m=+3.087371422 container cleanup 67904393a6f4cd214ebcbcd064a173ad5316ccc7a474f8d4fdcee2507f9de675 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-57676-3-bee299f) 2024-05-08T06:17:43.528 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 08 06:17:43 smithi192 bash[70444]: ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-f 2024-05-08T06:17:43.528 INFO:journalctl@ceph.mds.f.smithi192.stdout:May 08 06:17:43 smithi192 podman[70458]: 2024-05-08 06:17:43.42510388 +0000 UTC m=+0.504146279 container remove 67904393a6f4cd214ebcbcd064a173ad5316ccc7a474f8d4fdcee2507f9de675 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-f, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-05-08T06:17:43.549 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mds.f.service' 2024-05-08T06:17:43.581 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:17:43.581 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-08T06:17:43.581 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-08T06:17:43.582 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mds.i 2024-05-08T06:17:43.800 INFO:journalctl@ceph.mds.i.smithi192.stdout:May 08 06:17:43 smithi192 systemd[1]: Stopping Ceph mds.i for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T06:17:44.300 INFO:journalctl@ceph.mds.i.smithi192.stdout:May 08 06:17:43 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-i[65766]: 2024-05-08T06:17:43.902+0000 7f85e9f1b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:17:44.300 INFO:journalctl@ceph.mds.i.smithi192.stdout:May 08 06:17:43 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-i[65766]: 2024-05-08T06:17:43.902+0000 7f85e9f1b700 -1 mds.i *** got signal Terminated *** 2024-05-08T06:17:47.300 INFO:journalctl@ceph.mds.i.smithi192.stdout:May 08 06:17:46 smithi192 podman[70566]: 2024-05-08 06:17:46.929258253 +0000 UTC m=+3.123200622 container died 58e62652450fed29bf744f85ca016d6faa802712571fb2978e96a6e7159aeee2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-i, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-08T06:17:47.800 INFO:journalctl@ceph.mds.i.smithi192.stdout:May 08 06:17:47 smithi192 podman[70566]: 2024-05-08 06:17:47.585474393 +0000 UTC m=+3.779416762 container cleanup 58e62652450fed29bf744f85ca016d6faa802712571fb2978e96a6e7159aeee2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-i, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-08T06:17:47.800 INFO:journalctl@ceph.mds.i.smithi192.stdout:May 08 06:17:47 smithi192 bash[70566]: ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-i 2024-05-08T06:17:48.019 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mds.i.service' 2024-05-08T06:17:48.051 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:17:48.051 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-08T06:17:48.051 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-08T06:17:48.051 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mds.l 2024-05-08T06:17:48.388 INFO:journalctl@ceph.mds.l.smithi192.stdout:May 08 06:17:48 smithi192 systemd[1]: Stopping Ceph mds.l for 85cfb2e6-0cff-11ef-bc97-c7b262605968... 2024-05-08T06:17:48.774 INFO:journalctl@ceph.mds.l.smithi192.stdout:May 08 06:17:48 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-l[64822]: 2024-05-08T06:17:48.388+0000 7f30d7464700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:17:48.774 INFO:journalctl@ceph.mds.l.smithi192.stdout:May 08 06:17:48 smithi192 ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-l[64822]: 2024-05-08T06:17:48.388+0000 7f30d7464700 -1 mds.l *** got signal Terminated *** 2024-05-08T06:17:52.772 INFO:journalctl@ceph.mds.l.smithi192.stdout:May 08 06:17:52 smithi192 podman[70687]: 2024-05-08 06:17:52.402544679 +0000 UTC m=+4.120905668 container died 9f56e97b86fe48a424c5af5d83b46338e4c82ba64f856a35248c1c0da853edd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-l, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f) 2024-05-08T06:17:53.038 INFO:journalctl@ceph.mds.l.smithi192.stdout:May 08 06:17:52 smithi192 podman[70687]: 2024-05-08 06:17:52.838776504 +0000 UTC m=+4.557137492 container cleanup 9f56e97b86fe48a424c5af5d83b46338e4c82ba64f856a35248c1c0da853edd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-l, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-08T06:17:53.039 INFO:journalctl@ceph.mds.l.smithi192.stdout:May 08 06:17:52 smithi192 bash[70687]: ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968-mds-l 2024-05-08T06:17:53.263 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-85cfb2e6-0cff-11ef-bc97-c7b262605968@mds.l.service' 2024-05-08T06:17:53.294 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:17:53.294 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-08T06:17:53.294 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-08T06:17:53.610 INFO:teuthology.orchestra.run.smithi123.stdout:Deleting cluster with fsid: 85cfb2e6-0cff-11ef-bc97-c7b262605968 2024-05-08T06:17:58.228 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-08T06:17:58.419 INFO:teuthology.orchestra.run.smithi163.stdout:Deleting cluster with fsid: 85cfb2e6-0cff-11ef-bc97-c7b262605968 2024-05-08T06:18:00.821 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-08T06:18:01.012 INFO:teuthology.orchestra.run.smithi192.stdout:Deleting cluster with fsid: 85cfb2e6-0cff-11ef-bc97-c7b262605968 2024-05-08T06:18:03.481 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T06:18:03.574 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T06:18:03.606 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T06:18:03.631 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-08T06:18:03.633 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697251/remote/smithi123/crash 2024-05-08T06:18:03.634 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/crash -- . 2024-05-08T06:18:03.740 INFO:teuthology.orchestra.run.smithi123.stderr:tar: /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-08T06:18:03.741 INFO:teuthology.orchestra.run.smithi123.stderr:tar: Error is not recoverable: exiting now 2024-05-08T06:18:03.745 DEBUG:teuthology.misc:Transferring archived files from smithi163:/var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697251/remote/smithi163/crash 2024-05-08T06:18:03.746 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/crash -- . 2024-05-08T06:18:03.788 INFO:teuthology.orchestra.run.smithi163.stderr:tar: /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-08T06:18:03.789 INFO:teuthology.orchestra.run.smithi163.stderr:tar: Error is not recoverable: exiting now 2024-05-08T06:18:03.791 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697251/remote/smithi192/crash 2024-05-08T06:18:03.792 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/crash -- . 2024-05-08T06:18:03.821 INFO:teuthology.orchestra.run.smithi192.stderr:tar: /var/lib/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-08T06:18:03.821 INFO:teuthology.orchestra.run.smithi192.stderr:tar: Error is not recoverable: exiting now 2024-05-08T06:18:03.822 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-08T06:18:03.823 DEBUG:teuthology.orchestra.run.smithi123:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-08T06:18:03.935 INFO:tasks.cephadm:Compressing logs... 2024-05-08T06:18:03.935 DEBUG:teuthology.orchestra.run.smithi123:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-08T06:18:03.939 DEBUG:teuthology.orchestra.run.smithi163:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-08T06:18:03.942 DEBUG:teuthology.orchestra.run.smithi192:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-08T06:18:03.966 INFO:teuthology.orchestra.run.smithi192.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-08T06:18:03.966 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph.log 2024-05-08T06:18:03.966 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-osd.2.log 2024-05-08T06:18:03.966 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose/var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph.log: -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mds.f.log 2024-05-08T06:18:03.967 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-08T06:18:03.967 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-osd.5.log 2024-05-08T06:18:03.967 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mds.c.log 2024-05-08T06:18:03.967 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph.audit.log: /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mon.c.log 2024-05-08T06:18:03.970 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mgr.z.log 2024-05-08T06:18:03.970 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- 87.0% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-08T06:18:03.970 INFO:teuthology.orchestra.run.smithi192.stderr: /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mds.i.log 2024-05-08T06:18:03.970 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mgr.z.log: /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mds.i.log: 90.4% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-08T06:18:03.974 INFO:teuthology.orchestra.run.smithi163.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph.log 2024-05-08T06:18:03.974 INFO:teuthology.orchestra.run.smithi163.stderr:: No such file or directory 2024-05-08T06:18:03.974 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-08T06:18:03.974 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-osd.7.log 2024-05-08T06:18:03.974 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph.log: /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mds.b.log 2024-05-08T06:18:03.974 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-osd.4.log 2024-05-08T06:18:03.975 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-osd.1.log 2024-05-08T06:18:03.975 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mgr.y.log 2024-05-08T06:18:03.975 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mds.h.log 2024-05-08T06:18:03.975 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mgr.y.log: 87.1% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-08T06:18:03.975 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-osd.10.log 2024-05-08T06:18:03.975 INFO:teuthology.orchestra.run.smithi163.stderr: 90.3% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-08T06:18:03.975 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-08T06:18:03.975 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5/var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-osd.10.log: --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mds.e.log 2024-05-08T06:18:03.975 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph.cephadm.log: 82.5% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-08T06:18:03.975 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-08T06:18:03.981 INFO:teuthology.orchestra.run.smithi192.stderr: 92.7% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mgr.z.log.gz 2024-05-08T06:18:03.981 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-08T06:18:03.981 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-osd.11.log 2024-05-08T06:18:03.981 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-osd.8.log 2024-05-08T06:18:03.981 INFO:teuthology.orchestra.run.smithi192.stderr: 82.4% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-08T06:18:03.981 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-08T06:18:03.983 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mds.e.log: 92.7% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mgr.y.log.gz 2024-05-08T06:18:03.984 INFO:teuthology.orchestra.run.smithi163.stderr: 94.5% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mds.e.log.gz 2024-05-08T06:18:03.984 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mon.b.log 2024-05-08T06:18:03.984 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mds.k.log 2024-05-08T06:18:03.988 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-osd.8.log: /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mds.l.log 2024-05-08T06:18:03.991 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-08T06:18:03.992 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mds.k.log: 91.0% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-08T06:18:03.996 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-08T06:18:03.996 INFO:teuthology.orchestra.run.smithi163.stderr: 94.5% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mds.k.log.gz 2024-05-08T06:18:03.998 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-08T06:18:03.999 INFO:teuthology.orchestra.run.smithi123.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-08T06:18:03.999 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph.log 2024-05-08T06:18:04.000 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-08T06:18:04.002 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mds.g.log 2024-05-08T06:18:04.003 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mds.j.log 2024-05-08T06:18:04.004 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mds.g.log: 87.0% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-08T06:18:04.004 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-osd.3.log 2024-05-08T06:18:04.005 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-osd.0.log 2024-05-08T06:18:04.005 INFO:teuthology.orchestra.run.smithi123.stderr: 90.2% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-08T06:18:04.006 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mgr.x.log 2024-05-08T06:18:04.006 INFO:teuthology.orchestra.run.smithi123.stderr: 94.7% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mds.g.log.gz 2024-05-08T06:18:04.007 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mon.a.log 2024-05-08T06:18:04.008 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 92.5% 94.7% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mds.l.log.gz 2024-05-08T06:18:04.009 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mgr.x.log: 94.6% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mds.j.log.gz 2024-05-08T06:18:04.009 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-osd.6.log 2024-05-08T06:18:04.010 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-08T06:18:04.011 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mon.a.log: /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mds.a.log 2024-05-08T06:18:04.013 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-08T06:18:04.013 INFO:teuthology.orchestra.run.smithi123.stderr: 82.8% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-08T06:18:04.014 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mds.d.log 2024-05-08T06:18:04.016 INFO:teuthology.orchestra.run.smithi192.stderr: 94.3% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mds.c.log.gz 2024-05-08T06:18:04.016 INFO:teuthology.orchestra.run.smithi192.stderr: -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-08T06:18:04.016 INFO:teuthology.orchestra.run.smithi192.stderr: 91.0% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-08T06:18:04.017 INFO:teuthology.orchestra.run.smithi192.stderr: 94.2% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mds.f.log.gz 2024-05-08T06:18:04.018 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-volume.log: /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-osd.9.log 2024-05-08T06:18:04.020 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-08T06:18:04.020 INFO:teuthology.orchestra.run.smithi192.stderr: 92.6% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mds.i.log.gz 2024-05-08T06:18:04.029 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-osd.9.log: /var/log/ceph/cephadm.log: 94.5% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mds.d.log.gz 2024-05-08T06:18:04.033 INFO:teuthology.orchestra.run.smithi123.stderr: 90.9% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-08T06:18:04.048 INFO:teuthology.orchestra.run.smithi123.stderr: 93.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-08T06:18:04.077 INFO:teuthology.orchestra.run.smithi123.stderr: 92.5% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mds.a.log.gz 2024-05-08T06:18:04.211 INFO:teuthology.orchestra.run.smithi123.stderr: 90.1% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mgr.x.log.gz 2024-05-08T06:18:04.356 INFO:teuthology.orchestra.run.smithi192.stderr: 91.8% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mon.c.log.gz 2024-05-08T06:18:04.374 INFO:teuthology.orchestra.run.smithi163.stderr: 91.8% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mon.b.log.gz 2024-05-08T06:18:04.424 INFO:teuthology.orchestra.run.smithi163.stderr: 95.6% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mds.h.log.gz 2024-05-08T06:18:05.062 INFO:teuthology.orchestra.run.smithi123.stderr: 90.3% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mon.a.log.gz 2024-05-08T06:18:05.160 INFO:teuthology.orchestra.run.smithi163.stderr: 92.1% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-osd.7.log.gz 2024-05-08T06:18:05.219 INFO:teuthology.orchestra.run.smithi192.stderr: 92.9% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-osd.11.log.gz 2024-05-08T06:18:05.287 INFO:teuthology.orchestra.run.smithi123.stderr: 92.6% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-osd.9.log.gz 2024-05-08T06:18:05.322 INFO:teuthology.orchestra.run.smithi163.stderr: 92.2% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-osd.4.log.gz 2024-05-08T06:18:05.388 INFO:teuthology.orchestra.run.smithi123.stderr: 92.4% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-osd.6.log.gz 2024-05-08T06:18:05.408 INFO:teuthology.orchestra.run.smithi123.stderr: 92.4% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-osd.3.log.gz 2024-05-08T06:18:05.441 INFO:teuthology.orchestra.run.smithi123.stderr: 92.2% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-osd.0.log.gz 2024-05-08T06:18:05.444 INFO:teuthology.orchestra.run.smithi123.stderr: 2024-05-08T06:18:05.444 INFO:teuthology.orchestra.run.smithi123.stderr:real 0m1.470s 2024-05-08T06:18:05.445 INFO:teuthology.orchestra.run.smithi123.stderr:user 0m6.299s 2024-05-08T06:18:05.445 INFO:teuthology.orchestra.run.smithi123.stderr:sys 0m0.618s 2024-05-08T06:18:05.545 INFO:teuthology.orchestra.run.smithi192.stderr: 92.0% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-osd.8.log.gz 2024-05-08T06:18:05.561 INFO:teuthology.orchestra.run.smithi192.stderr: 92.7% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-osd.2.log.gz 2024-05-08T06:18:05.621 INFO:teuthology.orchestra.run.smithi163.stderr: 92.5% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-osd.1.log.gz 2024-05-08T06:18:06.107 INFO:teuthology.orchestra.run.smithi163.stderr: 92.0% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-osd.10.log.gz 2024-05-08T06:18:06.419 INFO:teuthology.orchestra.run.smithi192.stderr: 94.3% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-osd.5.log.gz 2024-05-08T06:18:06.420 INFO:teuthology.orchestra.run.smithi192.stderr: 2024-05-08T06:18:06.420 INFO:teuthology.orchestra.run.smithi192.stderr:real 0m2.465s 2024-05-08T06:18:06.421 INFO:teuthology.orchestra.run.smithi192.stderr:user 0m7.168s 2024-05-08T06:18:06.421 INFO:teuthology.orchestra.run.smithi192.stderr:sys 0m0.259s 2024-05-08T06:18:07.519 INFO:teuthology.orchestra.run.smithi163.stderr: 93.4% -- replaced with /var/log/ceph/85cfb2e6-0cff-11ef-bc97-c7b262605968/ceph-mds.b.log.gz 2024-05-08T06:18:07.521 INFO:teuthology.orchestra.run.smithi163.stderr: 2024-05-08T06:18:07.522 INFO:teuthology.orchestra.run.smithi163.stderr:real 0m3.565s 2024-05-08T06:18:07.522 INFO:teuthology.orchestra.run.smithi163.stderr:user 0m10.412s 2024-05-08T06:18:07.522 INFO:teuthology.orchestra.run.smithi163.stderr:sys 0m0.367s 2024-05-08T06:18:07.522 INFO:tasks.cephadm:Archiving logs... 2024-05-08T06:18:07.522 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697251/remote/smithi123/log 2024-05-08T06:18:07.523 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T06:18:08.544 DEBUG:teuthology.misc:Transferring archived files from smithi163:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697251/remote/smithi163/log 2024-05-08T06:18:08.546 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T06:18:10.044 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697251/remote/smithi192/log 2024-05-08T06:18:10.045 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T06:18:11.170 INFO:tasks.cephadm:Removing cluster... 2024-05-08T06:18:11.170 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 --force 2024-05-08T06:18:11.492 INFO:teuthology.orchestra.run.smithi123.stdout:Deleting cluster with fsid: 85cfb2e6-0cff-11ef-bc97-c7b262605968 2024-05-08T06:18:12.119 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 --force 2024-05-08T06:18:12.312 INFO:teuthology.orchestra.run.smithi163.stdout:Deleting cluster with fsid: 85cfb2e6-0cff-11ef-bc97-c7b262605968 2024-05-08T06:18:12.589 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 85cfb2e6-0cff-11ef-bc97-c7b262605968 --force 2024-05-08T06:18:12.779 INFO:teuthology.orchestra.run.smithi192.stdout:Deleting cluster with fsid: 85cfb2e6-0cff-11ef-bc97-c7b262605968 2024-05-08T06:18:13.032 INFO:tasks.cephadm:Removing cephadm ... 2024-05-08T06:18:13.032 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-08T06:18:13.091 DEBUG:teuthology.orchestra.run.smithi163:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-08T06:18:13.106 DEBUG:teuthology.orchestra.run.smithi192:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-08T06:18:13.121 INFO:tasks.cephadm:Teardown complete 2024-05-08T06:18:13.122 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-08T06:18:13.139 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-08T06:18:13.140 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T06:18:13.145 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T06:18:13.149 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T06:18:13.297 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-08T06:18:13.308 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-08T06:18:13.308 DEBUG:teuthology.orchestra.run.smithi123:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T06:18:13.315 DEBUG:teuthology.orchestra.run.smithi163:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T06:18:13.318 DEBUG:teuthology.orchestra.run.smithi192:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T06:18:13.330 INFO:teuthology.orchestra.run.smithi163.stderr:bash: line 1: ntpq: command not found 2024-05-08T06:18:13.331 INFO:teuthology.orchestra.run.smithi192.stderr:bash: line 1: ntpq: command not found 2024-05-08T06:18:13.334 INFO:teuthology.orchestra.run.smithi163.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T06:18:13.335 INFO:teuthology.orchestra.run.smithi163.stdout:=============================================================================== 2024-05-08T06:18:13.335 INFO:teuthology.orchestra.run.smithi163.stdout:^* hv01.front.sepia.ceph.com 3 6 377 24 +500us[ +480us] +/- 66ms 2024-05-08T06:18:13.335 INFO:teuthology.orchestra.run.smithi163.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 22 -793us[ -793us] +/- 64ms 2024-05-08T06:18:13.335 INFO:teuthology.orchestra.run.smithi163.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 24 +497us[ +476us] +/- 89ms 2024-05-08T06:18:13.335 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T06:18:13.336 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T06:18:13.336 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2024-05-08T06:18:13.336 INFO:teuthology.orchestra.run.smithi192.stdout:^* hv01.front.sepia.ceph.com 3 6 377 23 +502us[ +483us] +/- 66ms 2024-05-08T06:18:13.336 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 23 -822us[ -822us] +/- 64ms 2024-05-08T06:18:13.336 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 23 +469us[ +469us] +/- 89ms 2024-05-08T06:18:13.336 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T06:18:13.363 INFO:teuthology.orchestra.run.smithi123.stderr:bash: line 1: ntpq: command not found 2024-05-08T06:18:13.373 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T06:18:13.373 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2024-05-08T06:18:13.373 INFO:teuthology.orchestra.run.smithi123.stdout:^* hv01.front.sepia.ceph.com 3 7 377 86 +427us[ +410us] +/- 65ms 2024-05-08T06:18:13.373 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 24 -890us[ -890us] +/- 64ms 2024-05-08T06:18:13.374 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 22 +465us[ +465us] +/- 89ms 2024-05-08T06:18:13.374 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T06:18:13.375 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-08T06:18:13.386 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-08T06:18:13.386 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-08T06:18:13.413 DEBUG:teuthology.orchestra.run.smithi123:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T06:18:13.598 DEBUG:teuthology.orchestra.run.smithi163:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T06:18:13.662 DEBUG:teuthology.orchestra.run.smithi192:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T06:18:13.726 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T06:18:13.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T06:18:13.819 DEBUG:teuthology.orchestra.run.smithi163:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T06:18:13.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T06:18:13.845 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T06:18:13.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T06:18:13.872 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-08T06:18:13.883 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-08T06:18:13.894 INFO:teuthology.task.internal:Duration was 1853.799113 seconds 2024-05-08T06:18:13.894 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-08T06:18:13.906 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-08T06:18:13.906 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T06:18:13.914 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T06:18:13.916 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T06:18:13.950 INFO:teuthology.orchestra.run.smithi163.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T06:18:13.952 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T06:18:14.041 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T06:18:14.341 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-08T06:18:14.341 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi123.front.sepia.ceph.com 2024-05-08T06:18:14.341 DEBUG:teuthology.orchestra.run.smithi123:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T06:18:14.416 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi163.front.sepia.ceph.com 2024-05-08T06:18:14.417 DEBUG:teuthology.orchestra.run.smithi163:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T06:18:14.442 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi192.front.sepia.ceph.com 2024-05-08T06:18:14.443 DEBUG:teuthology.orchestra.run.smithi192:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T06:18:14.469 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-08T06:18:14.469 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T06:18:14.477 DEBUG:teuthology.orchestra.run.smithi163:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T06:18:14.484 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T06:18:14.563 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-08T06:18:14.563 DEBUG:teuthology.orchestra.run.smithi123:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T06:18:14.568 DEBUG:teuthology.orchestra.run.smithi163:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T06:18:14.570 DEBUG:teuthology.orchestra.run.smithi192:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T06:18:15.345 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-08T06:18:15.356 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-08T06:18:15.356 DEBUG:teuthology.orchestra.run.smithi123:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T06:18:15.439 DEBUG:teuthology.orchestra.run.smithi163:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T06:18:15.464 DEBUG:teuthology.orchestra.run.smithi192:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T06:18:15.490 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-08T06:18:15.502 DEBUG:teuthology.orchestra.run.smithi123:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T06:18:15.510 DEBUG:teuthology.orchestra.run.smithi163:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T06:18:15.512 DEBUG:teuthology.orchestra.run.smithi192:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T06:18:15.533 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern = core 2024-05-08T06:18:15.553 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = core 2024-05-08T06:18:15.599 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = core 2024-05-08T06:18:15.641 DEBUG:teuthology.orchestra.run.smithi123:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T06:18:15.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T06:18:15.688 DEBUG:teuthology.orchestra.run.smithi163:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T06:18:15.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T06:18:15.702 DEBUG:teuthology.orchestra.run.smithi192:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T06:18:15.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T06:18:15.717 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-08T06:18:15.728 INFO:teuthology.task.internal:Transferring archived files... 2024-05-08T06:18:15.729 DEBUG:teuthology.misc:Transferring archived files from smithi123:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697251/remote/smithi123 2024-05-08T06:18:15.729 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T06:18:15.841 DEBUG:teuthology.misc:Transferring archived files from smithi163:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697251/remote/smithi163 2024-05-08T06:18:15.841 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T06:18:15.877 DEBUG:teuthology.misc:Transferring archived files from smithi192:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697251/remote/smithi192 2024-05-08T06:18:15.877 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T06:18:15.915 INFO:teuthology.task.internal:Removing archive directory... 2024-05-08T06:18:15.915 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T06:18:15.923 DEBUG:teuthology.orchestra.run.smithi163:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T06:18:15.925 DEBUG:teuthology.orchestra.run.smithi192:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T06:18:15.974 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-08T06:18:15.990 INFO:teuthology.task.internal:Not uploading archives. 2024-05-08T06:18:15.990 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-08T06:18:16.011 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-08T06:18:16.011 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T06:18:16.019 DEBUG:teuthology.orchestra.run.smithi163:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T06:18:16.021 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T06:18:16.033 INFO:teuthology.orchestra.run.smithi163.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 06:18 /home/ubuntu/cephtest 2024-05-08T06:18:16.036 INFO:teuthology.orchestra.run.smithi192.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 06:18 /home/ubuntu/cephtest 2024-05-08T06:18:16.068 INFO:teuthology.orchestra.run.smithi123.stdout: 393249 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 06:18 /home/ubuntu/cephtest 2024-05-08T06:18:16.072 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-08T06:18:16.084 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-08T06:18:16.116 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs:workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/direct_io}} duration: 1853.7991125583649 flavor: default owner: scheduled_vshankar@teuthology success: true 2024-05-08T06:18:16.116 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T06:18:16.202 INFO:teuthology.run:pass